DSXchange

Sorry but that just looks like math to me... as in "add one". What's the actual problem you're having here - how to combine the two sources in a serial fashion? What have you tried? It would help if you let us know what your sources actually are... files or tables.

Not a problem... same answer applies anyway. And stop piping cats, there's absolutely no need to be doing that. If you want to know the number of records in a file, just do:

wc -l <filename>

Perform the 'cat' from the command line yourself with an invalid filename and note what gets written to the screen. That would be the 'Command Output' in that case. Why a 'cat'? What exactly are you trying to capture, does your file only have a single record in it and you need to contents of that re...

Conduct a test. Write your records to a flat file, note the time it takes. Remove all of the 'set null' handling, rerun.

Did you confirm it actually did a BULK load? (DIRECT=TRUE)

<toned down the shouting in the topic>

How will you recognize the schema being used? Do they have a fixed set of configurations or can they be pretty much any random thing?

You also said you need to parse it "and create different column"... what exactly does that mean?

In other words, you need to design that capability into your jobs. How you do that depends on many factors such as the nature of both the load and the data being loaded. Primary question in my mind is - is your source data static? Will record #65000001 be the same on the rerun as it was on the first...

SKP - I'd really like to see your answer to the second question of mine as it will change the logic you'd need to build. Repeating:

S: "abcdefghij"
R: "abddefghij" - is this a 20% match or a 90% match?

Bhanu - please use the Reply to topic button, there's no need to constantly 'Reply with quote' when we can all see the text of the message you are replying to. Thanks.

So... this is resolved?

You can't do a DIRECT (i.e. bulk) load on a table with indexes so it switched to a conventional load - inserts. You would need to add the index options it needs to allow the bulk load to happen.

You're going to have to explain how to compute this percentage, were you given any requirements or guidelines along with this task? How about examples? Source: "abcdefghij" Reference: "abcdewqar" - is this a 50% match? Reference: "abddefghij" - is this a 20% match or a ...

Not even really sure what 'percentage of match' would be. Is this QualityStage related?

You wouldn't be able to bulk load if you went the delete and insert route as that turns every insert into two operations - first a transactional delete and then the insert. Better to 'bulk' delete and then bulk load.

Not sure there's a need for that. How familiar are you with stage variables? This is pretty basic 'group change detection' logic you would need to leverage. There are a number of examples that you could adapt for this, here is one.