DSXchange

The external source stage lets you call up any program you want, the stage merely calls up that program and then reads whatever is sent to the program's standard input; meaning whatever the program outputs to its stdout is read by datastage. This can be a simple "cat filename.txt" or a use...

What stages does the job have? Are you doing any sorting in the job (either explicit sorts or the inserted kind, i.e. for a JOIN)? If you have many stages, does the error message look different if you include the parameter $APT_DISABLE_COMBINATION and turn that to "true"? If you replace yo...

1. What is your data type in MySql? 2. What is your data type in the ODBC stage 3. Does "View Data" in the job show 1 character or the full length? 4. Are you using standard ASCII or multibyte characters? 5. If you write to a MySql database using the same ODBC settings, do the strings also...

The two stage variables I supplied do a simple group change incremental counting - each time a key changes the counter is bumped by 1. If you want to use two counters then you need to add another stage variable. If you want different logic then please explain what you want to do.

The after-job subroutine just passes your command to the OS, just as ExecSH does. The problem doesn't lie within DataStage but somewhere in the OS. Which LINUX are you using and do you have support? It must have something to do with either forking processes or machine load that is causing sporadic i...

I would use 2 stage variables

svGroupNo If(In.Key=svLastKey) THEN svGroupNo ELSE svGroupNo+1
svLastKey In.Key

I'm not sure exactly what you want, but in a transform stage if you want to generate a unique number regardless of how many partitions you have the formula is:
Note that this will start at 1 each run.

Replace ":TAG:-" with "" and then import the copybooks.

Please show us the column definitions, without knowing how you've defined the file it is difficult to explain the error message.

I think that using this method you don't have a choice, the file is created using fixed widths (which can be parallelized easily, unlike variable width files) so preserving the original number of trailing spaces won't be possible. Is using the normal insert method an option?

I think Kryt0n might have the correct idea - what method are you using to load into Oracle - if using BULK it might be using a fixed-width file which in turn might be causing the inserting of extra spaces.

Did you write a DataStage Routine or a Function? I will assume you wrote a function, as you are writing about a "return" value. In DataStage the design interface automatically returns the value of variable "Ans". Thus we need to know how you are assigning this value in your code ...

I had forgotten about not being able to assign variable values in the stage var. Although it wastes CPU cycles, you change the derivations to read: svLastC3 = IF ((In.ID=svLastID) OR @INROWNUM=1) THEN NullToValue(In.C3,svLastC3) ELSE NullToValue(In.C3,"") etc. The OR condition would stop c...

Given your example in the initial post, what are your results now? The logic should function as expected.

Addendum: For the first row of data you should give your svLastID an initial value of the first key in the stage variable definition.

I'd give it a go with some test programs with bulk data (the Row Generator is your friend) to see the actual timings on (a) splitting inserts/upates, (b) doing the deletes then inserts and (c) doing upserts with update first. I am fairly certain that the deletes-then-inserts is going to be the slowe...