DSXchange

Gain more idea about software testing here. http://en.wikipedia.org/wiki/Software_testing

kashif007 - At max system integration test will be done by you and UAT will be done only by users. (Else it shouldnt be UAT ). During unit testing, you might have tested each job to work with the sample data given at your development site. Make sure you dont have any warnings or error in this level....

The column (not nullable) in the sequential file for which the warning is emerging needs a null handling, you have DefaultNullValue option in column properties. You can set to any default value. Out of three option that you have tried, just the first will help not the rest. Or you can try with handl...

rradunz wrote:Not a Shared Container, but the output from a Modify stage, yes.
Since I am new to DS, I thought the stage wasn't being run.
Many thanks to DSXchange and your prompt replies!

Is it a two modify stage, condected next to next?

You cannot change the metadata dynamically, but you can achive you requirement by some work arround. Read all the 250 column as single varchar column along with delimeter. Check for '' between two delimeter. If so exclude it. Say if you delimeter is ',' then something like if inputfield[i,2]=',,' th...

What is the output you got with 'name_#dsjobTimestamp#.xml'. If format is not as expected, try to user Iconv/Oconv to change it.

If you have append type output, you might have wrongly refferd to the previous run data, the only recored with the correct value might be for current.

If you dont wish to land three files, you need to compromise with number of stages. User transformer to split the recoreds in 3 links, user another transformer for each link to restructure, user link collector to merge all the three. Then user the sort with the sort order you per-specified.

Your code seem to be ok. Try to recompile the both the job and the routine and try again. If this the same case for most of other job, tyr REINDEX with complet access to the project.

In the 4-node example above, the order of the disks is purposely shifted for each node, in an attempt to minimize I/O contention

Hi Avishek,
Do you mean that, DS prefers the intensive I/O operation in first disk specified in the list?

Get the system date (SYSDATE) while you start the batch, and subtract with 1 day and pass it as an parameter to the job which runs for the various period. So that it wont varies for the whole flow.

Iam limited to PX access, so couldnt justy whats been said. balajisr - Dosent your job any sort (explict sort stage/ inline sort thorugh link). If no, it shows that, the stages like join, insert an implict sort during runtime (irrespective of the partition been ment). And this can be controlled only...

Check whether your file have any header information. Is it totally a 40 line file.

If so calculate the space available in scratch disk as suggested by Arnd (df -g /tmp), during the job run (parallaly). If you reach 90%, you should be ready to increase the space.

Each stage can be of single operator or of many. So if you have n stages in a job, with m nodes, you will end up with n*m process running simulteneously. There are some operators which can be combined during runtime. By allowing the combinability to true you make them to combine and hence, the requi...