Why and Why?

[DS_FocusGroup]

why is it that when i disable the APT_Disable_combination operator it doesnt reject the records with null values and when i enable it it does.

why is it that when i have done ulimit -f unlimited and it says fsize unlimited i am getting errors such as output file is full ?

p.s i knwo i will be battered up by ray for putting 2 questions in one forum !

[ray.wurlod]

You have not given us sufficient information. What is "it"? What is the job design for the first question? Disabling operator combination alone should not make any difference to null handling - you would have to look at the score to determine precisely what was being executed.

ulimit -f sets the maximum file size, but does not prevent "disk full" from occurring. Nor does ulimit -f unlimited prevent a "file full" error if the application itself sets an upper limit on the size of a file. For example some applications limit file size to 2GB irrespective of whether the operating system has large file support or not. So which file was it that filled? A data file of a Data Set perhaps?

[ray.wurlod]

Castigation Notes

One question per thread please. It makes it easier for future searchers.

For the same reason please try to use a meaningful subject.

Please get the details right. APT_DISABLE_COMBINATION is an environment variable, not an operator, and its name is given in all upper case.

Properly formed English sentences begin with a captial letter.

[vijayrc]

Castigation Notes

One question per thread please. It makes it easier for future searchers.

For the same reason please try to use a meaningful subject.

Please get the details right. APT_DISABLE_COMBINATION is an environment variable, not an operator, and its name is given in all upper case.

Properly formed English sentences begin with a captial letter.

I see Ray's location as Bangalore.India now
Ray , you are in Bangalore now ??

[Teej]

why is it that when i disable the APT_Disable_combination operator it doesnt reject the records with null values and when i enable it it does.

why is it that when i have done ulimit -f unlimited and it says fsize unlimited i am getting errors such as output file is full ?

p.s i knwo i will be battered up by ray for putting 2 questions in one forum !

For your first question, this could be indicating many different things. To summarize:

APT_DISABLE_COMBINATION does what it say -- it tells the engine to not combine certain set of stages in order to reduce the length of time it take to run a job.

You may be witnessing a bug, so further details of the job's structure, along with the specific version of DataStage and the operating system would help.

* * *

As for the second question, where exactly are you doing this ulimit command? The engine is not exactly using your user account to run the job (or rather, it is not using the configuration settings for your account.) It is using the administrative account (usually dsadm).

[ray.wurlod]

My location as stated is always correct. So, yes, currently I am in Bangalore.

[DS_FocusGroup]

Ok.

I got the output file full message when i was joining (left join) a table of 28 million records with a table of 4 records( more of a lookup). i was sorting both left and right link of the join before providing input to the join stage. Sort stage was used for this. Both inputs were coming from teradata enterprise stage. so the only file i think that was invloved were some temp files used during sorting or join ?

Would appreciate some help on how to solve this as the count would be increasing to 60 or 70M.

cheers.

[shershahkhan]

If i understand your problem correct, you mean to say that you have two file and you want to join them , from my point of view it will be better to join the file through lookup stage, becuase it will not require sorting if the partion of left file is round robin that will be much more effiecient becuase the right file will be present on all partions(entire partition), so the joining will be much more fast. i didn't understand your question of NULL feild drop with APT_DISABLE_COMBINATION problem. If you explain your question in more detail we will be able to help you out. If your TD node is more powerfull then do the join in TD enterprise stage and then pass the result to next stage.