Number of warning crossed 50 while reading from a sequential

harikhk · Post by **harikhk** » Fri Oct 02, 2015 2:31 pm

While reading data from a fixed width sequential file, have got warnings and crossed 50 and the job got aborted.

The next stage to sequential file is a transformer stage and also there is a reject link on the source sequential file stage.

The flow of execution is as below
A script executes the command dsjob to initiate a sequence and the sequence in turn calls the above job(which is getting aborted due to warnings limit).

Even though I have called this script from datastage director by setting the warning to No limt...The job still fails.

I have come across some blogs that to avoid this warning limit(say 200), dsjob to be used with warn parameter or by demoting the warnings.

But this may be counter productive when the warning limit crosses he defined.

* The source file is out of my control for modification
* When there is a reject link on the source file(fixed width) isn't it not sufficient to handle this?

Please help me with your thoughts how to overcome this problem

chulett · Post by **chulett** » Fri Oct 02, 2015 3:10 pm

You can't "call a script" from the Director, so a little lost there. When the script runs the job, why not use the -warn option there to change the number of warnings from 50? You can use -warn 0 for unlimited or something higher than 50 to increase it.

harikhk · Post by **harikhk** » Fri Oct 02, 2015 3:27 pm

Hey Craig,

To clarify..when I say running a script from director, I have created a sequence job which has Execute_Command wherein I mention the script.

Yes I can use warn(0) parameters along with the dsjob command which may not be a good practice to adopt.

The topic which I am confused is "When there is a reject file from the sequential file stage does it not ignore the warnings and continue the process"

chulett · Post by **chulett** » Fri Oct 02, 2015 4:38 pm

I agree that setting warnings to unlimited is not a good practice to adopt as a general statement but on a case by case basis? Perhaps.

Sorry, can't really answer the reject link question other than to say that if it generates warnings there's no way for the logging mechanism to ignore them based on where they come from - it's an all or nothing affair. AFAIK, you either need to suppress the warnings or adjust (or eliminate) the warning threshold accordingly.

Others may have more better information for you, though. Let's see!

ray.wurlod · Post by **ray.wurlod** » Fri Oct 02, 2015 4:52 pm

Better practice is to identify the causes of the warnings and to try to eliminate them.

When you run a job from Director or Designer the Job Run Options dialog allows you to set a non-default limit on the number of warnings.

qt_ky · Post by **qt_ky** » Fri Oct 02, 2015 5:37 pm

ray.wurlod wrote:Better practice is to identify the causes of the warnings and to try to eliminate them.

Agreed. Perhaps that is the best practice.

ray.wurlod · Post by **ray.wurlod** » Sat Oct 03, 2015 2:06 am

<grammar nazi>
It would be best if there were three or more options. Here, however, there were only two options (to fix, or not to fix, that is the question), so the comparative adjective "better" should be used.
</grammar nazi>

qt_ky · Post by **qt_ky** » Sat Oct 03, 2015 7:14 am

And Craig's "more better" slides by?

qt_ky · Post by **qt_ky** » Sat Oct 03, 2015 7:29 am

Having a reject file from the Sequential File stage is good in that it will let you examine which records were rejected.

Having warnings when records are rejected is good in that it serves as your alert. It is alerting someone to wake up and examine why records are rejected.

One use for that is to take the information back to the source of the records and let them know they are sending you junk. If they don't care and aren't going to fix the problem, then you could demote the warnings to informational status.

So, it seems like you have three or more options:

1) do nothing

2) have the source system fix the records before creating the file(s)

3) create a preprocessing job to tidy up the files yourself before the existing job runs

4) use message handler to make warnings informational

5) use the dsjob -run -warn 0 option

One of the choices is likely "more better" than the other...

priyadarshikunal · Post by **priyadarshikunal** » Sun Oct 04, 2015 1:16 am

qt_ky wrote:And Craig's "more better" slides by?

He already suggested a better approach than his last approach which may not be the best, hence "more better"

PS: it just fell shy of being much better

ray.wurlod · Post by **ray.wurlod** » Sun Oct 04, 2015 3:26 am

I gave up on Craig and grammar many moons ago.

chulett · Post by **chulett** » Sun Oct 04, 2015 8:09 am

harikhk · Post by **harikhk** » Mon Oct 05, 2015 8:20 am

Thank you all for your suggestions.
Would try the possible solutions mentioned and implement the suitable one.
Marking the topic closed