DSXchange

While you can cat the contents of the filelist to the stage to give it a list of files to process, you cannot at the same time pipe it through grep like that. Well, you can (obviously) but it's not doing what you think it is doing. All the grep is being applied to are the filenames themselves, not t...

Wait... over and over? (sniff) Note to self...

And yet another edit - now that you mention it, I do believe we had this same conversation last time this came up, at least up to the "hey, it actually does exist!" part. Thanks for taking the time to actually go through the workings of it.

It is somewhat equivalent. The Folder stage (from what I recall) was originally (only?) meant to be used with the XML stages. It only supports 2 input ports, one contains the filename being processed and the other supplies the entire contents of the file all in one burst. Hence the fit for XML proce...

Sadly, probably not. And you might want to check your date math routine.

Nice... if it does support a file containing "a list of files" to read then it is exactly like Informatica's "indirect" read option. Wonder when that was added?

However, does it handle a header record on every file in that case?

Had to look your dang Aussie slang up... so squib - a "firecracker that does not go off" seems to be the appropriate translation here, yes? This rather than the word I know - a device used in movies to simulate a gunshot wound. :D Mainframes have been 'going away' and COBOL has been a 'dea...

I agree that setting warnings to unlimited is not a good practice to adopt as a general statement but on a case by case basis? Perhaps. Sorry, can't really answer the reject link question other than to say that if it generates warnings there's no way for the logging mechanism to ignore them based on...

You can still use the Filter in this case as long as whatever is supplying the data to stdout - say Perl or awk or sed or your utility of choice - strips the first and/or last record as needed from each file.

It's a bit Old School but it does work. Or can be made to.

You can't "call a script" from the Director, so a little lost there. When the script runs the job, why not use the -warn option there to change the number of warnings from 50? You can use -warn 0 for unlimited or something higher than 50 to increase it.

There's no "filelist" concept in DataStage like the indirect reads that Informatica supports, which I think is too damn bad. You may be able to leverage the Filter option in the Sequential stage, you'd just need something at the command line to send the files to stdout with the first recor...

From the command line you use dsjob and leverage the various "getlog" functions it supports to query / extract the information you need from the various job log tables. It's all well documented in the chapter (manual?) on the DataStage Command Line Interface.

Not all are listed "out of the box". Create it and see what happens.

Sorry... "Project_Plus"?

There are no "special characters" in that string. If you want the dashes removed, look into a function like Convert() as one option.