DSXchange

This problem has been reported here a couple of times but with no resolution that I remember seeing. Best to report it to Support and see what they say.

And if you do get a resolution - post it here!

Post some examples, before and after. The pivot stage really isn't about de-duping but examples would help us understand your situation.

Ok... keep in mind the OS designator in that first post is meant to tell us what platform your Server is installed on. That way we can give proper UNIX versus DOS/Windows advise in situations like this.

So, a nice little korn shell script (or your shell of choice) seems to be in your future.

No, the Server Sequential stage does not support wildcards, as least not as easily as the PX version does. Read up on the Filter option, there are times it can be leveraged for that. Use a command like: cat AA_*.dat there to send all files, one after another, to 'standard out'. The stage will then r...

I would have suggested that but have run into problems trying to use wildcarded cat commands in the Filter. Perhaps it works mo betta on Windows.

You can't just type in something like that all on one line and expect it to work there - is that what you are saying? If so, do as your subject line suggests - create a script with that as the content and execute the script via EXECSH. Or a batch file if you are on Windows. Clarify something - your ...

Heck if I know... but I would think you could Google something up rather easily.

Using the type command and redirection or copy with wildcards or the append (+) operator.

How exactly are you 'running this script in EXECSH line'? Hopefully not by putting this entire thing there, yes? Post your syntax, cut and paste directly from the Input Value box in the job to here please.

Look into remote shell execution - rsh for example.

It's just a select statement, how hard is that to implement? You don't even need 'custom' sql for it. Execute the select statement and then send the number that is returned somewhere as noted previously.

Job Parameters.

Make sure you annotate this is happening on the canvas. Before/After job/stage routines are pretty much invisible to the neked eye.

Yes... all you've handled is the Date part. You'd need to either add in a Time() function call or switch to the combo one - TimeDate? We use that one in a custom routine to get a System timestamp.

Thanks, but the table may have about 3,500,000 records, and the file about 20,000 records. The process is a daily process, and I think that this can take an important time to load the table. This is a common requirement and will not be slow - if done properly. For a file with 20,000 records, the mo...

Good to know... I'm not anywhere that I can actually test anything.