DSXchange

Reimporting all jobs can be excessive for so simple of a problem. The log settings are a single row in each and every log file for a job. Change a couple of attributes (columns) in that row and viola! Purge settings are updated. It's a shame that release 7.5 still doesn't have a feature in DS Manage...

If the data is sorted in ascending order, and you're starting with 10M rows and will only end up with 7M when done, your 30% of repeated rows are almost negligle. Just process all of the rows, writing to a hash file using the unique identifier (but not occurence number) as the key. The last row writ...

I've seen this exact behavior when the total number of characters in a function/routine exceeded a certain amount. No error messages, nothing, just missing text. You could type in a lot of text, but upon saving and re-opening all of it was gone. I ended up breaking the routine up with some external ...

The job ran for 2 1/2 hours trying to pull out 13 million rows. Doesn't this seem to be an excessive task? This means for 2 1/2 hours you have an open cursor on 13 million source rows holding all of that data in rollback. This is bad. You need to NOT have that data held that long, I don't care what ...

This is a point of style. Relying on the stage to behave poorly (IMO), meaning to silently toss data, is poor style. You should affirmatively handle all data. If you don't want a row, build the logic to purposely remove the remove. If yo do want specific rows, build the logic to purposely include on...

Why complicate something so simple?

DataStage jobs DO something. All you need to do is run the jobs. What would you vary, job parameters? Different data in the source and target? The jobs are static.

Just run the jobs when necessary via the command line interface program dsjob.

count is a reserved word because it's an internal function. You need to called your function something else. See if that clears up your problem.

Rather than counters you should use modular programing techniques. Nested subroutines and functions are essential parts of good programming practices. You could periodically call a function that checks a directory for the existence of a file. That file could be a STOP, PAUSE, ABORT, STATUS, etc set ...

Write a ksh script that uses the results of a wc -l to issue the appropriate head and tail commands, streaming to stdout. Put that ksh into the filter on the Sequential stage.

Did you check the job log for reject messages? Did you add a reject link to collect DB rejects?

If the same exact link with constraint can write to a sequential text file, then the only issue is the database. What do the link statistics show for the job?

You're doing a concatentation using a variable that has not be previously declared. Chances are that you have either a misspelled link.column name or a stage variable that is spelled in a different case than how it is declared. You'll find it, go back into the transformer and figure it out.

Go over to the IBM website and find the documentation for the Universe product. The DS Engine is a code-branch of that, most of the documentation is still relative.

Not to sound cruel, but if you allowed 2 weeks of work to happen without periodically exporting and backing up your work, then it's really your fault you're in this situation. As a best practice, you should: 1. Automate nightly exports of all development projects 2. Automate nightly file system back...

Creating files with embedded reserved characters is usually a no-no. These would be chars such as:

In the uvconfig file there is a temp variable that defaults to /tmp. You could consider mapping this to a different directory if you wish. There's also the variables in the dsenv.

Server's pretty negligible on /tmp usage. You'll probably be fine.