DSXchange

eldonp - At my last site, a large AIX implementation, I noticed that several times on some of the engine that dynamic files were corrupted, upon further inspection it seems that the DATA.30 files were *poof* gone. Over the course of months this happened several times and I was never able to track do...

Just cat them together from the Windows command line; i.e. "a.txt + b.txt + c.txt"

Are you stating that if you take the same query and put it into your PX sybase stage and write that to a sequential file with no other stages in the job, you get only half as many rows?

There isn't enough information to make any sort of an analysis; it is like asking a technician at the auto shop to "sometimes my car won't start and sometimes it stalls in traffic. What is wrong?".

Replace your database output stages with peek stages to see if the hangs are caused by writes.

Although this is a topic that has seen a lot of posts, I'll make a stab at an answer. Somewhere in your stage you are using DMA_CODE without having handled nulls. Remember, this also includes use of constraints.

No advantages to using all 5 at all, just performance disadvantages! You've hit the nail on the head, it is sufficient to partition on just one column.

Do you expect a column called "sortRec" in your input stream? What stages do you have? Have you tried to disable combination to see if you get a stage name that is causing the error?

The &PH& file (or directory) comes from the term "phantom" occasionally seen in DataStage; this was the name given to processes executed in the background and their standard output (at the time called como files, hence the existence of the now little-used &COMO& directory) would be s...

You can issue the UNIX command "orchadmin dump {yourfile} > {sequential file}" to create a sequential file. But doing this from within the job that generates the dataset is useless. How about generate a fileset - you get sequential files but still have the advantage given by PX parallelism?

Ideally you have your data hash partitioned on your 5 columns, so that each node at runtime only needs to sort data in its own stream and does not need to repartition.

That is too broad a question, similar in scope to me asking "Can someone tell me about what approach to take with politics".
DataStage does what you tell it to do; so you need to know what you would like to do and then the appropriate question is "How can I make DataStage do it that way".

Why would you want to repeatedly backup your production projects? The only things that should change regularly are the log contents. I'd only backup after applying new jobs or changes and perhaps ensure quarterly backups - but daily or weekly would be overkill and only cause overhead. On an annual b...

Yes, the line numbers are referring to the generate code. And yes, there is an offset that sometimes needs to be applied. Ever since the dawn of time (well, call it 1982 when I started using UniVerse) there has been an issue with the compiler getting its line numbers incorrect on some lines (stupidl...

You mean add the record length of each row to the end of the row, or keep a total of the sum of all lengths in a stage variable?

The important one is to ensure that the &PH& directory gets flushed regularly. This is usually done by setting the autopurge at a project and job level, but if that fails for some reason this directory can fill up quickly on a busy system and can cause real job slowdowns. If you put in a dai...