DSXchange

You don't need any extra software, what you have is sufficient. But give yourself several days to set up the server and configure connectivity to the database.

Ahh, I see what you meant.

As you've discovered, the rows/second cannot be used in the way you want. If you add a sequential file stage between important stages in your job you will see that you can use rows/second to identify your slowest stages, since this breaks up the flow into distinct elements. This is not necessarily t...

The seed value is used to ensure that you get the same sampled records each run (assuming you use the same seed). Random number generators are actually "pseudo" random generators that use complex forumulas to generate sequences of numbers. With the same seed you will get the same results each time. ...

Ray - I understand your explanation and that is how I see the aggregation stage working. But I'm not sure which of the two assertions I made isn't correct - that a partially sorted table is faster than a nonsorted table or that the aggregator stage still needs to sort on those extra columns?

Each 10th row of every partition is sampled, just as you have stated.

If you use the manager to move the job to a new category does the problem go away? Also, if you have no other users on the system, you can try using the DS.TOOLS option 2 to rebuild your indices as that is sometimes a cause of existing jobs not being found.

Is the project set up in PX so that transforms will compile and run? If yes, it would seem that the include file should import the definitions - can you try to add -bloadmap or -bnoquiet to the compile for additional information on why the DllImport didn't work as expected.

sachin1 - that would work only for a server job or if a BASIC transform stage were used in a PX job. The method shown by JoshGeorge above is a common way to abort a job with a given data condition by splitting the stream and setting a trigger to abort the job if any data goes down the error stream.

It will still need to sort on 3 of the 5 keys, but partially sorted data will sort much quicker than nonsorted data.

Try this to prove the point that Ray and Craig are trying to make. Do an ANALYZE.FILE to get the modulo for the current file. Then do a "CREATE.FILE NewFile DYNAMIC GENERAL MINIMUM.MODULO {mod} 64BIT". This will create an empty file that is presized. If you can create the file, then you can issue a ...

Durga - could you explain your rule in more detail? Do you want to remove all occurrences of "100"and "***" in this column? So would the string "I have 141002 *******s to remove" turn into "I have 142 *s to remove"?

The OP stated "discard and warn", so that is what I based my response on. I think the "column contains terminators" would only work with a quoted string column or fixed length as well.

The record is discarded in the read stage, so the transform stage would never see it. I like defining the record as a single large string, then in a transform stage use the delimiter count to decide whether a record is valid, if not then concatenate it with the previous one (using a stage variable);...

Rakesh, although you haven't specified your source or target, when jobs slow down over time it is most often due to the output stage. If you are writing to a database, it might be that there are indices being built in the background or activities like buffering or rollback/before images. Try using a...