DSXchange

FTP a very large file from your database server to your datastage server right now and take that as a benchmark. Once they move the servers FTP the same file and complain if its slower! You need to take DataStage out of the equation and just talk to your network guys about bandwidth and throughput o...

Sure, demonstrate using DataStage EE instead of server edition. Partitioning is automatically handled in EE. If you want to partition in server jobs you have to do it manually. Since you can't cut and dice an XML file for fear of creating invalid documents you need to read it in its entirety and the...

DataStage XML Input stage flattens XML sources, ie. Where a complex hierarchy exists DataStage turns it into one type of relational record. By split do you mean output five different record types or do you mean partition the data into five chunks? To split an XML file into five different record type...

Your client / server performance should be okay, you may see slower startup and compile times but job development on the client should be okay. Your server - server performance, as Ray suggests, is your major concern. You want data to travel directly from your database server to your ETL server with...

You can do it in a transformer, use the right mouse click menu to find a list of functions, you can do it a bit more quickly in a Modify stage. Use the Parallel Job Developers Guide to find a list of functions valid for a Modify stage spec. Note that the functions used by the Modify stage are differ...

Also be aware that occassionally a PX job will issue a megawarning and suppress further instances of that warning. So instead of seeing 100,000 warning messages for a row being rejected due to poor null handling you see just one warning and it tells you further warnings have been suppressed. That on...

Looks like you are in a Transform stage, wrap a NullToValue around your string fields to turn them into zeros or reject them out of the transformer. If you were using a Modify you would do something like decimal_from_string(handle_null(stringfield, 0)).

Do a forum search for DSExecute or look in the DataStage BASIC Guide for help writing a routine that renames a file. You can pass file names in as an argument or find file names by peeking in the directory.

I would be a bit worried about partitioning warnings, in case your partitioning meant missed lookups or aggregation rows. I would also be concerned about database upserts/inserts/updates that did not have reject links because when rows get rejected you either get a warning message you ignore or no w...

Try asking an Ascential sales person. Any benchmarks from any of us will depend on the architecture and the environment and may not be indicative of what you can expect.

Concatenate the fields in a transformer. input.key1:input.key2.

You will find Windows easier if you have a lot of Excel and MS Access sources. It depends on how your hardware support teams are setup and what the company standard platform is.

If you are talking about using search and replace on the data in the file you are better off doing it in the job via a transformer. Otherwise the easiest way to do grep is to call the grep command via dsexecute. Have a look at the DataStage BASIC Guide or do a forum search for the dozens of dsexecut...

Both ways will work and both ways will be very easy to build. It comes down to load and bottlenecks. The user-defined SQL method has the benefit of reducing the amount of data to be transferred to the DataStage server, this reduces load on the network and certainly reduces the load on the DS server....

If you partition and sort by KEY1 and KEY2 then all records belonging to a particular KEY1 / KEY2 value will be on the same partition in a sorted order. This means you can use the transformer stage variables to keep the previous value of KEY1/KEY2 and compare it to the incoming row. svNewKey input.K...