DSXchange

By "hits daylight savings time" you mean the system clock changes while the transformation job and batch job are running? If that's the case you should contact tech support. If the DSWaitForJob API is hanging that needs to be determined. That infinite wait is only satisfied by the transformation job...

That depends on what you're doing in your custom job control logic. Is it polling in a loop the jobs it is executing, or it is infinitely hung with a DSWaitForJob API call? If it's in a polling loop, do you have any time logic going on, such as monitor job execution time? As for running jobs during ...

There's absolute metaphysical certitude that your rows are ordered correctly, and then there's everything else. Even if performance is 1/10th your current row/sec, you're still chugging along quite well. You roll those dice. I'm fortunate in that my data is never sorted going in, so I just end up co...

The collector sort/merge reads multiple sorted input links. Since your data is sorted, use this collection type instead of round robin. Designate the sort key column, and you're all set. It should maintain the sort order when collating the link rows. In result it will just do a round-robin, but with...

1. The utility works well. It changes your OCI8 to OCI stages in release 7.1+. The OCI stage formerly was for Oracle 7.3, but that support is dropped in 7.1+ so they rebranded OCI for 9. There's no OCI10 stage. To work with 10g database, use OCI9 and Oracle 9.2 client 32 bit libraries. 2. No. OCIs h...

Don't trust the alternate without using the sorting to guarantee it. You might get them out of order. I didn't suggest stage variables because the aggregator is more scalable if you have a lot of data columns. The AGG works well even with 100's of columns. I use this technique quite often to great s...

I would use a SEQ stage to read file 1 and go to a XFM and leave the data in the column position it's in, but add a third column with a derivation of "". I would add another SEQ stage to read file 2 and go to a XFM and map the data into the third column created just like above, but put "" for the de...

It's a factor of the average number of characters in a typical row, packet sizes, etc. You can hit the help button on the ODBC/OCI stages and see the explanation more clearly. You're going to need to profile your data to figure out the average row size and how many rows will fit into the array optim...

Knowing nothing about Teradata, my guess would be that the stage has the option to act like a quasi-active stage if using the pipe. As rows complete in the transformation portion of the job they'll be sent to the pipe, which means that the load is active during transformation. Writing to a file mean...

Typically batch processing is what people think of ETL and relates to OLAP. Realtime, where singular transactions are sent over message queues, was called EAI and relates to OLTP. Most realtime data warehouses are of the ETL nature, using microbatches, which mean every 60 seconds or so the process w...

Well, I know nothing of Teradata SQL, but thought it was a good shot. What seems weird is that yor data range implies that the dates of the data have to be in that range just in order to be selected, however, your commentary is that a julian value is what is output. To reverse the value 1721424, bre...

Why don't you go read the existing answers to the same post you left earlier today?

http://www.dsxchange.com/viewtopic.php?t=96179

By first scanning your source file for the large table keys, bulk loading into work table in Oracle, and then inner-join the big table to the work table and spool to hash files, you reduce their volume to what works well with hash files. In all cases, only spool the columns necessary for your trans...

Are the number of jobs in dev and test the same? Have you tried creating a new project and using that as test? Potentially, you could have filled up the project directory with a bunch of junk files and such. Also, thru years of repeated importing, the underlying files may be dynamically sized poorly...