DSXchange

I'm about to build a 4 terabyte datawarehouse for a customer. Their requirements are for realtime loading. The first question ss: Does realtime mean within seconds, or within minutes? The second is: Are you doing OLTP to data warehouse or OLTP to OLTP? OLTP to OLTP is data bridging and EAI, I won't ...

Read this post, I put logic for a routine that doesn't have the overhead issues you're talking about.

http://www.dsxchange.com/viewtopic.php? ... eyparallel

Folks, the problem here is simple. There is a single Oracle process spooling data once a query has resolved and produces output. There is a finite rate at which a single process can spool data out of Oracle. Once you've achieved that top rate, NOTHING will speed it up. The ONLY SOLUTION is to have m...

Here's some SQL to demonstrate a way to do what you want: create table kcb_values (increasing_nbr integer); insert into kcb_values (select rownum from all_tables where rownum <= 100); create table kcb_work (your_key integer, max_nbr integer); insert into kcb_work (select ...

Are you going from Server to Server, or Server to Parallel? The answer is easy, Server to Server upgrades pretty easily by just installing the new version and importing the jobs and routines. However, you need to deal with upgrading any to any plugins that are now supported under the new release. In...

I see two options, one easy, the other scalable. Easy solution: Put N output links from a transformer to a link collector or a series of files. Each link has a constraint that only allows the row if its fixed value is less than your looping value. Scalable solution: Load your source data into a hash...

The stage does not rewrite your query to take advantage of SQL extensions specific to the database. So, make sure you use portable SQL that works across all platforms. If you need values like a system date, you will need to use job parameters and feed them into your job/query. You will NOT be able t...

Ayyyeeesh. 24+ hours??? 8 row/s???

Horrorcle needs a bulk-unloader. Visit m and get his code for a pro*C bulk-unloader. Or, shell out some bucks and buy CoSorts multi-threaded bulk-unloader for Horrorcle. Another option is to upgrade to PX, which has similar multi-processing capabilities to get data out quickly. Otherwise, it's the "...

Me, I like simple SQL to pivot the data: select ID, MAX(colA) "colA", MAX(colB) "colB", MAX(colC) "colC" from (select ID, case when Colname = 'colA' then New_Value else NULL end "colA", case when Colname = 'colB' then New_Value else...

I'm going to avoid asking why you find a runtime of anything measured in days to be acceptable performance. You may consider that your process died and doesn't know it. It could be that you exceeded a 2.2 gigabyte limit for your particular release of DS and OS if you're constrained by 32-bit files. ...

Write SQL appropriate to the database you are using.

Sorry You Bought A Slow Engine

You're not improving the speed. What you will be doing is distributing the effort across multiple identical processes, each handling a specific set of the data. This concept is called "partitioned parallelism". You have partitioned the data, and by running multiple identical process simultaneously (...

You need to add a ranging where clause and run multiple instances of your job, that way you have multiple connections to the table each extracting a portion of the data.