DSXchange

Hi,

I am wondering how to add a palette group? I guess its just about creating a new folder somewhere or is there something else I need to do apart from that.

Thanks

Hi Ernie,

Thanks for the suggestion, Custom operator approach might be perceivable.

At a high level could you please highlight the steps involved in building the custom operator so that I may get direction to proceed further.

Thanks

Hi, I have implemented encryption and decryption in parallel routines but the process runs slow as it has to pick up the key from disk also the transformer does not free the memory of returned pointer so the job aborts on very high volume. To enhance the speed I am planning to have a build stage whe...

DataStage 7.5.X is not IBM product and not a part of IBM Infosphere (or the earlier Websphere), it was owned by Ascential Software. Post IBM purchase of DataStage they made it a part of Websphere and released Version 8.0 and then later it was made a part of Infosphere. So the question was to have As...

If they are pcode instructions then the executable should be platform independent meaning which if I export with executable a job from unix machine and import onto windows would that work fine ?

Thanks

It might not as DataStage jobs dont have actual executables (unlike c/c++ which create executables that can be run on any like machines). When you import the job into test along with the output of compilation (with executable option of export) would still have to go through the linking phase which t...

Some components of the job like the transformer, build op's, parallel routines etc are compiled to created object files which contain binaries and if you are migrating your job to an environmnt where you dont have a compiler there is no way to get your job to run unless you have those binaries. Bina...

Use a Tranformer and put your condition as link constraint, user the "Abort After Rows" property, set it to 1 and the job will abort when there is even a single row for which you SK is greater than the preset value.

If you dont want to settle with holes in the generated column then you can either run the job on sequential mode (which you wouldn't as if it was then this thread would not have existed :D ) or use NextSurrogateKey() with a block size of 1 which again kind of serializes your processing.

The pain area while I was performing an upgrade was when the job is coded to pick a random one amongst the duplicates and this feed was used in multiple places and we had to document every such instance.

I am not sure If I understood it correctly, you need to count the number of unique records base on a column for the entire input data ? If so, in the transformer as you are doing hash partitioning, have a stage variable to count the unique records per partition pass it to output and finally have a a...

Below is a simple routine I wrote to check if my actual routine had any other memory leak or was it really due to transformer not free'ing up the returned pointer. A simple job with row generator ---> transformer --->sequential file was designed and a call to this routine was made from the transform...

I have jobs that have processed hundreds of millions of rows and never had to do that. The above is intriguing as how that is working, I can clearly see that transformer is not freeing up the returned pointer in my case. APT_CombinedOperatorController,0: Current heap size: 1,208,487,600 bytes in 25...

vamsi.4a6 wrote:What is the key column i need to mention in aggregator?

Looks like you dont have a column to group by ... !!! so what, you can always generate one. have a column generator to generate a dummy column with a static value say 1 and group by on that.

1. IF your source is a database handle this in the SQL by generating the sum and joining back to your main source on a ever true condition (1=1) 2. Non Database: you may follow a traditional fork join meathod of taking a copy and then aggregator and then joinin back to second output of copy. 3. If y...