Ofcourse if you want Optimize suggestion from DS, yours is the way. Just so you know, have you thought of doing the sorting and duplicate removals in source Teradata SQL itself if you have the liberty to modify the SQL.
There are ORDER BY NULLS FIRST and RANK() functions which satisfy your criteria.
Search found 203 matches
- Thu May 23, 2019 1:38 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Balance Optimization fails
- Replies: 5
- Views: 4205
- Thu May 23, 2019 12:43 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: NLS issues in DataStage with WIN-1252 characters
- Replies: 3
- Views: 5695
Re: NLS issues in DataStage with WIN-1252 characters
As mentioned earlier, this is a multiple iteration process till the compatible character encoding is known, unless someone responded who exactly worked on the very same encoding. I had similar situations for latin and french accented characters and it looks sometime to finish that trial. You could d...
- Wed May 22, 2019 2:55 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: NLS issues in DataStage with WIN-1252 characters
- Replies: 3
- Views: 5695
Re: NLS issues in DataStage with WIN-1252 characters
Couple of things wanted to mention: 1. You changed the DB stage setting to have required NLS. I think it should be changed at the job level to try, rather at a single stage level. Try storing sample output in a temp file to see those ? character and this could confirm if it is an issue with Netezza ...
- Wed May 22, 2019 1:49 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Need help in creating transposed file
- Replies: 8
- Views: 6732
Re: Need help in creating transposed file
Yes you could write such IF-THEN-ELSE logic with looping but when to come to think of it, it feels strongly that DS wouldnt support 5000 columns metadata and it includes my previous MODIFY stage option too where 5000 columns are created. Do check the maximum number of columns DS can support. In that...
- Wed May 22, 2019 1:33 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: job compilation and connector issue on newly added server
- Replies: 6
- Views: 5615
While you are connecting with IBM support as per Paul suggestion, there is one verification you can perform among the exported/imported environment variables as in #3. There are some APT variables defined for compilers like APT_COMPILER which you can see in Admin client. You can cross check those va...
- Wed May 22, 2019 1:18 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Remove Carriage Return in Sequential Stage Filter
- Replies: 10
- Views: 7587
- Sun May 19, 2019 4:09 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Sending an Excel Sheet to Amazon S3
- Replies: 5
- Views: 4358
Re: Sending an Excel Sheet to Amazon S3
Just a rhetorical thought for a worst case scenario considering the columns of one tab is overwritten by other tab.
Is there a way to separate those 2 tabs and process them separately and then send them to S3?
Is there a way to separate those 2 tabs and process them separately and then send them to S3?
- Wed May 15, 2019 12:12 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Remove Carriage Return in Sequential Stage Filter
- Replies: 10
- Views: 7587
- Tue May 14, 2019 3:19 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Need help in creating transposed file
- Replies: 8
- Views: 6732
Re: Need help in creating transposed file
This certainly a challenging scenario which is almost a pivoting scenario. I think this can be done in 2 steps for 2 needs in your result. one need is to create header row with key columns and new columns created based on Code and the other need is to map the Value to each code not sure if it really...
- Tue May 14, 2019 2:24 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Field length more than 8000
- Replies: 3
- Views: 3759
You positioned a question such that it is about exceeding the length of string data types (MAX is 8000 as we see). As Craig mentioned and as you know, your choice of data type in DS is usually based on the data type defined in DB table. But in this case,as its about the maximum in which SQL Server D...
- Mon May 13, 2019 1:17 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: NEED LOGIC IN DATASTGE
- Replies: 2
- Views: 3230
Re: NEED LOGIC IN DATASTGE
Transformer Looping is an apt choice for this scenario since there is no other direct feature in DS which helps with the result. But if you want to explore the option on the DB side, you could try LEAD() and LAG() functions depending on the type of DB you are using and play with the query usage opti...
- Mon May 13, 2019 12:42 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Converting column data into Columns and performing count
- Replies: 3
- Views: 3443
Re: Converting column data into Columns and performing count
If you have a Transformer already in the job, you could use that with LOOPING logic inside the Transformer itself and get the count directly for each JobId (if you find Aggregator and Pivoting combination becomes heavy)
- Mon May 13, 2019 12:03 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Throwing exception: APT_BadAlloc: Heap allocation failed.
- Replies: 7
- Views: 6918
Re: Throwing exception: APT_BadAlloc: Heap allocation failed
Heap allocation is a generic error related to buffering and memory allocation. Not much direct answer for this error as the job needs to be completely checked. But to begin with, are you getting this issue when using ODBC stage only? Can you try replacing it with some other stage as source and check...
- Sun May 12, 2019 4:15 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Remove Carriage Return in Sequential Stage Filter
- Replies: 10
- Views: 7587
Re: Remove Carriage Return in Sequential Stage Filter
Based on Craig's and Ray's response focusing on carriage returns, I am slightly confused of your need. I think you are looking for a command pattern which does both header/trailer removal and CR removal in one shot. If so, this seems a unix command related question which is to be put in SeqFile stag...
- Mon Apr 22, 2019 12:20 pm
- Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
- Topic: Trying to extract row with certain criteria
- Replies: 2
- Views: 3070
Transformer stage is an option for this scenario and it should work with your constraint. I suspect the format of the input values coming for 'STAT CODE' is not correctly compared. - Can you can check the actual values just right before the Transformer stage? You may be storing in Integer but compar...