Search found 1419 matches

by asorrell
Fri May 28, 2021 11:48 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Removing duplicates with RCP
Replies: 1
Views: 9711

Re: Removing duplicates with RCP

There is no way to remove duplicates without defining the columns that need duplicates removed. You'll need to define those columns so the data can be sorted and duplicates can be removed. RCP is easy for lift and shift, but details are required for those operations.
by asorrell
Thu Apr 29, 2021 10:15 am
Forum: General
Topic: Some code to help compare jobs
Replies: 0
Views: 12454

Some code to help compare jobs

Command line comparison of a job in two different projects, placing the output in an html file: (Note: Windows!) diffapicmdline.exe /lhscd "/d="HOSTNAME:PORT" /h="ENGINE" /u="USERID" /p="PASSWORD" "PROJECT" "JOBNAME"" /rhscd "...
by asorrell
Wed Apr 28, 2021 2:05 pm
Forum: General
Topic: Is this site still in Use??
Replies: 1
Views: 10306

Re: Is this site still in Use??

Yes - it was out for a while with technical issues, but was re-loaded into a newer version of bulletin board software. If you have a relevant question please post it!

Andy
by asorrell
Sun Feb 16, 2020 5:40 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Generic Stage in FastTrack generated job
Replies: 5
Views: 6854

You can retain the metadata on the input and output links, and temporarily keep a copy of the generic stage for a reference in case it has additional details. 1) Right click on the Generic Stage and copy it, then paste it back into an empty spot on the canvas. That should place it there with its inp...
by asorrell
Tue Feb 11, 2020 1:55 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Hierarchical stage - XML parsing
Replies: 2
Views: 4949

Only other way to isolate the problem is to gradually remove sections from the Hierarchical stage to see when the error message goes away. That would tell you at least what section caused the error.

But yes - you might want to call customer service on it to get their take on the problem.
by asorrell
Tue Feb 11, 2020 1:41 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Generic Stage in FastTrack generated job
Replies: 5
Views: 6854

You can replace the Generic stage with the Copy stage or any other stage required for job completion. The Generic is just a placeholder.

FastTrack provides a starting point, but you are expected to take the beginning job and add to it and modify it as required.
by asorrell
Tue Feb 11, 2020 1:37 pm
Forum: General
Topic: Datastage Client Install issues
Replies: 3
Views: 12867

1) You must specifically have admin rights for your PC assigned to you.
2) This does not include being in an admin "group" - that doesn't work. Your id must be in the admin list.
3) User Account Control (UAC) must be off (at least temporarily, can be turned back on after the installation)
by asorrell
Tue Feb 11, 2020 1:34 pm
Forum: General
Topic: Datastage V8 certification - training material ?
Replies: 6
Views: 14822

I would recommend going for the most recent certifications - which are all version 11. Rev 8 is no longer supported.

https://www-01.ibm.com/software/support ... 7Q91753O35
by asorrell
Tue Feb 11, 2020 1:20 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Multi-instance job with Oracle source and DB2 target
Replies: 6
Views: 6629

I concur with Craig. You need to use the correct stage for the correct source / target. 1) If your source is an Oracle table, then use an Oracle Connector 2) If your source is a file, use a Sequential File stage with a schema file 3) If your target is DB2 table, use DB2 stage If you are doing no dat...
by asorrell
Wed Jan 15, 2020 10:28 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: How to handle huge historical record loads in Azure DataLake
Replies: 2
Views: 3965

My first thought is to request they add a timestamp column to the source. Is your problem that you are trying to identify and update changed records? I have the same problem with another client with a feed way larger than yours. They say they can't modify the source since its a mainframe file. They ...
by asorrell
Wed Jan 15, 2020 9:34 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Sorting and sending to Entire Partition
Replies: 5
Views: 5755

Couldn't you just do a join with it being set to sequential rather than parallel? Nope - its processing in excess of a billion rows, sometimes up to double digit billions! Found a solution by the way. Used a partitioned Sort stage, fed it to a sequential Copy stage using "Sort-Merge" Coll...
by asorrell
Sun Jan 05, 2020 10:22 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Sorting and sending to Entire Partition
Replies: 5
Views: 5755

And the answer seems to be "no". We ran a test and it looks like it dropped about 5 million records, so I'm assuming that DataStage doesn't keep the records in sorted order as it goes from many (Hash) to many (entire). Inserting a single-threaded copy stage works, but is slow (run time goe...
by asorrell
Sun Jan 05, 2020 6:49 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Sorting and sending to Entire Partition
Replies: 5
Views: 5755

Sorting and sending to Entire Partition

We have a situation where we are sorting a moderately large dataset on about 100 nodes. However, we want to send the resulting sorted results to a join with "Entire" for the left leg of a right outer join. -YES- I know this sounds odd, but trust me, there are some extreme variants in the d...
by asorrell
Mon Dec 03, 2018 9:00 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Reading SAS code through datastage
Replies: 5
Views: 4205

So, just to insure I understand... You have been tasked to replace a SAS process that reads a COBOL binary file with a DataStage process? If so, feeding DataStage from a SAS application won't accomplish your goal - which was to stop using SAS... Regardless, you can't insert SAS code into a DataStage...
by asorrell
Mon Dec 03, 2018 8:41 pm
Forum: General
Topic: DB2 vs SQL Server for repositories?
Replies: 5
Views: 4537

Craig! Stop talking about Ray like that! :-)