DSXchange

Folks,

One of the ETL guys here was able to secure a patch from IBM that allowed him to create a DS 9.1.2 job in AIX that connected to HDFS.

Rob

Looks like the concierge program is done, as is the migration tool.

The InfoSphere Information Server Concierge Program is withdrawn as of Dec 21, 2013

The "migration tool" is no longer generally available

See this link: http://www-01.ibm.com/support/docview.w ... wg21445403

Oops, double post. Can you fix it Craig?

Thanks.

See this post that does an excellent job of summarizing the interaction between mainframes and DataStage.

viewtopic.php?t=143596

This character corresponds to xA7 in the 8859-1 (Latin 1) code page. It has been my experience that it's best to avoid characters above the standard set, i.e. anything above x7F, especially for delimiters. This avoids problems such as the one you're are experiencing. My favourite delimiter over the ...

Hi HemaV, Your data source is using a different code page than your DataStage environment. The pound Sterling sign is Hex A3 in the extended characters of the Latin 1 (8859-1) code table (the most commonly used code table). Your source and/or your DataStage environment is using a code table that has...

From the IBM doco for 8.5 =========== Restrict memory usage This is set to 20 by default. It causes the Sort stage to restrict itself to the specified number of megabytes of virtual memory on a processing node. The number of megabytes specified should be smaller than the amount of physical memory on...

The conclusion to this thread seems to be that the necessary bits to accomplish this are available in DataStage and it would be a great idea if someone were to build something, however, at this time is no tool that accomplishes this.

Rob

Thanks for your response Craig.

Does the sort stage default to a higher amount of allocated memory? We saw significantly better performance in the stage. Is there some risk in using too much memory when lots of sort stages are invoked at the same time?

Rob

Folks, The sort Stage in 8.X works much better than it used to in 7.X. We did some benchmarks which indicated that for our test file / environment the sort stage was about twice as fast, presumably because the sort stage can allocate more memory to the sort. Is everyone seeing this kind of performan...

My approach has always been to create templates of each data flow pattern. I ensure that the pattern adheres to all the standards. Each detail design specifies a starting pattern and adherence to the pattern / standards is enforced at code review time. IBM is suggesting much the same approach. Some ...

Folks, On a bit of a trolling mission here... I'm documenting ETL best practices and standards at my current client. One of the managers had experience with a tool that would check 3GL code for naming conventions, code layout, commenting, etc. and wondered if such a thing exists in our world. While ...

Within the Balanced Optimizer dialog you can specify the range of stages to push into the source/target database. Have you investigated this feature which, I believe, is new in version 9.1? Hi Ray, We're still getting access to 9.1. The admin hasn't yet gotten the GCC compiler installed, this is th...

You can't force datastage to push down when it doesn't want to do so. Your only option would be to manually manipulate the job if it didn't push as much logic into the database as you were hoping. Hi Keith, That's exactly what I'm talking about. Determining the tricks needed to manually manipulate ...

Hi Ray, While we may find some jobs that we can push down without any logic that would require a transform stage, I don't want to limit myself. My expectation is that I would very quickly get to jobs that have that sort of complexity. Of course, this will lead to a whole different exercise. We will ...