DSXchange

Check your unix settings for items such as nproc, noproc etc. If the process cannot allocate the resources required to start the process, you could encounter something like this. Also, based on the IBM link, be aware that Redhat linux changes the location of nfiles and nofiles. We first setup Redhat...

The Enterprise stage will present the generated BTEQ script for review. We have an enhancement request with IBM that the connector stage be enhanced to support the same functionality as Enterprise stage. The issue is the binaries used are different between Connector and Enterprise stage. Enterprise ...

The question is, what are you trying to accomplish. I am Admin for a large DataStage site. We have 2 independent headnodes for each environment. We do this as we are a 24x7 site and need the flexibility to shift load between our "A" and "B" systems for Software patching. A normal...

Installing the Grid Resource Toolkit, you substitute scripts for the 'osh' process which is renamed to osh.exe. You also create a "prototype" apt file with dummy values that may be substituted. When submitting jobs under DataStage, you execute the dsjob command and when the osh script is i...

We have a large grid, so what we do is create APT files for each compute node on the grid.

Disable the GRID thru the environment variable and then resubmit the job for each compute node.

Ray D

Another way to speed up processing when reading sequential files is read the data as a single column since reading the CSV is a sequential operation. Then parse the data so that the access of the data is sequential but the parsing of the data into columns is done in parallel.

Regards

Ray D

At 8.5 that is not an option.

However you may look at the environment variables in unix. cd to the project and perform a "more DSParams"

That is where the variables are stored.

Ray D

Hi All

I was wondering how many Greenplum / Pivotal users there are working with both DataStage and Greenplum?

I was hoping to exchange war stories and compare uses of the tool.

Regards

Ray D

One of our contractors here believes that if NLS is disabled, we could see upward of a 30% performance improvement. I'm not sure how to test it. We have a performance system and I can set NLS to 0 in uvconfig, but I still have every directory containing the ".NLS" flag. I also have disable...

You don't mention the volume of data and that has a part in designing the solution. You could always create a dataset which contains ONLY KEYS for the data being inserted. When the job loads, load the keys as a lookup and then perform a lookup on the table. If the key exists, ignore the row. If it d...

Try defining the sequential file correctly. You have not defined a correct delimiter for parsing the columns or you have not defined the blocks of data. Fix your definition and the reading of the data by Datastage will be correct.

Regards

Ray D

I very rarely have problems with the stop on director..... maybe about 5 times. grep for the jobname. You will see the shell executing, then the su to your execution userid, then dsjob command executing. I usually do the following: kill -15 bin/sh So that should be close to the top of the tree. Make...

We have the development teams break up the datastage jobs to a single element in the SCCS and the job must have been compiled in the Development system only, not the QA environment. 1 Datastage component (EE Job, Sequencer, Shell Script, or Parameter set) and it's object is migrated into the system ...

Usually just the stop button on the director will stop the job. We do go on a weekly basis looking for osh or osh.exe jobs on all compute nodes and kill lingering processes. Also, we make sure we purge the scratch directory. Many times the tsort will leave a bunch of files on the system. Thanks in a...

I am working for a small retail company and we execute over 250k jobs per week on a Datastage grid environment. Outside of the &PH&, searching for hung OSH processes on each compute node, and disabling the monitoring process, are there any weekly maintance tasks that admins are doing out the...