Search found 15603 matches

by ArndW
Mon Aug 06, 2012 7:44 am
Forum: General
Topic: Big Data File Stage
Replies: 7
Views: 3987

by ArndW
Mon Aug 06, 2012 1:58 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Sequential File Stage
Replies: 7
Views: 2605

Usually such jobs are limited by I/O speed and not CPU, and in both cases the same amount of I/O is being done. Parsing the columns directly in the stage should be somewhat more efficient than doing using a field() function, so the results that Chandra has seen are what I would expect; the increase ...
by ArndW
Mon Aug 06, 2012 1:28 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Poor Performance post new DataStage Installation
Replies: 14
Views: 6170

I'd stick with analyzing your example (b), since that shows a big unexplained difference in speeds.

What are the sources to the lookup in that job? Database connections? If so, do a simple dummy job DB -> peek stage with a large volumn on both machines and compare the throughput.
by ArndW
Mon Aug 06, 2012 12:17 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Records in sequential order...
Replies: 11
Views: 4355

Sura - I'm curious as to how you would solve this problem using a loop in the transform stage. Could you expand on your solution?
by ArndW
Fri Aug 03, 2012 10:18 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: subquery from file in oci stage
Replies: 11
Views: 3756

Then back to my request for the full error message.
by ArndW
Fri Aug 03, 2012 9:34 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: subquery from file in oci stage
Replies: 11
Views: 3756

In cases like this the only way to get a good answer is to cut-and-paste the entire error message, those two words "missing expression" aren't quite enough information.
by ArndW
Fri Aug 03, 2012 6:23 am
Forum: General
Topic: DSGetparamInfo error
Replies: 3
Views: 1186

And after you add the $INCLUDE, remember that the routine expects a Job Handle as the first parameter and your variable "seq_TMW_MID_INVENTORYHIST_ONHAND" looks suspiciously like a job name. You need to use DSAttachJob() to get a handle given a name.
by ArndW
Fri Aug 03, 2012 5:07 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Poor Performance post new DataStage Installation
Replies: 14
Views: 6170

Example (a) tells me that the new disk target for datasets is slower. Is it a SAN, or the same SAN and the same virtual disk?. Note that this test is only useful for comparison purposes if it runs for at least a minute, longer is better. Example (b) doesn't necessarily identify any specific bottlene...
by ArndW
Fri Aug 03, 2012 5:02 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Runtime Column Propagation Passing Metadata in runtime
Replies: 11
Views: 4732

What exactly is the error message you are getting (try cut-and-paste). What is your database output stage? There might be a way around the nullability issue.
by ArndW
Fri Aug 03, 2012 4:39 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Runtime Column Propagation Passing Metadata in runtime
Replies: 11
Views: 4732

What stage are you using and what is the exact message? You might be able to get by with nullable columns in this case.
by ArndW
Fri Aug 03, 2012 2:50 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Runtime Column Propagation Passing Metadata in runtime
Replies: 11
Views: 4732

What exactly is the error message you are getting. In some cases with nullability differences you can set default values, but in others the error is a "hard" one which cannot be circumvented. Dynamically handling any nullable columns in a generic job is a bit more complicated and would inv...
by ArndW
Fri Aug 03, 2012 2:17 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Runtime Column Propagation Passing Metadata in runtime
Replies: 11
Views: 4732

You don't need a schema file. The schema is embedded in the dataset and the columns are automatically used when RCP is turned on at this stage. Try a test job, declare your dataset and write to a peek stage, compile & run and ...voila... your columns are all visible in the peek.
by ArndW
Fri Aug 03, 2012 1:42 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Runtime Column Propagation Passing Metadata in runtime
Replies: 11
Views: 4732

If the names and datatypes for the datasets and their corresponding tables are identical then you have a perfect situation and the solution is very easy. You declare your source and target with RCP turned on and define no columns at all, datastage takes care of the rest. If columns are missing in th...
by ArndW
Fri Aug 03, 2012 1:23 am
Forum: General
Topic: Resetting the Job from Director
Replies: 9
Views: 2780

In both cases the job has no information about the previous run or state. The answer to the question depends upon how you read your input data and how you performed commits on the aborted job.