Compare the number of records

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
srini.dw
Premium Member
Premium Member
Posts: 186
Joined: Fri Aug 18, 2006 1:59 am
Location: Chennai

Compare the number of records

Post by srini.dw »

Hi,

I have 2 files CART input file and CART control file.

1. The CART input data file will have an number of rows.
Eg. VIS.DAT

CART control file will be having the told number of records present in CARTinput file.
Eg.. VIS.DAT 55000

Question: How to compare the number of records present in CART input data file & CART control file are equal.

Any ideas..

Thanks
Raghavendra
Participant
Posts: 147
Joined: Sat Apr 30, 2005 1:23 am
Location: Bangalore,India

Post by Raghavendra »

Can you please let me know what is CART input file & Control file?
Is CART some different file type?
Raghavendra
Dare to dream and care to achieve ...
srini.dw
Premium Member
Premium Member
Posts: 186
Joined: Fri Aug 18, 2006 1:59 am
Location: Chennai

Post by srini.dw »

Please ignore the word CART, take 1 input file and other Control file
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

In a word, beforehand. Or afterwards.

If beforehand, get the line count from the input file and compare it with the value in the control file. You can use operating system utilities to get both values, and compare them in a Nested Condition activity.

If afterwards you can compare one of the row counts captured by DataStage (presumably on the file-reading link) with the number in the control file. You can do this in an after-job subroutine, or back in the job sequence once the job has finished.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
BugFree
Participant
Posts: 82
Joined: Wed Dec 13, 2006 6:02 am

Post by BugFree »

So is this CART file a text file? You have a data file with the records and a control file with the data file name and record count. you want to cross check if the record counts match? Is that all?

Use an external source stage. "wc -l " will be the command followed by your data file path and name. Use a column generator and generate a dummy column with value, say 'A'. Now in another stream use a sequential file stage to read the control file, followed by a column generator again to generate a dummy column with the value 'A'. Join the two streams using a join stage. Now you have both the counts (control file count and the actual count). Compare them using a transformer.
Ping me if I am wrong...
srini.dw
Premium Member
Premium Member
Posts: 186
Joined: Fri Aug 18, 2006 1:59 am
Location: Chennai

Post by srini.dw »

I will check with option provided and come back if it works, Thanks for the information provided
Post Reply