Sequential file validation question

[jrawsterne]

We need to receive a sequential file in the following format:

File Header Record
Item Header Record
Item Detail Record
Item Detail Record
Item Detail Record
Item Header Record
Item Detail Record
Item Detail Record
Item Header Record
Item Detail Record
Item Header Record
Item Detail Record
Item Detail Record
Item Detail Record
etc...
File Trailer Record

We need to validate all the detail lines for each item header. If an error is found we want to move the item header and the associated detail records to an error file. The valid records should continue to the next stage of processing. What's the best way to do this?

Thanks in advance
JR

[ray.wurlod]

You really should have either posted this on the Server forum or indicate that it's a parallel job. Which engine it is will govern the answer.

[jrawsterne]

Slip of the finger. It is a parallel job.
Thanks
JR

[T42]

Quick and dirty method - Do it in two jobs:

1. First job - Place key fields from Item Header Record to every record immediately afterward. You will need to run this transformer in sequential mode to prevent any partitioning. You can use the Stage Variables to handle the awareness of the header records. For all records that are rejected, output just the header's key fields to an output source you can read later. Output records INCLUDING the header key field to another output source.

2. From the actual data, do a lookup (or Join/Merge, dependent on size of rejects you expect to see). For matches, output records to reject file without the key fields. For non-matches, do whatever you want with it.

Enjoy.

[Andet]

I thought of two ways:

1. Cycle the records through a transformer, and use staging variables to hold the information from prior records to compare against current records.

2. Split the file into two streams, one of header records and one of detail - send the detail through transformer/aggregator and merge the streams using a lookup, compare, or whatever you prefer.

Good luck!

Ande