DSXchange

How did you find out the difference or discrepency?

Parag,

What if they have HYD_MS_C14_VTTF1988.data ?

len-14 is best.

What is the datatype for the column?

What if you set the null value as 'Unknown' ?

Suggest commenting substring and moving transfer of record outside for loop (with a dummy value set) and check the speed.

Also the location instruction starts from beginning for each run. Maybe a simple C construct to ease it.

What about writing into an intermediate unix pipe file and reading it as newline record delimiter?

or create a "Date2 - Date1" column and use it in the sort.

In that case, would't the second record be discarded by the stage ?

There was a C++ routine which Priyadharshi wrote. Search.

This is where naming conventions come handy.

What are your keys for remove duplicates ?

EVAL should do the trick.

Is the If-Then-Else combination be the same always with with different range checks?

Alternatively use server jobs.

Did you check whether this is the impact of 'First' in remove duplicates?

What does the 'sometimes' have in common ?

What incorrect results do you receive? i.e. does it pick wrong record or both?

By doing in sequence, you are avoiding any other jobs.