Recursive Fold ! Tricky
Posted: Thu Jun 07, 2018 1:25 pm
I have a Bunch of Rows grouped by a key and multiple such bunches.
The requirement
For a given Grouping Key , start the first group at the lowest date and and it at the row where the date is highest value less than 365 - store in first grouping bucket
next start with the row that is the First Value date > 365 and iterate upto 365 days further and bucket into the second grouping
and so on and so forth
Im not sure if Transformer looping would help me achieve this, Modulo groups calculated from the earliest date of the groups range would not work because the start date of iteration n+1 would reset beyond an absolute 365 day cutoff - but can still make acrude solution.
The only thing I have in mind here is to run the first iteration through a transformer and write a 'reject' file that would overwrite the input file next time via after job command, and continue looping through residual records till no records exist- this of course calls for a sequencer with a loop which is do able, but is there anyway of doing it via a Transformer looping itself ?
Alternatively I thought an array of all dates in a group would help me create recursive Folds once I pivot all dates into a Variable Length Vector via a combine records stage & Split Vector and column export the vector column into a delimited string list where the Field Function and Dcount would help me achive certain results , but that's just a thought - have people in the forum have any better ideas to run with this?
The requirement
For a given Grouping Key , start the first group at the lowest date and and it at the row where the date is highest value less than 365 - store in first grouping bucket
next start with the row that is the First Value date > 365 and iterate upto 365 days further and bucket into the second grouping
and so on and so forth
Im not sure if Transformer looping would help me achieve this, Modulo groups calculated from the earliest date of the groups range would not work because the start date of iteration n+1 would reset beyond an absolute 365 day cutoff - but can still make acrude solution.
The only thing I have in mind here is to run the first iteration through a transformer and write a 'reject' file that would overwrite the input file next time via after job command, and continue looping through residual records till no records exist- this of course calls for a sequencer with a loop which is do able, but is there anyway of doing it via a Transformer looping itself ?
Alternatively I thought an array of all dates in a group would help me create recursive Folds once I pivot all dates into a Variable Length Vector via a combine records stage & Split Vector and column export the vector column into a delimited string list where the Field Function and Dcount would help me achive certain results , but that's just a thought - have people in the forum have any better ideas to run with this?