how to dynamically create sequential files from the source?
Moderators: chulett, rschirm, roy
how to dynamically create sequential files from the source?
depending upon the number of rows in the source how to create files dynamically?
Assuming you mean that you need to create separate sequential files based on row counts, I have used two methods in the past.
1) a Post-Job UNIX script that counts rows and cuts the file into separate pieces.
2) Have the job count the rows and add a key column that changes every time you need to change file names. Then in the sequential file stage change the Write method to Generate Multiple Files, and add the Key property (with your key column name). You'll need to set your path and any file prefix in the Root File String property. You'll still need to strip out the key column from your data in a post job command.
1) a Post-Job UNIX script that counts rows and cuts the file into separate pieces.
2) Have the job count the rows and add a key column that changes every time you need to change file names. Then in the sequential file stage change the Write method to Generate Multiple Files, and add the Key property (with your key column name). You'll need to set your path and any file prefix in the Root File String property. You'll still need to strip out the key column from your data in a post job command.