Wait for file activity

[kennyapril]

I am trying to build a sequence job with the below requirement

A parallel job has CSV file as source and after some transformations the data is loaded into a table.

In the sequence job I need to call this parallel job but the constraint is I need to check for the file in a directory on other server and if it sees a file the sequence job should trigger which will trigger the parallel job as well.

So no one will run the job but instead when ever a file is appeared in that directory on the other server, the sequence job should trigger by itself.

Please let me know the ways to do it...
Thanks!

[SURA]

You already said Wait for file activity. Then what else you need?

Code: Select all

Create one new Sequence. Start with Wait for file --> Job Activity. In the Job Activity you can call another Sequence / job / whatever you want!!

[ray.wurlod]

Is the "directory on the other server" visible from (mounted on) the current server? That is, is the file for which you have to wait accessible via a regular or UNC pathname? If so, you can happily use the WaitForFile activity. If not, use a third-party enterprise scheduler with file detection capability to initiate your parallel job using a dsjob command line.

[chulett]

So no one will run the job but instead when ever a file is appeared in that directory on the other server, the sequence job should trigger by itself.

Just wanted to point out that nothing can 'trigger by itself'. Something must trigger it, so something must be watching for the file to arrive. Forgetting the 'no one will run the job' part, a typical solution would be a Sequence job starting with a Wait For File Activity stage that you run and that sits running waiting for the file. Once found, the rest of the sequence triggers.

Or, as noted by others, something else is running waiting for the file - another Sequence job, a cron script, an Enterprise Scheduler - something that initiates the sequence in question when the file shows up.

[kennyapril]

Thank you!

The file will be on the same DS server, the scenario is
When ever some one clicks a button on GUI a file is uploaded on the server and as soon as the file appears the DS job should read the file and do some validations load it into DB.
1) when you said another sequencer....so the job should be only one file activity stage and next stage will call another sequence job...once we run this job it will not be running always.. as no one know when will the file appear as it depends on the user...? Please correct me if I am wrong..
Another option is write a script which willbe running always and when ever it sees a file it will trigger the job...

2) In the validations...if any validation fails...an error msg should be sent to the user, any way to do it?



Thanks again!

[chulett]

1) Do you still need help with this?

2) Sure... depends on what you do in that event. In your shoes I'd probably do something with an error file and in the event it is non-empty after processing, use an Email Notification stage to send it out as an attachment.

[BI-RMA]

When ever some one clicks a button on GUI a file is uploaded on the server ...

This looks like a typical web-service scenario and this is what Information Services Director is made for. There you'd have your "always on"-job that would be watching for new instances of an xml-input-file in a specific folder or an MQ-series-message or ...

[kennyapril]

Thank you Chulett!
Will use the process as given below.

Thank you BI-RMA!
I did creates web services in Information services Director but I think the WISD input with be waiting for single records each time from all the fields.

But my scenario is reading complete file at a time which is .CSV...Please suggest if this is possible

Thanks again!

[chulett]

The service is waiting for input - which could be as simple as the name of the file to process. Which it then goes off and does.

[MrBlack]

I agree with the others that have posted, you want to build a web service. But just in the off chance that a web service is unavailable to you you could accomplish this using a wait for file activity and either scheduling the job to run every minute or use a never ending loop with a wait for file activity so once it has processed one file it'll loop forever waiting for the next file.

[SURA]

The file will be on the same DS server, the scenario is When ever some one clicks a button on GUI a file is uploaded on the server and as soon as the file appears the DS job should read the file and do some validations load it into DB.

You can handle the file upload process using UNIX. Execute a script in the command activity stage and check the file placed properly and once that is done, you can run the Job. That means whenever and whoever click the file load process, it will load the file as well as trigger the next job to load the data.

[kennyapril]

Thank you all very much!

Will work on it and get back if I miss anything