File properties as source columns

chandu123 · Post by **chandu123** » Fri Sep 16, 2016 3:34 pm

I have a requirement where I need to compare the Modified Timestamp or Created Timestamp of the input file and If there is a new file, Job needs to be run, if not another flow.

Can we use the file properties as source columns in the Datastage job or as job parameters so that I can use Nested Condition activity of the sequence to check for the condition and decide the flow?

chulett · Post by **chulett** » Fri Sep 16, 2016 5:14 pm

Sorry but you need to compare the timestamps to what - each other? A list of filenames you've already processed and their timestamps? Mostly curious what a 'new' file means, exactly. If that also includes a modified version of a file you've already processed, for instance.

There are ways to get just the file properties into a job, UNIX would make it easier than Windows and I can't get to the Windows command line here to play. Blocked by Group Policy. Yay.

SURA · Post by **SURA** » Sat Sep 17, 2016 4:52 am

You have MKS tool kit. So you can still run most of the unix commands. One way of doing; get the modified time stamp using command activity stages, cleanse the command output for both files and compare.

In the last project i did the same validation.

chulett · Post by **chulett** » Sat Sep 17, 2016 7:23 am

Not to derail the conversation but I thought I had read that the MKS Toolkit was no longer part of the more recent versions. No?

SURA · Post by **SURA** » Sat Sep 17, 2016 12:28 pm

I know it is there in 11.3, but just checked the latest version and it seems still using.

http://www.ibm.com/support/knowledgecen ... rsion.html

SURA · Post by **SURA** » Sat Sep 17, 2016 12:31 pm

chulett wrote:MKS Toolkit was no longer ...

This is really a good point. I always use unix commands, but i need to think twice before us it! At some point it may hit me back.

chulett · Post by **chulett** » Sun Sep 18, 2016 7:26 am

Okay, must have been something about it being re-architected. However, it can still be pretty mysterious if all someone has known is Windows. And that through the mouse-driven GUI without all of that goofy type-y type-y command line DOS stuff old folks like me use to live and die with.

Still waiting for requirement clarifications from our OP...

IBM Analytics Champion 2009 - 2020 · Post by **asorrell** » Sun Sep 18, 2016 11:10 am

Hmm - I also saw presentations last year from IBM saying they were removing MKS Toolkit from future versions DataStage. However, it does seem to be there.

Must have reversed course or delayed the change. I do know it would cause problems for several customers that have designs that use certain UNIX commands even when on Windows....

ray.wurlod · Post by **ray.wurlod** » Sun Sep 18, 2016 6:17 pm

Create one or more DataStage server routines that you can call from Routine activities in a sequence job.

In these use OPENSEQ to open the file(s), then the STATUS statement to obtain the property values. Don't forget CLOSESEQ before returning from the routine. I suggest that you return all the information you require using a dynamic array (delimited string).

chulett · Post by **chulett** » Sun Sep 18, 2016 7:01 pm

Saved that one for you, Ray.

ray.wurlod · Post by **ray.wurlod** » Mon Sep 19, 2016 12:41 am

DataStage itself is no longer relying on MKS Toolkit. All previous "UNIX" calls from DataStage code have now been rewritten to invoke native Windows equivalents (as much as possible, with some custom code to cope the the remainder).

However, MKS Toolkit continues to ship because of all those folks who have chosen to make use of its functionality.

chulett · Post by **chulett** » Mon Sep 19, 2016 7:02 am

Aha! That was the 're-architected' bit I was trying to recall.