DSXchange

Craig has forgotten that server jobs should APPEND to /dev/null, rather than try to overwrite. Heck, Craig doesn't even remember that he forgot anything of the sort and doesn't recall ever worrying about append vs. overwrite when dumping stuff in the magic disappearing bucket. Maybe we just wrote i...

Yes. And that's what the custom routine you need to create (that the links mention) would do for you. You call it in your transformer, passing it the string to save and then the call to DSSetUserStatus inside the routine would write the result to USERSTATS, i.e. "park" it there. My old job...

Use a Server job to fetch the value(s) you need and park them in that job's USERSTATUS area. One such discussion on that process is here. Then any downstream objects in the Sequence job can easily pull information from there for their parameter values - it is built in to the External Parameter Helpe...

Aha! That was the 're-architected' bit I was trying to recall.

Saved that one for you, Ray.

Sweet. Go all Old Skool on it.

Makes me think of my adventures under the covers back in the day with Karen Powers leading the way after I'd trashed a project. Or two.

Okay, must have been something about it being re-architected. However, it can still be pretty mysterious if all someone has known is Windows. And that through the mouse-driven GUI without all of that goofy type-y type-y command line DOS stuff old folks like me use to live and die with. :wink: Still ...

Not to derail the conversation but I thought I had read that the MKS Toolkit was no longer part of the more recent versions. No?

Sorry but you need to compare the timestamps to what - each other? A list of filenames you've already processed and their timestamps? Mostly curious what a 'new' file means, exactly. If that also includes a modified version of a file you've already processed, for instance. There are ways to get just...

No need to keep posting this in other forums, any help will show up here.

Good to know. Informatica? Concepts are so very similar that we could speak in that language. :wink: So, it sounds like you don't really need to create another file for use by downstream processes, just validate that the original file has no duplicates. Which means you could write any duplicates to ...

Welcome! While you could do 'all' of this in a transformer, it's not really a beginner topic. Either way, sorting would be required. However, what is the rest of your requirement - you haven't specified what your output needs to be or what happens if a concatenated value isn't unique. ps. 100k is a ...

Welcome!

Rows per transaction just tells it when to commit. If you have an 'Array Size' property there, that would be what controls how many to send to the target at a time.

TonyInFrance wrote:Maybe its my server configuration (SMP/MPP) which is something I don't know how to ascertain.

YES! That architecture is exactly what controls whether you get everything up in shared memory or everything spread out, duplicated on each node.

viewtopic.php?t=96993