setting a peek stage in a developed job

suneyes · Post by **suneyes** » Fri Apr 10, 2009 12:34 am

Hi ,
I have a doubt regarding the usage of peek stage.
I have a DS job that was productionzed and I need to analyse it for some business requirements.
I thought of putting a peek stage in between the stages of the job but I am unable to do it without disturbing the link properties.
I finally ended up using a sequential file stage for capture the data and analyse it.
Can you suggest a way by which I can use a peek stage without affecting the link properties.

chulett · Post by **chulett** » Fri Apr 10, 2009 7:17 am

Perhaps it's obvious to everyone else, but I'm curious what "disturbing the link properties" means here.

ray.wurlod · Post by **ray.wurlod** » Fri Apr 10, 2009 10:04 am

Are you saying that you can add one stage type (Sequential File) without disturbing the link properties but cannot add a different stage type (Peek) without doing so? Do you mean, perhaps that you are replacing a database stage with a Sequential File stage? Then, at the very least, you are disturbing the link properties to the extent that the table name will be lost so that you can introduce a File property.

You should not be making ANY changes in production in any case - this breaks the rules of any decent software development life cycle. Create a version of the job in development that DOES include the diagnostic/audit captures (by whatever means) and promote that job properly through test into production.

suneyes · Post by **suneyes** » Sun Apr 12, 2009 10:46 pm

chulett wrote:Perhaps it's obvious to everyone else, but I'm curious what "disturbing the link properties" means here.

I mean,if I have a job flow like this.

seq.File---->transformer------->oracle

when I am trying to keep a peek stage in the middle of transformer and oracle stages but by doing that I losing the mapping between transformer and oracle stages.can you suggest some way by which I can place a peek stageithout disturbing these properties

sima79 · Post by **sima79** » Mon Apr 13, 2009 3:28 am

Just add an extra link coming out of the transformer stage, so that one link (the undistributed link) still goes to the Oracle stage and the new link goes to the peek stage. You can then just copy the columns and derivations onto the peek stage link.

e.g.

Code: Select all

seq.File---->transformer------->oracle 
                  |
                  |
                 \/
                Peek

sbass1 · Post by **sbass1** » Mon Apr 13, 2009 3:57 am

What's nice about the transformer stage is it's easy to drag columns from the input to output links and vice versa.

When I'm working with a stage that doesn't support this, what works for me is to save the metadata from the desired (old) link, then copying that to the new link.

So, in this scenario, you could try dragging the link off the oracle stage, drag the peek stage onto the canvas, then create the new link from the peek to oracle. Then open the old link, click the columns tab, then click the Save... button. Then drag the saved metadata onto the new link (will be in the Tables, Saved category).

This assumes parallel jobs build similarly to server jobs.

HTH,
Scott