DataStage Client Slow response

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

roblew
Charter Member
Charter Member
Posts: 123
Joined: Mon Mar 27, 2006 7:32 pm
Location: San Ramon

DataStage Client Slow response

Post by roblew »

Hello,

I'd like any tips or recommendations for running the DataStage client overseas (India, Africa, Asia, etc.).

We have some developers in remote locations, who have mentioned slow response working with the DataStage Designer. Just opening a job for editing in Designer can take up to 30 seconds. Besides the obvious network limitations, are there any configurations on the server within DataStage which we should pay attention to? any parameters? any OS kernel settings?

Would there be a reason why all client connections to particular projects in the DataStage repository are slow? Connections were fast when the project was small, but gradually worsened. We've got ~100 jobs per project. The jobs themselves run fast.

Also, I've noticed quite a few hung client connections both from overseas and locally. I tried setting the inactivity timeout to 2 hours, which seemed to cause connection problems at all times. Is there a recommendation for this? Has anyone else run into this same problem?


Thanks,
Rob

We're running DataStage EE 7.5.1A on RedHatEnterprise Linux AS3.
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

Are the overseas developers connecting through citrix or they have a client installed in their machines and they are in the network?
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Post by kcbland »

Running jobs slow down the response time on the node that houses the job repository. The increase in jobs may be an indication that more work is being done on the server, and thus response time decreases. If no one is executing jobs, then the response time opening a Designer session should be reasonable if the job design itself is not large.
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Most of the difference is in the network, as you can verify by comparing local versus remote users' experiences.

With only 100 jobs per project, there is little within DataStage that would be causing such delay.

Sometimes I use a similar (non-Citrix) connection between Australia and the USA (just connecting to the remote machine via TCP/IP), and it definitely seems to be a function of how otherwise busy the network is how good or bad my DataStage response is.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
roblew
Charter Member
Charter Member
Posts: 123
Joined: Mon Mar 27, 2006 7:32 pm
Location: San Ramon

Post by roblew »

The developers overseas have the client installed on their local clients, connected to our network. We did consider using a citrix setup, where they would log into a session and run it from the network. Would that be preferred (faster)? We are also considering seting up a local workstation or server in California with DataStage installed, and have the remote developers use Remote Desktop to log into the local machine. Just checking here to see if anyone has suggestions...

I'd like to mention that we experience the slow response locally as well. I may open up a support ticket with IBM, since I don't think we should be experiencing this problem. The projects are not huge, yet some jobs are fairly complex (25-30 stages). I still think the development environment feels very sluggish with larger jobs.

And by the way, it is slow whether jobs are running or not.

Thanks for all the feedback!! I think I'll have to get a premium membership to see Ray's full post. =(
vijayrc
Participant
Posts: 197
Joined: Sun Apr 02, 2006 10:31 am
Location: NJ

Re: DataStage Client Slow response

Post by vijayrc »

roblew wrote:Hello,

I'd like any tips or recommendations for running the DataStage client overseas (India, Africa, Asia, etc.).

We have some developers in remote locations, who have mentioned slow response working with the DataStage Designer. Just opening a job for editing in Designer can take up to 30 seconds. Besides the obvious network limitations, are there any configurations on the server within DataStage which we should pay attention to? any parameters? any OS kernel settings?

Would there be a reason why all client connections to particular projects in the DataStage repository are slow? Connections were fast when the project was small, but gradually worsened. We've got ~100 jobs per project. The jobs themselves run fast.

Also, I've noticed quite a few hung client connections both from overseas and locally. I tried setting the inactivity timeout to 2 hours, which seemed to cause connection problems at all times. Is there a recommendation for this? Has anyone else run into this same problem?


Thanks,
Rob

We're running DataStage EE 7.5.1A on RedHatEnterprise Linux AS3.
In the same line, though slightly deviating from this topic: What's the miminum requirement for the Client PC [RAM/Processor etc] to run DataStage EE Designer/Manager/Director Client..Thanks, V
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

roblew wrote:We did consider using a citrix setup, where they would log into a session and run it from the network. Would that be preferred (faster)?
Yes, both. Enormously. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Previous post didn't really deserve premium status, so I've removed same. There's nothing technical in it.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
richdhan
Premium Member
Premium Member
Posts: 364
Joined: Thu Feb 12, 2004 12:24 am

Post by richdhan »

Hi Rob,

We access DataStage 7.1 client components using Citrix Metaframe XP.

The performance is very good.

HTH
--Rich
roblew
Charter Member
Charter Member
Posts: 123
Joined: Mon Mar 27, 2006 7:32 pm
Location: San Ramon

Post by roblew »

thanks all. I'll definitely try to get the DS client on Citrix. Sounds like that's our best option.
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Post by kcbland »

The only way to identify the issues are to eliminate as many issues as possible. When your client is particularly sluggish, look at system utilization using something like top, prstat, or glance on the repository node. Maybe jobs aren't running, but you can't discount that the machine may be bottlenecked.

One of my customers had terrible local performance with Designer because the database coordinator node was also the job design repository node. A query in UDB would kill Designer response time on every single click that had a roundtrip to the repository (opening stages, importing metadata, selecting from the right-click picklist menu, etc).
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
roblew
Charter Member
Charter Member
Posts: 123
Joined: Mon Mar 27, 2006 7:32 pm
Location: San Ramon

Post by roblew »

kcbland wrote:When your client is particularly sluggish, look at system utilization using something like top, prstat, or glance on the repository node. Maybe jobs aren't running, but you can't discount that the machine may be bottlenecked.
I have done this (top), when users complain and it looks as idle as could be (97-99% IDLE). The servers are dedicated DataStage servers (2 node network cluster), and it's located in our data center. One thing I have noticed is the memory allocation looks high (1589156k used of 3GB), and yet nothing is running.

16:32:10 up 20 days, 20:44, 2 users, load average: 0.01, 0.00, 0.00
101 processes: 99 sleeping, 2 running, 0 zombie, 0 stopped
CPU states: cpu user nice system irq softirq iowait idle
total 0.0% 0.0% 0.0% 0.0% 0.0% 1.1% 98.9%
cpu00 0.0% 0.0% 0.0% 0.0% 0.0% 1.2% 98.8%
cpu01 0.0% 0.0% 0.0% 0.0% 0.0% 1.0% 99.0%
Mem: 3947892k av, 1589156k used, 2358736k free, 0k shrd, 269788k buff
653672k active, 511704k inactive
Swap: 8388472k av, 0k used, 8388472k free 958776k cached
kcbland wrote:One of my customers had terrible local performance with Designer because the database coordinator node was also the job design repository node. A query in UDB would kill Designer response time on every single click that had a roundtrip to the repository (opening stages, importing metadata, selecting from the right-click picklist menu, etc).
Could you please explain a little more about this? I'm not sure if I understood that correctly. I didn't know you could separate the nodes for the job design repository and db coordinator node (keep in mind that this is a dedicated DataStage server. all databases are remote servers). Although we notice sluggish performance all around, the problem would theoretically only get worse as more developers run more jobs during the day. Maybe there is a way to give preference to the designer processes over the running jobs (this would be only for development env.).

Note: don't feel that you have to give me the exact technical details if I can find it in some manuals somewhere or I can get support from IBM directly.
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Post by kcbland »

Since the DS Engine was installed on the coordinator node, and the other 17 nodes for the database also were PX nodes, you'd see tremendous degradation in Designer when queries were swamping the coordinator node. Jobs took a long time to open, close, and edit when the coordinator node was busy doing DB work, running PX jobs, etc. Compiles took forever, 20 minutes for the simplest job. Developers would come in at off-hours just to be able to work because business hours were too difficult to do anything. Director was unusable, because a screen refresh would take 5 minutes, even on a small category (folder) of jobs. The contractors loved it because you didn't have to do much except listen to headphones.

Since top shows nothing going on, you're probably looking at network issues. A local client shouldn't be slow under your demonstrated conditions. Can you confirm a client on the local network is acceptable?
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
roblew
Charter Member
Charter Member
Posts: 123
Joined: Mon Mar 27, 2006 7:32 pm
Location: San Ramon

Post by roblew »

I cannot confirm that local connections to the DS server is acceptible. Even local connections are aweful for this particular project.

For example, it just took me 1.5 minutes to open a job with 7 stages in it. This particular project has 229 px and server jobs.

By the way, I thought for a second it could be because we have QualityStage on the same server. So I stopped the daemon just in case. Nobody is using QS although it is installed.

This doesn't seem to be the same scenario as yours, since there's no database hosted on the same environment. Although it seems like the symtoms are similar.

I would hate to think it's a network problem when the data center is only the next building over from me.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

roblew wrote:For example, it just took me 1.5 minutes to open a job with 7 stages in it. This particular project has 229 px and server jobs.
That's still a 'small' number of jobs and by itself won't account for the delay in opening the job. For example, my current project has 1,439 jobs and it just took 40 seconds to open a job there with 32 stages. And this is remote over VPN and cable. :wink:

In my experience, the two biggest offenders when looking into client side speed killers are 1) how heavily taxed the box hosting the DS server is and 2) the network. And the culprit usually turns to to be the network connectivity.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply