DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
roblew


since June 2006

Group memberships:
Premium Members

Joined: 27 Mar 2006
Posts: 115
Location: San Ramon
Points: 1460

Post Posted: Wed Apr 12, 2006 1:05 pm Reply with quote    Back to top    

DataStage® Release: 7x
Job Type: Parallel
OS: Unix
Additional info: slow Designer response times
Hello,

I'd like any tips or recommendations for running the DataStage client overseas (India, Africa, Asia, etc.).

We have some developers in remote locations, who have mentioned slow response working with the DataStage Designer. Just opening a job for editing in Designer can take up to 30 seconds. Besides the obvious network limitations, are there any configurations on the server within DataStage which we should pay attention to? any parameters? any OS kernel settings?

Would there be a reason why all client connections to particular projects in the DataStage repository are slow? Connections were fast when the project was small, but gradually worsened. We've got ~100 jobs per project. The jobs themselves run fast.

Also, I've noticed quite a few hung client connections both from overseas and locally. I tried setting the inactivity timeout to 2 hours, which seemed to cause connection problems at all times. Is there a recommendation for this? Has anyone else run into this same problem?


Thanks,
Rob

We're running DataStage EE 7.5.1A on RedHatEnterprise Linux AS3.
DSguru2B

Premium Poster


since February 2006

Group memberships:
Premium Members, Heartland Usergroup

Joined: 09 Feb 2005
Posts: 6854
Location: Houston, TX
Points: 35666

Post Posted: Wed Apr 12, 2006 1:27 pm Reply with quote    Back to top    

Are the overseas developers connecting through citrix or they have a client installed in their machines and they are in the network?

_________________
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
Rate this response:  
Not yet rated
kcbland

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 15 Jan 2003
Posts: 5209
Location: Lutz, FL
Points: 39192

Post Posted: Wed Apr 12, 2006 1:29 pm Reply with quote    Back to top    

Running jobs slow down the response time on the node that houses the job repository. The increase in jobs may be an indication that more work is being done on the server, and thus response time decreases. If no one is executing jobs, then the response time opening a Designer session should be reasonable if the job design itself is not large.

_________________
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 52036
Location: Sydney, Australia
Points: 282486

Post Posted: Wed Apr 12, 2006 4:36 pm Reply with quote    Back to top    

Most of the difference is in the network, as you can verify by comparing local versus remote users' experiences.

With only 100 jobs per project, there is little within DataStage that would be causing such delay.

Sometimes I use a similar (non-Citrix) connection between Australia and the USA (just connecting to the remote machine via TCP/IP), and it definitely seems to be a function of how otherwise busy the network is how good or bad my DataStage response is.

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
currently hiring: Sydney and Melbourne
Rate this response:  
Not yet rated
roblew


since June 2006

Group memberships:
Premium Members

Joined: 27 Mar 2006
Posts: 115
Location: San Ramon
Points: 1460

Post Posted: Wed Apr 12, 2006 5:20 pm Reply with quote    Back to top    

The developers overseas have the client installed on their local clients, connected to our network. We did consider using a citrix setup, where they would log into a session and run it from the network. Would that be preferred (faster)? We are also considering seting up a local workstation or server in California with DataStage installed, and have the remote developers use Remote Desktop to log into the local machine. Just checking here to see if anyone has suggestions...

I'd like to mention that we experience the slow response locally as well. I may open up a support ticket with IBM, since I don't think we should be experiencing this problem. The projects are not huge, yet some jobs are fairly complex (25-30 stages). I still think the development environment feels very sluggish with larger jobs.

And by the way, it is slow whether jobs are running or not.

Thanks for all the feedback!! I think I'll have to get a premium membership to see Ray's full post. =(
Rate this response:  
Not yet rated
vijayrc
Participant



Joined: 02 Apr 2006
Posts: 197
Location: NJ
Points: 1438

Post Posted: Wed Apr 12, 2006 6:05 pm Reply with quote    Back to top    

roblew wrote:
Hello,

I'd like any tips or recommendations for running the DataStage client overseas (India, Africa, Asia, etc.).

We have some developers in remote locations, who have mentioned slow response working with the DataStage Designer. Just opening a job for editing in Designer can take up to 30 seconds. Besides the obvious network limitations, are there any configurations on the server within DataStage which we should pay attention to? any parameters? any OS kernel settings?

Would there be a reason why all client connections to particular projects in the DataStage repository are slow? Connections were fast when the project was small, but gradually worsened. We've got ~100 jobs per project. The jobs themselves run fast.

Also, I've noticed quite a few hung client connections both from overseas and locally. I tried setting the inactivity timeout to 2 hours, which seemed to cause connection problems at all times. Is there a recommendation for this? Has anyone else run into this same problem?


Thanks,
Rob

We're running DataStage EE 7.5.1A on RedHatEnterprise Linux AS3.


In the same line, though slightly deviating from this topic: What's the miminum requirement for the Client PC [RAM/Processor etc] to run DataStage EE Designer/Manager/Director Client..Thanks, V
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 39274
Location: Denver, CO
Points: 200594

Post Posted: Wed Apr 12, 2006 7:23 pm Reply with quote    Back to top    

roblew wrote:
We did consider using a citrix setup, where they would log into a session and run it from the network. Would that be preferred (faster)?

Yes, both. Enormously. Wink

_________________
-craig

If you try and take a cat apart to see how it works, the first thing you have on your hands is a non-working cat. -- Douglas Adams
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 52036
Location: Sydney, Australia
Points: 282486

Post Posted: Wed Apr 12, 2006 8:40 pm Reply with quote    Back to top    

Previous post didn't really deserve premium status, so I've removed same. There's nothing technical in it.

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
currently hiring: Sydney and Melbourne
Rate this response:  
Not yet rated
richdhan



Group memberships:
Premium Members

Joined: 12 Feb 2004
Posts: 364

Points: 2614

Post Posted: Thu Apr 13, 2006 2:32 am Reply with quote    Back to top    

Hi Rob,

We access DataStage 7.1 client components using Citrix Metaframe XP.

The performance is very good.

HTH
--Rich
Rate this response:  
Not yet rated
roblew


since June 2006

Group memberships:
Premium Members

Joined: 27 Mar 2006
Posts: 115
Location: San Ramon
Points: 1460

Post Posted: Thu Apr 13, 2006 12:09 pm Reply with quote    Back to top    

thanks all. I'll definitely try to get the DS client on Citrix. Sounds like that's our best option.
Rate this response:  
Not yet rated
kcbland

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 15 Jan 2003
Posts: 5209
Location: Lutz, FL
Points: 39192

Post Posted: Thu Apr 13, 2006 12:16 pm Reply with quote    Back to top    

The only way to identify the issues are to eliminate as many issues as possible. When your client is particularly sluggish, look at system utilization using something like top, prstat, or glance on the repository node. Maybe jobs aren't running, but you can't discount that the machine may be bottlenecked.

One of my customers had terrible local performance with Designer because the database coordinator node was also the job design repository node. A query in UDB would kill Designer response time on every single click that had a roundtrip to the repository (opening stages, importing metadata, selecting from the right-click picklist menu, etc).

_________________
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Rate this response:  
Not yet rated
roblew


since June 2006

Group memberships:
Premium Members

Joined: 27 Mar 2006
Posts: 115
Location: San Ramon
Points: 1460

Post Posted: Thu Apr 13, 2006 5:49 pm Reply with quote    Back to top    

kcbland wrote:
When your client is particularly sluggish, look at system utilization using something like top, prstat, or glance on the repository node. Maybe jobs aren't running, but you can't discount that the machine may be bottlenecked.

I have done this (top), when users complain and it looks as idle as could be (97-99% IDLE). The servers are dedicated DataStage servers (2 node network cluster), and it's located in our data center. One thing I have noticed is the memory allocation looks high (1589156k used of 3GB), and yet nothing is running.

16:32:10 up 20 days, 20:44, 2 users, load average: 0.01, 0.00, 0.00
101 processes: 99 sleeping, 2 running, 0 zombie, 0 stopped
CPU states: cpu user nice system irq softirq iowait idle
total 0.0% 0.0% 0.0% 0.0% 0.0% 1.1% 98.9%
cpu00 0.0% 0.0% 0.0% 0.0% 0.0% 1.2% 98.8%
cpu01 0.0% 0.0% 0.0% 0.0% 0.0% 1.0% 99.0%
Mem: 3947892k av, 1589156k used, 2358736k free, 0k shrd, 269788k buff
653672k active, 511704k inactive
Swap: 8388472k av, 0k used, 8388472k free 958776k cached

kcbland wrote:
One of my customers had terrible local performance with Designer because the database coordinator node was also the job design repository node. A query in UDB would kill Designer response time on every single click that had a roundtrip to the repository (opening stages, importing metadata, selecting from the right-click picklist menu, etc).

Could you please explain a little more about this? I'm not sure if I understood that correctly. I didn't know you could separate the nodes for the job design repository and db coordinator node (keep in mind that this is a dedicated DataStage server. all databases are remote servers). Although we notice sluggish performance all around, the problem would theoretically only get worse as more developers run more jobs during the day. Maybe there is a way to give preference to the designer processes over the running jobs (this would be only for development env.).

Note: don't feel that you have to give me the exact technical details if I can find it in some manuals somewhere or I can get support from IBM directly.
Rate this response:  
Not yet rated
kcbland

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 15 Jan 2003
Posts: 5209
Location: Lutz, FL
Points: 39192

Post Posted: Thu Apr 13, 2006 6:08 pm Reply with quote    Back to top    

Since the DS Engine was installed on the coordinator node, and the other 17 nodes for the database also were PX nodes, you'd see tremendous degradation in Designer when queries were swamping the coordinator node. Jobs took a long time to open, close, and edit when the coordinator node was busy doing DB work, running PX jobs, etc. Compiles took forever, 20 minutes for the simplest job. Developers would come in at off-hours just to be able to work because business hours were too difficult to do anything. Director was unusable, because a screen refresh would take 5 minutes, even on a small category (folder) of jobs. The contractors loved it because you didn't have to do much except listen to headphones.

Since top shows nothing going on, you're probably looking at network issues. A local client shouldn't be slow under your demonstrated conditions. Can you confirm a client on the local network is acceptable?

_________________
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Rate this response:  
Not yet rated
roblew


since June 2006

Group memberships:
Premium Members

Joined: 27 Mar 2006
Posts: 115
Location: San Ramon
Points: 1460

Post Posted: Thu Apr 13, 2006 6:51 pm Reply with quote    Back to top    

I cannot confirm that local connections to the DS server is acceptible. Even local connections are aweful for this particular project.

For example, it just took me 1.5 minutes to open a job with 7 stages in it. This particular project has 229 px and server jobs.

By the way, I thought for a second it could be because we have QualityStage on the same server. So I stopped the daemon just in case. Nobody is using QS although it is installed.

This doesn't seem to be the same scenario as yours, since there's no database hosted on the same environment. Although it seems like the symtoms are similar.

I would hate to think it's a network problem when the data center is only the next building over from me.
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 39274
Location: Denver, CO
Points: 200594

Post Posted: Thu Apr 13, 2006 7:38 pm Reply with quote    Back to top    

roblew wrote:
For example, it just took me 1.5 minutes to open a job with 7 stages in it. This particular project has 229 px and server jobs.

That's still a 'small' number of jobs and by itself won't account for the delay in opening the job. For example, my current project has 1,439 jobs and it just took 40 seconds to open a job there with 32 stages. And this is remote over VPN and cable. Wink

In my experience, the two biggest offenders when looking into client side speed killers are 1) how heavily taxed the box hosting the DS server is and 2) the network. And the culprit usually turns to to be the network connectivity.

_________________
-craig

If you try and take a cat apart to see how it works, the first thing you have on your hands is a non-working cat. -- Douglas Adams
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours