DataStage application is running too slow

A forum for discussing DataStage<sup>®</sup> basics. If you're not sure where your question goes, start here.

Moderators: chulett, rschirm, roy

Post Reply
Chon
Participant
Posts: 19
Joined: Mon Mar 12, 2018 4:01 am

DataStage application is running too slow

Post by Chon »

Our DataStage application and jobs are running too slow
1. I tried clearing up some files in scratch disk
2. Cleared the &PH& using CLEAR.FILE &PH& in DataStage Administrator
3. Checked the cpu usage. Usage around 30 to 50%

Hoping Someone Could Help Me With This Problem.
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

Can you narrow down the problem to say more specifically what is running too slow? Is it Server jobs or Parallel jobs? Is it all jobs or just a subset of jobs that read from a particular database, etc.? Is this a new server or a new problem on an existing server? Is to too slow all the time or just sometimes?
Choose a job you love, and you will never have to work a day in your life. - Confucius
Chon
Participant
Posts: 19
Joined: Mon Mar 12, 2018 4:01 am

Post by Chon »

The application it self sometimes is not responding or receiving very slow responding during logon. And parallel job is running too slow(eg. the job was running 30 mins before, but it is running 4 hours now)
We experienced this issue yesterday. What we normally do is restart the DS services monthly or sometimes two weeks after the last restart. Is there things I need to check to avoid the DS restart every week or every other week?
PaulVL
Premium Member
Premium Member
Posts: 1315
Joined: Fri Dec 17, 2010 4:36 pm

Post by PaulVL »

Are you running on a physical host or a Virtual machine?

How many PIDs are running on the host?

Is it a stand alone setup or a Cluster/Grid?

NAS or SAN mounts?
Chon
Participant
Posts: 19
Joined: Mon Mar 12, 2018 4:01 am

Post by Chon »

Are you running on a physical host or a Virtual machine?
- Physical host

How many PIDs are running on the host?
- maybe more than 100

Is it a stand alone setup or a Cluster/Grid?
- no idea. how to check this?

NAS or SAN mounts?
- no idea. how to check this?
PaulVL
Premium Member
Premium Member
Posts: 1315
Joined: Fri Dec 17, 2010 4:36 pm

Post by PaulVL »

I would now say that you need to talk to your DataStage admin to see his take on your "slow" environment.

for pid count, type this at the command line:


ps -ef | wc -l
Chon
Participant
Posts: 19
Joined: Mon Mar 12, 2018 4:01 am

Post by Chon »

I'm the current DS admin in our team, but my primary role is developer.

PID count = 650
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

Restarting the server is rarely a good solution. I would call that the "Windows way" and try to avoid the practice, unless there is some known issue that cannot be resolved...

In case you don't know how to monitor the health and performance details on your server, then get with a server admin for your operating system. Walk over to them or open a ticket and get some attention on it. Maybe it's storage related. Someone with hands on at your organization needs to help dive into this, as at this point, it could be caused by any number of variables.

Also open a Support case and be prepared to provide a lot of details, like when the problem started and what changed just prior to that point in time. You've got to start some where... In the meantime, go to the IBM Support portal web page and do a bunch of searching on keywords like "DataStage" and "slow" and variations.

Good luck and let us know what you find out.
Choose a job you love, and you will never have to work a day in your life. - Confucius
PaulVL
Premium Member
Premium Member
Posts: 1315
Joined: Fri Dec 17, 2010 4:36 pm

Post by PaulVL »

So you're the admin and you do not know if you are clustered or grid? hmmm... Look at your APT file. If you see more than one fastname host than you are Cluster. If you use APT_GRID_<anything> than you are Grid.


SAN = local storage
NAS = Network Attached Storage

Look at your "df -T" output and look at the device type. NAS would have NFS and SAN would have EXT4 (or something like that).


But ya... I agree with qt_ky, don't just reboot, that is not solving anything. Hell I'd seriously think about firing anyone who proposed that as a solution.

I have DataStage installs that are going on two years since their last reboot and even then it was because of an OS level patch and they HAD to be restarted.



Break down the issue into steps. List what the factors are. Is it failing because of concurrent quantity of jobs? Time of day? After X amount of jobs? Do the slow jobs have something in common? Are there background tasks that linger when you have your slowness?

Start to debug a lot more.
Post Reply