DS SERVER/CLIENT PROBLEM - too slow

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
raj_cipher
Participant
Posts: 90
Joined: Mon Dec 08, 2003 4:48 am
Location: Chennai

DS SERVER/CLIENT PROBLEM - too slow

Post by raj_cipher »

Hi,

My DS server is too slow.I hv a doubt in my client also.
My job consists of 60 transformer stage,10 link collectors and 60 look-ups.
It takes 15 min to compile.10 min to open the job itself.I'm really frustrated dealing with this.

What might be the pbl. ? My DB is SQL SERVER.
Only three person are using the DS server.Both DS & DB runs on the same machine.My repository consists of lot of jobs some of which can be deleted.Does cleaning the repository will help or any other means.
Even viewing the DS designer is consuming lot of time.
What can be done to this ?
Think Ahead,
Raj.D
WoMaWil
Participant
Posts: 482
Joined: Thu Mar 13, 2003 7:17 am
Location: Amsterdam

Post by WoMaWil »

Hi raj,

a 130+ stage- Job wow. My first smell, without having seen the job: do you need everything so complicate. DataStage is transforming all your grafical code into p-code and then starting to compile this.

Such a big job may be feasible with Datastage, but should not be the rule.

Not on behalf on DataStage, but on behalf of you and the people maniging it may be helpful to splitt the giant in smaller parts.

For compiling and opening a Job you cannot blame your DB.

We have also many jobs and complicate ones in our company, sometime we also like the director a bit quicker, but your problems are far not ours.

Jobs with more than 10 Stages are in our company exeption not the rule.

When I started with DataStage 7 years ago, I was proud about my first 70 stage-Job, but time has learn me, that something like this is feasible, but not sensefull.

Wolfgang
ogmios
Participant
Posts: 659
Joined: Tue Mar 11, 2003 3:40 pm

Re: DS SERVER/CLIENT PROBLEM - too slow

Post by ogmios »

Rolling on the floor laughing. Sorry, DataStage is slow but I think you should split up your jobs in seperate smaller jobs.

In most of my jobs I only have 1 transformer and possible 1 or 2 lookups.
The problem with your slowness is probably because DataStage was never really "meant" to be used like that.

I have a solid hunch your execution speed will also be < 1.

I've seen people writing bigs jobs in DataStage but most of them did not dare to go farther than using between 10 and 15 stages at most.

Ogmios
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Post by kcbland »

Your job is way too big. You're doing too much in it, probably have no modularity. Trust me on this.
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Willo
Participant
Posts: 10
Joined: Fri Jan 02, 2004 2:07 am

Post by Willo »

I agree that your job is way to big, do what ogmios suggests and break it down into smaller jobs. Also when comparing loads into SQL Server and Oracle, SQL Server is pants and Oracle will always beat it.

The fact that you have both DS and SQL Server on the same machine is also cause for concern, thats going to be your primary bottleneck.
Post Reply