DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
raj_cipher
Participant



Joined: 08 Dec 2003
Posts: 90
Location: Chennai
Points: 945

Post Posted: Fri Mar 05, 2004 2:58 am Reply with quote    Back to top    

DataStage® Release: 7x
Job Type: Server
OS: Windows
Hi,

My DS server is too slow.I hv a doubt in my client also.
My job consists of 60 transformer stage,10 link collectors and 60 look-ups.
It takes 15 min to compile.10 min to open the job itself.I'm really frustrated dealing with this.

What might be the pbl. ? My DB is SQL SERVER.
Only three person are using the DS server.Both DS & DB runs on the same machine.My repository consists of lot of jobs some of which can be deleted.Does cleaning the repository will help or any other means.
Even viewing the DS designer is consuming lot of time.
What can be done to this ?

_________________
Think Ahead,
Raj.D
WoMaWil
Participant



Joined: 13 Mar 2003
Posts: 482
Location: Amsterdam
Points: 2224

Post Posted: Fri Mar 05, 2004 3:26 am Reply with quote    Back to top    

Hi raj,

a 130+ stage- Job wow. My first smell, without having seen the job: do you need everything so complicate. DataStage is transforming all your grafical code into p-code and then starting to compile this.

Such a big job may be feasible with Datastage, but should not be the rule.

Not on behalf on DataStage, but on behalf of you and the people maniging it may be helpful to splitt the giant in smaller parts.

For compiling and opening a Job you cannot blame your DB.

We have also many jobs and complicate ones in our company, sometime we also like the director a bit quicker, but your problems are far not ours.

Jobs with more than 10 Stages are in our company exeption not the rule.

When I started with DataStage 7 years ago, I was proud about my first 70 stage-Job, but time has learn me, that something like this is feasible, but not sensefull.

Wolfgang
Rate this response:  
Not yet rated
ogmios
Participant



Joined: 11 Mar 2003
Posts: 659

Points: 5352

Post Posted: Fri Mar 05, 2004 3:31 am Reply with quote    Back to top    

Rolling on the floor laughing. Sorry, DataStage is slow but I think you should split up your jobs in seperate smaller jobs.

In most of my jobs I only have 1 transformer and possible 1 or 2 lookups.
The problem with your slowness is probably because DataStage was never really "meant" to be used like that.

I have a solid hunch your execution speed will also be < 1.

I've seen people writing bigs jobs in DataStage but most of them did not dare to go farther than using between 10 and 15 stages at most.

Ogmios
Rate this response:  
Not yet rated
kcbland

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 15 Jan 2003
Posts: 5209
Location: Lutz, FL
Points: 39183

Post Posted: Fri Mar 05, 2004 6:33 am Reply with quote    Back to top    

Your job is way too big. You're doing too much in it, probably have no modularity. Trust me on this.

_________________
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Rate this response:  
Not yet rated
Willo
Participant



Joined: 02 Jan 2004
Posts: 10

Points: 71

Post Posted: Fri Mar 05, 2004 10:05 am Reply with quote    Back to top    

I agree that your job is way to big, do what ogmios suggests and break it down into smaller jobs. Also when comparing loads into SQL Server and Oracle, SQL Server is pants and Oracle will always beat it.

The fact that you have both DS and SQL Server on the same machine is also cause for concern, thats going to be your primary bottleneck.
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours