Performance Tune Teradata Load

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
horserider
Participant
Posts: 71
Joined: Mon Jul 09, 2007 1:12 pm

Performance Tune Teradata Load

Post by horserider »

Reading a mainframe binary file using a CFF stage and simply dumping 5 million rows to a teradata table using Enterprise Stage. Takes anywhere 35-40 minutes to load. No transformation is happening.

(1) Any way to reduce the load time?
(2) Is Enterprise Stage the best way to load 5 mil rows in teradata table?
John Smith
Charter Member
Charter Member
Posts: 193
Joined: Tue Sep 05, 2006 8:01 pm
Location: Australia

Post by John Smith »

Well you need to do your own benchmarks then. As a general rule the Enterprise stage is supposed to be the fastest as it is based on Teradata Fast Load. To tune it you may need to engage your Teradata DBA as Teradata do provide a number of ways of tuning your data loads.
thamark
Premium Member
Premium Member
Posts: 43
Joined: Thu Jan 29, 2004 12:12 am
Location: US

Re: Performance Tune Teradata Load

Post by thamark »

Enterprise stage will be much faster and in fact it may take only few minutes or even less and I hope this is just an truncate and load to the table.
Hmm i will fill this when ever i get one
toshea
Participant
Posts: 79
Joined: Thu Aug 14, 2008 6:46 pm

Post by toshea »

Try setting the requestedsessions option to a smaller value. It defaults to the number of AMPs on your server, and the number of player processes is computed by requestedsessions divided by the sessionsperplayer. If you have a lot of AMPs, it results in too many player processes all competing to update the terasync table. Set requestedsessions to a smaller value like 8. How many nodes do you have in your PX config file? Optimally, you'd want to set requestedsessions to the number of nodes multipled by sessionsperplayer. That way you will not have any repartitioning.
venkatvelpula
Participant
Posts: 80
Joined: Sat Mar 18, 2006 10:27 pm

Post by venkatvelpula »

Try to write that into sequentail file and load it from there..It's not optimum solution but it might work for your case.
Post Reply