Job Gets hanged or running slow

[jpraveen]

Hi ,

My DS job gets frequently hanged and sometimes it run very slow rather than expected time.

The job is reading 8 xml files and loading data in Teradata Table.

--8 xml files---Funnel--- Tfm--- TD table---

and it is using 8 node config file and the data is approximately 14940000 records and it takes 2 hours to complete(regularly it should be completed in 1 hour)

[ray.wurlod]

So what's your question?

[jpraveen]

I mean what are the scenarios that my job is getting Hanged,
i asked my Network team and Unix team and they said there is no critical issues seen while thie job is running.

and its get hanged after importing 95% of data from xml files

[pnpmarques]

XML stages are tipically heavy in my project. Despite this, they dont' take hours to run.
Identify in your log entries which step is taking longer, for example, did you find any log entry with "Requesting delayed metadata"?

[rohitagarwal15]

Also please check with your database team also if there is any issue in table or database level at the time of loading data in the table.

[PaulVL]

Look at the partitioning of the data. Is each of the 8 partitions getting data or just one?

That can greatly affect your speed.

[XRAY]

would you try to check the time for

--8 xml files---Funnel--- Tfm--Copy stage

alone ?

If this job run fast, that has prove issue in db stage