DSXchange

did you try increase number of nodes

"I want to create new record for values having duplicates from reference dataset"

What about unique records??

combination of sort stage and remove duplicate should work.

first hash partition on the key columns then use sort stage on key columns and the effective date Desc. and in the remove duplicate stage keep the 1st record.

yes. i was also having the similar issue. and found that oracle connecter stage does not work for synonyms.

how about sparse lookup.

not enough information.
Post the log from director

i have a user defined sql in oracle reference table, using sparse lookup. but it seems to be not working. it just hanged.
but it works when i change it to automated generated query is sparse lookup.

does the files contains column names?

yes if you can make sure that you are not updating the same record, you are inserting from other link.

yes

update table
set col3=1234
col1=orchestrate.col1
col2=orchestrate.col2
in col3('1','2')

is this the actual query?
should be something like :

update table <Table-name>
set col3=1234
WHERE col1=orchestrate.col1
AND col2=orchestrate.col2

etc.

What database? how many nodes?

ALL columns should be CHAR to read fixed width file.

i need to build this in Job as we need to improve the performance of the Jobs

Not Sure if you get it.


do you want top 2 salaried employee of all department, or top 2 salaried employee in each department??

what is the value of open_cursors parameters in Oracle.. can you increase that