DSXchange

hmm.... hold on, things now get complicated. Say I need to join two streams (A & B) with the same join keys student_id and course_id. stream A is hashed on student_id, and stream B is hashed on course_id, But when coming into the join stage, they all both sort on student_id first and then course...

there's no strict rule that it must be one of the key columns

hmm..., I was thinking of the fully normalized BNF3 tables, in which no any sub group of the fields in the table is functionally dependent on any fields other than the candidate keys.

Keep in mind that this warning is from the sequential file stage, not the lookup stage and concerns your reject output

but the solution I found is to fix the lookup stage, hence my question.

Thanks

double post deleted.

I see the source of confusion, though, if you treat the key list as a set of fields then POSTCODE is a subset of (POSTCODE, HOUSENUM)

Thanks for the clarification, Phil.

Following up on http://www.dsxchange.com/viewtopic.php?t=142981 with a different focus. So my understanding of the hash partitioning and sorting usage and best practice so far is, EDITED, 5th edition: Best practice & goal: choose as minimum keys to partition as possible as long as rows can be di...

Partitioning must be a superset of the key that you are using for joining, or aggregating, or whatever.

Hi Phil,

Did you mean subset of the keys?
Sorry, not trying to be nitpicky here but to make it crystal clear.

Thanks

Hi, If I use a sequential file as the reject file for the lookup stage, I always get the following warning: Warning: A sequential operator cannot preserve the partitioning of input data set on input port 0 How to solve the problem? What if I do want to preserve the partitioning for the lookup stage'...

left outer join throughout all the tables

thanks Ray.

cmd = 'kill -9 ':out:''

That's pretty heavy-handed method. FYI you can also write a shell script to kill those un-killable jobs. Make your code simpler.

or, if they have to stay protected, then export & import into a different server, then compare.

Hi, I pay extra attention to where I place my link labels, so that the job execution statistics can show up nicely. However, I noticed that my carefully placed labels sometimes will revert to its default locations after I reopen the job, after I've save & compile it of course. Has anyone noticed...

Hi, Just trying to understand how the Job Compilation and Execution basically works. To my understanding, basically, Job Compilation does 2 things, 1. Generates OSH script that represents data flow and stages 2. Generates transform code for each Transformer, then compiled them into C++ and then to c...

Hi, Just to confirm, Is there any special requirement for oracle database to use the Partition Table option in oracle Enterprise stage? As per craig, Well... there will be specific indexes that support the partitioning, but yes - only if the table itself is partitioned. ... there are special require...