I have a job to join data from two different streams. On first stream I am reading data from file , doing sort in ETL, removing duplicates then passing it to join stage.
Sorting data on Key-1, partition on Key-1 with Case Insensitive option.
On join right link, reading data from DB and sorting on link on Key-1, partition on Key-1 with Case Insensitive option.
While running the job, I am getting the following warning.
Join_36: When checking operator: User inserted sort "Join_36.DSLink38_Sort" does not fulfill the sort requirements of the downstream operator "APT_JoinSubOperatorNC in Join_36"
When I checked the dump score, I see in Primary link as
ds0: {op0[1p] (sequential Sequential_File_13)
eOther(APT_HashPartitioner { key={ value=Key-1,
subArgs={ ci }
}
And secondary link on Join as
ds1: {op1[1p] (sequential DB2_Connector_37)
eOther(APT_HashPartitioner { key={ value=Key-1 }
On secondary link, Link sort with Case Insensitive didn't include subArgs={ ci } on background.
Is anyone have an idea why subArgs={ ci } option is not included on second link in score?
Job design as below.
Code: Select all
Seq->Sort->RemoveDuplicate->Join->Copy
^
|
DB