Warnings in PX

[johnorr]

Hi
I am working on DS EE 7.1 version. I have developed a job which retrieves data from sources and does a change data capture with another table find the records that have been inserted/updated or deleted.
The job runs with a few warnings but no errors.

They are listed as per stage:

Change data capture-
CDC_TBL: When checking operator: Defaulting "BFDMD40_DELDT_S" in transfer from "beforeRec" to "outputRec"



Shared container to generate sequence keys
SCNT_BEDWFARCMC001SEQKEYGENC22.SEND_SCNT: When checking operator: Sequential operator cannot preserve partitioning of parallel dataset on input port 0

Join Stage to join these records and sequence keys-
JN_GENKEYS.LN_CHG9HREC_Sort: When checking operator: Operator of type "APT_TSortOperator": Will partition despite preserve-partitioning flag on dataset on input port 0


Inaddition the job does not show any outputs (or flow of blue color while its running) eventhough there are some records available in source.

Can you please explain what these errors mean?


Thanks

[cbrandel]

See my comments [cb]

Hi
I am working on DS EE 7.1 version. I have developed a job which retrieves data from sources and does a change data capture with another table find the records that have been inserted/updated or deleted.
The job runs with a few warnings but no errors.

They are listed as per stage:

Change data capture-
CDC_TBL: When checking operator: Defaulting "BFDMD40_DELDT_S" in transfer from "beforeRec" to "outputRec"

[cb] This means that a colum is present in the output (or the input of the next stage) but not in any input and therefore no value can be put into it. Check Your input/output columns.


Shared container to generate sequence keys
SCNT_BEDWFARCMC001SEQKEYGENC22.SEND_SCNT: When checking operator: Sequential operator cannot preserve partitioning of parallel dataset on input port 0

[cb] The Shared container runs sequential, correct? The incoming data per default has the "preserve partitioning flag" set. Since this can potentially mean >1 nodes the partitioning scheme cannot be kept since for the sequential stage the data has to be collected. try setting the "preserve Partitioning" to "clear" in the stage preceding the shared container.



Join Stage to join these records and sequence keys-
JN_GENKEYS.LN_CHG9HREC_Sort: When checking operator: Operator of type "APT_TSortOperator": Will partition despite preserve-partitioning flag on dataset on input port 0

[cb] The Join Stage tries to be smart about partitioning data. Since Your job shows no rows, You may have to go back and make sure that the data that enters the join stage is sorted and hash partitioned on the column(s) that You want to join over.



Inaddition the job does not show any outputs (or flow of blue color while its running) eventhough there are some records available in source.

Can you please explain what these errors mean?


Thanks

hope that helps,
Chris

[Madhav_M]

in transfer from "beforeRec" to "outputRec"..

I am also getting the same warning while using CC stage.
FYI both the input page of CC stage has the column(s) which is there in output of CC stage but still the warning is populated.

Can anyone advice me on this?

[thebird]

Hi,

I had the same problem with the Change Capture stage. This usually and probably happens when you pass all the fields to the stage and set the the Change Mode to Explicit Keys&Values.

Because of this we decide to change the design of the job and passed only the key and change values to the Change Capture stage. The other fields where taken to Join stages and where joined to the output of the change capture stage which was split using a Switch stage into (1) Delete records and (2) Insert and Update records.

In this manner the whole record was brought in to the main stream again.

This approach is not recommended for very high volumes of data, as this design takes up a lot of load.


The Join stage takes a hash partition mode by default i.e, when the the partition mode is set to Auto, if the stages linking to the join stage have the preserve partition flag set to "clear".

Even if the partitioning mode is set to "Hash" in the join stage, the preserve partitioning flag for the preceeding 2 stages would have to be set to "clear".

This is because most of the stages have the preserve partitioning flag set to "propagate" by default. In this case, the join would take the partitioning mode of the previous stage if set to Auto. And if the Join is set to Hash partition explicitly, the DataStage would have to repartition the data from the stage which has its preserve partition set to "propagate" and this would throw warnings.

Hope this has helped.

Regards,

The Bird.