CFF Stage and special or non-printable characters (again)
Posted: Wed Sep 06, 2017 10:15 am
Greetings,
I've been saddled with a group of DataStage EE and Server jobs that are suffering of the same aliment; seeing special or non-printable characters in various fields in the source records, and can't find a way to eradicate them once in DataStage. I've read the various posts in this forum on the topic but can't say I came away with a clear answer, and not being a recent Mainframe guy (programmed in COBOL a few decades ago on non-MF systems), not sure what path to go down.
Anyway, let me get on with it. to illustrate an example of the problem I'm having, I have the following sample copybook layout:
7 columns - level 2 - DISPLAY_NUMERIC
1 column - level 2 - GROUP
3 columns - level 3 - DISPLAY_NUMERIC
1 column - level 2 - GROUP
2 columns - level 3 - DISPLAY_NUMERIC
1 column - level 2 - DECIMAL - length 5
5 columns - level 2 - CHARACTER
1 column - level 2 - DECIMAL - length 7 Scale 4
4 columns - level 2 - CHARACTER
problem col - level 2 - CHARACTER - length 12 - seeing hex 86 - daggers
And in other columns later seeing hex 18 - CANCEL - and in the same record later, other types of non-printable chars..
I checked the source file in UltraEdit outside of DataStage and see the same thing. I've also been told we pull the file via FTP in BIN mode. So Question is, how do I:
A: Get the MF system providing the file to stop inserting non-printable chars in the fields they currently are when no data is provided in them? I say that because in the DataStage debugger I see empty fields.
B: If they persist, how can I configure the CFF stage or in the DataStage job, eliminate these non-printable chars?
I also stumbled on the following technote, albeit we are on 11.5 and not 8.7:
http://www-01.ibm.com/support/docview.w ... wg21613309
Is it possible on the MF the customer has set unicode for character fields? I wasn't aware the MF was unicode aware/capable. Oh and I tried the peek stage trick and still see the same non-printable chars.
TIA for any insight or revelations.
ff
I've been saddled with a group of DataStage EE and Server jobs that are suffering of the same aliment; seeing special or non-printable characters in various fields in the source records, and can't find a way to eradicate them once in DataStage. I've read the various posts in this forum on the topic but can't say I came away with a clear answer, and not being a recent Mainframe guy (programmed in COBOL a few decades ago on non-MF systems), not sure what path to go down.
Anyway, let me get on with it. to illustrate an example of the problem I'm having, I have the following sample copybook layout:
7 columns - level 2 - DISPLAY_NUMERIC
1 column - level 2 - GROUP
3 columns - level 3 - DISPLAY_NUMERIC
1 column - level 2 - GROUP
2 columns - level 3 - DISPLAY_NUMERIC
1 column - level 2 - DECIMAL - length 5
5 columns - level 2 - CHARACTER
1 column - level 2 - DECIMAL - length 7 Scale 4
4 columns - level 2 - CHARACTER
problem col - level 2 - CHARACTER - length 12 - seeing hex 86 - daggers
And in other columns later seeing hex 18 - CANCEL - and in the same record later, other types of non-printable chars..
I checked the source file in UltraEdit outside of DataStage and see the same thing. I've also been told we pull the file via FTP in BIN mode. So Question is, how do I:
A: Get the MF system providing the file to stop inserting non-printable chars in the fields they currently are when no data is provided in them? I say that because in the DataStage debugger I see empty fields.
B: If they persist, how can I configure the CFF stage or in the DataStage job, eliminate these non-printable chars?
I also stumbled on the following technote, albeit we are on 11.5 and not 8.7:
http://www-01.ibm.com/support/docview.w ... wg21613309
Is it possible on the MF the customer has set unicode for character fields? I wasn't aware the MF was unicode aware/capable. Oh and I tried the peek stage trick and still see the same non-printable chars.
TIA for any insight or revelations.
ff