HEX and ASCII, SEQ() Translation

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
horserider
Participant
Posts: 71
Joined: Mon Jul 09, 2007 1:12 pm

HEX and ASCII, SEQ() Translation

Post by horserider »

I have a mainframe source file that has a column defined as PICX(1). The values on those columns can be anything like 1,R,Y,>,L,{,},[,] etc.

There is 1 character that is somehow not translated properly. Below are the details.

(1) On Mainframe Editor (HEX ON) that character shows as ] with HEX
VALUES BB.
(2) In Teradata when I see the HEX VALUE is displays as 1A using the CHAR2HexInt().
(3) In Parallel Job the same character is displayed as ASCII Value 26
using SEQ ().

So ] in mainframe BB as HEX on is translated as SUB (substitute) through a parallel job.

Is this the right translation?
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

The mainframe will use EBCDIC and the character0xBB is not defined.
0x1A in ASCII is defined as "SUB"
Decimal 26 is 0x1A, the same as above.
The EBCDIC hex value for "]" is 0x5A.
Some 3270 emulators will display the character as "]", though.

Unless you specify a conversion explicitly for this non-EBCDIC character, the resulting ASCII character is undefined.
horserider
Participant
Posts: 71
Joined: Mon Jul 09, 2007 1:12 pm

Post by horserider »

(1)
Is there a list that explains what characters of mainframe (EBCDIC) will NOT be converted properly.

(2)
Is there a way to solve this in Parallel Job?

(3)
Can Server Job is used to convert instead?
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Use Google to search for "ASCII EBCDIC conversion" and you will get many hits, or look in the BASIC Job developer which also has a conversion table.

Where are you doing your EBCDIC conversion? In some places you can explicitly modify this convsersion mapping (i.e. within DataStage). If you cannot control the mapping of your (fake) "]" to "SUB" then you could just convert all "SUB" to the ASCII "]" in DataStage using the replace() function call in PX.
horserider
Participant
Posts: 71
Joined: Mon Jul 09, 2007 1:12 pm

Post by horserider »

Here is what I am doing

Parallel Job 1: Getting the file as binary/EBCDIC on ETL Server (File A)
through FTP Enterprise Stage

Parallel Job 2: Reading File A through CFF stage and using the Cobol File
Definition and then loading the data into a table.

The translation is happening fine except for 2 characters

All ! are becoming ] HEX 5D
All ] are becoming HEX 1A

Every other character so far is being translated properly.

Currently I am using below logic in a variable but what happens when the list grows?
If Seq(DSLink148.VO_CODE)=93 Then '!' Else If Seq(DSLink148.VO_CODE) =26 Then ']' Else DSLink148.VO_EDIT_CODE

QUESTION

As suggested by previous posted to explicity convert some characters like 26, is there a list that explains what characters in EBCDIC won't be converted to proper ASCII?
shamshad
Premium Member
Premium Member
Posts: 147
Joined: Wed Aug 25, 2004 1:39 pm
Location: Detroit,MI

Post by shamshad »

I am also looking for a list of characters that WON'T be translated using a CFF stage on a binary downloaded Mainframe file.
flashgordon
Premium Member
Premium Member
Posts: 99
Joined: Tue Aug 17, 2004 7:50 am
Location: Boulder, Colorado

Post by flashgordon »

I would argue that cff stage will translate all "normal" characters numbers, letters, special characters and not much else. You seem to be able to isolate the characters that are not translating. You may know about od -x against small ebcdic files that are residing on unix to see what you really have as input to cff stage. I would think that cff stage on se and ee do pretty much the same thing. Since you seem to know what hex you have in cff stage I will show you this that worked for me to handle special hex values in the transformer after cff stage. Isolate the column that has the hex in it and then "if Input.MyVar = char(n) then MyAction" where n is the decimal value between 1 and 256 that corresponds to the hex value you know is in the input file. With the hex value isolated you can set the result in Teradata that you want. You seem experienced at this but I would have to say that it is unusual for an mvs file to have hex values other than packed decimal, a binary integer. or a floating point number.

... Flash
Flash Gordon
Hyperborean Software Solution
flashgordon
Premium Member
Premium Member
Posts: 99
Joined: Tue Aug 17, 2004 7:50 am
Location: Boulder, Colorado

Post by flashgordon »

I would argue that cff stage will translate all "normal" characters numbers, letters, special characters and not much else. You seem to be able to isolate the characters that are not translating. You may know about od -x against small ebcdic files that are residing on unix to see what you really have as input to cff stage. I would think that cff stage on se and ee do pretty much the same thing. Since you seem to know what hex you have in cff stage I will show you this that worked for me to handle special hex values in the transformer after cff stage. Isolate the column that has the hex in it and then "if Input.MyVar = char(n) then MyAction" where n is the decimal value between 1 and 256 that corresponds to the hex value you know is in the input file. With the hex value isolated you can set the result in Teradata that you want. You seem experienced at this but I would have to say that it is unusual for an mvs file to have hex values other than packed decimal, a binary integer. or a floating point number.

... Flash
Flash Gordon
Hyperborean Software Solution
Post Reply