DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
Sandeep.pendem
Participant



Joined: 02 May 2008
Posts: 27
Location: Mumbai
Points: 220

Post Posted: Thu Dec 18, 2008 5:32 pm Reply with quote    Back to top    

DataStage® Release: 7x
Job Type: Server
OS: Unix
Additional info: EBCDIC (binary data PIC S9(4) COMP) to ASCII from CFF to Seq
Hi,

I have tried almost all the transform Datatype functions to read the SmallInt(2) i.e a EBCDIC(binary) data from Complex flat file stage in the data format option selected ASCII and type binary to convert the field into ASCII data, none of the function is giving me expected results.

Does anyone knows which transform datatype function i should be using or we dont any such conversion function in DS Version 7.1?

Thanks very much,
Sandeep
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 39347
Location: Denver, CO
Points: 200968

Post Posted: Thu Dec 18, 2008 5:48 pm Reply with quote    Back to top    

Packed data (any of the COMP family) isn't EBCDIC nor ASCII - it's packed. And 'converting' it in any fashion will corrupt it. You just need to 'unpack' it. Define the input field as an appropria ...

_________________
-craig

The moments seemed lost in all the noise, a snow storm, a stimulating voice
Of warmth of the sky, of warmth when you die
Rate this response:  
Not yet rated
crouse


since February 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 05 Oct 2003
Posts: 204

Points: 1412

Post Posted: Thu Dec 18, 2008 6:11 pm Reply with quote    Back to top    

but if you're using CFF stage like the additional note says, it will do the unpacking for you if your copy book (input meta data for the CFF stage) is defined correctly.

If you decide to do it via the routine, remember it takes up more room (more bytes) unpacked than packed (I guess I just stated the obvious).

Depending on how it is packed determines how big it is unpacked... which can cause you grief (I did this stuff before the CFF stage was developed).

Best bet is to make the CFF stage do it, that's one of the best reasons for the CFF stage.

_________________
Craig Rouse
Griffin Resouces, Inc
www.griffinresources.com
Rate this response:  
Not yet rated
Sandeep.pendem
Participant



Joined: 02 May 2008
Posts: 27
Location: Mumbai
Points: 220

Post Posted: Thu Dec 18, 2008 7:23 pm Reply with quote    Back to top    

Thanks for the response, I quite didnt understood regarding unpacked and packed stuff? can you please elaborate little more with some examples? Dont we have a trasnform/routine which can convert my binary data to a readable format on datastage? I am curious to know?
Rate this response:  
Not yet rated
crouse


since February 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 05 Oct 2003
Posts: 204

Points: 1412

Post Posted: Thu Dec 18, 2008 8:06 pm Reply with quote    Back to top    

Have you tried using the CFF stage yet?

_________________
Craig Rouse
Griffin Resouces, Inc
www.griffinresources.com
Rate this response:  
Not yet rated
Sandeep.pendem
Participant



Joined: 02 May 2008
Posts: 27
Location: Mumbai
Points: 220

Post Posted: Thu Dec 18, 2008 8:12 pm Reply with quote    Back to top    

Yes, I am using CFF stage with ASCII as data format , Record Style as binary and I have Binary fields in my source...as such I am getting data in a flat file with ebcdic and binary mode format, however when I am using CFF stage, I could view most of the other character fields as ASCII but Binary field data is still an issue

Thats the reason I am looking for any routine or transform function which will convert this binary fields into ASCII human readable format

Thanks much for the help, please advice if i can use any such functions?
Rate this response:  
Not yet rated
crouse


since February 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 05 Oct 2003
Posts: 204

Points: 1412

Post Posted: Thu Dec 18, 2008 8:48 pm Reply with quote    Back to top    

Sorry to keep bringing up the CFF stage, but if you have it configured right and the meta data truely
reflects your data then the CFF stage works.

Else fall back to using the transform/routine mr. Hulett refers to. Their use is pretty straight forward.

_________________
Craig Rouse
Griffin Resouces, Inc
www.griffinresources.com
Rate this response:  
Not yet rated
Sandeep.pendem
Participant



Joined: 02 May 2008
Posts: 27
Location: Mumbai
Points: 220

Post Posted: Thu Dec 18, 2008 8:56 pm Reply with quote    Back to top    

Thanks for the response,

I have tried almost all the functions on the server transform/routines, none of this is working fine for me, i have a binary field, length is 2 I am unable convert it to ASCII format, Can you please suggest some way? alternate or right way of using this functions?

I have tried DataTypePicComp,DataTypePic9,DataTypeEbcidicToAscii,DataTypePicS9,etc...
Rate this response:  
Not yet rated
Sandeep.pendem
Participant



Joined: 02 May 2008
Posts: 27
Location: Mumbai
Points: 220

Post Posted: Fri Dec 19, 2008 4:00 pm Reply with quote    Back to top    

Hello,

Experts please suggest some alternative, or right way of using the transform functions to reac ebcdic(binary) data to convert it into Ascii, I am still struggling with this encdic to ascii conversion..any help would be great..
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 39347
Location: Denver, CO
Points: 200968

Post Posted: Fri Dec 19, 2008 4:28 pm Reply with quote    Back to top    

Again, if we're still just talking about packed decimal fields, they're neither EBCDIC nor ASCII. Tell us exactly how you tried to use the DataTypePicComp SDK routine and how it didn't work for you, i.e. what came out the other end. Stages, data types and sizes, derivations.

Examples are always good. Details are always good. Saying something just "didn't work" for you... not so good.

ps. As that other Craig noted, none of this is necessary if you are using the CFF stage. Set it up correctly with the proper metadata and all this is automagic.

_________________
-craig

The moments seemed lost in all the noise, a snow storm, a stimulating voice
Of warmth of the sky, of warmth when you die
Rate this response:  
Not yet rated
crouse


since February 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 05 Oct 2003
Posts: 204

Points: 1412

Post Posted: Fri Dec 19, 2008 4:40 pm Reply with quote    Back to top    

I've finally got to a machine to bring up the CFF stage and the settings.

First of all, how did you acquire the data?
I'm assuming it is from an IBM mainframe, right?
Did you get it via FTP? Was the FTP performed in ASCII or BINARY mode?
If in ASCII mode, your packed/binary data is no good anymore and you can't convert it no matter what you do, and the CFF stage won't help either.
If you FTP'd the data in BINARY mode, you should use the EBCDIC option for Data Format in the CFF stage, and BINARY for Record Style. The CFF stage figures out the record length in this mode, assuming you're copybook is correct (you do have a copybook defining the mainframe data, right?)

Now, the routines/transforms... if the data was FTP'd in ascii format, the data is no good and can never be unpacked using any routine/transform.

Now, let's assume the data was FTP'd in Binary and you don't want to use the CFF stage. Well, the packed/binary data can be unpacked using the transforms/routines. The trick? Knowing just exactly where the data is in the row. The copybook says it is PIC S9(4) COMP, right? Well, it doesn't take up 4 or 5 bytes in your raw data file from the mainframe. It takes up 2 or 3 or 4, I forget. So you need to make sure you're looking at the right/correct bytes in order to unpack it. This is precisely why you use the CFF stage. If you have more than a handful of packed fields, defining the correct file layout in the Sequential file stage is really hard and prone to data shifting big time. Think about it, you define the first packed field incorrectly and the entire remainder of the row meta data is screwed up.

I may be going off into left field. Does any of this help with your issue?

_________________
Craig Rouse
Griffin Resouces, Inc
www.griffinresources.com
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 39347
Location: Denver, CO
Points: 200968

Post Posted: Fri Dec 19, 2008 5:13 pm Reply with quote    Back to top    

psssttt... it takes up 2. Wink

_________________
-craig

The moments seemed lost in all the noise, a snow storm, a stimulating voice
Of warmth of the sky, of warmth when you die
Rate this response:  
Not yet rated
Sandeep.pendem
Participant



Joined: 02 May 2008
Posts: 27
Location: Mumbai
Points: 220

Post Posted: Fri Dec 19, 2008 6:41 pm Reply with quote    Back to top    

Thanks a lot experts,

I have FTPied it using mainframe in EBCDIC and MODE is binary
Here is an example I am struglling with
my source column MONTHKEY is PIC S9(4), I have used Binary(4) in CFF stage and it shows me valyue as 7397, but the expected value is 7329

I have used following functions convert it into binary
1) DataTypeAsciiPic9(DSLink8.MONTHKEY) the traget column MONTHKEY is smallInt(5)
2)DataTypePicS9(DSLink8.MONTHKEY) the target column is MONTHKEY is smallInt(5)
3)DatatypePicComp(DSLink8.MONTHKEY) the target column is MONTHKEY is smallInt(5)
4) DatatypeEbcdictoasciiDSLink8.MONTHKEY) the target column is MONTHKEY is smallInt(5)
...
....
Likewaise I tried almost all the functions, any idea what is correct way of using this functions? or I am defining my metadat incorrectly? however all the character data in the EBCDIC file shows up as expected...dont know why this binary is troubling me so much...

Thank much
Rate this response:  
Not yet rated
Sandeep.pendem
Participant



Joined: 02 May 2008
Posts: 27
Location: Mumbai
Points: 220

Post Posted: Fri Dec 19, 2008 6:42 pm Reply with quote    Back to top    

Thanks a lot experts,

I have FTPied it using mainframe in EBCDIC and MODE is binary
Here is an example I am struglling with
my source column MONTHKEY is PIC S9(4), I have used Binary(4) in CFF stage and it shows me valyue as 7397, but the expected value is 7329

I have used following functions convert it into binary
1) DataTypeAsciiPic9(DSLink8.MONTHKEY) the traget column MONTHKEY is smallInt(5)
2)DataTypePicS9(DSLink8.MONTHKEY) the target column is MONTHKEY is smallInt(5)
3)DatatypePicComp(DSLink8.MONTHKEY) the target column is MONTHKEY is smallInt(5)
4) DatatypeEbcdictoasciiDSLink8.MONTHKEY) the target column is MONTHKEY is smallInt(5)
...
....
Likewaise I tried almost all the functions, any idea what is correct way of using this functions? or I am defining my metadat incorrectly? however all the character data in the EBCDIC file shows up as expected...dont know why this binary is troubling me so much...

Thank much
Rate this response:  
Not yet rated
Sandeep.pendem
Participant



Joined: 02 May 2008
Posts: 27
Location: Mumbai
Points: 220

Post Posted: Sun Dec 21, 2008 9:46 am Reply with quote    Back to top    

Hello Experts,

Could you please throw somelight here....is this doable? I mean can we use those DS transform function to convert it into Ascii, I am wondering why none of this functions give me accurate results.....please advice....
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours