DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
wblack



Group memberships:
Premium Members

Joined: 23 Sep 2010
Posts: 30

Points: 285

Post Posted: Wed Dec 21, 2011 4:11 pm Reply with quote    Back to top    

DataStage® Release: 8x
Job Type: Parallel
OS: Unix
Additional info: 8.5 fix pack 1
I'm using a column import that takes a schema file and reads in a file that's partially ascii and partially ebcdic. Most columns are either ascii or ebcdic and can be specified which in the schema file but I have one that consist of both ascii and ebcdic characters so I have to read it in that way. Is there a way of translating data character by character from within a single column. I was thinking about using the looping portion of the transformer and then translating it. Is there a better way or can this even be done?

record {binary, big_endian, record_format={type=implicit}, delim=none, final_delim=none}
(
irlstno:uint32;
ircmstrt:uint8 {link_keep};
ircfnm:uint8 {link_keep};
ircuser:uint8 {link_keep};
irstrt:string[max=255] {ebcdic, reference='ircmstrt'};
iruser:string[max=255] {reference='ircuser'};
}

_________________
William Black
jwiles



Group memberships:
Premium Members

Joined: 14 Nov 2004
Posts: 1274

Points: 10406

Post Posted: Wed Dec 21, 2011 5:18 pm Reply with quote    Back to top    

Are you saying that:

1) The column can contain a string which is either all ebcdic or all ascii and that can vary from row to row, or
2) The column can contain a string which is mixed ebcdic and ascii?

If #1, is there a separate indicator which can tell you whether the string is ascii or ebcdic? If so, simply use that to split the row to one of two column import stages to convert the column data then funnel the rows back together.

You could also use a BuildOp, or maybe a parallel routine called from a transformer (both are C++), for both situations. This would be more efficient that what could be coded with transformer logic alone.

Regards,

_________________
- james wiles


All generalizations are false, including this one - Mark Twain.
Rate this response:  
Not yet rated
wblack



Group memberships:
Premium Members

Joined: 23 Sep 2010
Posts: 30

Points: 285

Post Posted: Wed Dec 21, 2011 6:47 pm Reply with quote    Back to top    

I'm saying that some fields are ascii and some are ebcdic, but there's one field that's a combination of ascii and ebcdic. The first byte is the description of the field followed by a length byte of the data followed by the data segment. The first two bytes are ascii and the data segment is ascii.

_________________
William Black
Rate this response:  
Not yet rated
shamshad



Group memberships:
Premium Members

Joined: 25 Aug 2004
Posts: 147
Location: Detroit,MI
Points: 1383

Post Posted: Wed Dec 21, 2011 6:49 pm Reply with quote    Back to top    

Are you downloading the file from mainframe system?

_________________
Datawarehouse Consultant
Rate this response:  
Not yet rated
wblack



Group memberships:
Premium Members

Joined: 23 Sep 2010
Posts: 30

Points: 285

Post Posted: Wed Dec 21, 2011 6:51 pm Reply with quote    Back to top    

No, the file isn't from a mainframe/

_________________
William Black
Rate this response:  
Not yet rated
wblack



Group memberships:
Premium Members

Joined: 23 Sep 2010
Posts: 30

Points: 285

Post Posted: Thu Dec 22, 2011 5:51 am Reply with quote    Back to top    

The first byte of the column in question has a binary indicator (description of the field), followed by a byte for the binary length of the data segment followed by the ebcdic data segment where the length byte tells the length.

_________________
William Black
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours