Amount field in Complex Flat File in Parallel Vs Server

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

FranklinE
Premium Member
Premium Member
Posts: 739
Joined: Tue Nov 25, 2008 2:19 pm
Location: Malvern, PA

Post by FranklinE »

I had to reread your opening post to see what I missed. I apologize for not seeing it on the first read.

The character "{" represents the EBCDIC hex code xC0. The only way you see that character is if you are viewing the data with the EBCDIC character set. The ASCII code for "{" is x7B.

You indicate in your opening post that the Record Options tab has ASCII as the character set. The only thing I can think of that is preventing you from resolving your problem is this confusion between character sets. If your native file is stored using ASCII, you will not be able to read the numerics correctly.

Confirm the character set in the physical file first. If it's ASCII, you might want to re-copy the original but use EBCDIC to store it. If it's already stored with EBCDIC, I can't do anything further for you without having access to your system, and that is not an option. Sorry.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson

Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
kaps
Participant
Posts: 452
Joined: Tue May 10, 2005 12:36 pm

Post by kaps »

It's a vendor file we receive so I can't change anything. It's Ascii file with packed decimal amounts is what I thought. Please see below for part(some columns) of the actual file. Last record is the Trailer record which I am not taking into account now as I am trying to get other records to work first.

Code: Select all

VA2332101EF75  20151109  REF000000000000000000000{000000{000000{000000{000000{000000{000000{002310}000000{000000{ 500194049  00000000
IN4713002SF71  20150805  VOI000000000000000000000{000000{000000{000000{000000{000000{000000{000450}000000{000000{ 500472457  00000000
NC2814601EF69  20151028  VOI000000000000000000000{000000{000000{000000{000000{000000{000000{000660}000000{000000{ 500396210  00000000
SC2903601EM73  20151118  VOI000000000000000000000{000000{000000{000000{000000{000000{000000{000880}000000{000000{ 500403562  00000000
NC2770301EM68  20150910  VOI000000000000000000000{000000{000000{000000{000000{000000{000000{000370}000000{000000{ 500482603  00000000
ND5870301EF41  20150728  VOI000000000000000000000{000000{000000{000000{000000{000000{000000{004690}000000{000000{ 500447459  00000000
KY4021901EF66  20151026  REF000000000000000000000{000000{000000{000000{000000{000000{000000{000990}000000{000000{ 500192968  00000000
MS3905601EF552A20151117  SP 043410000000000002250{002250{001800{000450{000000{000000{000000{000450{000000{000000{ 340341515TR20151231
MS3905601EF552A20151117  SP 043410000000000002250{002250{001800{000450{000000{000000{000000{000450{000000{000000{ 340341515BR20151231
AR7221201EM751A20151204  SP 001500000000000000850{000850{000610{000240{000000{000000{000000{000240{000000{000000{ 340343326  20151231
AR7221201EM751A20151204  SP 001500000000000000850{000850{000610{000240{000000{000000{000000{000240{000000{000000{ 340343326  20151231
AR7221201EM751A20151204  SP 002740000000000000710{000710{000490{000220{000000{000000{000000{000220{000000{000000{ 340343326  20151231
AR7221201EM751A20151204  SP 011100000000000000750{000750{000410{000340{000000{000000{000000{000340{000000{000000{ 340343326  20151231
F320160128TPM0616910105633081D0105504689D0027124689H0000000000{0000000000{0000000000{0026795540F
By the way, Thanks for all your help and I am not expecting you to do my work either :-) I think I have couple of options to work around this but was just wondering If I did anything wrong. I am also going to create PMR with IBM about this and also talk to vendor to see if this can be a csv file instead of a file like this :-)

Thanks again for your time !
FranklinE
Premium Member
Premium Member
Posts: 739
Joined: Tue Nov 25, 2008 2:19 pm
Location: Malvern, PA

Post by FranklinE »

Since it's a vendor file, and you can't control how you receive it, your options are definitely limited.

The final byte in each amount field is workable as suggested earlier with Convert. I would suggest, to avoid future glitches, that you create a conversion for all 20 (ten negatives, ten positives) possible values in that last byte. Unless you are absolutely sure the final number will always be zero, you should code for the other numbers.

Another possibility is to reverse-translate the entire file back to EBCDIC. I've never tried that, but it seems possible to put it back to the format it should be in for Cobol-sourced data. CFF is designed for it. If it works, it would be more efficient than having a convert derivation for every amount field.

I didn't mean to imply that you are hoping for me to do your work for you. You have my sympathy here, and I'm glad to help.

Edit: I don't believe this is a PMR problem. I won't second-guess what IBM will say, but I don't think they can help. The root cause is the vendor process, so I also don't believe you've done anything wrong. If you read the FAQ, you'll see that working with mainframe data is just not as easy as it looks. Good luck.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson

Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
kaps
Participant
Posts: 452
Joined: Tue May 10, 2005 12:36 pm

Post by kaps »

Thanks for your note Franklin. I thought you said that there is a flaw in 8.x in your first post and moreover this works in Server which is why I thought I will give a try with IBM.

I just have one question. Can you tell me why I had to divide the numbers by 100 and 10 like I mentioned in my other post to match the value which is displayed in Server ? If I convert StringToDecimal with the target column defined as Decimal 7.2 should not it work ?

Thanks again...
FranklinE
Premium Member
Premium Member
Posts: 739
Joined: Tue Nov 25, 2008 2:19 pm
Location: Malvern, PA

Post by FranklinE »

There is a flaw, but your problem happens before you get to that point.

It turns out I had an old job that does most of what I suggested. It converts an ASCII file to an equivalent EBCDIC file. The assumption is that you want it that way, and conform with the COBOL/EBCDIC requirements. In your case, it should work.

Code: Select all

ASCII sequential file stage ===> EBCDIC sequential file stage:

Table definition in both: single column, Char for length of record.

ASCII file format attributes:
Record level -- Record delimiter = UNIX newline; record length = fixed
Field defaults -- delimiter = none
Type Defaults -- General -- Character set = ASCII; Data format = text

EBCDIC file format attributes: after "remove all", right-click on Record level, select Format as/Mainframe(COBOL)
I used your posted data successfully in creating a sequential file that I could then read with CFF.

I didn't mind doing this at all, because it helps something else I'm doing.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson

Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
FranklinE
Premium Member
Premium Member
Posts: 739
Joined: Tue Nov 25, 2008 2:19 pm
Location: Malvern, PA

Post by FranklinE »

I don't know why your conversion needs the divide by 100, etc. I believe it's due to something going on in the StringToDecimal function, and the temporary variables it creates when compiled.

My method above works (you will ask why, I'm sure) because: :lol:

Your ASCII file is a "direct" conversion from EBCDIC characters. It is not a binary conversion, which is why my ASCII format attribute is set to "text". The vendor replaced the hex value of the character "{" xC0 with the hex value x7B. It preserved the character, not the binary value.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson

Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
FranklinE
Premium Member
Premium Member
Posts: 739
Joined: Tue Nov 25, 2008 2:19 pm
Location: Malvern, PA

Post by FranklinE »

For your posted data, here's the cfd I created:

Code: Select all

       01 IN-REC.
           05 FILLER PIC X(43).
           05 AMT-1  PIC S9(5)V9(2).
           05 AMT-2  PIC S9(5)V9(2).
           05 AMT-3  PIC S9(5)V9(2).
           05 AMT-4  PIC S9(5)V9(2).
           05 AMT-5  PIC S9(5)V9(2).
           05 AMT-6  PIC S9(5)V9(2).
           05 AMT-7  PIC S9(5)V9(2).
           05 AMT-8  PIC S9(5)V9(2).
           05 AMT-9  PIC S9(5)V9(2).
           05 AMT-10 PIC S9(5)V9(2).
           05 FILLER PIC X(20).

I was only interested in getting the amount fields to behave properly. The FILLER fields still display the data correctly.

Good luck.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson

Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
Post Reply