Amount field in Complex Flat File in Parallel Vs Server

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

kaps
Participant
Posts: 452
Joined: Tue May 10, 2005 12:36 pm

Amount field in Complex Flat File in Parallel Vs Server

Post by kaps »

I am having trouble converting a server job to a parallel(v8.7) job which uses Complex flat file stage.

Problem is that all of the Amount fields does not show up correct when I do view data in Parallel but works fine in Server. These fields are designed as DISPLAY_NUMBERIC in both versions with the Length 7 and scale 2. We have slected 'Signed' in both versions in Sign Indicator. It's same table definition imported into Parallel job but Parallel version shows only 0's instead of the actual amount.

In 'Record Options' tab, it's defined as 'Big-endian' for Byte order, Chracter set of 'ASCII' and Data format of 'Binary' and the record delimiter is 'Unix Newline'. For Decimal fields, Rounding is 'Nearest Value' and Seperator is 'Project Default' and 'Allow all zeros' option is checked.

Only difference I see is that in PArallel NLS is defined as Project Default(ISO-8859-1) whereas in Server it's defined as Project Default(NONE) which is not available in Parallel.

Any idea why Parallel is not showing the correct results ?

Appreciate your time in advance.
FranklinE
Premium Member
Premium Member
Posts: 739
Joined: Tue Nov 25, 2008 2:19 pm
Location: Malvern, PA

Post by FranklinE »

viewtopic.php?p=468568#468568

There is a flaw in 8.x concerning display numeric fields. See if the linked post is helpful.

I'd need to see the original Cobol definition of the field to comment further, but if it is something like PIC -9(5).99, my suggestion in that post will help.

The link to the using mainframe data FAQ is in my signature line.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson

Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
kaps
Participant
Posts: 452
Joined: Tue May 10, 2005 12:36 pm

Post by kaps »

I actually don't have a CFD. I exported the table definition from the server job and imported into Parallel job. I am not able to change the Display_Numeric to AlphaNumeric as there is no data type like that. I tried to change it to Varchar but it did not work.
kduke
Charter Member
Charter Member
Posts: 5227
Joined: Thu May 29, 2003 9:47 am
Location: Dallas, TX
Contact:

Post by kduke »

Try is as a CHAR or VARCHAR then convert to decimal.
Mamu Kim
FranklinE
Premium Member
Premium Member
Posts: 739
Joined: Tue Nov 25, 2008 2:19 pm
Location: Malvern, PA

Post by FranklinE »

In the CFF stage, Records tab, you can make the change in the "Native Type" column. Try using CHARACTER. This is the CFF equivalent to alphanumeric, and to the Cobol clause PIC X().

On the Layout tab, click the Cobol radial. Right click on the first line for the record, and save as html file. This will give you the equivalent to the cfd, whether it was imported or entered manually doesn't matter.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson

Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
kaps
Participant
Posts: 452
Joined: Tue May 10, 2005 12:36 pm

Post by kaps »

Thanks for the suggestions. When I made it as CHAR(7) then I am able to see the values when I view the data.

In Parallel it shows like 002250{ whereas in server it shows correctly like +00225.00

Then I converted the amount in Transformer like StringToDecimal(DSLink2.FieldName) in which the target is defined as Decimal 7 with Scale 2 but what I see in the target sequential file is 00000.00. I tried mentioning packed and signed in the properties of the target file it did not make any difference. I have given the CDF as requested below.

Thanks for your time.

Code: Select all

Column Picture clause Starting column Ending column Storage length 
 01 RECORD_1 (273)     
   02 RecordType PIC X(2). 1 2 2 
   02 Subsidiary PIC X(3). 3 5 3 
   02 Policy PIC X(6). 6 11 6 
   02 DivisionNo PIC X(5). 12 16 5 
   02 Cert PIC X(9). 17 25 9 
   02 ClassNo PIC X(2). 26 27 2 
   02 PlanNo PIC X(3). 28 30 3 
   02 PlanType PIC X(1). 31 31 1 
   02 PlanEffDt PIC X(8). 32 39 8 
   02 CoverageType PIC X(3). 40 42 3 
   02 MemberSsn PIC X(9). 43 51 9 
   02 PolicyType PIC X(2). 52 53 2 
   02 TransType PIC X(1). 54 54 1 
   02 ClaimType PIC X(1). 55 55 1 
   02 ClaimNumber PIC X(12). 56 67 12 
   02 AcctngDate PIC X(8). 68 75 8 
   02 PPO PIC X(6). 76 81 6 
   02 ProvSsn PIC X(9). 82 90 9 
   02 ProvAin PIC X(9). 91 99 9 
   02 NationalProviderId PIC X(10). 100 109 10 
   02 ProvSt PIC X(2). 110 111 2 
   02 ProvZip5 PIC X(5). 112 116 5 
   02 ClmntNo PIC X(2). 117 118 2 
   02 ClmntRelation PIC X(1). 119 119 1 
   02 ClmntSex PIC X(1). 120 120 1 
   02 ClmntAge PIC X(2). 121 122 2 
   02 BenType PIC X(2). 123 124 2 
   02 DateOfServ PIC X(8). 125 132 8 
   02 ExclCode PIC X(2). 133 134 2 
   02 ExclGrp PIC X(3). 135 137 3 
   02 ProcCode PIC X(5). 138 142 5 
   02 ListCode PIC X(5). 143 147 5 
   02 PayasCode PIC X(5). 148 152 5 
   02 SubmitCharge PIC X(7). 153 159 7 
   02 ConsidCharge PIC X(7). 160 166 7 
   02 AmtDisallow PIC X(7). 167 173 7 
   02 AllowAmt PIC X(7). 174 180 7 
   02 DeducAmt PIC X(7). 181 187 7 
   02 CopayAmt PIC X(7). 188 194 7 
   02 CobAmt PIC X(7). 195 201 7 
   02 AmtPaid PIC X(7). 202 208 7 
   02 ExclByMax PIC X(7). 209 215 7 
   02 CoMaxUsed PIC X(7). 216 222 7 
   02 HPSPolicyNumber PIC X(10). 223 232 10 
   02 ToothQuadArch PIC X(2). 233 234 2 
   02 Adjdate PIC X(8). 235 242 8 
   02 ClmntName PIC X(23). 243 265 23 
   02 RecvDate PIC X(8). 266 273 8 
Total File Length: 273 
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

kaps wrote:In Parallel it shows like 002250{ whereas in server it shows correctly like +00225.00
Those are two different flavors of the same correct. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
kaps
Participant
Posts: 452
Joined: Tue May 10, 2005 12:36 pm

Post by kaps »

I kind of got that craig...My question is more towards why am I getting 0's after applying StringToDecimal function ?
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Because "002250{" is not a valid Decimal number. The "{" character indirectly indicates that the number is positive. It's made up of the nybbles for "0" and "{".

If you're sure that all values will be positive, simply convert the "{" to 0 before applying the StringToDecimal() function.

Code: Select all

StringToDecimal(Convert("{","0",DSLink2.FieldName))
If you have numbers of either sign it's only marginally more complex.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

You can't just handle "{" - seems to me you'd need to handle the full range of the signed overpunch values... "A" being a positive 1, "B" positive 2, etc. And then there's the negatives. And you'd have to handle the implied decimal. Shouldn't the stage be doing that for you? :?
-craig

"You can never have too many knives" -- Logan Nine Fingers
FranklinE
Premium Member
Premium Member
Posts: 739
Joined: Tue Nov 25, 2008 2:19 pm
Location: Malvern, PA

Post by FranklinE »

Ray and Craig are helpful as far as they go. Here's a bit of background and what should be the correct parallel format for your decimal field:

Code: Select all

decimal[7,2] {zoned, default=0}
Cobol equivalent: PIC S9(5)V9(2)
The field is not display numeric, it's zoned numeric. The overpunch aspect will be correctly read by CFF. You can confirm this by creating the CFD manually from what you showed above (copy/paste the text, make sure the first line starting with "01" starts in position 8, remove the position and length numbers from each line). Make each amount field as I showed above. The "V" is the implied position of the decimal, which is not stored in the physical file. Import the CFD and test it with your data.

You can do that in the table definition or the CFF stage. I'm suggesting you go through that as an exercise, because the most efficient way to make sure of your table definition is in the CFD, and knowing how to edit it before import.

FYI:

"Zoned" is a variant of signed. You don't need to add that attribute. It is what defines the "overpuch". Parallel format has total length with decimal portion, Cobol shows each portion explicitly.

EBCDIC hex values for numerals are F1 through F9. "F" is also the place holder for an unsigned number. The overpunch replaces the F with "C" for positive and "D" for negative, hence the change in the character being displayed. The same usage appears in packed decimals. See the FAQ.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson

Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
FranklinE
Premium Member
Premium Member
Posts: 739
Joined: Tue Nov 25, 2008 2:19 pm
Location: Malvern, PA

Post by FranklinE »

chulett wrote:Shouldn't the stage be doing that for you? :?
Craig, I took some mercy on the person because of the server-to-parallel thing. CFF and CFDs in general are a pain in the butt, with some non-intuitive things.

Besides, I like making posts that score more than 10 points. :lol:
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson

Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

LOL

(3 points!)
-craig

"You can never have too many knives" -- Logan Nine Fingers
kaps
Participant
Posts: 452
Joined: Tue May 10, 2005 12:36 pm

Post by kaps »

Ray/Craig,
I am able to convert the values using the convert function like advised. Please see below.

Code: Select all

If Index(DSLink2.AmtPaid,"{", 1) >= 1
Then StringToDecimal(Convert("{","0", DSLink2.AmtPaid)/100)
Else 
          If Index(DSLink2.AmtPaid,"}", 1) >= 1 
          Then StringToDecimal(Convert("}","", DSLink2.AmtPaid)*-1/10)
          Else ''
I had to divide by 100 for Positive and divide by 10 for Negative numbers to get the correct value though it's defined as Decimal length 7 with scale of 2. What am I doing wrong ?
kaps
Participant
Posts: 452
Joined: Tue May 10, 2005 12:36 pm

Post by kaps »

Franklin

I completed the exercise as you suggested but it still shows 0's when I view data. Have given the CFD after changing it...

Code: Select all

01 RECORD_1 (273)     
   02 RECORDTYPE PIC X(2). 1 2 2 
   02 SUBSIDIARY PIC X(3). 3 5 3 
   02 POLICY PIC X(6). 6 11 6 
   02 DIVISIONNO PIC X(5). 12 16 5 
   02 CERT PIC X(9). 17 25 9 
   02 CLASSNO PIC X(2). 26 27 2 
   02 PLANNO PIC X(3). 28 30 3 
   02 PLANTYPE PIC X(1). 31 31 1 
   02 PLANEFFDT PIC X(8). 32 39 8 
   02 COVERAGETYPE PIC X(3). 40 42 3 
   02 MEMBERSSN PIC X(9). 43 51 9 
   02 POLICYTYPE PIC X(2). 52 53 2 
   02 TRANSTYPE PIC X(1). 54 54 1 
   02 CLAIMTYPE PIC X(1). 55 55 1 
   02 CLAIMNUMBER PIC X(12). 56 67 12 
   02 ACCTNGDATE PIC X(8). 68 75 8 
   02 PPO PIC X(6). 76 81 6 
   02 PROVSSN PIC X(9). 82 90 9 
   02 PROVAIN PIC X(9). 91 99 9 
   02 NATIONALPROVIDERID PIC X(10). 100 109 10 
   02 PROVST PIC X(2). 110 111 2 
   02 PROVZIP5 PIC X(5). 112 116 5 
   02 CLMNTNO PIC X(2). 117 118 2 
   02 CLMNTRELATION PIC X(1). 119 119 1 
   02 CLMNTSEX PIC X(1). 120 120 1 
   02 CLMNTAGE PIC X(2). 121 122 2 
   02 BENTYPE PIC X(2). 123 124 2 
   02 DATEOFSERV PIC X(8). 125 132 8 
   02 EXCLCODE PIC X(2). 133 134 2 
   02 EXCLGRP PIC X(3). 135 137 3 
   02 PROCCODE PIC X(5). 138 142 5 
   02 LISTCODE PIC X(5). 143 147 5 
   02 PAYASCODE PIC X(5). 148 152 5 
   02 SUBMITCHARGE PIC S9(5)V9(2). 153 159 7 
   02 CONSIDCHARGE PIC S9(5)V9(2). 160 166 7 
   02 AMTDISALLOW PIC S9(5)V9(2). 167 173 7 
   02 ALLOWAMT PIC S9(5)V9(2). 174 180 7 
   02 DEDUCAMT PIC S9(5)V9(2). 181 187 7 
   02 COPAYAMT PIC S9(5)V9(2). 188 194 7 
   02 COBAMT PIC S9(5)V9(2). 195 201 7 
   02 AMTPAID PIC S9(5)V9(2). 202 208 7 
   02 EXCLBYMAX PIC S9(5)V9(2). 209 215 7 
   02 COMAXUSED PIC S9(5)V9(2). 216 222 7 
   02 HPSPOLICYNUMBER PIC X(10). 223 232 10 
   02 TOOTHQUADARCH PIC X(2). 233 234 2 
   02 ADJDATE PIC X(8). 235 242 8 
   02 CLMNTNAME PIC X(23). 243 265 23 
   02 RECVDATE PIC X(8). 266 273 8 
Thanks for your help.
Post Reply