New XML Stage Issue
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 32
- Joined: Tue Mar 09, 2010 5:56 am
- Location: Bangalore
New XML Stage Issue
I have a job like the following.
1. External source stage to read the xml file path.
2. Parsing the xml by using new xml stage in datastage 8.5.
3. Write the parsing records into one file.
But here I am getting only one output record, though there are number of records in the XML file.
Is there anything I have missed to configure in XML stage.
1. External source stage to read the xml file path.
2. Parsing the xml by using new xml stage in datastage 8.5.
3. Write the parsing records into one file.
But here I am getting only one output record, though there are number of records in the XML file.
Is there anything I have missed to configure in XML stage.
Thanks & Regards,
Partha
Partha
-
- Premium Member
- Posts: 47
- Joined: Tue Sep 13, 2005 4:28 am
-
- Premium Member
- Posts: 32
- Joined: Tue Mar 09, 2010 5:56 am
- Location: Bangalore
1)In extenal source stage I am using this command which will read the file staring with TST.
ls /app/DEV_SET0/Temp/TST
This command is giving the list of files with path.
2)The in XML stage in XML parser step I am using file set option to read the filepath which I have selected from the drop down list.
3)In the document root I have browsed the XSD for the XML files , then in validation I am using the minimum validation option.
4)I have done the validation of the data by clicking on Test Assembly--->Run Test-->Test Data.
5)Then in output step mapping i am using automapping and deleting the unwanted columns, then click ok and run the job.
It is fetching only one record from the XML stage.
ls /app/DEV_SET0/Temp/TST
This command is giving the list of files with path.
2)The in XML stage in XML parser step I am using file set option to read the filepath which I have selected from the drop down list.
3)In the document root I have browsed the XSD for the XML files , then in validation I am using the minimum validation option.
4)I have done the validation of the data by clicking on Test Assembly--->Run Test-->Test Data.
5)Then in output step mapping i am using automapping and deleting the unwanted columns, then click ok and run the job.
It is fetching only one record from the XML stage.
Thanks & Regards,
Partha
Partha
It's probably a list mapping issue.
Look at your xsd as loaded into the Assembly. One place you can do this is in beginning of the xml Parser step where you selected your document root.
Open up the whole thing. Look for the little blue "list" icons that outline your "repeating nodes".
You say that you have multiple "records" in your xml. Usually this means that you have multiple "instances" of some lower level node --- such as several employees, assets, visits, calls --- whatever the node is containing. It helps if you are also looking at an actual document opened in your browser.
What "level" are those "records" at? This is the most critical thing to consider.....like the "repeating element" of the xmlInput Stage, the key to mapping your multiple instances of a node inside of the xml Stage is to map the list for the "lowest level that you care about for repeating data" to the list that represents your output link. Do this mapping BEFORE you map or do ANYTHING else.
List-to-List --- number of occurring nodes to number of rows.
If the notes above aren't making sense, perhaps we don't understand what you mean by multiple "records" in your document.
Ernie
Look at your xsd as loaded into the Assembly. One place you can do this is in beginning of the xml Parser step where you selected your document root.
Open up the whole thing. Look for the little blue "list" icons that outline your "repeating nodes".
You say that you have multiple "records" in your xml. Usually this means that you have multiple "instances" of some lower level node --- such as several employees, assets, visits, calls --- whatever the node is containing. It helps if you are also looking at an actual document opened in your browser.
What "level" are those "records" at? This is the most critical thing to consider.....like the "repeating element" of the xmlInput Stage, the key to mapping your multiple instances of a node inside of the xml Stage is to map the list for the "lowest level that you care about for repeating data" to the list that represents your output link. Do this mapping BEFORE you map or do ANYTHING else.
List-to-List --- number of occurring nodes to number of rows.
If the notes above aren't making sense, perhaps we don't understand what you mean by multiple "records" in your document.
Ernie
Ernie Ostic
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
-
- Premium Member
- Posts: 32
- Joined: Tue Mar 09, 2010 5:56 am
- Location: Bangalore
eostic wrote:It's probably a list mapping issue.
Look at your xsd as loaded into the Assembly. One place you can do this is in beginning of the xml Parser step where you selected your document root.
Open up ...
I am not able to see the full content, could you please send the details in my mail id, if possible, my mail id is partha.natta@igate.com
Thanks & Regards,
Partha
Partha
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Please obtain premium membership, which will allow you to see the entire response from the five premium posters.
Premium membership is the mechanism used to fund the hosting and bandwidth costs incurred by DSXchange. Without this, DSXchange cannot continue to exist.
Premium membership is the mechanism used to fund the hosting and bandwidth costs incurred by DSXchange. Without this, DSXchange cannot continue to exist.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Premium Member
- Posts: 32
- Joined: Tue Mar 09, 2010 5:56 am
- Location: Bangalore
I have taken the premium membership for one month, but still I am not able to see the content, even after follow up also I am not getting any reply from them.
Invoice# 0000002805
Invoice Date: 06/12/2014 03:00:45
Items
Qty Item # Description Price Each Total
1 1 Month Premium DSXchange 1 Month Premium Access
DSXchange User Name:partha.natta
$14.95 $14.95
Subtotal: $14.95
Tax : $0.00
Total: $14.95
Balance Due: $0.00
Invoice# 0000002805
Invoice Date: 06/12/2014 03:00:45
Items
Qty Item # Description Price Each Total
1 1 Month Premium DSXchange 1 Month Premium Access
DSXchange User Name:partha.natta
$14.95 $14.95
Subtotal: $14.95
Tax : $0.00
Total: $14.95
Balance Due: $0.00
Thanks & Regards,
Partha
Partha
-
- Premium Member
- Posts: 32
- Joined: Tue Mar 09, 2010 5:56 am
- Location: Bangalore
Sampple XML:
<?xml version="1.0" encoding="UTF-8"?>
<TrxnRecon xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="nsrt" xsi:schemaLocation="nsrt Reconciliation.xsd" Version="24">
<CreateDate>20140521</CreateDate>
<MgmtCode>UN6</MgmtCode>
<DlrCode>9190</DlrCode>
<TrxnRec><ProcessDate>20140520</ProcessDate><FundAcctID>42069310</FundAcctID><AcctDesig>2</AcctDesig><DlrCode>9190</DlrCode><DlrAcctID>D140508150501</DlrAcctID><RepCode>AAB</RepCode><AcctType>01</AcctType><OrdID>2390886</OrdID><OrdSrc>D</OrdSrc><OrdType>W</OrdType><SrcID>NEFC1757635194</SrcID><BuyCof><TrxnTyp>5</TrxnTyp><BuyFund><FundID>3406</FundID><Currency>00</Currency><DivOpt>1</DivOpt><SequenceNum>27020000</SequenceNum><GrossAmt>10004.44</GrossAmt><FundPdComm>100.04</FundPdComm><NetAmt>10004.44</NetAmt><NAV>12.8453</NAV><UnitTrxnd>778.8410</UnitTrxnd><TotalIssued>0.0000</TotalIssued><TotalUnissued>3311.4950</TotalUnissued><AveCost>12.0805</AveCost><SettlMethd>1</SettlMethd><SettlAmt>9904.40</SettlAmt></BuyFund><TradeDate>20140515</TradeDate><SettlDate>20140520</SettlDate></BuyCof></TrxnRec>
<TrxnRec><ProcessDate>20140521</ProcessDate><FundAcctID>42169310</FundAcctID><AcctDesig>3</AcctDesig><DlrCode>9191</DlrCode><DlrAcctID>E140508150501</DlrAcctID><RepCode>AAC</RepCode><AcctType>02</AcctType><OrdID>2390887</OrdID><OrdSrc>E</OrdSrc><OrdType>Z</OrdType><SrcID>NEFC1757635195</SrcID><BuyCof><TrxnTyp>6</TrxnTyp><BuyFund><FundID>3407</FundID><Currency>01</Currency><DivOpt>2</DivOpt><SequenceNum>27020001</SequenceNum><GrossAmt>10005.44</GrossAmt><FundPdComm>100.05</FundPdComm><NetAmt>10005.44</NetAmt><NAV>13.8453</NAV><UnitTrxnd>779.8410</UnitTrxnd><TotalIssued>1.0000</TotalIssued><TotalUnissued>3312.4950</TotalUnissued><AveCost>13.0805</AveCost><SettlMethd>2</SettlMethd><SettlAmt>9905.40</SettlAmt></BuyFund><TradeDate>20140516</TradeDate><SettlDate>20140521</SettlDate></BuyCof></TrxnRec>
</TrxnRecon>
XSD File:
<?xml version="1.0"?>
<xs:schema xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" attributeFormDefault="unqualified" elementFormDefault="qualified" targetNamespace="nsrt" xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="TrxnRecon">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="CreateDate" type="xs:string" />
<xs:element minOccurs="0" name="MgmtCode" type="xs:string" />
<xs:element minOccurs="0" name="DlrCode" type="xs:string" />
<xs:element minOccurs="0" name="TrxnRec">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="ProcessDate" type="xs:string" />
<xs:element minOccurs="0" name="FundAcctID" type="xs:string" />
<xs:element minOccurs="0" name="AcctDesig" type="xs:string" />
<xs:element minOccurs="0" name="DlrCode" type="xs:string" />
<xs:element minOccurs="0" name="DlrAcctID" type="xs:string" />
<xs:element minOccurs="0" name="RepCode" type="xs:string" />
<xs:element minOccurs="0" name="AcctType" type="xs:string" />
<xs:element minOccurs="0" name="OrdID" type="xs:string" />
<xs:element minOccurs="0" name="OrdSrc" type="xs:string" />
<xs:element minOccurs="0" name="OrdType" type="xs:string" />
<xs:element minOccurs="0" name="SrcID" type="xs:string" />
<xs:element minOccurs="0" name="BuyCof">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="TrxnTyp" type="xs:string" />
<xs:element minOccurs="0" name="BuyFund">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="FundID" type="xs:string" />
<xs:element minOccurs="0" name="Currency" type="xs:string" />
<xs:element minOccurs="0" name="DivOpt" type="xs:string" />
<xs:element minOccurs="0" name="SequenceNum" type="xs:string" />
<xs:element minOccurs="0" name="GrossAmt" type="xs:string" />
<xs:element minOccurs="0" name="FundPdComm" type="xs:string" />
<xs:element minOccurs="0" name="NetAmt" type="xs:string" />
<xs:element minOccurs="0" name="NAV" type="xs:string" />
<xs:element minOccurs="0" name="UnitTrxnd" type="xs:string" />
<xs:element minOccurs="0" name="TotalIssued" type="xs:string" />
<xs:element minOccurs="0" name="TotalUnissued" type="xs:string" />
<xs:element minOccurs="0" name="AveCost" type="xs:string" />
<xs:element minOccurs="0" name="SettlMethd" type="xs:string" />
<xs:element minOccurs="0" name="SettlAmt" type="xs:string" />
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element minOccurs="0" name="TradeDate" type="xs:string" />
<xs:element minOccurs="0" name="SettlDate" type="xs:string" />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
<xs:attribute name="Version" type="xs:string" use="optional" />
</xs:complexType>
</xs:element>
</xs:schema>
As you can see there are two records in the xml file, but I am getting only the first record in output.
<?xml version="1.0" encoding="UTF-8"?>
<TrxnRecon xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="nsrt" xsi:schemaLocation="nsrt Reconciliation.xsd" Version="24">
<CreateDate>20140521</CreateDate>
<MgmtCode>UN6</MgmtCode>
<DlrCode>9190</DlrCode>
<TrxnRec><ProcessDate>20140520</ProcessDate><FundAcctID>42069310</FundAcctID><AcctDesig>2</AcctDesig><DlrCode>9190</DlrCode><DlrAcctID>D140508150501</DlrAcctID><RepCode>AAB</RepCode><AcctType>01</AcctType><OrdID>2390886</OrdID><OrdSrc>D</OrdSrc><OrdType>W</OrdType><SrcID>NEFC1757635194</SrcID><BuyCof><TrxnTyp>5</TrxnTyp><BuyFund><FundID>3406</FundID><Currency>00</Currency><DivOpt>1</DivOpt><SequenceNum>27020000</SequenceNum><GrossAmt>10004.44</GrossAmt><FundPdComm>100.04</FundPdComm><NetAmt>10004.44</NetAmt><NAV>12.8453</NAV><UnitTrxnd>778.8410</UnitTrxnd><TotalIssued>0.0000</TotalIssued><TotalUnissued>3311.4950</TotalUnissued><AveCost>12.0805</AveCost><SettlMethd>1</SettlMethd><SettlAmt>9904.40</SettlAmt></BuyFund><TradeDate>20140515</TradeDate><SettlDate>20140520</SettlDate></BuyCof></TrxnRec>
<TrxnRec><ProcessDate>20140521</ProcessDate><FundAcctID>42169310</FundAcctID><AcctDesig>3</AcctDesig><DlrCode>9191</DlrCode><DlrAcctID>E140508150501</DlrAcctID><RepCode>AAC</RepCode><AcctType>02</AcctType><OrdID>2390887</OrdID><OrdSrc>E</OrdSrc><OrdType>Z</OrdType><SrcID>NEFC1757635195</SrcID><BuyCof><TrxnTyp>6</TrxnTyp><BuyFund><FundID>3407</FundID><Currency>01</Currency><DivOpt>2</DivOpt><SequenceNum>27020001</SequenceNum><GrossAmt>10005.44</GrossAmt><FundPdComm>100.05</FundPdComm><NetAmt>10005.44</NetAmt><NAV>13.8453</NAV><UnitTrxnd>779.8410</UnitTrxnd><TotalIssued>1.0000</TotalIssued><TotalUnissued>3312.4950</TotalUnissued><AveCost>13.0805</AveCost><SettlMethd>2</SettlMethd><SettlAmt>9905.40</SettlAmt></BuyFund><TradeDate>20140516</TradeDate><SettlDate>20140521</SettlDate></BuyCof></TrxnRec>
</TrxnRecon>
XSD File:
<?xml version="1.0"?>
<xs:schema xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" attributeFormDefault="unqualified" elementFormDefault="qualified" targetNamespace="nsrt" xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="TrxnRecon">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="CreateDate" type="xs:string" />
<xs:element minOccurs="0" name="MgmtCode" type="xs:string" />
<xs:element minOccurs="0" name="DlrCode" type="xs:string" />
<xs:element minOccurs="0" name="TrxnRec">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="ProcessDate" type="xs:string" />
<xs:element minOccurs="0" name="FundAcctID" type="xs:string" />
<xs:element minOccurs="0" name="AcctDesig" type="xs:string" />
<xs:element minOccurs="0" name="DlrCode" type="xs:string" />
<xs:element minOccurs="0" name="DlrAcctID" type="xs:string" />
<xs:element minOccurs="0" name="RepCode" type="xs:string" />
<xs:element minOccurs="0" name="AcctType" type="xs:string" />
<xs:element minOccurs="0" name="OrdID" type="xs:string" />
<xs:element minOccurs="0" name="OrdSrc" type="xs:string" />
<xs:element minOccurs="0" name="OrdType" type="xs:string" />
<xs:element minOccurs="0" name="SrcID" type="xs:string" />
<xs:element minOccurs="0" name="BuyCof">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="TrxnTyp" type="xs:string" />
<xs:element minOccurs="0" name="BuyFund">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="FundID" type="xs:string" />
<xs:element minOccurs="0" name="Currency" type="xs:string" />
<xs:element minOccurs="0" name="DivOpt" type="xs:string" />
<xs:element minOccurs="0" name="SequenceNum" type="xs:string" />
<xs:element minOccurs="0" name="GrossAmt" type="xs:string" />
<xs:element minOccurs="0" name="FundPdComm" type="xs:string" />
<xs:element minOccurs="0" name="NetAmt" type="xs:string" />
<xs:element minOccurs="0" name="NAV" type="xs:string" />
<xs:element minOccurs="0" name="UnitTrxnd" type="xs:string" />
<xs:element minOccurs="0" name="TotalIssued" type="xs:string" />
<xs:element minOccurs="0" name="TotalUnissued" type="xs:string" />
<xs:element minOccurs="0" name="AveCost" type="xs:string" />
<xs:element minOccurs="0" name="SettlMethd" type="xs:string" />
<xs:element minOccurs="0" name="SettlAmt" type="xs:string" />
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element minOccurs="0" name="TradeDate" type="xs:string" />
<xs:element minOccurs="0" name="SettlDate" type="xs:string" />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
<xs:attribute name="Version" type="xs:string" use="optional" />
</xs:complexType>
</xs:element>
</xs:schema>
As you can see there are two records in the xml file, but I am getting only the first record in output.
Thanks & Regards,
Partha
Partha
-
- Premium Member
- Posts: 47
- Joined: Tue Sep 13, 2005 4:28 am
Hi Partha,
As per my knowledge your XSD element TrxnRec should contain below statement.
<xs:element name="TrxnRec" maxOccurs="unbounded" minOccurs="0">
after adding that I am able to map Trxn array element from left hand to right in output mapping window . (I have not tested)
Hope might helpful...
As per my knowledge your XSD element TrxnRec should contain below statement.
<xs:element name="TrxnRec" maxOccurs="unbounded" minOccurs="0">
after adding that I am able to map Trxn array element from left hand to right in output mapping window . (I have not tested)
Hope might helpful...
Kiran Kumar
Good catch! MaxOccurs defaults to 1 if not specified. Your xsd is saying that multiple instances of that node aren't allowed.
...so you either have an invalid xml document, or a bad xsd.
Ernie
...so you either have an invalid xml document, or a bad xsd.
Ernie
Ernie Ostic
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>