Decoding a URL Encoded POST String

pbttbis · Posted: Fri Jan 08, 2016 3:23 am

Hi,

Is there any functionality in Datastage to allow me to decode an url encoded post string such as the below:

calcData=%3cMotorInsuranceRequest+xmlns%3d%22http%3a%2f%2fwww.insurethebox.com%2fMotorInsuranceRequest.xsd%22+xmlns%3axsi%3d%22http%3a%2f%2fwww.w3.org%2f2001%2fXMLSchema-instance%22+xmlns%3axsd%3d%22http%3a%2f%2fwww.w3.org%2f2001%2fXMLSchema%22+xmlns%3aemp%3d%22http%3a%2f%2fwww.insurethebox.com%2fEmployment.xsd%22+xmlns%3adrv%3d%22http%3a%2f%2fwww.insurethebox.com%2fDriver.xsd%22+xmlns%3aclm%3d%22http%3a%2f%2fwww.insurethebox.com%2fClaims.xsd%22+xmlns%3aconv%3d%22http%3a%2f%2fwww.insurethebox.com%2fConvictions.xsd%22+xmlns%3amad%3d%22http%3a%2f%2fwww.insurethebox.com%2fMainAdditionalDriver.xsd%22+xmlns%3aveh%3d%22http%3a%2f%2fwww.insurethebox.com%2fVehicleTypes.xsd%22+xmlns%3apol%3d%22http%3a%2f%2fwww.insurethebox.com%2fPolicy.xsd%22%3e%0d%0a++%3cProposer%3e%0d%0a++++%3cemp%3aEmploymentTitle%3e824%3c%2femp%3aEmploymentTitle%3e%0d%0a++++%3cdrv%3aHasClaim%3eNo%3c%2fdrv%3aHasClaim%3e%0d%0a++++%3cdrv%3aHasConviction%3eNo%3c%2fdrv%3aHasConviction%3e%0d%0a++++%3cdrv%3aGender%3eM%3c%2fdrv%3aGender%3e%0d%0a++++%3cdrv%3aMaritalStatus%3e1%3c%2fdrv%3aMaritalStatus%3e%0d%0a++++%3cdrv%3aLicenceType%3e1%3c%2fdrv%3aLicenceType%3e%0d%0a++++%3cdrv%3aDrivingQualifications%3e3%3c%2fdrv%3aDrivingQualifications%3e%0d%0a++++%3cdrv%3aTitle%3eMr%3c%2fdrv%3aTitle%3e%0d%0a++++%3cdrv%3aForeName%3eShaun%3c%2fdrv%3aForeName%3e%0d%0a++++%3cdrv%3aSurname%3etest%3c%2fdrv%3aSurname%3e%0d%0a++++%3cdrv%3aDateOfBirth%3e02%2f04%2f1993%3c%2fdrv%3aDateOfBirth%3e%0d%0a++++%3cdrv%3aIsMainDriver%3etrue%3c%2fdrv%3aIsMainDriver%3e%0d%0a++++%3cdrv%3aLicenceHeldSinceMonths%3e01%3c%2fdrv%3aLicenceHeldSinceMonths%3e%0d%0a++++%3cdrv%3aLicenceHeldSinceYears%3e2012%3c%2fdrv%3aLicenceHeldSinceYears%3e%0d%0a++++%3cdrv%3aMedicalConditions%3e0%3c%2fdrv%3aMedicalConditions%3e%0d%0a++++%3cdrv%3aDriveOtherVehicles%3eNo%3c%2fdrv%3aDriveOtherVehicles%3e%0d%0a++++%3cdrv%3aResidentSinceMonths%3e04%3c%2fdrv%3aResidentSinceMonths%3e%0d%0a++++%3cdrv%3aResidentSinceYears%3e1993%3c%2fdrv%3aResidentSinceYears%3e%0d%0a++++%3cdrv%3aHasCriminalConviction%3eNo%3c%2fdrv%3aHasCriminalConviction%3e%0d%0a++++%3cmad%3aAddress%3e%0d%0a++++++%3cmad%3aHouseNumberName%3e57%3c%2fmad%3aHouseNumberName%3e%0d%0a++++++%3cmad%3aAddressLineOne%3eWhite+Swan+Close%3c%2fmad%3aAddressLineOne%3e%0d%0a++++++%3cmad%3aAddressLineTwo%3eKillingworth%3c%2fmad%3aAddressLineTwo%3e%0d%0a++++++%3cmad%3aAddressLineThree%3eNewcastle+upon+Tyne%3c%2fmad%3aAddressLineThree%3e%0d%0a++++++%3cmad%3aAddressLineFour%3e%3c%2fmad%3aAddressLineFour%3e%0d%0a++++++%3cmad%3aPostalCode%3eNE12+6UG%3c%2fmad%3aPostalCode%3e%0d%0a++++%3c%2fmad%3aAddress%3e%0d%0a++++%3cmad%3aMobilePhone%3e07414+545854%3c%2fmad%3aMobilePhone%3e%0d%0a++++%3cmad%3aEmailAddress%3eamc080120161%40bigggg.com%3c%2fmad%3aEmailAddress%3e%0d%0a++++%3cmad%3aPreferredPaymentMethod%3eAnnually%3c%2fmad%3aPreferredPaymentMethod%3e%0d%0a++++%3cmad%3aNoClaimDiscountBonus%3e1%3c%2fmad%3aNoClaimDiscountBonus%3e%0d%0a++++%3cmad%3aDeclinedMotorInsurance%3eNo%3c%2fmad%3aDeclinedMotorInsurance%3e%0d%0a++++%3cmad%3aHomeOwner%3eF%3c%2fmad%3aHomeOwner%3e%0d%0a++%3c%2fProposer%3e%0d%0a++%3cAdditionalDriverCollection+%2f%3e%0d%0a++%3cVehicle%3e%0d%0a++++%3cveh%3aAnnualMileage%3e8000%3c%2fveh%3aAnnualMileage%3e%0d%0a++++%3cveh%3aIsVehicleModified%3eNo%3c%2fveh%3aIsVehicleModified%3e%0d%0a++++%3cveh%3aVehicleNightLocation%3e2%3c%2fveh%3aVehicleNightLocation%3e%0d%0a++++%3cveh%3aVehicleKey%3e17553502%3c%2fveh%3aVehicleKey%3e%0d%0a++++%3cveh%3aVehicleRegistration%3eNH08UHR%3c%2fveh%3aVehicleRegistration%3e%0d%0a++++%3cveh%3aSeats%3e5%3c%2fveh%3aSeats%3e%0d%0a++++%3cveh%3aHasTracker%3eNo%3c%2fveh%3aHasTracker%3e%0d%0a++++%3cveh%3aVehicleDrivingPosition%3e2%3c%2fveh%3aVehicleDrivingPosition%3e%0d%0a++++%3cveh%3aVehicleSecurityDevice%3e5%3c%2fveh%3aVehicleSecurityDevice%3e%0d%0a++++%3cveh%3aYearOfManufacture%3e2008%3c%2fveh%3aYearOfManufacture%3e%0d%0a++++%3cveh%3aVehicleDayLocation%3e7%3c%2fveh%3aVehicleDayLocation%3e%0d%0a++++%3cveh%3aBusinessMileage%3e0%3c%2fveh%3aBusinessMileage%3e%0d%0a++++%3cveh%3aVehicleImported%3eNo%3c%2fveh%3aVehicleImported%3e%0d%0a++++%3cveh%3aVehicleMarketValue%3e4045%3c%2fveh%3aVehicleMarketValue%3e%0d%0a++%3c%2fVehicle%3e%0d%0a++%3cPolicyDetail%3e%0d%0a++++%3cpol%3aIncludeProtectedNoClaimsBonus%3eF%3c%2fpol%3aIncludeProtectedNoClaimsBonus%3e%0d%0a++++%3cpol%3aPolicyStartDate%3e08%2f01%2f2016%3c%2fpol%3aPolicyStartDate%3e%0d%0a++++%3cpol%3aRegisteredKeeper%3e0%3c%2fpol%3aRegisteredKeeper%3e%0d%0a++++%3cpol%3aRegisteredOwner%3e0%3c%2fpol%3aRegisteredOwner%3e%0d%0a++++%3cpol%3aVehiclePurchasedDate%3e01%2f05%2f2015%3c%2fpol%3aVehiclePurchasedDate%3e%0d%0a++++%3cpol%3aPolicyUsage%3e1%3c%2fpol%3aPolicyUsage%3e%0d%0a++++%3cpol%3aCoverLevel%3e01%3c%2fpol%3aCoverLevel%3e%0d%0a++++%3cpol%3aVoluntaryExcess%3e2%3c%2fpol%3aVoluntaryExcess%3e%0d%0a++%3c%2fPolicyDetail%3e%0d%0a%3c%2fMotorInsuranceRequest%3e&calcType=Full+Premium+Calc+TBB&transform=%2fTransforms%2fctmTransformTBB&calcResponse=%2fresponses%2fctmResponseTBB&source=ctm&test=test

Thanks,

Shaun

eostic · Posted: Fri Jan 08, 2016 8:47 am

I haven't seen anyone do it, but it shouldn't be that difficult, provided you know all the different "translations" for the various escape sequences that are in here. I am not familiar with this, but ...

What's this?

pbttbis · Posted: Fri Jan 08, 2016 9:54 am

well there are 224 replacement pairs I believe based on http://www.w3schools.com/tags/ref_urlencode.asp

Some options we were considering:

1) Write a routine
2) Use the JAVA stage and make use of String result = java.net.URLDecoder.decode(url, "UTF-8");
3) Perhaps using the transformer looping through the pairs and doing the replacement with ereplace

Currently we have a working job that does the decode using the External Filter step and the following perl script:

perl -pi.back -e 'y/+/ /;s/%([\\da-f]{2})/pack H2,$1/gie;s/[\r\n]//g'