Page 1 of 1

My real time job does not return the nested JSON format data

Posted: Tue May 17, 2016 12:19 pm
by yliu@mitre.org
Hi,
1) my real time job design is "IDS input -> lookup stage with sparse type -> XML stage -> ISD output'
Sample Output result:
[{"name":"Econ Analysis & Acquisition Mg","specialties":["Risk Management","Business and Investment Analysis","Contracting Analysis","Program Management [Acquisition]","Cost Analysis","Economic Analysis","Financial Management Analysis","Economic Modeling","Econometrics","Benefit Cost Analysis","Health Economics"]}]

After deploying in ISD console, I run it in the browser. The first time, I got the result as the same as the above. BUT when I run it in the Browser the second time, I got the wrong data like this:
[{"name":"Econ Analysis & Acquisition Mg"},{"name":"SW Eng, Comp Sci & Mathematics"}]

Can anyone know why?

2) If I make it as a batch job, I do not have such issue. But batch job is slower than the real time job.

Thank you in advance!
Jane

Posted: Thu Aug 11, 2016 6:59 am
by atul9806
What is your output array properties ?

Posted: Thu Aug 11, 2016 11:14 am
by eostic
Hard to say. Could be lots of things. For initial debugging, I would have you put a transformer after the xml stage, and split the output into two places....first to a sequential file (use "NONE" for delimter and for quote character) and then the other link to ISDoutput.

[you can make that change fairly simply, by "disabling" the service, change and recompile the Job and then "re-enable". You don't have to entirely un-deploy, provided that your output link to ISDoutput remains the same in terms of columns and structure].

Then see what goes to the sequential file. Hard to say what this would be, based on your description, but it looks like for some reason your sparse lookup is failing. Be sure it is truly sparse so that it is doing SQL each time, per row/per request. This is a requirement of lookups in ISD jobs (and no funnels or other complex scnearios that require initial reading of a 'file' as I/O --- ONLY the ISDinput link can be the DRIVER of the job).

Ernie