Creating ISD/REST service with pagination; data shaping

Dedicated to DataStage and DataStage TX editions featuring IBM<sup>®</sup> Service-Oriented Architectures.

Moderators: chulett, rschirm

Post Reply
rasampath
Premium Member
Premium Member
Posts: 2
Joined: Thu Nov 10, 2016 3:42 pm
Location: Toronto

Creating ISD/REST service with pagination; data shaping

Post by rasampath »

Hi all,

We've created a couple of simple REST 2.0 services via ISD to retrieve data from a database via a DataStage job and have got them working.

We're now looking to see how we can implement the following and I'm looking for guidance/recommended approaches for doing this:

1. Pagination - When the underlying query returns more than a certain number of rows (e.g. 100), how can the resultset be paginated. REST practices recommend including a Limit/Offset or a next/previous link; however I'm curious as to the approaches members have taken with ISD and how they've been implemented.

2. Data shaping - for wide tables, how can the fields returned be controlled based on the fields required by the service consumer? For example, I have a customer table with 100's of fields, however, based on the list of fields specified via a query string parameter (e.g. ?fields=custName, custID,custCity), I'd like to return only the fields specified.

Thanks in advance.
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

ISD Jobs are stateless; so there is no way to "buffer" or save rows for a particular user or session to get them on a subsequent call. You would have to code such logic yourself...perhaps saving rows in a table somewhere and then retrieving them again, in batches, for "that" user, or "that" request with logic that you would design yourself.

On the other.....do "whatever you need" in the DataStage Job. However...being able to specify your own columns means taking responsibility for the signature of the service. It's tricky to drop columns unless you take complete control of the payload and pass back a single "chunk" of information (perhaps comma delimited with one giant column on the output link to ISDOutput) or else take 100% responsibility for the payload and use the text over http binding instead.
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
rasampath
Premium Member
Premium Member
Posts: 2
Joined: Thu Nov 10, 2016 3:42 pm
Location: Toronto

Post by rasampath »

Thanks Ernie, for your response.

We're trying to steer the design away from these approaches primarily due to the complexity and customization associated with implementing the logic to achieve this.

We'll keep the options you've provided in mind when finalizing the design.

Thanks
Post Reply