DSXchange

Posted: **Wed Mar 15, 2017 8:30 am**

Hi all,

I ran into problem of not able to process all records in one go, I am trying to process a large ResultSet, I am also using setFetchSize parameter for tuning purposes. However I want to know if a single process() call can handle the complete set of input records.As per the docs, its Yes, however I couldn't configure it such way that it handles complete set of input records.

I tried using while loop on ResultSet with no luck to process all the records in one process() method call. process() method is getting invoked again and again even if it is the only method in the class.

Please let me know if anyone did anything similar.

Thanks & Regards
Koti

Posted: **Thu Mar 16, 2017 1:37 pm**

A couple of things....

1. Lets change your subject, if possible, to mention the java integration stage...a lot of folks wont know that this is what you are ultimately asking abut.

2. Tell us more about what you are doing......number of input links, number of output links, goal of your class, overall description of the task....

3. More detail about your goal and why it is important......it may be fine and very do-able.....we just need more context in order to provide thoughts and ideas.

Ernie

Posted: **Thu Mar 16, 2017 3:22 pm**

Re: #1... added to the Additional Info area.

Posted: **Thu Mar 16, 2017 7:50 pm**

Thanks Craig.....brings up an interesting question --- exactly what release are we talking about, and is this the Java Integration Stage, or its predecessor, an implementation of the Java Client or Java Transformer Stage.

Ernie

Posted: **Fri Mar 17, 2017 6:13 pm**

Hi Craig,

Thanks for changing the subject.

Hi Ernie,

I am trying to use java client stage as source and load dataset.
My class file is compiling fine and able to call it successfully without any problem.I am using setFetchSize through get more data for ResultSet interface for tuning purposes in my Initialize Method. Please find the skeleton of my code and what i want to achieve. I want to add while loop in process() method to loop through all records in one shot instead of just calling resultset.next() at the begining of process() method which is also working but record by record.

Code: Select all

public class Mytransformer extends Stage
{
   public void initialize()
      {
         //Intialized the JDBC Connection
	statement.setFetchSize(1000); //My Result set is pretty big
      }
   public int process()
      {
         while(resultset.next()) {                             //<--
         Row outputRow = createOutputRow();
	 //Looping through List of Columns using metadata using for loop then
	 outputRow.setValueAsString(columnNumber,resultset.getString(columnNumber+1));	 
         writeRow(outputRow);
         return OUTPUT_STATUS_READY;
         }                                                     //<--
      }
}

Posted: **Fri Mar 17, 2017 6:24 pm**

No worries.

Note that I added

Code: Select all

 tags to your post since they preserve whitespace and thus formatting but they don't allow bold or italics so swapped in a couple of arrows to mark the loop. No charge. :wink:

DSXchange

A single process() call can handle the complete set of input

A single process() call can handle the complete set of input

Java Client Stage, Datastage 8.5, JDBC, JDK1.6