Hi all,
I ran into problem of not able to process all records in one go, I am trying to process a large ResultSet, I am also using setFetchSize parameter for tuning purposes. However I want to know if a single process() call can handle the complete set of input records.As per the docs, its Yes, however I couldn't configure it such way that it handles complete set of input records.
I tried using while loop on ResultSet with no luck to process all the records in one process() method call. process() method is getting invoked again and again even if it is the only method in the class.
Please let me know if anyone did anything similar.
Thanks & Regards
Koti
A single process() call can handle the complete set of input
Moderators: chulett, rschirm, roy
A couple of things....
1. Lets change your subject, if possible, to mention the java integration stage...a lot of folks wont know that this is what you are ultimately asking abut.
2. Tell us more about what you are doing......number of input links, number of output links, goal of your class, overall description of the task....
3. More detail about your goal and why it is important......it may be fine and very do-able.....we just need more context in order to provide thoughts and ideas.
Ernie
1. Lets change your subject, if possible, to mention the java integration stage...a lot of folks wont know that this is what you are ultimately asking abut.
2. Tell us more about what you are doing......number of input links, number of output links, goal of your class, overall description of the task....
3. More detail about your goal and why it is important......it may be fine and very do-able.....we just need more context in order to provide thoughts and ideas.
Ernie
Ernie Ostic
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Thanks Craig.....brings up an interesting question --- exactly what release are we talking about, and is this the Java Integration Stage, or its predecessor, an implementation of the Java Client or Java Transformer Stage.
Ernie
Ernie
Ernie Ostic
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Java Client Stage, Datastage 8.5, JDBC, JDK1.6
Hi Craig,
Thanks for changing the subject.
Hi Ernie,
I am trying to use java client stage as source and load dataset.
My class file is compiling fine and able to call it successfully without any problem.I am using setFetchSize through get more data for ResultSet interface for tuning purposes in my Initialize Method. Please find the skeleton of my code and what i want to achieve. I want to add while loop in process() method to loop through all records in one shot instead of just calling resultset.next() at the begining of process() method which is also working but record by record.
Thanks for changing the subject.
Hi Ernie,
I am trying to use java client stage as source and load dataset.
My class file is compiling fine and able to call it successfully without any problem.I am using setFetchSize through get more data for ResultSet interface for tuning purposes in my Initialize Method. Please find the skeleton of my code and what i want to achieve. I want to add while loop in process() method to loop through all records in one shot instead of just calling resultset.next() at the begining of process() method which is also working but record by record.
Code: Select all
public class Mytransformer extends Stage
{
public void initialize()
{
//Intialized the JDBC Connection
statement.setFetchSize(1000); //My Result set is pretty big
}
public int process()
{
while(resultset.next()) { //<--
Row outputRow = createOutputRow();
//Looping through List of Columns using metadata using for loop then
outputRow.setValueAsString(columnNumber,resultset.getString(columnNumber+1));
writeRow(outputRow);
return OUTPUT_STATUS_READY;
} //<--
}
}
No worries.
Note that I added
Note that I added
Code: Select all
tags to your post since they preserve whitespace and thus formatting but they don't allow bold or italics so swapped in a couple of arrows to mark the loop. No charge. :wink:
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers