JavaPack to request messages/logs from JMS/RIB

Dedicated to DataStage and DataStage TX editions featuring IBM<sup>®</sup> Service-Oriented Architectures.

Moderators: chulett, rschirm

Post Reply
peddidsx
Premium Member
Premium Member
Posts: 55
Joined: Wed Dec 26, 2007 8:20 am

JavaPack to request messages/logs from JMS/RIB

Post by peddidsx »

Hello,

We are trying to request and receive messages/xmllogs from the RIB (Retail Integration Bus) server using the JMS (Java Messaging Service) in real time. The only thing that we have available to us is the JavaPack, which came with our 8.x installation. We were able to replicate what the JMS does, by going through the shell scripts, grabbing the .jar files from JMS, putting them on our DS machine and executing from the command line on the DS server, and receiving the XML file to a particular folder on our server.

When we try using the command in the Java Client using the same command we get errors: java.lang.ClassNotFoundException

Here are my questions:
1. How do we pass in the parameters in the Java Client stage? If .jar file expects parameters, can they be passed in to the User's Classpath? And how?
2. If the command outputs multiple rows, how do we handle it in DS, using Java Client or Transformer? Examples, would be greatly appreciated.
3. Does the JMS client need to be installed on the DataStage server? Or can we bypass by using the existing jar programs which exist in JMS and DS?
4. Is there a particular .jar name that you can recommend, with which we can use JMS classes to pull the messages from JMS.
5. Do you have more .dsx examples which use the Java Client and Java Transformer? That would be greatly appreciated.
6. Also, any code examples using Initialize, Process, Terminate, would also be helpful.

Ernie, I have gone through your blog, which was very helpful. I have also gone through most of the posts on this site trying to answer these questions but I am still unclear.

Thanks in advance.
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

You have to "write" your own java class that reads from the JMS queue. The JavaPack doesn't just let you call "any" class/jar ....it is a bit more complex than that --- but ultimately, more powerful. Instead, it lets you directly interact with the DataStage "machinery". The Stage will give your class control when it needs a row.....you'll get the message from JMS, do whatever you need to with it inside of java, and then "write" that row to your output link (thus giving control back to DataStage).

Check out the samples I have on my blog...they are simple...and just pass data in and out...but they show the basics of what you need to do.

Ping me offline and I can also send you a JMS example...I just haven't ever had the time to put a blog entry together on it. If the JMS provider isn't picky about it's client environment (most are ok, but WebSphere JMS can be a bit problematic), you should find this whole exercise fairly simple, since you already have a java class on your own that you wrote.

...but play with the ExamineRows sample first...just to fully understand the mechanics of the JavaPack.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

...the other point I may have not made clear is that you have to use specific method names and use the javaApi that we provide with JavaPack...it contains the specific read() and write() methods that you will use to get or put data onto specific links within a DS Job.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
kirkboy
Participant
Posts: 6
Joined: Tue Aug 25, 2009 11:59 am

Post by kirkboy »

I have been working with Ron on this (peddidsx) and want to share some of what we figured out.

It was a mess figuring out how to make a java JMS client communicate with the RIB... what we found out was that oracle's AQ topics require a special connection factory--you cannot use the default JMS connection factory class to make this work.

In an java client connecting to an Oracle AQ topic (as used by RIB) you need to include these .jar:
jms.jar & j2ee.jar (unless you are creating a j2ee enterprise application, which has jms classes integrated)
aqapi.jar (this is the oracle aq library that provides oracles customized jms classes)
odbc6.jar (another custom oracle library, used for establishing connections without JNDI namespace, which was unavailable to our remote client)

And then when creating a Datastage Accessible version, we of course had to include tr4j.jar

Instead of using a normal connection Factory, we had to use AQjmsFactory class.

Code: Select all

        // Initialize connection to specified topic
        // Create Connection Factory using AQjmsFactory
        Properties jmsProp = new Properties();
        jmsProp.put("user", userName);
        jmsProp.put("password", password);
        
        info("Creating Connection Factory...");
        
        if (connFactory == null) {
            try{
                connFactory = AQjmsFactory.getTopicConnectionFactory(
                        providerUrl, jmsProp);                        
                        
                info("Created Connection Factory");
                
            } catch (Exception e){
                // report connection factory creation error
                warn("Error Creating Connection Factory: "+e.getMessage());
            }
        }
The providerUrl was a jdbc connection string, like
jdbc:oracle:thin:@erpndevdb1:1521:rmsdev1

That was the crux of it. Once we had the connection factory, JMS code worked as usual. In the datastage Java Client app we created the connection factory, connection, session, topic and subscriber all in the Initialize() method, then pulled messages from the Oracle AQ Topic in the process() method.

Pushing a message onto the topic has proven trickier. We have been able to do it in a stand alone java application, but when we integrate it into a Datastage Compliant java app it runs as expected and creates no errors, but the message never makes it to the topic. Still working on that one...
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

Yep.... agree with the all the points... Ultimately, you need all the pieces to "talk" to the JMS provider in question. So, at best, find someone who has already authored a connection to that system via Java..... getting it into DataStage is usually the easy part. The connection factory and JNI stuff can be nasty at times, depending on the provider of the JMS, it's security requirements, etc. All of that has nothing to do with DS.

As for targeting, it shouldn't be any different........but start small...send one little column into your java class via the JavaClient Stage input link from a transformer with the string "hello"...make sure you can get that into your message queue......and then go from there.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Post Reply