Is Datastage compliant with Oracle RAC?

rleishman · Post by **rleishman** » Thu Jun 19, 2008 12:13 am

I thought this was going to be an easily confirmed thing, but I can't find anything on ibm.com, Google, or - surprise surprise - here.

I have used DS Enterprise in its simplest form - parallel processing on a single box, writing to a single Oracle instance.

I understand that it can be configured to run on a multi-node environment, with parallel processes farmed out to separate servers.

In this multi-node configuration, of course it would be possible for all nodes to write to the same Oracle instance, but it would create a bottleneck at the database. I imagine that Enterprise supports Oracle RAC, so that each node could write to a separate database instance. Is this true?

If so, can anyone direct me to some doco that explains the architecture.

chulett · Post by **chulett** » Thu Jun 19, 2008 5:51 am

Hmmm... I'm not sure anything like DataStage needs to explicitly support or be 'compliant' with Real Application Clusters, it should be transparent to it. There's still only one physical database, it's just that there's multiple logical instances of it running and Oracle is deciding who / what runs where and manages the 'inter-virutal-instance' (made that up) communications. But everything from the access side just sees one instance like 'normal'. You really don't have the ability to pick which 'virtual instance' you connect to, AFAIK.

Have you chatted with your DBA on this? It's been ages since I was physically involved/trained on RAC but we use it here and DataStage is none the wiser.

rleishman · Post by **rleishman** » Thu Jun 19, 2008 5:12 pm

Thanks Craig, that makes sense.

My DBA hasn't worked with RAC before - nor have I. I was making some round assumptions about how it would work based on a rudimentary knowledge of the architecture.

I'm not actually looking to do this - yet. I just want to be able to document a plausible upgrade path for scaleability.

I'll do some more reasearch on the RAC side rather than the DS side.

rleishman · Post by **rleishman** » Thu Jun 19, 2008 5:55 pm

OK. I think I have enough information for my purposes. It seems likely that Craig and I were both right. Why is that not surprising with Oracle?

In a RAC, I believe that the NORMAL operating procedure is for applications to nominate a single database instance in the RAC that it will use. This is done via a Virtual IP (VIP), which will failover to another node if that node is broken (high availability).

Under this model, each DS parallel process would need to pick a RAC node that it wanted to use. This was my original question - is it possible to configure parallel DS processes to pick a RAC node? But wait...

Oracle offers another approach called "Services", where you tell the database beforehand which applications run on which nodes - Oracle then does the switching for you. Since a parallel DS job is a single application, this might be a bit difficult to exploit. But wait...

There is another thing called Connection Load Balancing:

Oracle 10g doco wrote:Oracle Net Services provides connection load balancing for database connections. Connection load balancing occurs when the connection is created. Connections for a given service are balanced across all of the running instances that offer the service. You should define how you want connections to be balanced in the service definition. However, you must still configure Oracle Net Services. When you enable the load balancing advisory, the Listener uses the load balancing advisory for connection load balancing.

This sounds more like what Craig was talking about, and also sounds like the best solution for a parallel DS environment.

chulett · Post by **chulett** » Thu Jun 19, 2008 6:15 pm

Interesting... sounds like it has growed up quite a bit since last I looked at it. From what I recall, many of the connection options you have (like for failover) are specified in the tnsnames.ora file. Found this online for Oracle CM but it seems to do a nice job of walking you through a RAC setup. It includes Client and Listener load balancing and "TAF" or Transparent Application Failover. Cool stuff. :D

Have fun!

ps. You've probably already seen this but thought I would link to it for grins.

Post by **daignault** » Wed Jul 02, 2008 12:26 pm

The way Datastage EE operates within a DB2 environment (YMMV) is EE bypasses the co-ordinator node as it is familiar with the partitioning of a clustered DB2 instance.

By bypassing the co-ordinator items may be processed in parallell by each database instance in the cluster.

If it holds true in Oracle (and I believe that it does) then bypassing the co-ordinator node should invalidate Oracle RAC.

I've got a great document on setting up a DB2 Release 8 cluster on Linux if anyone is interested. Available to all premium members. Send me a mail message and I'll send the document out.

Ray Daignault

ray.wurlod · Post by **ray.wurlod** » Wed Jul 02, 2008 4:00 pm

It does not true in Oracle. That's why you need select privilege on certain Oracle system tables. So that partitioning information can be discovered at runtime.