Sun C++ routine Orchestrate Error Message

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
flashgordon
Premium Member
Premium Member
Posts: 99
Joined: Tue Aug 17, 2004 7:50 am
Location: Boulder, Colorado

Sun C++ routine Orchestrate Error Message

Post by flashgordon »

Hi,

Just running a test routine to see if I can get C++ routines working on our Hawk install. Does anyone have a Sun c++ sample routine that works?

this was the c++ routine:
#include "string.h"
#include "stdio.h"
#include "stdlib.h"
// declares this as a integer routine
int xretstr(int argIN){
int xyz;
xyz = argIN +1;
return xyz; }


this was how the routine was compiled:
/xxxx/bin/CC -O -c xretint.cc -o xretint


this was the error message from the compile
Output from transformer compilation follows:

##I IIS-DSEE-TFCN-00001 17:27:35(000) <main_program>
IBM WebSphere DataStage Enterprise Edition 8.0.1.4665
Copyright (c) 2001, 2005-2007 IBM Corporation. All rights reserved



##I IIS-DSEE-TUTL-00031 17:27:35(001) <main_program> The open files limit is 100; raising to 65536.
##I IIS-DSEE-TOSH-00002 17:27:35(002) <main_program> orchgeneral: loaded
##I IIS-DSEE-TOSH-00002 17:27:35(003) <main_program> orchsort: loaded
##I IIS-DSEE-TOSH-00002 17:27:35(004) <main_program> orchstats: loaded
##W IIS-DSEE-TOSH-00049 17:27:35(007) <main_program> Parameter specified but not used in flow: DSPXWorkingDir
##E IIS-DSEE-TBLD-00076 17:27:41(000) <main_program> Error when checking composite operator: Subprocess command failed with exit status 256.
##E IIS-DSEE-TFSR-00019 17:27:41(001) <main_program> Could not check all operators because of previous error(s)
##W IIS-DSEE-TFTM-00012 17:27:41(002) <transform> Error when checking composite operator: The number of reject datasets "0" is less than the number of input datasets "1".
##W IIS-DSEE-TBLD-00000 17:27:41(003) <main_program> Error when checking composite operator: Output from subprocess: ld: fatal: file /opt/IBM/dsadm/xtest.o: open failed: No such file or directory
ld: fatal: File processing errors. No output written to /opt/IBM/data/projects/testinstall/RT_BP16.O/V0S1_testxxx7_Transformer_1.so

##I IIS-DSEE-TBLD-00079 17:27:41(004) <transform> Error when checking composite operator: /opt/SUNWspro/bin/CC -L/opt/IBM/data/projects/testinstall/RT_BP16.O/ -L/opt/IBM/InformationServer/Server/PXEngine/lib -L/opt/IBM/InformationServer/Server/PXEngine/user_lib -G -library=iostream -lorchsun4 -lorchcoresun4 -lorchbuildopsun4 /opt/IBM/dsadm/xtest.o /opt/IBM/data/projects/testinstall/RT_BP16.O/V0S1_testxxx7_Transformer_1.tmp.o -o /opt/IBM/data/projects/testinstall/RT_BP16.O/V0S1_testxxx7_Transformer_1.so.
##E IIS-DSEE-TCOS-00029 17:27:41(005) <main_program> Creation of a step finished with status = FAILED. (testxxx7.Transformer_1)

*** Internal Generated Transformer Code follows:
0001: //
0002: // Generated file to implement the V0S1_testxxx7_Transformer_1 transform operator.
0003: //
0004:
0005: // define external functions used
0006: extern int32 xretint(int32 argIN);
0007:
0008: // define our input/output link names
0009: inputname 0 DSLink3;
0010: outputname 0 DSLink4;
0011:
0012: initialize {
0013: // define our row rejected variable
0014: int8 RowRejected0;
0015:
0016: // define our null set variable
0017: int8 NullSetVar0;
0018:
0019: }
0020:
0021: mainloop {
0022: // initialise our row rejected variable
0023: RowRejected0 = 1;
0024:
0025: // evaluate columns (no constraints) for link: DSLink4
0026: DSLink4.x2 = xretint(DSLink3.x1);
0027: writerecord 0;
0028: RowRejected0 = 0;
0029: }
0030:
0031: finish {
0032: }
0033:
*** End of Internal Generated Transformer Code
Flash Gordon
Hyperborean Software Solution
flashgordon
Premium Member
Premium Member
Posts: 99
Joined: Tue Aug 17, 2004 7:50 am
Location: Boulder, Colorado

Post by flashgordon »

Hi,

I figured this out myself. The problem was the "Library path:" field in the routine. "path" suggests to me they just want a directory name and then they are going to look for the "External subroutine name:" in the "Library path:". The IBM documentation clearly shows they want a fully qualified reference including the binary file name in "Libary path:". When I did that everything started working as advertising. For people looking for a working example on a current Sun box including the compile syntax, see below:

$ more xin.txt
0,"one"
1,"two"
2,"three"
3,"four"
$ more xtest.cc
#include "string.h"
#include "stdio.h"
#include "stdlib.h"
#include "string.h"
// declares this as a character routine
char * xtest(char *inString)
{
if (strstr(inString, "one"))
return "five";
else
if (strstr(inString, "two"))
return "six";
else
if (strstr(inString, "three"))
return "seven";
else
return "eight";
}
$ ls how*
howtocompileroutine.txt
$ more howtocompileroutine.txt
// the .o extension is not necessary, but this is how the compile must be done:
/opt/SUNWspro/bin/CC -O -c xtest.cc -o xtest.o
$ more xout.txt
"1","six"
"3","eight"
"0","five"
"2","seven"
Flash Gordon
Hyperborean Software Solution
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

That's a good point. Yes, the "library path" is the pathname of the library object itself, in which the function is exposed. But it's an unfortunate choice of terminology because, as we both did, it can be so readily confused with the name of the environment variable that specifies the shared library search list.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
flashgordon
Premium Member
Premium Member
Posts: 99
Joined: Tue Aug 17, 2004 7:50 am
Location: Boulder, Colorado

Post by flashgordon »

Wanted to add one more thing to make this complete. IBM has published a documement "Extending DSEE +Wrapper.ppt" that has a brief section on c++ functions in it. The examples of how to code fields for the Routine in Datastage show what needs to be done from the Datastage side and is very good.

... Tom
Flash Gordon
Hyperborean Software Solution
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Care to share the URL? A search at ibm.com didn't find it.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
flashgordon
Premium Member
Premium Member
Posts: 99
Joined: Tue Aug 17, 2004 7:50 am
Location: Boulder, Colorado

Post by flashgordon »

Ray,

I found out this came from the advanced ee class. The zip for the whole thing including some make files and other stuff is called: "Extending DSEE.zip". IBM probably wants you to take the class to get the material but those in good stead with their Datastage support/sales person can probably just get it.

I am going to do the cliff notes version of filling out the Routine portion in datastage below for the non-intuitive fill in boxes on the screen. You obviously have to be able to use c++ at the native Unix level to create a function.

There are 2 tabs you need to fille in for the Routine, General and Arguments.

For the General tab:

Routine name: is the name of the routine, can be anything, is what will show in the repository and how you call it in a transformer

Exernal subroutine name: I would make this what is before the open paren in the line below
char * xtest(char *inString)

I would call my binary the same thing

Type: Select "External Function"

Object type: Click "Object", warning "Object" is not the default

Library path: the fully qualified name of the binary, compiled routine including the name of the file itself. For my example it was:
/opt/IBM/dsadm/EEroutines/xtest

this shows what Solaris thinks xtest is:
$ file xtest
xtest: ELF 32-bit MSB relocatable SPARC32PLUS Version 1, V8+ Required

The binary file can or cannot have a .o extension. It doesn't seem to matter.

Argument Tab:

Argument name: for my example the value was "inString"

I/O type: is "I"

Native type: for my example was char*

Hope this helps. ... Flash
Flash Gordon
Hyperborean Software Solution
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Interesting. I teach DX436 but there is no such zip file in the materials. There are two routines (Keywords.c and Keywords2.c) in the solutions, but no zip file. Was it on the student CD or handed out by the instructor?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply