DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
This topic has been marked "Resolved."
Author Message
flashgordon



Group memberships:
Premium Members

Joined: 17 Aug 2004
Posts: 99
Location: Boulder, Colorado
Points: 1188

Post Posted: Fri Sep 05, 2008 4:31 pm Reply with quote    Back to top    

DataStage® Release: 8x
Job Type: Parallel
OS: Unix
Additional info: SunOS machiname 5.10 Generic_125100-10 sun4u sparc SUNW,Sun-
Hi,

Just running a test routine to see if I can get C++ routines working on our Hawk install. Does anyone have a Sun c++ sample routine that works?

this was the c++ routine:
#include "string.h"
#include "stdio.h"
#include "stdlib.h"
// declares this as a integer routine
int xretstr(int argIN){
int xyz;
xyz = argIN +1;
return xyz; }


this was how the routine was compiled:
/xxxx/bin/CC -O -c xretint.cc -o xretint


this was the error message from the compile
Output from transformer compilation follows:

##I IIS-DSEE-TFCN-00001 17:27:35(000) <main_program>
IBM WebSphere DataStage Enterprise Edition 8.0.1.4665
Copyright (c) 2001, 2005-2007 IBM Corporation. All rights reserved



##I IIS-DSEE-TUTL-00031 17:27:35(001) <main_program> The open files limit is 100; raising to 65536.
##I IIS-DSEE-TOSH-00002 17:27:35(002) <main_program> orchgeneral: loaded
##I IIS-DSEE-TOSH-00002 17:27:35(003) <main_program> orchsort: loaded
##I IIS-DSEE-TOSH-00002 17:27:35(004) <main_program> orchstats: loaded
##W IIS-DSEE-TOSH-00049 17:27:35(007) <main_program> Parameter specified but not used in flow: DSPXWorkingDir
##E IIS-DSEE-TBLD-00076 17:27:41(000) <main_program> Error when checking composite operator: Subprocess command failed with exit status 256.
##E IIS-DSEE-TFSR-00019 17:27:41(001) <main_program> Could not check all operators because of previous error(s)
##W IIS-DSEE-TFTM-00012 17:27:41(002) <transform> Error when checking composite operator: The number of reject datasets "0" is less than the number of input datasets "1".
##W IIS-DSEE-TBLD-00000 17:27:41(003) <main_program> Error when checking composite operator: Output from subprocess: ld: fatal: file /opt/IBM/dsadm/xtest.o: open failed: No such file or directory
ld: fatal: File processing errors. No output written to /opt/IBM/data/projects/testinstall/RT_BP16.O/V0S1_testxxx7_Transformer_1.so

##I IIS-DSEE-TBLD-00079 17:27:41(004) <transform> Error when checking composite operator: /opt/SUNWspro/bin/CC -L/opt/IBM/data/projects/testinstall/RT_BP16.O/ -L/opt/IBM/InformationServer/Server/PXEngine/lib -L/opt/IBM/InformationServer/Server/PXEngine/user_lib -G -library=iostream -lorchsun4 -lorchcoresun4 -lorchbuildopsun4 /opt/IBM/dsadm/xtest.o /opt/IBM/data/projects/testinstall/RT_BP16.O/V0S1_testxxx7_Transformer_1.tmp.o -o /opt/IBM/data/projects/testinstall/RT_BP16.O/V0S1_testxxx7_Transformer_1.so.
##E IIS-DSEE-TCOS-00029 17:27:41(005) <main_program> Creation of a step finished with status = FAILED. (testxxx7.Transformer_1)

*** Internal Generated Transformer Code follows:
0001: //
0002: // Generated file to implement the V0S1_testxxx7_Transformer_1 transform operator.
0003: //
0004:
0005: // define external functions used
0006: extern int32 xretint(int32 argIN);
0007:
0008: // define our input/output link names
0009: inputname 0 DSLink3;
0010: outputname 0 DSLink4;
0011:
0012: initialize {
0013: // define our row rejected variable
0014: int8 RowRejected0;
0015:
0016: // define our null set variable
0017: int8 NullSetVar0;
0018:
0019: }
0020:
0021: mainloop {
0022: // initialise our row rejected variable
0023: RowRejected0 = 1;
0024:
0025: // evaluate columns (no constraints) for link: DSLink4
0026: DSLink4.x2 = xretint(DSLink3.x1);
0027: writerecord 0;
0028: RowRejected0 = 0;
0029: }
0030:
0031: finish {
0032: }
0033:
*** End of Internal Generated Transformer Code

_________________
Flash Gordon
Hyperborean Software Solution
flashgordon



Group memberships:
Premium Members

Joined: 17 Aug 2004
Posts: 99
Location: Boulder, Colorado
Points: 1188

Post Posted: Wed Sep 10, 2008 2:54 pm Reply with quote    Back to top    

Hi,

I figured this out myself. The problem was the "Library path:" field in the routine. "path" suggests to me they just want a directory name and then they are going to look for the "External subroutine name:" in the "Library path:". The IBM documentation clearly shows they want a fully qualified reference including the binary file name in "Libary path:". When I did that everything started working as advertising. For people looking for a working example on a current Sun box including the compile syntax, see below:

$ more xin.txt
0,"one"
1,"two"
2,"three"
3,"four"
$ more xtest.cc
#include "string.h"
#include "stdio.h"
#include "stdlib.h"
#include "string.h"
// declares this as a character routine
char * xtest(char *inString)
{
if (strstr(inString, "one"))
return "five";
else
if (strstr(inString, "two"))
return "six";
else
if (strstr(inString, "three"))
return "seven";
else
return "eight";
}
$ ls how*
howtocompileroutine.txt
$ more howtocompileroutine.txt
// the .o extension is not necessary, but this is how the compile must be done:
/opt/SUNWspro/bin/CC -O -c xtest.cc -o xtest.o
$ more xout.txt
"1","six"
"3","eight"
"0","five"
"2","seven"

_________________
Flash Gordon
Hyperborean Software Solution
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54582
Location: Sydney, Australia
Points: 295988

Post Posted: Wed Sep 10, 2008 3:25 pm Reply with quote    Back to top    

That's a good point. Yes, the "library path" is the pathname of the library object itself, in which the function is exposed. But it's an unfortunate choice of terminology because, as we both did, it ...

_________________
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Rate this response:  
Not yet rated
flashgordon



Group memberships:
Premium Members

Joined: 17 Aug 2004
Posts: 99
Location: Boulder, Colorado
Points: 1188

Post Posted: Thu Sep 11, 2008 11:01 am Reply with quote    Back to top    

Wanted to add one more thing to make this complete. IBM has published a documement "Extending DSEE +Wrapper.ppt" that has a brief section on c++ functions in it. The examples of how to code fields for the Routine in Datastage show what needs to be done from the Datastage side and is very good.

... Tom

_________________
Flash Gordon
Hyperborean Software Solution
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54582
Location: Sydney, Australia
Points: 295988

Post Posted: Thu Sep 11, 2008 4:00 pm Reply with quote    Back to top    

Care to share the URL? A search at ibm.com didn't find it.

_________________
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Rate this response:  
Not yet rated
flashgordon



Group memberships:
Premium Members

Joined: 17 Aug 2004
Posts: 99
Location: Boulder, Colorado
Points: 1188

Post Posted: Fri Sep 12, 2008 10:43 am Reply with quote    Back to top    

Ray,

I found out this came from the advanced ee class. The zip for the whole thing including some make files and other stuff is called: "Extending DSEE.zip". IBM probably wants you to take the class to get the material but those in good stead with their Datastage support/sales person can probably just get it.

I am going to do the cliff notes version of filling out the Routine portion in datastage below for the non-intuitive fill in boxes on the screen. You obviously have to be able to use c++ at the native Unix level to create a function.

There are 2 tabs you need to fille in for the Routine, General and Arguments.

For the General tab:

Routine name: is the name of the routine, can be anything, is what will show in the repository and how you call it in a transformer

Exernal subroutine name: I would make this what is before the open paren in the line below
char * xtest(char *inString)

I would call my binary the same thing

Type: Select "External Function"

Object type: Click "Object", warning "Object" is not the default

Library path: the fully qualified name of the binary, compiled routine including the name of the file itself. For my example it was:
/opt/IBM/dsadm/EEroutines/xtest

this shows what Solaris thinks xtest is:
$ file xtest
xtest: ELF 32-bit MSB relocatable SPARC32PLUS Version 1, V8+ Required

The binary file can or cannot have a .o extension. It doesn't seem to matter.

Argument Tab:

Argument name: for my example the value was "inString"

I/O type: is "I"

Native type: for my example was char*

Hope this helps. ... Flash

_________________
Flash Gordon
Hyperborean Software Solution
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54582
Location: Sydney, Australia
Points: 295988

Post Posted: Fri Sep 12, 2008 5:58 pm Reply with quote    Back to top    

Interesting. I teach DX436 but there is no such zip file in the materials. There are two routines (Keywords.c and Keywords2.c) in the solutions, but no zip file. Was it on the student CD or handed ...

_________________
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours