Parallel job compilation error with a transformer in the job

Alethesnake · Post by **Alethesnake** » Mon Jul 20, 2009 2:50 am

Hi All,
I read a lot of things about this problem in the forum but I was not able to solve the problem by myself.

This is the error I get (from the more button after an unsuccessful compilation):

Code: Select all

Output from transformer compilation follows:

##I IIS-DSEE-TFCN-00001 10:25:16(000) <main_program> 
IBM WebSphere DataStage Enterprise Edition 8.1.0.5040 
Copyright (c) 2001, 2005-2008 IBM Corporation. All rights reserved
 


##I IIS-DSEE-TFCN-00006 10:25:16(001) <main_program> conductor uname: -s=AIX; -r=3; -v=5; -n=svil; -m=00C87F7F4C00
##I IIS-DSEE-TOSH-00002 10:25:16(002) <main_program> orchgeneral: loaded
##I IIS-DSEE-TOSH-00002 10:25:16(003) <main_program> orchsort: loaded
##I IIS-DSEE-TOSH-00002 10:25:16(004) <main_program> orchstats: loaded
##W IIS-DSEE-TOSH-00049 10:25:17(000) <main_program> Parameter specified but not used in flow: DSPXWorkingDir
##E IIS-DSEE-TBLD-00076 10:25:20(000) <main_program> Error when checking composite operator: Subprocess command failed with exit status 256.
##E IIS-DSEE-TFSR-00019 10:25:20(001) <main_program> Could not check all operators because of previous error(s)
##W IIS-DSEE-TFTM-00012 10:25:20(002) <transform> Error when checking composite operator: The number of reject datasets "0" is less than the number of input datasets "1".
##W IIS-DSEE-TBLD-00000 10:25:20(003) <main_program> Error when checking composite operator: Output from subprocess: munch: The input file /sw/IBM/InformationServer/Server/PXEngine/lib/liborchbuildopaix64.so is not valid in the current object mode.

##I IIS-DSEE-TBLD-00079 10:25:20(004) <transform> Error when checking composite operator: /usr/vacpp/bin/xlC_r    -L/sw/IBM/InformationServer/Server/Projects/DATAWAREHOUSE_CEDACRI/RT_BP255.O/ -L/sw/IBM/InformationServer/Server/PXEngine/lib -L/sw/IBM/InformationServer/Server/PXEngine/user_lib -G -lorchaix64 -lorchcoreaix64 -lorchbuildopaix64 /sw/IBM/InformationServer/Server/Projects/DATAWAREHOUSE_CEDACRI/RT_BP255.O/V0S0_PAR_001_EnvVariableInTransformer_Transformer_0.tmp.o -o /sw/IBM/InformationServer/Server/Projects/DATAWAREHOUSE_CEDACRI/RT_BP255.O/V0S0_PAR_001_EnvVariableInTransformer_Transformer_0.o.
##E IIS-DSEE-TCOS-00029 10:25:20(005) <main_program> Creation of a step finished with status = FAILED. (PAR_001_EnvVariableInTransformer.Transformer_0)

*** Internal Generated Transformer Code follows:
0001: //
0002: // Generated file to implement the V0S0_PAR_001_EnvVariableInTransformer_Transformer_0 transform operator.
0003: //
0004: 
0005: // define our input/output link names
0006: inputname 0 DSLink3;
0007: outputname 0 DSLink4;
0008: 
0009: initialize {
0010: 	// define our row rejected variable
0011: 	int8 RowRejected0;
0012: 
0013: 	// define our null set variable
0014: 	int8 NullSetVar0;
0015: 
0016: }
0017: 
0018: mainloop {
0019: 	// initialise our row rejected variable
0020: 	RowRejected0 = 1;
0021: 
0022: 	// evaluate columns (no constraints) for link: DSLink4
0023: 	writerecord 0;
0024: 	RowRejected0 = 0;
0025: }
0026: 
0027: finish {
0028: }
0029: 
*** End of Internal Generated Transformer Code

DS Information Server 8.1 is installed on an AIX 5.3 server.

lslpp -L | grep -i xlc gives:

Code: Select all

  xlC.adt.include            8.0.0.0    C     F    C Set ++ Application
  xlC.aix50.rte             10.1.0.0    C     F    XL C/C++ Runtime for AIX 5.3
  xlC.cpp                    9.0.0.0    C     F    C for AIX Preprocessor
  xlC.msg.en_US.cpp          9.0.0.0    C     F    C for AIX Preprocessor
  xlC.msg.en_US.rte         10.1.0.0    C     F    XL C/C++ Runtime
  xlC.rte                   10.1.0.0    C     F    XL C/C++ Runtime

and from the ibm web site the compiler versions availables are:

Code: Select all

- May 2008 XL C/C++ Enterprise Edition V8.0 for AIX PTF, Reference #4019338
- August 2008 XL C/C++ Enterprise Edition V9.0 for AIX, Reference #4020144 
- October 2008 XL C/C++ Enterprise Edition V10.1 for AIX, Reference #4021392

APT_LINKER and APT_COMPILER are set to /usr/vacpp/bin/xlC_r

If I execute /usr/vacpp/bin/xlC_r from the command line the compiler answers with its information page.
Compiler options are also set correctly.

Any suggestion on how can I solve the problem?

Thanks,

Ale.

ArndW · Post by **ArndW** » Mon Jul 20, 2009 3:19 am

Your APT_COMPILEOPT should be "-O -c" and APT_LINKOPT should be "-G". Can you execute the compile from the command line? I also see that the error refers to "munch" instead of directly to the compiler.

Alethesnake · Post by **Alethesnake** » Mon Jul 20, 2009 3:36 am

Hi ArndW,
Compiler options are those you mentioned.

If I execute this from command line:

Code: Select all

/usr/vacpp/bin/xlC_r -L/sw/IBM/InformationServer/Server/Projects/DATAWAREHOUSE_CEDACRI/RT_BP255.O/ -L/sw/IBM/InformationServer/Server/PXEngine/lib -L/sw/IBM/InformationServer/Server/PXEngine/user_lib -G -lorchaix64 -lorchcoreaix64 -lorchbuildopaix64 /sw/IBM/InformationServer/Server/Projects/DATAWAREHOUSE_CEDACRI/RT_BP255.O/V0S0_PAR_001_EnvVariableInTransformer_Transformer_0.tmp.o -o /sw/IBM/InformationServer/Server/Projects/DATAWAREHOUSE_CEDACRI/RT_BP255.O/V0S0_PAR_001_EnvVariableInTransformer_Transformer_0.o

I get an error:

/usr/vacpp/bin/xlC_r: 1501-228 input file /sw/IBM/InformationServer/Server/Projects/DATAWAREHOUSE_CEDACRI/RT_BP255.O/V0S0_PAR_001_EnvVariableInTransformer_Transformer_0.tmp.o not found

executing:

Code: Select all

dump -ohv /sw/IBM/InformationServer/Server/PXEngine/lib/liborchbuildopaix64.so

I get the same error mentioned in the report above:

dump: /sw/IBM/InformationServer/Server/PXEngine/lib/liborchbuildopaix64.so: 0654-108 file is not valid in the current object file mode.
Use the -X option to specify the desired object mode.

ArndW · Post by **ArndW** » Mon Jul 20, 2009 4:50 am

Hmmm... that object should be a 32 bit one, in my opinion. I am working on 8.0.1 so can't answer this problem, which looks like a 8.1 specific one. On my system I only have "liborchbuildopaix3.so" in that subdir, no "liborchbuildopaix64.so"

Alethesnake · Post by **Alethesnake** » Mon Jul 20, 2009 9:07 am

The problem was effettively linked to the 64bit objects linked to ds 8.1.
On last friday (friday the 17th

) we migrated from version 8.0.1 to 8.1 and we restored into the new installation all the old environment variables set before the migration.

The problem was that compiler options are slightly for 8.1 version:

Code: Select all

APT_COMPILEOPT=-O -q64 -c
APT_LINKOPT=-G -q64

Everything is ok now, thanks for support.

Bye,

Ale.

trokosz · Post by **trokosz** » Wed Dec 02, 2009 6:51 pm

So are you saying the following options worked or didn't?

APT_COMPILEOPT=-O -q64 -c
APT_LINKOPT=-G -q64

I have the same ones and get the same error.....So is the solution to remove the -q64 as someone indicated? or what did you actually do to fix?

I migrated nothing yet.....Thanks

Alethesnake · Post by **Alethesnake** » Thu Dec 03, 2009 11:03 am

Hi trokosz,

The -q64 parameter indicates the compiler to run in 64bit mode. Was your system booted in 64bit mode? (bootinfo -K)

Regards,

Alessandro.

trokosz · Post by **trokosz** » Thu Dec 03, 2009 7:46 pm

OK, I resolved....The OS was AIX 6.1 (64.bit) and IS 8.1 and XLC 10.1 so the 10.1 XLC plus adding the -q64 in this environemnt solved the issue. Thanks

trokosz · Post by **trokosz** » Thu Dec 03, 2009 7:47 pm

OK, I resolved....The OS was AIX 6.1 (64.bit) and IS 8.1 and XLC 10.1 so the 10.1 XLC plus adding the -q64 in this environemnt solved the issue. Thanks

hsahay · Post by **hsahay** » Tue May 25, 2010 3:12 pm

trokosz wrote:OK, I resolved....The OS was AIX 6.1 (64.bit) and IS 8.1 and XLC 10.1 so the 10.1 XLC plus adding the -q64 in this environemnt solved the issue. Thanks

Could you please give me the details of the environmental variables.

We were facing the same issue in the same environment as you have resolved on
OS was AIX 6.1 (64.bit) and IS 8.1 and XLC 10.1

but still facing the compilation errors while using the values that you have mentioned in the above post:
APT_COMPILEOPT=-O -q64 -c
APT_COMPILER=/usr/vacpp/bin/xlC-r
APT_LINKER=/usr/vacpp/bin/xlC-r
APT_LINKOPT=-G -q64

Could you please check the above values as you have resolved!

Thanks-

vinnz · Post by **vinnz** » Tue May 25, 2010 3:55 pm

hsahay wrote: APT_COMPILER=/usr/vacpp/bin/xlC-r
APT_LINKER=/usr/vacpp/bin/xlC-r

Is it a typo in your post or do you have a hyphen(-) in place of an underscore(_) ? I believe it needs to be /usr/vacpp/bin/xlC_r