Compilation Issue in Parallel Job

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Re: Compilation Issue in Parallel Job

Post by ArndW »

mpathak66 wrote:...Cannot able to compile the new Parallel Transformer jobs but able to compile the existing jobs...
Does this mean that existing jobs with Transform stages in them can be compiled? That would point to a problem other than just an invalid compiler specification.

There is most likely no LOCK issue in this job. Doing the type of UNLOCK command you specified is a very bad idea, since that can release locks which are being held for a reason, i.e. to prevent multiple users to edit the same object at the same time.

Is this an existing installation where the compilation used to work and now doesn't? If so, what might have changed between the time it worked and now?
mpathak66
Premium Member
Premium Member
Posts: 6
Joined: Wed May 14, 2008 10:50 pm

Re: Compilation Issue in Parallel Job

Post by mpathak66 »

We tried running one sample c program and its able to compile and generate the .o file. SO, it seems there is no issue with the compiler. When we checked the RT_BP_XXX.o
folder after compilation where the the .o file will be generated, we did not see the .o file. So, its not able generate the .o file for the jobs having parallel transformer.
Could you please let me know if you have faced this problem before and resolution.

Below is the compilation error:
ld: 0711-998 SEVERE ERROR: Unexpected exception 10 caught.
I/O error
ld: 0711-993 Error occurred while writing to the output file: /apps/ptadm/dstage/IBM/InformationServer/Server/Projects/FIN_BI/RT_BP199.O/V0S1_Demo7_Transformer_1.o

ld: 0706-010 The binder was killed by a signal: Bus error
MAP
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Have you verified that you are running a supported compiler for your DataStage version and OS / platform?

It's also important to let us know if you have this problem with every Parallel job with a transformer in it or if others work and this error is only for this particular job.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Mike
Premium Member
Premium Member
Posts: 1021
Joined: Sun Mar 03, 2002 6:01 pm
Location: Tampa, FL

Post by Mike »

XL C/C++ for AIX, V11.1 would be a supported compiler on AIX for version 8.5.

Question: Can you force compile an existing job with a transformer stage?

What are your settings in these project environment variables?
$APT_COMPILER
$APT_COMPILEOPT
$APT_LINKER
$APT_LINKOPT

Maybe someone at an AIX site can verify your environment variable settings.

Mike
mpathak66
Premium Member
Premium Member
Posts: 6
Joined: Wed May 14, 2008 10:50 pm

Post by mpathak66 »

Hi Mike,

Once we did force compile, we cannot able to compile any parallel job with Transformer. Earlier it was working.
Below are the current Compiler Parameters:
$APT_COMPILEOPT=-O -q64 -c
$APT_COMPILER=/usr/vacpp/bin/xlC_r
$APT_LINKER=/usr/vacpp/bin/xlC_r
$APT_LINKOPT=-G -q64

I have tested the compiler with a small C program and it is working fine. Attached below are the steps I tried.
Compiler 11.1 is the supported version for DataStage 8.5 and AIX 6.1

void main ()
{
printf("Hello, World!\n");
}
Save the file as 'hello.c' in /tmp dir
$ /usr/vacpp/bin/xlC_r hello.c -o hello
Once the file is created, executed the file and output was executed.
$ ./hello

So Compiler is working and supports the current version.

The main problem we see is , while compiling .C and .tmp.o files get created by not .o file which is causing the issue.

Can you please help to identify the issue
MAP
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

mpathak66 wrote:Once we did force compile, we cannot able to compile any parallel job with Transformer. Earlier it was working.
So this was something that worked in the past but then stopped working? The obvious next question is what changed? I can't imagine that this is something that simply doing a 'force compile' would cause. :?
-craig

"You can never have too many knives" -- Logan Nine Fingers
Mike
Premium Member
Premium Member
Posts: 1021
Joined: Sun Mar 03, 2002 6:01 pm
Location: Tampa, FL

Post by Mike »

mpathak66 wrote:Once we did force compile, we cannot able to compile any parallel job with Transformer
This essentially means that you have never been able to compile any job containing a transformer with your current configuration.

Existing compiled transformers only get recompiled when necessary unless you specify a force compile.

So what has changed in your environment from the time that transformers used to compile until now.

Your hello world compile test doesn't reveal much since it isn't done in the same environment (meaning the engine's environment) and isn't using the same compile and link options.

Did you upgrade the OS and/or compiler?

Are there two versions of the compiler on the server? (thinking a possible PATH/LIBPATH conflict)

Can you uninstall all of the C++ compilers and reinstall just the 11.1 compiler?

Mike
mpathak66
Premium Member
Premium Member
Posts: 6
Joined: Wed May 14, 2008 10:50 pm

Post by mpathak66 »

1. Before 02/20, we could able to compile any parallel job and run.
From 02/22, we cannot able to compile any parallel job with Transformer.
Except adding UNLOCK Entry to VOC file(Project level), there was no other change. IBM too confirmed that there is no relation with UNLOCK entry and compiler error issue.

2. We had installed only 1 version of Compiler 11.1 in December 2014 and it supported the existing configuration.

3. Compared PATH, LIBPATH from existing dsenv file and from Env var settings of last successful run of parallel job (from director logs). Those are good.
MAP
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

It sounds like you all may have installed a 60 day trial edition of the compiler and now your time is up.
Choose a job you love, and you will never have to work a day in your life. - Confucius
Mike
Premium Member
Premium Member
Posts: 1021
Joined: Sun Mar 03, 2002 6:01 pm
Location: Tampa, FL

Post by Mike »

Try this command:

Code: Select all

/usr/vacpp/bin/xlC -qversion
If it is an expired trial version, I think that command would report it.

Though if it is an expired license, I think your hello world program wouldn't compile either.

Mike
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

The timing is a bit suspicious but yes, it doesn't quite add up. See if the 'lslpp -l | grep -i vacpp" command shows 'IBM XL C/C++ Evaluation' edition.
Choose a job you love, and you will never have to work a day in your life. - Confucius
mpathak66
Premium Member
Premium Member
Posts: 6
Joined: Wed May 14, 2008 10:50 pm

Post by mpathak66 »

Below are the requested details:

Compiler Verison:
/apps/ptadm/dstage/IBM/InformationServer/Server/DSEngine ->/usr/vacpp/bin/xlC -qversion
IBM XL C/C++ for AIX, V11.1 (5724-X13)
Version: 11.01.0000.0000

Compiler Edition:
/apps/ptadm/dstage/IBM/InformationServer/Server/DSEngine ->lslpp -l | grep -i vacpp
vacpp.Bnd 11.1.0.0 COMMITTED IBM XL C/C++ Media Defined
vacpp.cmp.aix50.lib
vacpp.cmp.aix50.tools
vacpp.cmp.aix52.lib
vacpp.cmp.aix52.tools
vacpp.cmp.aix53.lib 11.1.0.0 COMMITTED IBM XL C/C++ Libraries for AIX
vacpp.cmp.aix53.tools 11.1.0.0 COMMITTED IBM XL C/C++ Tools for AIX 5.3
vacpp.cmp.core 11.1.0.0 COMMITTED IBM XL C/C++ Compiler
vacpp.cmp.include 11.1.0.0 COMMITTED IBM XL C/C++ Compiler Include
vacpp.cmp.lib 11.1.0.0 COMMITTED IBM XL C/C++ Libraries
vacpp.cmp.rte 11.1.0.0 COMMITTED IBM XL C/C++ Compiler
vacpp.cmp.tools 11.1.0.0 COMMITTED IBM XL C/C++ Tools
vacpp.html.common 11.1.0.0 COMMITTED Supersede entry, not installed
vacpp.html.en_US 11.1.0.0 COMMITTED Supersede entry, not installed
vacpp.html.ja_JP 11.1.0.0 COMMITTED Supersede entry, not installed
vacpp.html.zh_CN 11.1.0.0 COMMITTED Supersede entry, not installed
vacpp.lic 11.1.0.0 COMMITTED IBM XL C/C++ Licence Files
vacpp.licAgreement 11.1.0.0 COMMITTED IBM XL C++ Electronic License
vacpp.man.en_US 11.1.0.0 COMMITTED IBM XL C/C++ Compiler Man
vacpp.memdbg.aix50.lib
vacpp.memdbg.aix50.rte
vacpp.memdbg.aix52.lib
vacpp.memdbg.aix52.rte
vacpp.memdbg.aix53.lib 11.1.0.0 COMMITTED IBM XL C/C++ User Heap/Memory
vacpp.memdbg.aix53.rte 11.1.0.0 COMMITTED IBM XL C/C++ User Heap/Memory
vacpp.memdbg.lib 11.1.0.0 COMMITTED IBM XL C/C++ User Heap and
vacpp.memdbg.rte 11.1.0.0 COMMITTED IBM XL C/C++ User Heap and
vacpp.ndi 11.1.0.0 COMMITTED IBM XL C/C++ Non-Default
vacpp.pdf.en_US 11.1.0.0 COMMITTED IBM XL C/C++ Documentation
vacpp.samples.ansicl 11.1.0.0 COMMITTED IBM XL C/C++ Compiler ANSI
vacpp.cmp.core 11.1.0.0 COMMITTED IBM XL C/C++ Compiler
MAP
mpathak66
Premium Member
Premium Member
Posts: 6
Joined: Wed May 14, 2008 10:50 pm

Post by mpathak66 »

Looks like its not a trial version since IBM agreed Warranty Support for an year.
MAP
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

Check your error with Support.
Choose a job you love, and you will never have to work a day in your life. - Confucius
Mike
Premium Member
Premium Member
Posts: 1021
Joined: Sun Mar 03, 2002 6:01 pm
Location: Tampa, FL

Post by Mike »

I agree support is probably going to be the best bet to help you track down what changed or what is inconsistent.

When did you do the AIX upgrade from 5.3 to 6.1?

Your original post seemed to show a compiler for 6.1, but your last post makes it look like a compiler for 5.3.

Mike
Post Reply