DSXchange

yup. done that.

We've got the ulimit settings back to where they were before all the changes were applied but something still is causing this message to appear in the logs.

I was quite surprised to still see the msgs in the logs after resetting the ulimit values.

What setting is it that when changed would cause this warning to start appearing in logs. node_Compute1: The open files limit is 10240; raising to 1048575. A number of different changes were made recently to our grid servers and those changes resulted in the above message showing up in logs where pr...

Thanks Andy. that was a big help. I did discover that one of the sequential files has 150+ columns defined as varchar(255). however current testing is only using 8 records so not certain that alone would cause the issue. Aslo discovered that the output of the 2nd join includes all 150+ varchar colum...

Am getting the following messages: main_program: orchgeneral: loaded orchsort: loaded orchstats: loaded APT_BadAlloc: Heap allocation failed. RT_SC310/OshExecuter.sh: line 25: 14911 Aborted (core dumped) $APT_ORCHHOME/bin/osh "$@" 2>&1 Parallel job reports failure (code 134) Job CopyOf...

Check with a DBA and see if there are deadlocks occurring in the table. Maybe something is not ending properly and leaving lock(s) in place.

Honestly, it had more to do with increasing the font size on the monitor than it did with "sharp eyesight"

No other jobs in the sequence. But, I solved the problem. I finally opened my eyes and noticed that the colon was part of the output string from the sequencer.sh script. So, I just changed the delimiter in the FIELD command I was using to parse out the 3 values from a space to a colon and ..... voil...

I am attempting to write a sequencer to execute a shell script on a compute node of our grid. I execute the sequencer.sh in the first stage and then parse the name of the selected compute node with: Field(sequencer_sh.$CommandOutput," ", 1) in a user variable activity stage. Finally in ano...

I have created a new custom wrapped stage and now have to import it to LOTS of other projects and environments. I plan to write a script that will loop through all the projects and import the stage. But, it also has to be GENERATEd before it can be used. Can the generate be done from the command lin...

PaulVL wrote:I did the same approach in my old environment. You'll have to take an average runtime approach. run each test 10 times (at least) back to back.
.
.
.
Throw gigs of row_gen data at it.

Thanks for the input. I'll throw some more data at it and see how it goes.

Q. Any ideas why I am not seeing more consistent run times for a job that runs multiple times reading unchanged data? I would suspect the problem is that the routing of your job's activity occurred on the public network rather than the desired private network. So you can verify that by checking the...

I've been tasked with developing some instructional guidelines for our development teams regarding how many nodes and partitions to request for various types of jobs. I know this is as much art as science, but I thought I should start with gathering some information first. I started just trying to s...

Bobyon, There's two ways I see you doing your task. External source stage and limit the execution of the stage to Node1. This looks like the approach I am going to take. I'll let the load leveler determine which node to run it on but will set the grid parms to a 1x1 config. Talk to the datastage ad...

That's exactly how to (using compute node info passed from the sequencer.sh script) get your ExecCommand activity job processed on a compute node. I guess there is a piece of this that I am still not understanding. Are you saying that if I execute sequncer.sh then I can somehow coax a subsequent ex...

Is there a particular reason/need to run the script on a compute node rather than the head node? Yes, the processing that is being done in the shell script is causing high wait CPU % on the head node and running it on a compute node doesn't have that problem. We need to reserve the head node for se...