DSXchange

Hi Ray, Thanks for the reply. That helps in clearing up a few items. A few additional question regarding the nodes especially, in the GRID world. I am trying to understand how some of the PX GRID parameters boil down into code during execution and would greatly appreciate your feedback if i am not c...

Hi Group, My job design is as follows. This is in a 4 node GRID environment Seq_File ==> Transformer ==> ODBC || || \/ Rejected records The job was using a default one node config file and some of our larger files are loading for a longggg time. I am trying this with different values for the number ...

Hi lstsaur, I have some good news from my first tests. Increasing the partitions per node to 4 helped a good number of jobs to complete faster. Here is the config file that i pulled from the director log for this execution. <Dynamic_gird.sh> SEQFILE Host(s): ctpcqabdsc02p: ctpcqabdsc02p: { node &quo...

Thanks lstsaur. I will definitely try the increased partition value and post the results. Meantime, SGE is the resource manager tool in this shop. Let me know if there are additional steps to find the resource allocation steps. Right now, i can see which computing node got allocated for a particular...

Hi Birendra and lstsaur, Thanks for your inputs. I tend to agree with lstsaur as the increase in nodes with the current setup of our system will still keep the new jobs in the queue. The feedback from our "person" who is managing the queue is that only one job can run in a node. If all the...

I agree with the feedback on the drawback on gigantic files...

When we had a similar situation... I ended up coding a Perl script (Manual parsing) to convert this mega XML to a manageable form.... Again, in line with Craig's...

Good luck,

Hi Group, In our Linux GRID environment we are often running into a situation where jobs end up waiting for a longggg time to be allocated a node. The Head Node has 3 computing nodes at its service and all our jobs are using the default GRID configuration parameter values as follows $APT_GRID_ENABLE...

Could you provide a simple representation of your job design please? E.g.... External Source ==> XMLInput ==> Transformer ... Your original post says XML Input, but your message log has the stage name as XML_Output_78,0. Is it just a naming convention error or are you working with XML output stage i...

Did you try enabling the "Log reject error" checkbox?

Thanks ArndW. I checked all the params and values and they look correct. I have submitted a ticket to IBM on this and also trying to do a back-end script in the database to fix the newline in a post job as we are running out of time. I will keep this thread open and keep trying options that might co...

The diff returns one line as a difference - for the header row. Strangely the header seems to have converted to a mixed case on the target and I have no idea how and why. The header is completely lower case in the source, whereas the target is mixed. E.g Source is as follows empid|~empname|~address1...

Thanks lstsaur. One additional question:- If the source file is on the head node, will the use of the variable $APT_GRID_SEQFILE_HOST interfere with the source file location if it did not get the head node as the Host name during the exeuction? Does the error message I received point to such a sympt...

Ray & Minhajuddin, Thanks for the followup. Here is what I did as the latest test - by replacing the target from DB to Sequential file. In the Sequential File stage that is used to read the source file i have set the "Record delimiter string" = "DOS format" Right-Click the &q...

Ray & Minhajuddin, Thanks for the followup. Here is what I did as the latest test - by replacing the target from DB to Sequential file. In the Sequential File stage that is used to read the source file i have set the "Record delimiter string" = "DOS format" Right-Click the &q...

Good one to learn more about the runtime. One question, if I may use this thread. ... So if a job is compiled in a particular configuration and is exported into a DSX file using the "Export Design with Executable" option and gets deployed on another system for execution, it would not reall...