Recompiling all jobs following upgrades is slow.

Ultramundane · Post by **Ultramundane** » Fri Oct 13, 2006 1:49 pm

After upgrading to 7.5.2 I need to recompile all jobs else they fail. I have seen many posts on using dscc and using the manager mulitple compile tool.

These projects are protected and first need to be unprotected in order to use either method. I don't like this. Also, dscc doesn't work for me because the jobs are also read only. I am able to use the manager to do a force compile of the jobs however it is going to take a few days to get through all jobs and projects. Reason being is that it is taking about 5-15 seconds per job to do the compiles. Checking the server utilization during the compilations show that the server is running at 1% utilization, very little disk I/O, and very little memory usage. It seems that the biggest bottleneck is the manager not compiling many jobs at once and using between 50% and 100% of the cpu on my box.

I'd like to be able to tell manager to compile so many jobs concurrently. Else, a better approach. Seems like a way to compile on the backend server would be most appropriate as it is much faster than my pc.

This manager method is going to take at least one day to do the compiles.

I also tried rebuilding from the vc repository. Same results. Terribly slow because my pc is running at 100% utilization and the backend server is running 1% utilization.

kcbland · Post by **kcbland** » Fri Oct 13, 2006 2:12 pm

My compile utility job is multi-instance enabled. You can run multiple instances and give the parameter for the folder a different value. I do this so that I have the compile running server side, as well as parallel executing jobs. It gets it done a lot quicker. The only issue I've found is that some jobs being imported from 7.5.1 to 7.5.2 don't always compile without first opening the job in designer and compiling there. Can't figure out why, it's just a mystery. But, if you're going from 7.5.2 to 7.5.2, there's no problem.

It's on the Members Area download page on my website. It's free if you want it.

Ultramundane · Post by **Ultramundane** » Fri Oct 13, 2006 2:26 pm

I will try it.

Thanks.

chulett · Post by **chulett** » Fri Oct 13, 2006 3:16 pm

Ultramundane wrote:These projects are protected and first need to be unprotected in order to use either method.

That's not true. A protected project can be 'multi job compiled' using the dsadm userid. Granted, it is slow as molasses because of all the silly gui updating going on, but you don't have to unprotect the project.

There is a program the upgrade process runs that does all that work, if you can find out what it is it can be run from a TCL prompt. DS.UPGRADEPROJ is in my brain, but I don't think that's right.

However, as Ken notes there are bugs in it. Going from 7.0.1 to 7.5.1A I found several instances in the upgrade log where the compile failed for odd reasons, but the jobs would compile fine afterwards in Designer. Turns out to be a bug they've allegedly since fixed, but perhaps not. 78132 is the Ecase number if anyone cares.

Ultramundane · Post by **Ultramundane** » Fri Oct 13, 2006 11:19 pm

Well it must be another bug. The option is greyed out and I cannot select it if the projected is protected. And that is using the dsadm userid. So my original assertion is true. It doesn't work with the project protected.

kumar_s · Post by **kumar_s** » Fri Oct 13, 2006 11:33 pm

If you own dsadm, why cant you change the project to Unprotected and compile and later change it back to Protected project.

Ultramundane · Post by **Ultramundane** » Fri Oct 13, 2006 11:35 pm

Please read the postings carefully.

That is exactly what I did. The post is about what options are available to compile faster.

kumar_s · Post by **kumar_s** » Fri Oct 13, 2006 11:40 pm

Sorry I missed that. If you able to Unprotect it, you can fire the dscc for each category or set of jobs paralley from your command prompt.

Ultramundane · Post by **Ultramundane** » Fri Oct 13, 2006 11:43 pm

kumar_s wrote:Sorry I missed that. If you able to Unprotect it, you can fire the dscc for each category or set of jobs paralley from your command prompt.

I was hoping that as much, but because the jobs are also read only I could not get dscc to work. I also specified the force option(do all jobs). Do you know of a flag that can specified to do read only jobs as well?

kumar_s · Post by **kumar_s** » Fri Oct 13, 2006 11:48 pm

It is again just READONLY that will be set as 1 in case of Readonly and 0 in case of Non-Readonly in your dsx. If you unprotect your project, the state of the jobs should also automatically chagne to Non-Readonly.
If it doesnt work, try to update the same via TCL, update DS_JOBOBJECTS and update the READONLY flag as NRO from RO.

chulett · Post by **chulett** » Sat Oct 14, 2006 7:09 am

kumar_s wrote:If you unprotect your project, the state of the jobs should also automatically chagne to Non-Readonly.

Umm... no. Protected and Read Only are two separate things. One shouldn't have to unprotect the project and then flip all the RO bits to NRO just to recompile things. Goodness knows what some nasty developer type might do during that Window of Vulnerability?

I don't see why the OP is having these issues. They do sound vaguely familiar, like something I went through early on, but I certainly don't have those issues today. I'll ponder this. I can certainly compile read only jobs, even in a protected project, as long as I use 'dsadm' to do it. Granted, I've not tried via dscc, just speaking of via either MJC or VC.

I recall that wasn't the case in 7.0.1, I remember having to unprotect things, but that was fixed in the upgrade to 7.5.1A. Hopefully it's not a bug reintroduced in 7.5.2! Have you talked to Support about it? You must be on a first name basis with everyone there by now...

Ultramundane · Post by **Ultramundane** » Wed Oct 18, 2006 9:29 am

Ecase: 108440 being worked on.

1. dscc not able to compile readonly jobs.
2. GUI not able to recompile jobs in protected projects.
3. The GUI being terribly slow and modal.
4. No provided utility to recompile jobs from the server side.
5. No provided utility to switch jobs from readonly to nonreadonly and vice versa easily.

Update from IBM:
Full Description:
Customer complains that the dscc coammnd ignores any jobs that are read only. They can recompile from Manager but that is too slow and take too much CPU.

I tested this in 7.5.1 A also and found that dscc does not compile read only jobs in the prior versions than 7.5.2 also. I was not sure if this is a bug or enhancement but customer wants this to work.

ray.wurlod · Post by **ray.wurlod** » Wed Oct 18, 2006 12:36 pm

It is claimed (at IOD) that the multi-job compile through the GUI is "much faster" in Hawk. Did not ask (yet) about dscc.

Ultramundane · Post by **Ultramundane** » Wed Oct 18, 2006 9:36 pm

ray.wurlod wrote:It is claimed (at IOD) that the multi-job compile through the GUI is "much faster" in Hawk. Did not ask (yet) about dscc. ...

Wonderful! I really don't need command line from my pc client. I think it would be useful to have command line on server side, but not really necessary for what I was trying to accomplish.

DSXchange

Recompiling all jobs following upgrades is slow.

Recompiling all jobs following upgrades is slow.

Re: Recompiling all jobs following upgrades is slow.

Re: Recompiling all jobs following upgrades is slow.