Migrating a 7.1 project from one win server to another?

tnickasn · Post by **tnickasn** » Thu Nov 10, 2005 2:13 am

Greetings DSXchange Community,

I need to move a DS server 7.1 project from one Win2000 server to another (the original was a temp loaner box). I want to be able to move everything (including hash files and environment variables) to the new box. I know how to do the project export/import part. Also, by reading the DSXchange, I see that I can manipulate the DSParam file (and .keep file) as needed. For me the big issue is the HASH files. This is a JDE EPM implementation and all the hash files are stored within the project folder (as opposed to Ray and other mention of a separate unified directory). There are hundreds of hash files in the overall system!

If the new target server has the same directory location on drive x as the old, can I simply do an O/S level backup with DS down (or something similar) and restore to the new server? What would be the quickest steps to get this done. From the research I done on DSXchange, it sounds like it could get ugly

(because of the hash files being mixed with everything else)

I'm relatively new to DS (I can build average complexity jobs...etc but certainly don't understand the finer technical details this would involve).

PS: I could probably set the ETL to do a full refresh in the new environment (database changing as well), but I know it will take a long time doing it that way (probably 24hrs and I'd have to do it on a weekend). I rather not reload if I can avoid it.

Thanks in advance.

vmcburney · Post by **vmcburney** » Thu Nov 10, 2005 2:17 am

It's a very poor ETL design that requires permanent hash files. Usually hash files get built during each run, or can be rebuilt as a recovery operation by utility jobs. How many of your hash files do you need to move? How many are built by each batch schedule?

Hash files are not a database engine with backup and restore and should not be treated as one. Move all your jobs across and run them and see how many fall over due to missing hash files. Then find the jobs that built them.

ArndW · Post by **ArndW** » Thu Nov 10, 2005 2:29 am

I tend to disagree with Vincent on hashed files; good and fast DS ETL implementations make use of hashed files for such purposes as changed data detection and efficient lookups; reloading them daily from a database system just to not have any any permanent data stored in hashed files defeats their purpose.

Be that as it may, if you were to create your new project in the same path on the new machine you can do a UNIX level copy into the new project from the old one. Important is that you create the project first. It is sensible to put all the hashed files in a separate directory, but it's a bit too late for you to do that in your case.

tnickasn · Post by **tnickasn** » Thu Nov 10, 2005 3:02 am

Hi Again,

Well I didn't build the ETL. I comes from JDEdwards/Peoplesoft. I added a few jobs and changes to suite our needs. Anyways, some of the hash files are used simply as interm stora and are not an issue. Others (as Arnd said) are used to maintain the relationship of surrogate keys to system keys. Jobs that use these read in a table, lookup the hash to find the surrogate key. If a match is found then nothing happens (other than an update). If a match isn't found, a call is made to get a new key and the row is inserted.

Anyways...Arnd...you say to get the project on the target first (via Admin tool I assume). If I do that but then restore the old files on top, won't it get confused somewhere? Whats the purpose behind creating the Project on the target first? When not just copy it? (is it because DS Admin writes some registry data or stores data about the project elsewhere??)

Thanks

ArndW · Post by **ArndW** » Thu Nov 10, 2005 3:40 am

When you create a project a number of actions are done, but the most important one from your point of approach is that the DataStage schema is created and the appropriate master pointers point to the directory where the project resides.

I think it might work if you were to just copy the project directory and then use the admin tool to add a new project into this existing directory structure, but I am not sure; I feel more comfortable using the first procedure.

DataStage projects are quite encapsulated, no references apart from the path and the SQL schema information are stored centrally. The paths really do need to be identical, as some of the file pointers within the project use absolute paths. There are mainly the DS secondary indices which could be rebuilt to get the new path if it has changed, but I wouldn't rely on that.