DSXchange

Hello,
I am looking for suggestions on the shortest way to refresh our TEST environment with PROD data.

Apart from the DB side which is easily taken care of by a backup of PROD and a restore to TEST, in our PROD DataStage project we have lots of hashed files that need to be moved as well (we use CRC to load tables incrementally). If we leave them behind, the TEST environment's hashed files will be out of synch with the content in the restored DB (by the way, this is a PeopleSoft installation so all hashed files are in the project's directory).

Would it work to copy the PROD project onto the TEST one and then restore the DSPARAM and ODBC.INI to their TEST version?

What if PROD and TEST projects have different names?

Any suggestion is greatly appreciated.
Thanks.

Why not create a suite of DataStage jobs to synchronize the contents of the hashed files? Are the TEST and PROD environments on the same server?

Whether you can successfully use operating system commands to move hashed files depends on how they were created. However, simply copying the PROD project onto the TEST one will break pretty much everything and you will spend days effecting repairs. Don't even think about it.

Hello Ray,

PROD and TEST are on different servers that are on different network segments. Since the project copy is ruled out it sound like the best option would be to take an OS backup of the PRD project's folder and restore it on the TEST server. At that point we can either copy the files to the TEST project with a Unix script or, as you suggested, create a set of jobs to transfer the data.
How would you source from or write to a target hashed file that is in a different project than the one you are executing from?

Thanks

What you propose won't work. The project directory contains objects that are part of a database instance, and there need to be system table entries that describe them. The right way is export/import.

This does not, of course, synchronize hashed files. You can use format.conv to export/import these, or maybe construct DataStage jobs to do the same (move records from hashed file to sequential file in production and back again in test. You can also use the hashed file bulk unloader (UNLOAD.FILE) and bulk loader (loadfile), but these are undocumented "UniVerse" utilities and UNLOAD.FILE would need to be modified to work with pathed hashed files.

Sorry for the delay in answering.

What I am interested in is just the hashed files and a few control files from the PeopleSoft warehouse. Their TST names match the PRD names and that is why I was thinking about a restore of the PRD project's folder to the TST server, in a temp directory. That way, with a Unix script it would be easy to copy the production files from the temp directory to the TST project's directory thus overwriting the existing ones. That should not break the files' references in the TST repository.

I like your solution of coding jobs to move PRD data to sequential files and then uploading it to TST because it uses the normal DataStage functionality and therfore it is safer. I am sure that my boss would not like it as much because we would have to code hundreds of jobs.

Do you think that the OS way is feasable?
Thanks

Before starting to work on the production project I decided to first try on DEV. The method I chose was an OS copy of the TST project's hashed files to the DEV project. Both TST and DEV projects are on the same DataStage server. I analyzed the TST project's content and came up with a list of wildcarded strings that would target the files I was interested in. Then I backed up the DEV project, refreshed the DEV database with the TST one, ran the extract script of the files from TST and their copy (overwrite) to DEV. Finally, I tested the DEV jobs, ran DB counts and everything seemed to be working fine.

I will test in the DEV project a bit more just to make sure. Than I will elaborate a strategy for the PRD-to-TST refresh which entails moving data between different servers.

Thank you Ray for your help.