istool syntax

A forum for discussing DataStage<sup>®</sup> basics. If you're not sure where your question goes, start here.

Moderators: chulett, rschirm, roy

asorrell
Posts: 1707
Joined: Fri Apr 04, 2003 2:00 pm
Location: Colleyville, Texas

istool syntax

Post by asorrell »

In an earlier post I referred to a comprehensive 8.1 patch for IS Manager and istool. The good news is that it has resolved several of the issues that were preventing both tools from working properly.

However, the syntax on the istool command can be a bit tricky to figure out the first time because it uses a -datastage (-ds) option that has sub-options that must be passed properly. Because of this, I'm posting a few working examples of the istool command in operation. I'm just going to show using the "export" option since that is the most frequently used option and it will give you an idea of how the options work.

Note: In the examples below, I've updated it to show a three-tier architecture with a separate services tier (serviceshost) and an engine tier (enginehost). It can be a bit picky as to matching the engine host syntax (might have fully qualified .com name) and also is in upper case. If you are having issues matching the syntax, login to the InfoSphere Information Server Manager (the GUI front-end for istool) and use same syntax it displays for hostname after you connect.

First - here's some of the "help" options to show that there is help for each of the main istool operations (again - just showing export).

Code: Select all

$ pwd
/opt/IBM/InformationServer/Clients/istools/cli
$ ./istool -h
istool Command [ Command_Options ] [ Options ]

Possible values for: Command
    -                    : Provide usage information.
    help                 : Provide command-line help information.
                         : Istool Console.
    export               : Export to a file.
    import               : Import from a file.
    build package        : Build a Deployment Package.
    deploy package       : Deploy a Deployment Package.

To list Command_Options, enter:  istool Command -help

Possible values for: Options
    Long Name:         Short Name:     Description:
    -help            , -h              : print command usage
    -verbose         , -v              : display progress
    -silent          , -s              : silence command output
    -script          , -script         : file to read commands from.
 
Exit codes:
      0 - Success
      1 - Warning
      2 - Partial failure
      3 - Error reading from console
      4 - Invalid command history index
      5 - Error reading from script file
     11 - Invalid command syntax

$./istool export -help
istool export  [ Options ]

Possible values for: Options

    Long Name:         Short Name:     Description:
    -help           ,  -h              : print command usage
    -domain         ,  -dom            : Name of Information Server domain
    -username       ,  -u              : Name of user account on Information Server domain
    -password       ,  -p              : Password for Information Server account
    -verbose        ,  -v              : display progress
    -silent         ,  -s              : silence command output
    -archive        ,  -ar             : Archive file
    -preview        ,  -pre            : Preview export task without executing
    -abortIfError   ,  -abort          : Abort task on N errors
    -updatearchive  ,  -up             : Update any existing Archive file
    -datastage      ,  -ds             : DataStage Assets

            Options for: -datastage

              -includedependent      ,  -incdep    : Include dependent items
              -nodesign              ,  -nodes     : Exclude design items
              -includeexecutable     ,  -incexec   : Include executable items
              -base                  ,  -base      : Prefix for Paths
              DataStage item paths

Exit codes:
      0 - Success
      1 - Warning
      2 - Partial failure
      3 - Export failed
      4 - Export preview failed
      5 - Invalid archive file
     10 - Failed to connect to Information Server domain
     11 - Invalid command syntax
Now some working examples of an export. Note that I've "sanitized" the commands and output a bit to remove sensitive info. I'm also using the "-preview" option in a few cases that just shows what would be exported without actually doing the export. Also note that the suffixes are internal (ie: not part of the user supplied item names) - here are the ones I'm aware of:
  • qjb - Job Sequence
    sjb - Server Job
    pjb - Parallel Job
    srt - Server routine
    tbd - Table Definition
    tfm - Transform
In the first example I'm just backing up a single category called "Batch":

Code: Select all

$ ./istool export -domain serviceshost:9080 -u dsadm -p password -ar test.isx -pre -ds '-base="ENGINEHOST/Dev_Project" Jobs/Batch/*.*'
Beginning Export Preview
 [1/2] ENGINEHOST/Dev_AdHoc/Jobs/Batch/SeqIBIInvoices.qjb
 [2/2] ENGINEHOST/Dev_AdHoc/Jobs/Batch/SeqIBIOrders.qjb
Previewed 2 assets
In the second example I use the "-incdep" option that includes dependent jobs and routines for that category.

Code: Select all

$ ./istool export -domain serviceshost:9080 -u dsadm -p password -ar test.isx -pre -ds '-incdep -base="ENGINEHOST/Dev_Project" Jobs/Batch/*.*'
Beginning Export Preview
 [1/9] ENGINEHOST/Dev_Project/Jobs/Batch/SeqIBIInvoices.qjb
 [2/9] ENGINEHOST/Dev_Project/Jobs/Batch/SeqIBIOrders.qjb
 [3/9] ENGINEHOST/Dev_Project/Jobs/Control/tgtparmFileName.sjb
 [4/9] ENGINEHOST/Dev_Project/Jobs/Control/tgtparmSelectDate.sjb
 [5/9] ENGINEHOST/Dev_Project/Jobs/IBI Sales & Invoices/tgtinsIBIInvoices.sjb
 [6/9] ENGINEHOST/Dev_Project/Jobs/IBI Sales & Invoices/tgtinsIBISales.sjb
 [7/9] ENGINEHOST/Dev_Project/Jobs/IBI Sales & Invoices/tgtinsIBISalesVoids.sjb
 [8/9] ENGINEHOST/Dev_Project/Routines/Job Control/CreateFileFromDate.srt
 [9/9] ENGINEHOST/Dev_Project/Routines/Job Control/SetDateFromInput.srt
Previewed 9 assets
In the final example I show how you'd backup all the jobs in a project including any routines they call:

Code: Select all

$ ./istool export -domain serviceshost:9080 -u dsadm -p password -ar test.isx -ds '-incdep  -base="ENGINEHOST/Dev_Project" Jobs/*/*.*'
Beginning Export
 [1/11] ENGINEHOST/Dev_Project/Jobs/Batch/SeqIBIInvoices.qjb
 [2/11] ENGINEHOST/Dev_Project/Jobs/Batch/SeqIBIOrders.qjb
 [3/11] ENGINEHOST/Dev_Project/Jobs/Control/tgtparmFileName.sjb
 [4/11] ENGINEHOST/Dev_Project/Jobs/Control/tgtparmSelectDate.sjb
 [5/11] ENGINEHOST/Dev_Project/Jobs/IBI Sales & Invoices/tgtinsIBIInvoices.sjb
 [6/11] ENGINEHOST/Dev_Project/Jobs/IBI Sales & Invoices/tgtinsIBISales.sjb
 [7/11] ENGINEHOST/Dev_Project/Jobs/IBI Sales & Invoices/tgtinsIBISalesVoids.sjb
 [8/11] ENGINEHOST/Dev_Project/Jobs/XArchive/SeqIBIInvoices_ParameterChg_20070404.qjb
 [9/11] ENGINEHOST/Dev_Project/Jobs/XArchive/SeqIBIOrders_ParameterChg_20070404.qjb
 [10/11] ENGINEHOST/Dev_Project/Routines/Job Control/CreateFileFromDate.srt
 [11/11] ENGINEHOST/Dev_Project/Routines/Job Control/SetDateFromInput.srt
Elapsed time: 00:00:18
Exported 11 assets
$ ls -l test.isx
-rw-r--r--  1 dsadm dstage 115147 Jul  6 13:53 test.isx
Hopefully that's helpful and keeps you from wasting time trying to figure out the syntax on the command!

[Edit - Andy Sorrell 04/29/2014]
Adding some stuff I keep getting asked about...

Below syntax shows backing up EVERYTHING in a project (all routines, jobs, etc.). Also shows syntax to use authfile with encrypted password. Due to poor design, you must still specify domain on command line, even with an .authfile (yes I've complained to IBM about it!).

Code: Select all

./istool export -domain serviceshost:9080 -authfile /home/dsadm/.authfile -silent -archive /backups/projectname.isx -datastage "-incdep -base=ENGINEHOST/Dev_Project */*.*"
The contents of the .authfile would be something like this:
user=dsadm
password={iisenc}encryption_text_here==
domain=servicehost:9080
server=enginehost
Last edited by asorrell on Tue Apr 29, 2014 6:53 am, edited 2 times in total.
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2020
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Very nice, thank you Uncle Andy for taking point on that trail. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
roblew
Charter Member
Charter Member
Posts: 123
Joined: Mon Mar 27, 2006 7:32 pm
Location: San Ramon

Post by roblew »

Thanks. This helps quite a bit. However, I'm getting an error when I run the export saying that the project does not exist, even though I know it does. I'm wondering if I have a typo or something wrong with my syntax. Can you spot anything?

Code: Select all

[dsadm@cpgapl65: /vend/IBM/InformationServer/v81/Clients/istools/cli] $ ./istool export -domain cpgapl65 -u dsadm -p password -ar test.isx -ds '- base="cpgapl65.xsr.domain.net/SANDBOX" Jobs/*/*.*'
Beginning Export
Export failed: Invalid asset identifier: Jobs/*/*.*. Invalid project name.

[dsadm@cpgapl65: /vend/IBM/InformationServer/v81/Clients/istools/cli] $ dsjob -lprojects
SANDBOX

Status code = 0
asorrell
Posts: 1707
Joined: Fri Apr 04, 2003 2:00 pm
Location: Colleyville, Texas

Post by asorrell »

Two things - first of all I see a space before the word base - it should be "-base" not "- base".

Secondly - if you are in Unix and your password has any special characters like an @, & or $ you'll have to put a backslash in front of it (ie: \$) on the command line to submit it successfully.
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2020
Oritech
Premium Member
Premium Member
Posts: 140
Joined: Thu May 07, 2009 9:32 pm

Post by Oritech »

Andy,

Code: Select all

$ ./istool export -domain dsdevbox -u dsadm -p password -ar test.isx -ds '-incdep  -base="DSDEVBOX.MMM.COM/Dev_Project" Jobs/*/*.*' 
I tried,looks like its not doing the complete project export which I achieve
manually thro' old Export functionlity creating .dsx file by selecting all.

Need help to customise the above cole to do the complete project export.
DS_FocusGroup
Premium Member
Premium Member
Posts: 197
Joined: Sun Jul 15, 2007 11:45 pm
Location: Prague

Post by DS_FocusGroup »

And I assume that we cannot import a .dsx file through this?
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

That is correct. Deployment packages (which is what Information Server Manager (istool)) deals with, are different from DataStage export files.

Command line choices for importing from dsx files do exist, including dsjob, dscmdimport and dsimport.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
smosalakanti
Participant
Posts: 3
Joined: Tue Jan 08, 2008 2:20 pm
Location: Houston

Encrypt password

Post by smosalakanti »

thank you for the details on the utility "istool". Is it possible to encrypt password ?
asorrell
Posts: 1707
Joined: Fri Apr 04, 2003 2:00 pm
Location: Colleyville, Texas

Post by asorrell »

No method of encrypting passwords that I'm aware of. If your site uses individual login id's then you could
  • 1) store the password in a file
    2) make the file readable only by the dsadm id
    3) use file descriptors on the command line to read the password from the file
something like:

Code: Select all

-u dsadm -p 8 8</dir/dir/pw_file
That should probably work.

{Edited - 09/19/13}

The istool command now supports "-authfile" which allows you to put the id and password in a protected file so they won't be displayed. The password can be encrypted via ASBNode/bin/encrypt.sh.
Last edited by asorrell on Sat Apr 16, 2016 10:48 am, edited 2 times in total.
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2020
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

I wonder if Apple will sue IBM over use of its iStool product.

:lol:
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
asorrell
Posts: 1707
Joined: Fri Apr 04, 2003 2:00 pm
Location: Colleyville, Texas

Post by asorrell »

Probably send their stock right into the crapper...

:-)
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2020
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

-craig

"You can never have too many knives" -- Logan Nine Fingers
cbo0485
Participant
Posts: 1
Joined: Wed Apr 14, 2010 2:13 pm

Post by cbo0485 »

I'm trying to create a backupScript.sh that contains the following.

Note: The tmpVar - srvName/awk pieces are just b/c I thought I might have had some case sensitivity issues, so if I don't have to use them, great, if I do, it's already done and working.

I've verified the project exists and that the username/password works when pulling up the project through the Designer. It may not have anything to do with the issue, but I am using LDAP for my authentication into the DataStage Designer.

I get the following errors:


Beginning Export Preview
Export preview failed: Project not found: qa_PRNAME

Code: Select all

tmpVar=`hostname`
srvName=`echo $tmpVar | awk '{print toupper($0)}'`
#echo $srvName

#From http://www.dsxchange.com/viewtopic.php?t=128126
./istool export -domain $srvName -u <username> -p <pw:includes @> -ar /home/dsadm/backups/testBackupWWW.isx -pre -ds '-base="$srvName.DOMAIN-NAME.COM/qa_PRNAME" Jobs/*.*'
Any help would be greatly appreciated. This is 8.1 with FP1.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

I use the syntax

Code: Select all

istool export -domain $srvName:9080 -username xx -password yy -archive /home/dsadm/backups/testBackupWWW.isx -ds '"$srvName.DOMAIN.NAME.COM/qa_PRNAME/*/*.*"
probal
Participant
Posts: 24
Joined: Sun Oct 25, 2009 10:17 am
Location: kolkata

Post by probal »

asorrell:: As you have pointed out earlier that if the password contains '$' a backslash needs to be in front of it.

I don't think it is needed.I'm using a password that contains a '$' and without the '/' it is working fine for me.

and I guess if I am using istool utility to export assets I must use the istool utility to import those.
Probal
Post Reply