gunzip usage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
kum_d
Participant
Posts: 20
Joined: Tue Jul 17, 2007 4:53 am

gunzip usage

Post by kum_d »

Hi

I am getting the following error while i am trying to send the oputput of one command to another.

The command is : ls-rt1|tail -1| gunzip

here ls -rt1 |tail -1 is avle to give the out put with all .GZ files

i.e

total 74105
-rw-r--r-- 1 testapp netx 7337398 Oct 09 2006 Sep06.09Oct06_11:58.gz
-rw-r--r-- 1 testapp netx 7355629 Feb 01 2007 Jan07.18-19-Jan_Data_Corrupt.gz
-rw-r--r-- 1 ftptci staff 6806524 Jun 06 19:25 May07.gz
-rw-r--r-- 1 ftptci staff 7274381 Jul 01 06:00 Jun07.gz
-rw-r----- 1 ftptci staff 1220453 Jul 28 14:51 Feb07.gz
-rw-r--r-- 1 ftptci staff 7940110 Aug 01 06:00 Jul07.gz
the above all are in .gz format

But while i am sending the output of ls -rt1 | tail -1 to gunzip as :
ls -rt1 |tail -1|gunzip

but it is throwing the error message "gunzip: stdin: not in gzip format"

can any one please help me for " how to unzip Jul07.gz " using ls -rt1 |tail -1|gunzip.

is there any wrong with this command?
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

gunzip is trying to interpret stdin (your list of files) as a compressed stream so it cannot unzip it.

I am not at a UNIX box right now, but would recommend you use the find command, but I don't know the options from memory for selecting the appropriate single file, but the command would read something like

Code: Select all

find . -name \Feb07.gz -exec gunzip {} \;
kum_d
Participant
Posts: 20
Joined: Tue Jul 17, 2007 4:53 am

Post by kum_d »

Hi arnw,

Still i am getting same error.

In the above post i am giving the command ls -rt1 | tail -1which will givesthe output Jul07.gz which is a zip file.

the same .gz file i am paasing to gunzip command using | in unix box.

i am unable to understand why am i getting this error even i am passing the .gz file to gunzip.
kum_d
Participant
Posts: 20
Joined: Tue Jul 17, 2007 4:53 am

Post by kum_d »

Hi ArndW,

Can you also please suggest me how do i get previous month name using using date command .

Usualy date command give current date and time.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

kum_d - you are not passing the filename to the gunzip command, you are passing a string (containing the file name) to gunzip, which is detecting that the string is not compressed. What happened when you tried to use the find command I suggested?
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

I've been through this in the past. A Google search for that message turns up tons of people having the same problem and zero in the way of answers. Typical response? "Your file isn't really compressed". Umm, yes it is - I just gzipped it. [sigh]

Gunzip does support decompression from stdin, but you need to find a way to send the contents of the file that way, not the name. You are going to need to do as Arnd suggested, using some flavor of 'find'. From the official manual's Sample Output page:

The following command will find all gzip files in the current directory and subdirectories, and extract them in place without destroying the original:

Code: Select all

find . -name '*.gz' -print | sed 's/^\(.*\)[.]gz$/gunzip < "&" > "\1"/' | sh
Use that as a starting point.
-craig

"You can never have too many knives" -- Logan Nine Fingers
kum_d
Participant
Posts: 20
Joined: Tue Jul 17, 2007 4:53 am

Post by kum_d »

Hi Chulett,

Thanks for your reply.

I need one input from you for the find command:
find . -name '*.gz' -print | sed 's/^\(.*\)[.]gz$/gunzip < "&" > "\1"/' | sh

This will decompress all the zip files what it finds.

but My requirement is i need to decompress the previous months file only i.e Jul07.gz in the current mnoth.also to decompress aug07.gz in the month of september ...and need to access in my datastage job and pass the decompressed file name in my stage properties.

i am very helpless my self here.
Trying al ot .but no result.

can i get any help for this.
kum_d
Participant
Posts: 20
Joined: Tue Jul 17, 2007 4:53 am

Post by kum_d »

Hi

Can i know what sed and its parameters will do.

i have a bit knowledge about unix.

find . -name '*.gz' -print | sed 's/^\(.*\)[.]gz$/gunzip < "&" > "\1"/' | sh
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

sed is the stream editor for UNIX. Issuing the man sed command will tell you most of what you need to know in terms of syntax - after that it's a matter of experimentation and practice. A book such as UNIX for Dummies is also useful.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

kum_d wrote:My requirement is i need to decompress the previous months file only i.e Jul07.gz in the current mnoth.also to decompress aug07.gz in the month of september
When it's August and it's time for your job to run and the 'previous months file' is Jul07.gz - is that file the most recent file? Or could Aug07.gz also exist at that point?

The fact that you used "ls -rt1 | tail -1" ealier implies the former. Investigate the following find options to looks for gzip files in the current directory that are less than or equal to 30 days old:

Code: Select all

cd <TargetDirectory> && find . -name '*.gz' -type f -mtime +30 -print
Run that, modifying the number of days if it needs to be less than 30 until you think you've got what you need. Then combine it with sed and see what happens.
-craig

"You can never have too many knives" -- Logan Nine Fingers
kum_d
Participant
Posts: 20
Joined: Tue Jul 17, 2007 4:53 am

Post by kum_d »

Hi

Jul07 is the most recent and aug07 file does not exist bec it will be generated on sept first.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Combine Craig's find options and replace his -print option with what I posted earlier "-exec gunzip {} \;" and you will have something that works for you. But you really should read up on UNIX, particularly how pipes, redirection and command lines work since this is fundamental to working in that environment.
iDomz
Participant
Posts: 81
Joined: Wed Jul 25, 2007 5:25 am
Location: London

Post by iDomz »

Use xargs as follows

ls -rt1 |tail -1|xargs gunzip
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Thanks iDomz!
Post Reply