can't open .gzip archives retrieved with wget
I'm trying to download the content of a directory hosted on a server using wget in a cygwin terminal. I have no problem getting the files but they are all .gz archive and once downloaded they are impossible to open... here is the line i'm using:
wget -rnH --cut-dir=5 --header="Accept-Encoding:gzip" ""
Does someone have any idea how to retrieve usable archive using wget?
Any help would be appreciated, thanks in advance for giving it a thought ;)
1 Answer
I've found this one:
If you request gzip'ed content (using the accept-encoding: gzip header, which is correct), then it's my understanding that wget can't then read the content. So you will end up with a single, gzipped file on disk, for the first page you hit, but no other content.
i.e. you can't use wget to request gzipped content and to recurse the entire site at the same time.
I think there's a patch that allows wget to support this function but it's not in the default distribution version.
If you include the -S flag you can tell if the web server is responding with the correct type of content. For example,
wget -S --header="accept-encoding: gzip" wordpress.com
