[ftputil] host.download_if_newer problems

[Stefan Schwarzer]

Hello Yvan,

On 2008-05-13 14:39, Yvan Strahm wrote:
> I try to use ftputil to mirror biological databases. On the host the
> files are compressed (either .gz or .Z) on the local server they are
> uncompress. Is there a way of having uncompress file on the local server
> and using host.download_if_newer()?

download_if_newer (and more generally, ftputil) has no built-in
support for compressing or uncompressing files.

You write "host" and "local server" which makes me wonder which
computer is the client (i. e. the computer on which ftputil runs)
and which the server (i. e. the argument to FTPHost's
constructor) from ftputil's point of view. Below, I assume you
have the compressed files on a server and want to download them
to a client on which ftputil runs.

Probably the simplest way is to download the files with
download_if_newer and uncompress them locally, either with Python
libraries or with the subprocess module and command line tools.
Python supports uncompressing .gz files with the gzip and/or zlib
modules, but not, as far as I understand, uncompressing .Z files.
So for .Z files you will need the appropriate command line tools
or libraries which aren't part of Python.

To avoid the download followed by the local uncompressing, it
should in principle be possible to construct a file-like object
which supports uncompression on the fly, use ftputil to determine
modification timestamps (possibly considering time shift between
server and client) and use FTPHost.copyfileobj for the transfer.
However, probably the local decompression is so fast compared
to a download that the complex approach doesn't make sense.

> Thanks for your help

I don't know if you'll call that "help". ;-)

> And thanks a lot for ftputil.

Thank you! You're welcome. :-)

Please ask if you have more questions.

Best regards,
Stefan