[ftputil] dealing with server timeout + GMANE

[Stefan Schwarzer]

Hi Terrence,

On 2006-12-05 17:25, Terrence Brannon wrote:
> Hi, I filled out the subscription form at GMANE for this list so that it
> will be archived there as well as mail-archive.com... it is not easy to
> search mailman archives.

You are quite right, thanks for adding the Gmane interface for
the mailing list!

> Anyway, I have an issue where I am downloading some large files from a
> server and I am getting at "Connection timed out" error.

Do you get the error _during_ the download or after using the
FTPHost object _after_ the download?

I assume the latter. Ftputil opens an FTP session for the "main"
session when you construct an FTPHost object. For each file-like
object (i. e. also for up/downloads) a new FTP session is opened
behind the scenes. So I guess that after the download your main
session has timed out.

You could prevent this by giving the main session something to do
during the download. If your Python is compiled with threading
support, the following should work:

>>> import ftputil
>>> import time
>>> import threading
>>>
>>> def poll(host):
...   while True:
...     # getcwd won't work because it only fetches a cached value
...     host.chdir(host.curdir)
...     # must be less seconds than a server timeout
...     time.sleep(300)
...
>>> host = ftputil.FTPHost("ftp.debian.org", 'anonymous',
...                        'sschwarzer at sschwarzer.net')`
>>> t = threading.Thread(target=poll, args=(host,))
>>> t.start()
>>> t.join()

This isn't production-ready code (e. g. the thread isn't stopped
when the FTPHost instance is closed). Also note that server
timeouts are there for a reason, so be very careful about the
duration you maintain the connection.

Please let me know if that helps.

I had thought about "timeout safety" a while ago, see
http://codespeak.net/pipermail/ftputil/2006q1/000073.html . Since
I didn't get any responses, I removed the timeout functionality
in the face of the described implementation difficulty.

Stefan