There was a trick in your code I had never seen before. The comma was necessary to create a sequence here:<br> <br> >>> t = threading.Thread(target=poll, args=(host,))<br><br>Anyway, the program seems to head once I initiate the thread. I created a class for your code and then called it like below. I think throwing a simple chdir() in between each downloaded file should prevent the timeout, but I'd like to see if I can get the thread thing working anyway:
<br><br>import ftputil <br>import time <br>import threading <br> <br>class busyloop(object): <br> <br> def __init__(self,host): <br> self.host = host <br> <br> def poll(self, host): <br> while True: <br>
host.chdir(host.curdir) <br> time.sleep(300) <br> <br> def loop(self): <br> t = threading.Thread(target=self.poll, args=(self.host,)) <br> t.start() <br> t.join() <br><br><br>
def fetch_file(my): <br> #print "host = ftputil.FTPHost", my.cnf.dl.host, my.cnf.dl.user, my.c\<br>nf.dl.password <br> <br> host = ftputil.FTPHost(my.cnf.dl.host, my.cnf.dl.user, <br>
my.cnf.dl.password) <br> <br># t = data.localpy.ftputil.busyloop(host) <br># t.loop() <br> <br> print host <br> host.chdir(my.cnf.dl.root_cwd) <br> <br> expected_files = ['diseasecategories_export.txt',
<br> 'fdadruglistings_export.txt', <br> 'medicalareas_export.txt', <br> 'pipelinedrugs_export.txt', <br> 'pipelinedrugscategories_export.txt', <br> 'studylistings_export.txt',
<br> 'studyresults_export.txt'] <br> for f in host.listdir(host.curdir): <br> if re.compile(my.cnf.dl.filepattern).search(f): <br> print "Downloading", f <br>
host.chdir(host.curdir) <br> host.download(f, f, 't') <br> <br> try: <br> expected_files.remove(path(f).basename()) <br> except: <br> pass
<br> shutil.copy(f, my.storage.input) <br> shutil.move(f, my.storage.zip) <br> <br> if len(expected_files) != 0: <br> print "Expected files", expected_files, "not on",
my.cnf.dl.host <br> sys.exit() <br><br><br><div><span class="gmail_quote">On 12/5/06, <b class="gmail_sendername">Stefan Schwarzer</b> <<a href="mailto:sschwarzer@sschwarzer.net">sschwarzer@sschwarzer.net</a>
> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Hi Terrence,<br><br>On 2006-12-05 17:25, Terrence Brannon wrote:<br>> Hi, I filled out the subscription form at GMANE for this list so that it
<br>> will be archived there as well as mail-archive.com... it is not easy to<br>> search mailman archives.<br><br>You are quite right, thanks for adding the Gmane interface for<br>the mailing list!<br><br>> Anyway, I have an issue where I am downloading some large files from a
<br>> server and I am getting at "Connection timed out" error.<br><br>Do you get the error _during_ the download or after using the<br>FTPHost object _after_ the download?<br><br>I assume the latter. Ftputil opens an FTP session for the "main"
<br>session when you construct an FTPHost object. For each file-like<br>object (i. e. also for up/downloads) a new FTP session is opened<br>behind the scenes. So I guess that after the download your main<br>session has timed out.
<br><br>You could prevent this by giving the main session something to do<br>during the download. If your Python is compiled with threading<br>support, the following should work:<br><br>>>> import ftputil<br>>>> import time
<br>>>> import threading<br>>>><br>>>> def poll(host):<br>... while True:<br>... # getcwd won't work because it only fetches a cached value<br>... host.chdir(host.curdir)<br>... # must be less seconds than a server timeout
<br>... time.sleep(300)<br>...<br>>>> host = ftputil.FTPHost("<a href="http://ftp.debian.org">ftp.debian.org</a>", 'anonymous',<br>... '<a href="mailto:sschwarzer@sschwarzer.net">
sschwarzer@sschwarzer.net</a>')`<br>>>> t = threading.Thread(target=poll, args=(host,))<br>>>> t.start()<br>>>> t.join()<br><br>This isn't production-ready code (e. g. the thread isn't stopped<br>
when the FTPHost instance is closed). Also note that server<br>timeouts are there for a reason, so be very careful about the<br>duration you maintain the connection.<br><br>Please let me know if that helps.<br><br>I had thought about "timeout safety" a while ago, see
<br><a href="http://codespeak.net/pipermail/ftputil/2006q1/000073.html">http://codespeak.net/pipermail/ftputil/2006q1/000073.html</a> . Since<br>I didn't get any responses, I removed the timeout functionality<br>in the face of the described implementation difficulty.
<br><br>Stefan<br>_______________________________________________<br>ftputil mailing list<br><a href="mailto:ftputil@codespeak.net">ftputil@codespeak.net</a><br><a href="http://codespeak.net/mailman/listinfo/ftputil">http://codespeak.net/mailman/listinfo/ftputil
</a><br></blockquote></div><br>