From fbuchinger at gmail.com Fri Oct 19 22:21:08 2007 From: fbuchinger at gmail.com (Franz Buchinger) Date: Fri, 19 Oct 2007 22:21:08 +0200 Subject: [ftputil] FTP Transfer Progress // Multiple Threads Message-ID: Hello, thanks for this fascinating library! After reading the documentation, two questions remain: How can I measure the progress of an FTP down/upload? Can I do this with a callback function that is triggered by the FTPHost instance or are there other ways? As far as I understood the docs, FTPUtil is not 100% thread-safe, but I can have multiple file transfers at once by spawning a download thread for each FTPUtil file object. Is this correct? Thanx for your replies! Franz -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/ftputil/attachments/20071019/1c997e6e/attachment.htm From sschwarzer at sschwarzer.net Sat Oct 20 21:46:58 2007 From: sschwarzer at sschwarzer.net (Stefan Schwarzer) Date: Sat, 20 Oct 2007 21:46:58 +0200 Subject: [ftputil] FTP Transfer Progress // Multiple Threads In-Reply-To: References: Message-ID: <471A5B32.5010208@sschwarzer.net> Hello Franz, On 2007-10-19 22:21, Franz Buchinger wrote: > thanks for this fascinating library! :-) > How can I measure the progress of an FTP down/upload? Can I do this with > a callback function that is triggered by the FTPHost instance or are > there other ways? Unfortunately, there's no elegant way to track the progress of uploads/downloads. A workaround could be to "stat" the target file at certain intervals and use the results. Besides, there's actually a ticket on the subject ( http://ftputil.sschwarzer.net/trac/ticket/20 ), so I should work on this when I can get some time for it. > As far as I understood the docs, FTPUtil is not 100% thread-safe, but I > can have multiple file transfers at once by spawning a download thread > for each FTPUtil file object. Is this correct? If you use a distinct instance of FTPHost for each thread, that should be safe. I hope this helps a bit. Feel free to ask if you have any more questions. Regards, Stefan From sschwarzer at sschwarzer.net Sun Oct 21 10:02:31 2007 From: sschwarzer at sschwarzer.net (Stefan Schwarzer) Date: Sun, 21 Oct 2007 10:02:31 +0200 Subject: [ftputil] FTP Transfer Progress // Multiple Threads In-Reply-To: <471A5B32.5010208@sschwarzer.net> References: <471A5B32.5010208@sschwarzer.net> Message-ID: <471B0797.9070203@sschwarzer.net> Hi Franz, On 2007-10-20 21:46, Stefan Schwarzer wrote: > Unfortunately, there's no elegant way to track the progress of > uploads/downloads. A workaround could be to "stat" the target file > at certain intervals and use the results. To get the actual size of the file, remember to invalidate its path in the cache before calling the stat method [1]. Otherwise you would always get the cached value. [1] http://ftputil.sschwarzer.net/trac/wiki/Documentation#local-caching-of-file-system-information Regards, Stefan From fbuchinger at gmail.com Sat Nov 17 18:36:35 2007 From: fbuchinger at gmail.com (Franz Buchinger) Date: Sat, 17 Nov 2007 18:36:35 +0100 Subject: [ftputil] "Conditional" FTP Login Message-ID: Hello, I'm currently developing a script that connects to ftp servers using ftputil. i want to implement a "conditional" ftp login: if the ftp server is anonymous, the script should connect automatically. if the server requires a login, the user should be prompted to raw_input his credentials. I suppose I have to create a SessionFactory for this... can someone give further hints? thanks, Franz -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/ftputil/attachments/20071117/4f9cf968/attachment.htm From sschwarzer at sschwarzer.net Sun Nov 18 10:50:11 2007 From: sschwarzer at sschwarzer.net (Stefan Schwarzer) Date: Sun, 18 Nov 2007 10:50:11 +0100 Subject: [ftputil] "Conditional" FTP Login In-Reply-To: References: Message-ID: <47400AD3.80300@sschwarzer.net> Hello Franz, On 2007-11-17 18:36, Franz Buchinger wrote: > I'm currently developing a script that connects to ftp servers using > ftputil. i want to implement a "conditional" ftp login: if the ftp > server is anonymous, the script should connect automatically. if the > server requires a login, the user should be prompted to raw_input his > credentials. > > I suppose I have to create a SessionFactory for this... can someone give > further hints? depending on your needs, you may not need a SessionFactory. Possibly approach (untested): import getpass import ftputil from ftputil import ftp_error try: # try anonymous access ftp_host = ftputil.FTPHost(server, "anonymous", email_address) except ftp_error.PermanentError: # login failed, so try with userid and password userid = raw_input("Userid: ") password = getpass.getpass("Password: ") ftp_host = ftputil.FTPHost(server, userid, password) On the other hand, there may be other reasons for the first login attempt to fail with a PermanentError. So you would have to catch with "except ftp_error.PermanentError, exception" and examine (e. g. parse) the exception object. Depending on the server, though, there could be other exception classes. For "my" server software, PureFTPd, I get a general FTPOSError if I try an anonymous login. What are your results for the servers you tried? In summary, there doesn't seem to be an approach that is easy and robust at the same time. You should experiment with variants of the code above. Best regards, Stefan From smap2d at gmail.com Wed Nov 28 18:36:09 2007 From: smap2d at gmail.com (Chad Moore) Date: Wed, 28 Nov 2007 12:36:09 -0500 Subject: [ftputil] Newbie FTP question Message-ID: Hello all, I am wondering how to simply download an entire directory consisting of subdirectories and files. I am sure I can figure out how to "walk" through the directory structure, but isn't there an easier way to just suck down an entire direcotry from the FTP site? Here is my code and details. - - - - Description - - - - Log into FTP If a folder with todays date exists, download the entire directory to a local drectory If it doesnt exist return "nothing ready today" - - - - Error - - - - ftputil.ftp_error.FTPIOError: 550 2007_11_28/: Not a regular file - - - - Code - - - - import ftputil import os import sys from time import strftime from nt import mkdir localPath = "C:/Documents and Settings/USERNAME/My Documents/Downloads" #ftp host, login and password host = ftputil.FTPHost('ftp.host.com', 'user', 'pass') #host.set_directory_format("unix") #directories on the FTP site filesToDownloadFolder = 'subfolder01/subfolder02' host.chdir(filesToDownloadFolder) #get todays date todaysDate = strftime("%Y_%m_%d/") ##if folder exists, download it and send email notification. if not, send email, "nothing ready for today" host.download(todaysDate, localPath, 'b') # remote, local, binary mode host.close() Thanks, Chad -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/ftputil/attachments/20071128/6c289ff0/attachment.htm From sschwarzer at sschwarzer.net Wed Nov 28 19:16:28 2007 From: sschwarzer at sschwarzer.net (Stefan Schwarzer) Date: Wed, 28 Nov 2007 19:16:28 +0100 Subject: [ftputil] Newbie FTP question In-Reply-To: References: Message-ID: <474DB07C.6020201@sschwarzer.net> Hi Chad, On 2007-11-28 18:36, Chad Moore wrote: > I am wondering how to simply download an entire directory consisting of > subdirectories and files. I am sure I can figure out how to "walk" > through the directory structure, but isn't there an easier way to just > suck down an entire direcotry from the FTP site? There's some code for copying an entire directory tree in the ftputil source code repository: http://ftputil.sschwarzer.net/trac/browser/trunk/ftp_sync.py See http://ftputil.sschwarzer.net/trac/browser/trunk/_test_ftp_sync.py for the usage semantics. Due to the design, you can use the code to copy trees from local to local, local to remote, remote to local and remote to remote. As the name of the module implies, it's intended to become a syncing library, i. e. later it should only copy the files which aren't present in the target directory or have an older date/time. This would resolve issue #6 (http://ftputil.sschwarzer.net/trac/ticket/6). However, I'm very busy at the moment and haven't worked on the code for some time. Sorry. > Here is my code and > details. My unqualified comments on your code follow. :-) > - - - - Description - - - - > Log into FTP > If a folder with todays date exists, download the entire directory to a > local drectory > If it doesnt exist return "nothing ready today" > > - - - - Error - - - - > ftputil.ftp_error.FTPIOError: 550 2007_11_28/: Not a regular file > > - - - - Code - - - - > > import ftputil > import os > import sys > from time import strftime > from nt import mkdir Why don't you use mkdir from the os module? This should wrap platform differences. > localPath = "C:/Documents and Settings/USERNAME/My Documents/Downloads" > > #ftp host, login and password > host = ftputil.FTPHost(' ftp.host.com ', 'user', > 'pass') This host argument won't work, probably you didn't use something like "host " but just "ftp.host.com"? > #host.set_directory_format("unix") Explicitly setting the directory format is deprecated with newer versions of ftputil. You should set the parser if necessary (see the documentation on the website). Which version of ftputil do you use? > #directories on the FTP site > filesToDownloadFolder = 'subfolder01/subfolder02' > host.chdir(filesToDownloadFolder) > > #get todays date > todaysDate = strftime("%Y_%m_%d/") > ##if folder exists, download it and send email notification. if not, > send email, "nothing ready for today" > > host.download(todaysDate, localPath, 'b') # remote, local, binary mode Unfortunately, the download method only works on files and links to them, not on directories. Best regards, Stefan