From roger.demetrescu at gmail.com Wed Jul 23 14:29:33 2008 From: roger.demetrescu at gmail.com (Roger Demetrescu) Date: Wed, 23 Jul 2008 09:29:33 -0300 Subject: [ftputil] Implementing PEP 343: The 'with' statement Message-ID: <581f62ff0807230529l396eafdex96b5b815099ee2c8@mail.gmail.com> Hi, I have a suggestion: FTPHost could implement __enter__ and __exit__ methods necessary to allow the use of `with` statement like that: with ftputil.FTPHost('host', 'username', 'password') as host: host.chdir('/some/path') # do some more operations Cheers, Roger From sschwarzer at sschwarzer.net Wed Jul 23 21:05:53 2008 From: sschwarzer at sschwarzer.net (Stefan Schwarzer) Date: Wed, 23 Jul 2008 21:05:53 +0200 Subject: [ftputil] Implementing PEP 343: The 'with' statement In-Reply-To: <581f62ff0807230529l396eafdex96b5b815099ee2c8@mail.gmail.com> References: <581f62ff0807230529l396eafdex96b5b815099ee2c8@mail.gmail.com> Message-ID: <48878111.80601@sschwarzer.net> Hi Roger, On 2008-07-23 14:29, Roger Demetrescu wrote: > I have a suggestion: FTPHost could implement __enter__ and __exit__ > methods necessary to allow the use of `with` statement like that: > > with ftputil.FTPHost('host', 'username', 'password') as host: > host.chdir('/some/path') > # do some more operations I like the idea very much. Two years ago, I asked for opinions on the minimal Python version that should work with ftputil [1]. At that time, we concluded to stick to Python 2.3 as requirement. I had pondered how to make statements of newer Python versions available without causing a syntax error or cyclic imports when ftputil is run on older Python versions and thought it wouldn't be possible (I now see a way). In the special case of support of the with statement, however, it seems that I can simply add __enter__ and __exit__ methods. I've filed an issue on the topic under [2]. [1] http://codespeak.net/pipermail/ftputil/2006q3/000096.html [2] http://ftputil.sschwarzer.net/trac/ticket/30 Stefan From rholden at movenetworks.com Wed Jul 30 01:19:34 2008 From: rholden at movenetworks.com (Richard Holden) Date: Tue, 29 Jul 2008 17:19:34 -0600 Subject: [ftputil] FTPHost.makedirs has a bug with non-root directories when invoked on a Windows machine Message-ID: Example FTP Layout: /data/test_results/ is the ftp users home directory. If I use FTPHost.makedirs() as FTPHost.makedirs('1234/') from a Windows machine it creates the /data/test_results/\data/test_results/1234/ directory. This is caused by using os.sep to reinsert the seperator lost by the split. I am attaching a patch with a testcase that fixes the problem for me. -Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/ftputil/attachments/20080729/0ffaea05/attachment.htm -------------- next part -------------- A non-text attachment was scrubbed... Name: makedirs.patch Type: application/octet-stream Size: 1233 bytes Desc: makedirs.patch Url : http://codespeak.net/pipermail/ftputil/attachments/20080729/0ffaea05/attachment.obj From sschwarzer at sschwarzer.net Wed Jul 30 21:20:04 2008 From: sschwarzer at sschwarzer.net (Stefan Schwarzer) Date: Wed, 30 Jul 2008 21:20:04 +0200 Subject: [ftputil] FTPHost.makedirs has a bug with non-root directories when invoked on a Windows machine In-Reply-To: References: Message-ID: <4890BEE4.2060006@sschwarzer.net> Hello Richard, On 2008-07-30 01:19, Richard Holden wrote: > If I use FTPHost.makedirs() as FTPHost.makedirs(?1234/?) from a Windows > machine it creates the > /data/test_results/\data/test_results/1234/ directory. > > This is caused by using os.sep to reinsert the seperator lost by the > split. I am attaching a patch with a testcase that fixes the problem for me. Thanks a lot for the bug report and the patch. You are right. I applied a slightly modified variant of the patch (used try...finally instead of try...except to make sure the separator is restored if no exception is raised) and checked it in. I wonder why this bug hasn't been noticed for so long. Do so few people use ftputil on Windows or has somebody ran into the bug but hasn't reported it? To all the readers of the list: If you find a bug, _please_ report it. It helps you and others who encounter it. Thanks in advance. :-) Actually, it seems there was a report on this bug as ticket #27 ( http://ftputil.sschwarzer.net/trac/ticket/27 ), but the description was so incomplete that I didn't look into it, only wrote a comment to be read by the original reporter so he/she could provide more information. Best regards, Stefan From roger.demetrescu at gmail.com Wed Aug 20 06:13:23 2008 From: roger.demetrescu at gmail.com (Roger Demetrescu) Date: Wed, 20 Aug 2008 01:13:23 -0300 Subject: [ftputil] Implementing PEP 343: The 'with' statement In-Reply-To: <48878111.80601@sschwarzer.net> References: <581f62ff0807230529l396eafdex96b5b815099ee2c8@mail.gmail.com> <48878111.80601@sschwarzer.net> Message-ID: <581f62ff0808192113p291d9557je16e3a4d2ef2dae1@mail.gmail.com> Hi Stefan Finally, my final path for `with` statement support. Feel free to modify it (like putting some decent docstrings).. :) I decided to put all those tests in a separate file, so users or developers using python < 2.5 won't suffer when running the unit tests. Cheers, Roger -------------- next part -------------- A non-text attachment was scrubbed... Name: final_with_support.diff Type: text/x-diff Size: 5634 bytes Desc: not available Url : http://codespeak.net/pipermail/ftputil/attachments/20080820/593a769d/attachment.bin From hancock.robert at gmail.com Fri Aug 29 06:05:16 2008 From: hancock.robert at gmail.com (Robert Hancock) Date: Fri, 29 Aug 2008 00:05:16 -0400 Subject: [ftputil] python stat versus FTPHost.stat Difference Message-ID: <613d56580808282105v13c8f41erd7a92f7671d3a57f@mail.gmail.com> Python 2.5.2 (r252:60911, Aug 28 2008, 23:51:17) [GCC 4.3.0 20080428 (Red Hat 4.3.0-8)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import ftputil >>> host = ftputil.FTPHost('localhost', 'rhancock', 'test') >>> res = host.stat('/local/data/proftp/proftp.log') >>> res[8] 1188371100.0 >>> import os >>> res = os.stat('/local/data/proftp/proftp.log') >>> res[8] 1219979117 >>> Issuing a stat on a local file from ftputil and from python produces two different results. Why? Given the today's date, the python stat produces the correct result. Is there something I need to set in ftputil? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/ftputil/attachments/20080829/4e067707/attachment.htm From sschwarzer at sschwarzer.net Fri Aug 29 18:55:22 2008 From: sschwarzer at sschwarzer.net (Stefan Schwarzer) Date: Fri, 29 Aug 2008 18:55:22 +0200 Subject: [ftputil] python stat versus FTPHost.stat Difference In-Reply-To: <613d56580808282105v13c8f41erd7a92f7671d3a57f@mail.gmail.com> References: <613d56580808282105v13c8f41erd7a92f7671d3a57f@mail.gmail.com> Message-ID: <48B829FA.3040100@sschwarzer.net> Hello Robert, Thanks for your report! On 2008-08-29 06:05, Robert Hancock wrote: > Python 2.5.2 (r252:60911, Aug 28 2008, 23:51:17) > [GCC 4.3.0 20080428 (Red Hat 4.3.0-8)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> import ftputil >>>> host = ftputil.FTPHost('localhost', 'rhancock', 'test') >>>> res = host.stat('/local/data/proftp/proftp.log') >>>> res[8] > 1188371100.0 >>>> import os >>>> res = os.stat('/local/data/proftp/proftp.log') >>>> res[8] > 1219979117 This is quite interesting. That's a difference of "exactly" one year (365.83 days, if my calculation is correct). Could you please log in to the FTP server with an FTP command line client (if you hopefully have one) and send the output of a DIR command of the directory which contains the file? I would like to know if ftputil's parsing of the data fails or if the data going into the parser already isn't as expected. > Issuing a stat on a local file from ftputil and from python produces two > different results. Why? I don't know yet. :-) Please send me the output I asked for above. :) > Given the today's date, the python stat produces the correct result. Is > there something I need to set in ftputil? First, you may have to account for time shift between server and client (the computer running ftputil) by using FTPHost.synchronize_times or setting the time shift "manually" with FTPHost.set_time_shift (see the documentation for details). To my surprise, I found that you can have time shift issues even if server and client run on the same host: After an update some time ago, my local PureFTPd server always used UTC times in directory listings while the client code to test ftputil expected the local time zone (Middle European Time with DST). Second, depending on the age of the file on the server, it may return only the date, not the time, of the file. In that case, the best ftputil can do is to convert just this date to a float. This can result in stat'ed timestamp differences between the local and the remote file of up to a day! For instance, if ftputil gets a file's timestamp as "May 29 2000", it can't know if the modification _time_ actually was 0:01 or 23:59. Best regards, Stefan From sschwarzer at sschwarzer.net Sat Aug 30 22:06:41 2008 From: sschwarzer at sschwarzer.net (Stefan Schwarzer) Date: Sat, 30 Aug 2008 22:06:41 +0200 Subject: [ftputil] [ANN] ftputil 2.2.4 released Message-ID: <48B9A851.5040809@sschwarzer.net> ftputil 2.2.4 is now available from http://ftputil.sschwarzer.net/download . Changes since version 2.2.3 --------------------------- This release fixes a bug in the ``makedirs`` call (report and fix by Richard Holden). Upgrading is recommended. What is ftputil? ---------------- ftputil is a high-level FTP client library for the Python programming language. ftputil implements a virtual file system for accessing FTP servers, that is, it can generate file-like objects for remote files. The library supports many functions similar to those in the os, os.path and shutil modules. ftputil has convenience functions for conditional uploads and downloads, and handles FTP clients and servers in different timezones. Read the documentation at http://ftputil.sschwarzer.net/documentation . License ------- ftputil is Open Source software, released under the revised BSD license (see http://www.opensource.org/licenses/bsd-license.php ). Stefan From sschwarzer at sschwarzer.net Sat Aug 30 23:30:34 2008 From: sschwarzer at sschwarzer.net (Stefan Schwarzer) Date: Sat, 30 Aug 2008 23:30:34 +0200 Subject: [ftputil] python stat versus FTPHost.stat Difference In-Reply-To: <613d56580808301337v249cdac8mde1331c94732c492@mail.gmail.com> References: <613d56580808282105v13c8f41erd7a92f7671d3a57f@mail.gmail.com> <48B829FA.3040100@sschwarzer.net> <613d56580808301337v249cdac8mde1331c94732c492@mail.gmail.com> Message-ID: <48B9BBFA.8080006@sschwarzer.net> Hi Robert, On 2008-08-30 22:37, Robert Hancock wrote: >> This is quite interesting. That's a difference of "exactly" one >> year (365.83 days, if my calculation is correct). Could you >> please log in to the FTP server with an FTP command line client >> (if you hopefully have one) and send the output of a DIR command >> of the directory which contains the file? I would like to know if >> ftputil's parsing of the data fails or if the data going into the >> parser already isn't as expected. > > ftp> dir > 227 Entering Passive Mode (127,0,0,1,41,19). > 150 Here comes the directory listing. > drwxrwxr-x 2 500 500 24576 Aug 29 03:06 archive > -rw-r--r-- 1 500 500 4149948 Aug 29 03:05 proftp.log > 226 Directory send OK. > ftp> by > 221 Goodbye. Parsing seems to work: >>> import ftputil >>> import ftp_stat >>> up = ftp_stat.UnixParser() >>> up.parse_line("-rw-r--r-- 1 500 500 4149948 Aug 29 03:05 proftp.log") (33188, None, None, 1, '500', '500', 4149948, None, 1219971900.0, None) >>> t = _ >>> import time >>> time.localtime(t[-2]) (2008, 8, 29, 3, 5, 0, 4, 242, 1) > $ ls -la > total 4092 > drwxrwxr-x 3 rhancock rhancock 4096 2008-08-28 22:15 . > drwxrwxr-x 3 rhancock rhancock 4096 2008-08-28 21:48 .. > drwxrwxr-x 2 rhancock rhancock 24576 2008-08-28 23:06 archive > -rw-r--r-- 1 rhancock rhancock 4149948 2008-08-28 23:05 proftp.log Ah! This seems to be a time shift issue. The server time is four hours ahead of the client time, and I guess you haven't done anything (yet) to deal with time shift. Now, ftputil gets the directory listing from the server, and because the file's date on the server is "Aug 29" _and_ on the client August 29 _hasn't yet arrived_, ftputil assumes "Aug 29" must be August 29 in 2007! (In my experience, the directory output is made up according to this same rule, so I have implemented the parser that way.) This makes the one-year difference. Please read the ftputil documentation and try to use the time-shift-related commands. Please ask on the mailing list if any more difficulties come up. >> To my surprise, I found that you can have time shift issues even >> if server and client run on the same host: After an update some >> time ago, my local PureFTPd server always used UTC times in >> directory listings while the client code to test ftputil expected >> the local time zone (Middle European Time with DST). > > This will take me a while to investigate. I'm using the command line > utility of Fedora 9 to connect to vsfttp on the same machine. I have no problem with that. :) Do you have any objections to having me send your mail and my reply to the ftputil list. You (perhaps accidentally) sent your last mail only to me privately. Best regards, Stefan From sschwarzer at sschwarzer.net Sat Sep 6 22:04:55 2008 From: sschwarzer at sschwarzer.net (Stefan Schwarzer) Date: Sat, 06 Sep 2008 22:04:55 +0200 Subject: [ftputil] [ANN] ftputil 2.3 released Message-ID: <48C2E267.9080308@sschwarzer.net> ftputil 2.3 is now available from http://ftputil.sschwarzer.net/download . Changes since version 2.2.4 --------------------------- ftputil has got support for the ``with`` statement which was introduced by Python 2.5. You can now construct host and remote file objects in ``with`` statements and have them closed automatically (contributed by Roger Demetrescu). See the documentation for examples. What is ftputil? ---------------- ftputil is a high-level FTP client library for the Python programming language. ftputil implements a virtual file system for accessing FTP servers, that is, it can generate file-like objects for remote files. The library supports many functions similar to those in the os, os.path and shutil modules. ftputil has convenience functions for conditional uploads and downloads, and handles FTP clients and servers in different timezones. Read the documentation at http://ftputil.sschwarzer.net/documentation . License ------- ftputil is Open Source software, released under the revised BSD license (see http://www.opensource.org/licenses/bsd-license.php ). Stefan