From trac at ftputil.sschwarzer.net Sun Jan 2 18:32:03 2011 From: trac at ftputil.sschwarzer.net (ftputil) Date: Sun, 02 Jan 2011 17:32:03 -0000 Subject: [ftputil] #38: If cache size is zero, FTPHost.listdir causes an IndexError In-Reply-To: <046.a895107faa36162a3070c91517bab32b@ftputil.sschwarzer.net> References: <046.a895107faa36162a3070c91517bab32b@ftputil.sschwarzer.net> Message-ID: <055.24db3d9c2ac9537bf58b50ab322135d9@ftputil.sschwarzer.net> #38: If cache size is zero, FTPHost.listdir causes an IndexError ----------------------+----------------------------------------------------- Reporter: schwa | Owner: schwa Type: defect | Status: closed Priority: minor | Milestone: 2.4.2 Component: Library | Version: 2.4.2b Resolution: fixed | Keywords: cache, resize, IndexError ----------------------+----------------------------------------------------- Comment(by schwa): I changed the policy for zero size caches in [2640715ee9aa]. They're no longer allowed. Using a cache size of zero never was intuitive. Actually, it was only used to implicitly clear the cache. This now can be done with a new `clear` method. -- Ticket URL: ftputil Python FTP client library From trac at ftputil.sschwarzer.net Sun Jan 2 19:06:37 2011 From: trac at ftputil.sschwarzer.net (ftputil) Date: Sun, 02 Jan 2011 18:06:37 -0000 Subject: [ftputil] #54: Auto-increment functionality for stat cache (was: Auto-resizing functionality for stat cache) In-Reply-To: <046.ebc5bbcaac08ae35093137780a752e21@ftputil.sschwarzer.net> References: <046.ebc5bbcaac08ae35093137780a752e21@ftputil.sschwarzer.net> Message-ID: <055.bbac7562ae5c46a26f663cf7e50bb026@ftputil.sschwarzer.net> #54: Auto-increment functionality for stat cache --------------------------+------------------------------------------------- Reporter: schwa | Owner: schwa Type: enhancement | Status: closed Priority: major | Milestone: 2.6 Component: Library | Version: 2.5 Resolution: fixed | Keywords: cache, flushing, default size, stat --------------------------+------------------------------------------------- Comment(by schwa): Actually this feature should be called "auto-increment", not "auto- resize". The cache size is never decreased automatically, but can be set with `FTPHost.stat_cache.resize`. Decreasing the cache size automatically could counter all the positive effects of the increase as ftputil can't know if the cached entries are still needed. -- Ticket URL: ftputil Python FTP client library From trac at ftputil.sschwarzer.net Wed Jan 12 22:29:57 2011 From: trac at ftputil.sschwarzer.net (ftputil) Date: Wed, 12 Jan 2011 21:29:57 -0000 Subject: [ftputil] #55: synchronize_times() fails in western time zones when server GMT times Message-ID: <052.34308dce4002595061d1984fcf6ec2e9@ftputil.sschwarzer.net> #55: synchronize_times() fails in western time zones when server GMT times -----------------------------------+---------------------------------------- Reporter: ftputiluser | Owner: schwa Type: defect | Status: new Priority: major | Milestone: Component: Library | Version: 2.5 Keywords: GMT, mtime, assertion | -----------------------------------+---------------------------------------- synchronize_times() must fail with assertion ftputil.ftp_error.TimeShiftError: time shift (-31514426.37 s) > 1 day Debugging info: ftputil 2.5, Python 2.7 (win32) when the client and/or server are in western time zones and the server reports file/dir time as GMT. After creating a test file, synchronize_times() calls server_time = self.path.getmtime(helper_file_name) to get the server-relative time. getmtime() uses the normal functions to develop the time value. Within parse_unix_time() the code attempts to figure out which year a directory line references, e.g. line = -rw-r--r-- 1 500 500 0 Jan 12 20:43 _ftputil_sync_ when no year value is present in a listing line like above. If the time value developed by the code assuming "this year" is some impossible *future* time value, then it decides the correct year must be "last year". The code uses if st_mtime > time.time() + time_shift + 60.0: as a check for whether the year is correct. But if the server time is GMT, and the local client is two or more time zones west of GMT, then this check will always be true, and the year return will *always* be last year, and the __assert_valid_time_shift() rule 1 will always fail. As a check, I set the time_shift() value correctly, and then the problem didn't assert. I got some snapshots while stepping through the code, and for the directory line above. (Pdb) year, month, day, hour, minute (2011, 1, 12, 20, 43) (Pdb) st_mtime, time.asctime(time.localtime(st_mtime)) (1294886580.0, 'Wed Jan 12 20:43:00 2011') 197 -> if st_mtime > time.time() + time_shift + 60.0: (Pdb) st_mtime, time.time(), time_shift (1294886580.0, 1294865365.84, 0.0) 198 -> st_mtime = time.mktime( (year-1, month, day, 199 hour, minute, 0, 0, 0, -1) ) (Pdb) st_mtime, time.asctime(time.localtime(st_mtime)) (1263350580.0, 'Tue Jan 12 20:43:00 2010') ^^^^ I've seen mention of these symptoms with ftputil using Google, but don't remember where. I hope this explains the scenario. Maybe I'll have more insight to a solution later. -- Ticket URL: ftputil Python FTP client library From trac at ftputil.sschwarzer.net Wed Jan 12 22:57:11 2011 From: trac at ftputil.sschwarzer.net (ftputil) Date: Wed, 12 Jan 2011 21:57:11 -0000 Subject: [ftputil] #55: synchronize_times() fails in western time zones when server GMT times In-Reply-To: <052.34308dce4002595061d1984fcf6ec2e9@ftputil.sschwarzer.net> References: <052.34308dce4002595061d1984fcf6ec2e9@ftputil.sschwarzer.net> Message-ID: <061.7df9d0a0118946f0150c51a3bfa7cfed@ftputil.sschwarzer.net> #55: synchronize_times() fails in western time zones when server GMT times --------------------------+------------------------------------------------- Reporter: ftputiluser | Owner: schwa Type: defect | Status: new Priority: major | Milestone: Component: Library | Version: 2.5 Resolution: | Keywords: GMT, mtime, assertion --------------------------+------------------------------------------------- Old description: > synchronize_times() must fail with assertion > > ftputil.ftp_error.TimeShiftError: time shift (-31514426.37 s) > 1 day > Debugging info: ftputil 2.5, Python 2.7 (win32) > > when the client and/or server are in western time zones and the server > reports file/dir time as GMT. > > After creating a test file, synchronize_times() calls > server_time = self.path.getmtime(helper_file_name) > to get the server-relative time. getmtime() uses the normal functions to > develop the time value. > > Within parse_unix_time() the code attempts to figure out which year a > directory line references, e.g. > line = -rw-r--r-- 1 500 500 0 Jan 12 20:43 > _ftputil_sync_ > when no year value is present in a listing line like above. If the time > value developed by the code assuming "this year" is some impossible > *future* time value, then it decides the correct year must be "last > year". > > The code uses > if st_mtime > time.time() + time_shift + 60.0: > as a check for whether the year is correct. But if the server time is > GMT, and the local client is two or more time zones west of GMT, then > this check will always be true, and the year return will *always* be last > year, and the __assert_valid_time_shift() rule 1 will always fail. > > As a check, I set the time_shift() value correctly, and then the problem > didn't assert. > > I got some snapshots while stepping through the code, and for the > directory line above. > (Pdb) year, month, day, hour, minute > (2011, 1, 12, 20, 43) > > (Pdb) st_mtime, time.asctime(time.localtime(st_mtime)) > (1294886580.0, 'Wed Jan 12 20:43:00 2011') > > 197 -> if st_mtime > time.time() + time_shift + 60.0: > (Pdb) st_mtime, time.time(), time_shift > (1294886580.0, 1294865365.84, 0.0) > 198 -> st_mtime = time.mktime( (year-1, month, day, > 199 hour, minute, 0, 0, 0, > -1) ) > > (Pdb) st_mtime, time.asctime(time.localtime(st_mtime)) > (1263350580.0, 'Tue Jan 12 20:43:00 2010') > ^^^^ > I've seen mention of these symptoms with ftputil using Google, but don't > remember where. I hope this explains the scenario. Maybe I'll have more > insight to a solution later. New description: synchronize_times() must fail with assertion {{{ ftputil.ftp_error.TimeShiftError: time shift (-31514426.37 s) > 1 day Debugging info: ftputil 2.5, Python 2.7 (win32) }}} when the client and/or server are in western time zones and the server reports file/dir time as GMT. After creating a test file, synchronize_times() calls {{{ server_time = self.path.getmtime(helper_file_name) }}} to get the server-relative time. getmtime() uses the normal functions to develop the time value. Within parse_unix_time() the code attempts to figure out which year a directory line references, e.g. {{{ line = -rw-r--r-- 1 500 500 0 Jan 12 20:43 _ftputil_sync_ }}} when no year value is present in a listing line like above. If the time value developed by the code assuming "this year" is some impossible *future* time value, then it decides the correct year must be "last year". The code uses {{{ if st_mtime > time.time() + time_shift + 60.0: }}} as a check for whether the year is correct. But if the server time is GMT, and the local client is two or more time zones west of GMT, then this check will always be true, and the year return will *always* be last year, and the __assert_valid_time_shift() rule 1 will always fail. As a check, I set the time_shift() value correctly, and then the problem didn't assert. I got some snapshots while stepping through the code, and for the directory line above. {{{ (Pdb) year, month, day, hour, minute (2011, 1, 12, 20, 43) (Pdb) st_mtime, time.asctime(time.localtime(st_mtime)) (1294886580.0, 'Wed Jan 12 20:43:00 2011') 197 -> if st_mtime > time.time() + time_shift + 60.0: (Pdb) st_mtime, time.time(), time_shift (1294886580.0, 1294865365.84, 0.0) 198 -> st_mtime = time.mktime( (year-1, month, day, 199 hour, minute, 0, 0, 0, -1) ) (Pdb) st_mtime, time.asctime(time.localtime(st_mtime)) (1263350580.0, 'Tue Jan 12 20:43:00 2010') ^^^^ }}} I've seen mention of these symptoms with ftputil using Google, but don't remember where. I hope this explains the scenario. Maybe I'll have more insight to a solution later. -- Comment(by schwa): Many thanks for the detailed report! :-) (I added some code markup to the description for better readability.) -- Ticket URL: ftputil Python FTP client library From trac at ftputil.sschwarzer.net Thu Jan 13 18:30:12 2011 From: trac at ftputil.sschwarzer.net (ftputil) Date: Thu, 13 Jan 2011 17:30:12 -0000 Subject: [ftputil] #55: synchronize_times() fails in western time zones when server GMT times In-Reply-To: <052.34308dce4002595061d1984fcf6ec2e9@ftputil.sschwarzer.net> References: <052.34308dce4002595061d1984fcf6ec2e9@ftputil.sschwarzer.net> Message-ID: <061.096eb7b2a1e755303122f8b28fde54be@ftputil.sschwarzer.net> #55: synchronize_times() fails in western time zones when server GMT times --------------------------+------------------------------------------------- Reporter: ftputiluser | Owner: schwa Type: defect | Status: assigned Priority: major | Milestone: Component: Library | Version: 2.5 Resolution: | Keywords: GMT, mtime, assertion --------------------------+------------------------------------------------- Changes (by schwa): * status: new => assigned Old description: > synchronize_times() must fail with assertion > {{{ > ftputil.ftp_error.TimeShiftError: time shift (-31514426.37 s) > 1 day > Debugging info: ftputil 2.5, Python 2.7 (win32) > }}} > when the client and/or server are in western time zones and the server > reports file/dir time as GMT. > > After creating a test file, synchronize_times() calls > {{{ > server_time = self.path.getmtime(helper_file_name) > }}} > to get the server-relative time. getmtime() uses the normal functions to > develop the time value. > > Within parse_unix_time() the code attempts to figure out which year a > directory line references, e.g. > {{{ > line = -rw-r--r-- 1 500 500 0 Jan 12 20:43 > _ftputil_sync_ > }}} > when no year value is present in a listing line like above. If the time > value developed by the code assuming "this year" is some impossible > *future* time value, then it decides the correct year must be "last > year". > > The code uses > {{{ > if st_mtime > time.time() + time_shift + 60.0: > }}} > as a check for whether the year is correct. But if the server time is > GMT, and the local client is two or more time zones west of GMT, then > this check will always be true, and the year return will *always* be last > year, and the __assert_valid_time_shift() rule 1 will always fail. > > As a check, I set the time_shift() value correctly, and then the problem > didn't assert. > > I got some snapshots while stepping through the code, and for the > directory line above. > {{{ > (Pdb) year, month, day, hour, minute > (2011, 1, 12, 20, 43) > > (Pdb) st_mtime, time.asctime(time.localtime(st_mtime)) > (1294886580.0, 'Wed Jan 12 20:43:00 2011') > > 197 -> if st_mtime > time.time() + time_shift + 60.0: > (Pdb) st_mtime, time.time(), time_shift > (1294886580.0, 1294865365.84, 0.0) > 198 -> st_mtime = time.mktime( (year-1, month, day, > 199 hour, minute, 0, 0, 0, > -1) ) > > (Pdb) st_mtime, time.asctime(time.localtime(st_mtime)) > (1263350580.0, 'Tue Jan 12 20:43:00 2010') > ^^^^ > }}} > > I've seen mention of these symptoms with ftputil using Google, but don't > remember where. I hope this explains the scenario. Maybe I'll have more > insight to a solution later. New description: synchronize_times() must fail with assertion {{{ ftputil.ftp_error.TimeShiftError: time shift (-31514426.37 s) > 1 day Debugging info: ftputil 2.5, Python 2.7 (win32) }}} when the client and/or server are in western time zones and the server reports file/dir time as GMT. After creating a test file, synchronize_times() calls {{{ server_time = self.path.getmtime(helper_file_name) }}} to get the server-relative time. getmtime() uses the normal functions to develop the time value. Within parse_unix_time() the code attempts to figure out which year a directory line references, e.g. {{{ line = -rw-r--r-- 1 500 500 0 Jan 12 20:43 _ftputil_sync_ }}} when no year value is present in a listing line like above. If the time value developed by the code assuming "this year" is some impossible *future* time value, then it decides the correct year must be "last year". The code uses {{{ if st_mtime > time.time() + time_shift + 60.0: }}} as a check for whether the year is correct. But if the server time is GMT, and the local client is two or more time zones west of GMT, then this check will always be true, and the year return will *always* be last year, and the `__assert_valid_time_shift()` rule 1 will always fail. As a check, I set the time_shift() value correctly, and then the problem didn't assert. I got some snapshots while stepping through the code, and for the directory line above. {{{ (Pdb) year, month, day, hour, minute (2011, 1, 12, 20, 43) (Pdb) st_mtime, time.asctime(time.localtime(st_mtime)) (1294886580.0, 'Wed Jan 12 20:43:00 2011') 197 -> if st_mtime > time.time() + time_shift + 60.0: (Pdb) st_mtime, time.time(), time_shift (1294886580.0, 1294865365.84, 0.0) 198 -> st_mtime = time.mktime( (year-1, month, day, 199 hour, minute, 0, 0, 0, -1) ) (Pdb) st_mtime, time.asctime(time.localtime(st_mtime)) (1263350580.0, 'Tue Jan 12 20:43:00 2010') ^^^^ }}} I've seen mention of these symptoms with ftputil using Google, but don't remember where. I hope this explains the scenario. Maybe I'll have more insight to a solution later. -- Comment: Replying to [ticket:55 ftputiluser]: > The code uses `if st_mtime > time.time() + time_shift + 60.0:` as a check for whether the year is correct. But if the server time is GMT, and the local client is two or more time zones west of GMT, then this check will always be true, and the year return will *always* be last year, and the `__assert_valid_time_shift()` rule 1 will always fail. The problem is that `synchronize_times` calls `getmtime` which calls `parse_unix_time` - but `parse_unix_time` uses the very time shift value that `synchronize_times` is supposed to set in the first place. So the time shift value at the time of the year determination is still 0.0, regardless of the actual time zone difference. This, in turn, makes `parse_unix_time` conclude a wrong year. > As a check, I set the time_shift() value correctly, and then the problem didn't assert. This supports my interpretation nicely. :-) -- Ticket URL: ftputil Python FTP client library From trac at ftputil.sschwarzer.net Thu Jan 13 20:28:24 2011 From: trac at ftputil.sschwarzer.net (ftputil) Date: Thu, 13 Jan 2011 19:28:24 -0000 Subject: [ftputil] #55: synchronize_times() fails in western time zones when server GMT times In-Reply-To: <052.34308dce4002595061d1984fcf6ec2e9@ftputil.sschwarzer.net> References: <052.34308dce4002595061d1984fcf6ec2e9@ftputil.sschwarzer.net> Message-ID: <061.48d1faca33ed5a590cb31f88f0567ea3@ftputil.sschwarzer.net> #55: synchronize_times() fails in western time zones when server GMT times --------------------------+------------------------------------------------- Reporter: ftputiluser | Owner: schwa Type: defect | Status: assigned Priority: major | Milestone: 2.6 Component: Library | Version: 2.5 Resolution: | Keywords: GMT, mtime, assertion --------------------------+------------------------------------------------- Changes (by schwa): * milestone: => 2.6 Comment: The bug should be fixed in changeset [6d59213b9b9a]. It would be great if you could report if the problem is fixed for you. -- Ticket URL: ftputil Python FTP client library From trac at ftputil.sschwarzer.net Thu Jan 13 21:05:04 2011 From: trac at ftputil.sschwarzer.net (ftputil) Date: Thu, 13 Jan 2011 20:05:04 -0000 Subject: [ftputil] #55: synchronize_times() may fail when the server is east of the client (was: synchronize_times() fails in western time zones when server GMT times) In-Reply-To: <052.34308dce4002595061d1984fcf6ec2e9@ftputil.sschwarzer.net> References: <052.34308dce4002595061d1984fcf6ec2e9@ftputil.sschwarzer.net> Message-ID: <061.781f98570bed3c5681fa6b3b11ffa3cb@ftputil.sschwarzer.net> #55: synchronize_times() may fail when the server is east of the client --------------------------+------------------------------------------------- Reporter: ftputiluser | Owner: schwa Type: defect | Status: assigned Priority: major | Milestone: 2.6 Component: Library | Version: 2.5 Resolution: | Keywords: GMT, mtime, assertion --------------------------+------------------------------------------------- Comment(by schwa): To help other users finding a solution to similar problems, I changed the title to reflect that the server doesn't have to be in the GMT time zone. -- Ticket URL: ftputil Python FTP client library From trac at ftputil.sschwarzer.net Fri Jan 28 19:54:03 2011 From: trac at ftputil.sschwarzer.net (ftputil) Date: Fri, 28 Jan 2011 18:54:03 -0000 Subject: [ftputil] #55: synchronize_times() may fail when the server is east of the client In-Reply-To: <052.34308dce4002595061d1984fcf6ec2e9@ftputil.sschwarzer.net> References: <052.34308dce4002595061d1984fcf6ec2e9@ftputil.sschwarzer.net> Message-ID: <061.4c01bde85412b8b9b6a52faff9b3c0bd@ftputil.sschwarzer.net> #55: synchronize_times() may fail when the server is east of the client --------------------------+------------------------------------------------- Reporter: ftputiluser | Owner: schwa Type: defect | Status: closed Priority: major | Milestone: 2.6 Component: Library | Version: 2.5 Resolution: fixed | Keywords: GMT, mtime, assertion --------------------------+------------------------------------------------- Changes (by schwa): * status: assigned => closed * resolution: => fixed Comment: The problem seems to be fixed according to an e-mail by the reporter of this ticket. Thanks! -- Ticket URL: ftputil Python FTP client library From trac at ftputil.sschwarzer.net Wed Feb 16 01:42:08 2011 From: trac at ftputil.sschwarzer.net (ftputil) Date: Wed, 16 Feb 2011 00:42:08 -0000 Subject: [ftputil] #56: FTPHost.__del__ and ftp_path._Path._host wreak havoc with Python's circular garbage collection Message-ID: <052.8c48af72ac536036bdb00aeac96d33ef@ftputil.sschwarzer.net> #56: FTPHost.__del__ and ftp_path._Path._host wreak havoc with Python's circular garbage collection -------------------------+-------------------------------------------------- Reporter: ftputiluser | Owner: schwa Type: defect | Status: new Priority: major | Milestone: Component: Library | Version: 2.5 Keywords: | -------------------------+-------------------------------------------------- Since FTPHost defines a __del__ method and the ftp_path._Path instances have a reference to the FTPHost, Python's circular reference garbage collection can't cleanup. See http://docs.python.org/reference/datamodel.html#object.__del__ for details. This could represent a significant memory leak for anyone using ftputil in a long running app. -- Ticket URL: ftputil Python FTP client library From trac at ftputil.sschwarzer.net Sat Feb 19 10:54:44 2011 From: trac at ftputil.sschwarzer.net (ftputil) Date: Sat, 19 Feb 2011 09:54:44 -0000 Subject: [ftputil] #56: FTPHost.__del__ and ftp_path._Path._host wreak havoc with Python's circular garbage collection In-Reply-To: <052.8c48af72ac536036bdb00aeac96d33ef@ftputil.sschwarzer.net> References: <052.8c48af72ac536036bdb00aeac96d33ef@ftputil.sschwarzer.net> Message-ID: <061.70236a70b922e3b5dc244870337d0951@ftputil.sschwarzer.net> #56: FTPHost.__del__ and ftp_path._Path._host wreak havoc with Python's circular garbage collection --------------------------+------------------------------------------------- Reporter: ftputiluser | Owner: schwa Type: defect | Status: assigned Priority: major | Milestone: 2.6 Component: Library | Version: 2.5 Resolution: | Keywords: --------------------------+------------------------------------------------- Changes (by schwa): * status: new => assigned * milestone: => 2.6 -- Ticket URL: ftputil Python FTP client library From trac at ftputil.sschwarzer.net Sat Feb 19 13:18:25 2011 From: trac at ftputil.sschwarzer.net (ftputil) Date: Sat, 19 Feb 2011 12:18:25 -0000 Subject: [ftputil] #56: FTPHost.__del__ and ftp_path._Path._host wreaks havoc with Python's circular garbage collection (was: FTPHost.__del__ and ftp_path._Path._host wreak havoc with Python's circular garbage collection) In-Reply-To: <052.8c48af72ac536036bdb00aeac96d33ef@ftputil.sschwarzer.net> References: <052.8c48af72ac536036bdb00aeac96d33ef@ftputil.sschwarzer.net> Message-ID: <061.2ba2c36b11037e02321ecdefdb1d5187@ftputil.sschwarzer.net> #56: FTPHost.__del__ and ftp_path._Path._host wreaks havoc with Python's circular garbage collection --------------------------+------------------------------------------------- Reporter: ftputiluser | Owner: schwa Type: defect | Status: assigned Priority: major | Milestone: 2.6 Component: Library | Version: 2.5 Resolution: | Keywords: --------------------------+------------------------------------------------- Old description: > Since FTPHost defines a __del__ method and the ftp_path._Path instances > have a reference to the FTPHost, Python's circular reference garbage > collection can't cleanup. See > http://docs.python.org/reference/datamodel.html#object.__del__ for > details. > > This could represent a significant memory leak for anyone using ftputil > in a long running app. New description: Since FTPHost defines a `__del__` method and the ftp_path._Path instances have a reference to the FTPHost, Python's circular reference garbage collection can't cleanup. See [http://docs.python.org/reference/datamodel.html#object.__del__] for details. This could represent a significant memory leak for anyone using ftputil in a long running app. -- Comment(by schwa): Fixed description to work with Trac wiki formatting. -- Ticket URL: ftputil Python FTP client library From trac at ftputil.sschwarzer.net Sat Feb 19 13:26:30 2011 From: trac at ftputil.sschwarzer.net (ftputil) Date: Sat, 19 Feb 2011 12:26:30 -0000 Subject: [ftputil] #56: FTPHost.__del__ and ftp_path._Path._host wreaks havoc with Python's circular garbage collection In-Reply-To: <052.8c48af72ac536036bdb00aeac96d33ef@ftputil.sschwarzer.net> References: <052.8c48af72ac536036bdb00aeac96d33ef@ftputil.sschwarzer.net> Message-ID: <061.d14139c82540be12e998bddab4cefb2f@ftputil.sschwarzer.net> #56: FTPHost.__del__ and ftp_path._Path._host wreaks havoc with Python's circular garbage collection --------------------------+------------------------------------------------- Reporter: ftputiluser | Owner: schwa Type: defect | Status: assigned Priority: major | Milestone: 2.6 Component: Library | Version: 2.5 Resolution: | Keywords: --------------------------+------------------------------------------------- Comment(by schwa): Thank you for the bug report! Replying to [ticket:56 ftputiluser]: > Since FTPHost defines a `__del__` method and the ftp_path._Path instances have a reference to the FTPHost, Python's circular reference garbage collection can't cleanup. See [http://docs.python.org/reference/datamodel.html#object.__del__] for details. As far as I understand the documentation, uncollectable cycles occur only if ''several'' objects in the cycle have a `__del__` method and thus the garbage collector can't decide which one to delete first. As `_Path` doesn't have a `__del__` method, there are cycles, but they can be collected. > This could represent a significant memory leak for anyone using ftputil in a long running app. I guess even most long-running apps generate one or a few `FTPHost` instances and use them over and over. Of course, if you instantiated many `FTPHost` objects, non-collectable cycles would lead to a memory leak. As it happens, when I wrote a unit test for this bug, I found there ''is'' an uncollectable cycle regarding files and/or sockets (even though I had considered mutual references between `FTPHost` and `FTPFile` objects). I'll look into this. -- Ticket URL: ftputil Python FTP client library From trac at ftputil.sschwarzer.net Sat Feb 19 15:22:52 2011 From: trac at ftputil.sschwarzer.net (ftputil) Date: Sat, 19 Feb 2011 14:22:52 -0000 Subject: [ftputil] #56: FTPHost.__del__ and ftp_path._Path._host wreaks havoc with Python's circular garbage collection In-Reply-To: <052.8c48af72ac536036bdb00aeac96d33ef@ftputil.sschwarzer.net> References: <052.8c48af72ac536036bdb00aeac96d33ef@ftputil.sschwarzer.net> Message-ID: <061.5e86e09b38244498518b32e11ffae4ee@ftputil.sschwarzer.net> #56: FTPHost.__del__ and ftp_path._Path._host wreaks havoc with Python's circular garbage collection --------------------------+------------------------------------------------- Reporter: ftputiluser | Owner: schwa Type: defect | Status: assigned Priority: major | Milestone: 2.6 Component: Library | Version: 2.5 Resolution: | Keywords: --------------------------+------------------------------------------------- Comment(by schwa): I managed to fix the problem in changeset [dd9d9c95c7b6]. It seems there actually ''was'' a problem with the references held by `_Path` and `_Stat` but I was unable to find out why the garbage collection didn't work. In the end, I removed the `__del__` method from `FTPHost` because I think that's a cleaner solution than trying to break various cycles explicitly. -- Ticket URL: ftputil Python FTP client library From trac at ftputil.sschwarzer.net Sat Feb 19 15:23:06 2011 From: trac at ftputil.sschwarzer.net (ftputil) Date: Sat, 19 Feb 2011 14:23:06 -0000 Subject: [ftputil] #56: FTPHost.__del__ and ftp_path._Path._host wreaks havoc with Python's circular garbage collection In-Reply-To: <052.8c48af72ac536036bdb00aeac96d33ef@ftputil.sschwarzer.net> References: <052.8c48af72ac536036bdb00aeac96d33ef@ftputil.sschwarzer.net> Message-ID: <061.19431829e67e6fc4c9635069a10d3ed2@ftputil.sschwarzer.net> #56: FTPHost.__del__ and ftp_path._Path._host wreaks havoc with Python's circular garbage collection --------------------------+------------------------------------------------- Reporter: ftputiluser | Owner: schwa Type: defect | Status: closed Priority: major | Milestone: 2.6 Component: Library | Version: 2.5 Resolution: fixed | Keywords: --------------------------+------------------------------------------------- Changes (by schwa): * status: assigned => closed * resolution: => fixed -- Ticket URL: ftputil Python FTP client library From trac at ftputil.sschwarzer.net Sat Feb 19 15:32:23 2011 From: trac at ftputil.sschwarzer.net (ftputil) Date: Sat, 19 Feb 2011 14:32:23 -0000 Subject: [ftputil] #29: Distribute ftputil as a Debian/Ubuntu package In-Reply-To: <046.55dd1f0634bf00b870f56afca3014134@ftputil.sschwarzer.net> References: <046.55dd1f0634bf00b870f56afca3014134@ftputil.sschwarzer.net> Message-ID: <055.9bae223f98d7edfe0f8322b97d540b2a@ftputil.sschwarzer.net> #29: Distribute ftputil as a Debian/Ubuntu package --------------------+------------------------------------------------------- Reporter: schwa | Owner: schwa Type: task | Status: closed Priority: major | Milestone: 2.6 Component: Other | Version: 2.2.3 Resolution: fixed | Keywords: Debian, package, Ubuntu --------------------+------------------------------------------------------- Changes (by schwa): * status: assigned => closed * resolution: => fixed Comment: ftputil is now in Debian [http://packages.debian.org/testing/python /python-ftputil wheezy] and [http://packages.debian.org/unstable/python /python-ftputil sid]. However it's ftputil version 2.4.2 only. -- Ticket URL: ftputil Python FTP client library From trac at ftputil.sschwarzer.net Sat Feb 19 15:33:57 2011 From: trac at ftputil.sschwarzer.net (ftputil) Date: Sat, 19 Feb 2011 14:33:57 -0000 Subject: [ftputil] #15: Add support for FXP (FTP to FTP copy) In-Reply-To: <057.ccd6acfc7f01da6815edf970095d1b57@ftputil.sschwarzer.net> References: <057.ccd6acfc7f01da6815edf970095d1b57@ftputil.sschwarzer.net> Message-ID: <066.5176c0f22c86a8aedf5905dd73672b5c@ftputil.sschwarzer.net> #15: Add support for FXP (FTP to FTP copy) -------------------------------+-------------------------------------------- Reporter: swordp at gmail.com | Owner: schwa Type: enhancement | Status: assigned Priority: major | Milestone: 2.7 Component: Library | Version: Resolution: | Keywords: FXP, FTP-to-FTP-copy -------------------------------+-------------------------------------------- Changes (by schwa): * milestone: 2.6 => 2.7 -- Ticket URL: ftputil Python FTP client library From trac at ftputil.sschwarzer.net Sat Feb 19 15:47:08 2011 From: trac at ftputil.sschwarzer.net (ftputil) Date: Sat, 19 Feb 2011 14:47:08 -0000 Subject: [ftputil] #53: FTPHost.walk fails when the argument is a unicode string and the tree contains non-ASCII characters In-Reply-To: <046.c131001bbf3f163c5d12a0fe2912047a@ftputil.sschwarzer.net> References: <046.c131001bbf3f163c5d12a0fe2912047a@ftputil.sschwarzer.net> Message-ID: <055.1b63d0045a60989e51dee2ac65a5aeed@ftputil.sschwarzer.net> #53: FTPHost.walk fails when the argument is a unicode string and the tree contains non-ASCII characters ----------------------+----------------------------------------------------- Reporter: schwa | Owner: schwa Type: defect | Status: closed Priority: major | Milestone: 2.6 Component: Library | Version: 2.4.2 Resolution: fixed | Keywords: unicode, walk, UnicodeEncodeError ----------------------+----------------------------------------------------- Changes (by schwa): * milestone: 2.5.1 => 2.6 -- Ticket URL: ftputil Python FTP client library From trac at ftputil.sschwarzer.net Tue Mar 15 19:27:28 2011 From: trac at ftputil.sschwarzer.net (ftputil) Date: Tue, 15 Mar 2011 18:27:28 -0000 Subject: [ftputil] #57: Migrate ftputil mailing lists Message-ID: <046.a450c0d2f7fd666fd2635c979cab9bd3@ftputil.sschwarzer.net> #57: Migrate ftputil mailing lists -----------------------------------------------+---------------------------- Reporter: schwa | Owner: schwa Type: task | Status: new Priority: major | Milestone: Component: Other | Version: Keywords: ftputil, mailing lists, Codespeak | -----------------------------------------------+---------------------------- At the moment, the ftpuil mailing lists (general and ticket mailing list) are hosted by Codespeak. However, Codespeak is about to shut down its hosting services. Therefore, the ftputil lists have to be migrated to another provider. To do: * Find an alternative provider (wishlist: free of charge for open source projects; reliable; can migrate list archives; search function for archives). * Open new list accounts. * Configure lists. * If possible, provide forwarding from old to new lists. * Migrate archives. * Migrate list subscribers. * Announce switch on new list. -- Ticket URL: ftputil Python FTP client library From trac at ftputil.sschwarzer.net Tue Mar 15 22:32:25 2011 From: trac at ftputil.sschwarzer.net (ftputil) Date: Tue, 15 Mar 2011 21:32:25 -0000 Subject: [ftputil] #57: Migrate ftputil mailing lists In-Reply-To: <046.a450c0d2f7fd666fd2635c979cab9bd3@ftputil.sschwarzer.net> References: <046.a450c0d2f7fd666fd2635c979cab9bd3@ftputil.sschwarzer.net> Message-ID: <055.89546cb5ebfbfc739ce9c028a4297218@ftputil.sschwarzer.net> #57: Migrate ftputil mailing lists --------------------+------------------------------------------------------- Reporter: schwa | Owner: schwa Type: task | Status: assigned Priority: major | Milestone: Component: Other | Version: Resolution: | Keywords: ftputil, mailing lists, Codespeak --------------------+------------------------------------------------------- Changes (by schwa): * status: new => assigned Old description: > At the moment, the ftpuil mailing lists (general and ticket mailing list) > are hosted by Codespeak. However, Codespeak is about to shut down its > hosting services. Therefore, the ftputil lists have to be migrated to > another provider. > > To do: > > * Find an alternative provider (wishlist: free of charge for open source > projects; reliable; can migrate list archives; search function for > archives). > * Open new list accounts. > * Configure lists. > * If possible, provide forwarding from old to new lists. > * Migrate archives. > * Migrate list subscribers. > * Announce switch on new list. New description: At the moment, the ftpuil mailing lists (general and ticket mailing list) are hosted by Codespeak. However, Codespeak is about to shut down its hosting services. Therefore, the ftputil lists have to be migrated to another provider. To do: * Find an alternative provider (wishlist, ''ideally'': free of charge for open source projects; no advertisements; reliable; can migrate list archives; search function for archives). * Open new list accounts. * Configure lists. * If possible, provide forwarding from old to new lists. * Migrate archives. * Migrate list subscribers. * Announce switch on new list. -- -- Ticket URL: ftputil Python FTP client library From trac at ftputil.sschwarzer.net Tue Mar 15 22:37:19 2011 From: trac at ftputil.sschwarzer.net (ftputil) Date: Tue, 15 Mar 2011 21:37:19 -0000 Subject: [ftputil] #57: Migrate ftputil mailing lists In-Reply-To: <046.a450c0d2f7fd666fd2635c979cab9bd3@ftputil.sschwarzer.net> References: <046.a450c0d2f7fd666fd2635c979cab9bd3@ftputil.sschwarzer.net> Message-ID: <055.54a9edbb00b2f4139dd04d12bed46e92@ftputil.sschwarzer.net> #57: Migrate ftputil mailing lists --------------------+------------------------------------------------------- Reporter: schwa | Owner: schwa Type: task | Status: assigned Priority: major | Milestone: Component: Other | Version: Resolution: | Keywords: ftputil, mailing lists, Codespeak --------------------+------------------------------------------------------- Description changed by schwa: Old description: > At the moment, the ftpuil mailing lists (general and ticket mailing list) > are hosted by Codespeak. However, Codespeak is about to shut down its > hosting services. Therefore, the ftputil lists have to be migrated to > another provider. > > To do: > > * Find an alternative provider (wishlist, ''ideally'': free of charge > for open source projects; no advertisements; reliable; can migrate list > archives; search function for archives). > * Open new list accounts. > * Configure lists. > * If possible, provide forwarding from old to new lists. > * Migrate archives. > * Migrate list subscribers. > * Announce switch on new list. New description: At the moment, the ftpuil mailing lists (general and ticket mailing list) are hosted by Codespeak. However, Codespeak is about to shut down its hosting services. Therefore, the ftputil lists have to be migrated to another provider. Steps: * Find an alternative provider; wishlist, ''ideally'': * free of charge for open source projects * no advertisements (a little advertisement for the service itself is ok) * reliable * can migrate list archives * search function for archives * Open new list accounts. * Configure lists. * If possible, provide forwarding from old to new lists. * Migrate archives. * Migrate list subscribers. * Announce switch on new list. -- -- Ticket URL: ftputil Python FTP client library From trac at ftputil.sschwarzer.net Wed Mar 16 12:06:38 2011 From: trac at ftputil.sschwarzer.net (ftputil) Date: Wed, 16 Mar 2011 11:06:38 -0000 Subject: [ftputil] #57: Migrate ftputil mailing lists In-Reply-To: <046.a450c0d2f7fd666fd2635c979cab9bd3@ftputil.sschwarzer.net> References: <046.a450c0d2f7fd666fd2635c979cab9bd3@ftputil.sschwarzer.net> Message-ID: <055.b8d2fff64cc3f94130cdf7e1c555625e@ftputil.sschwarzer.net> #57: Migrate ftputil mailing lists --------------------+------------------------------------------------------- Reporter: schwa | Owner: schwa Type: task | Status: assigned Priority: major | Milestone: Component: Other | Version: Resolution: | Keywords: ftputil, mailing lists, Codespeak --------------------+------------------------------------------------------- Description changed by schwa: Old description: > At the moment, the ftpuil mailing lists (general and ticket mailing list) > are hosted by Codespeak. However, Codespeak is about to shut down its > hosting services. Therefore, the ftputil lists have to be migrated to > another provider. > > Steps: > > * Find an alternative provider; wishlist, ''ideally'': > * free of charge for open source projects > * no advertisements (a little advertisement for the service itself is > ok) > * reliable > * can migrate list archives > * search function for archives > * Open new list accounts. > * Configure lists. > * If possible, provide forwarding from old to new lists. > * Migrate archives. > * Migrate list subscribers. > * Announce switch on new list. New description: At the moment, the ftpuil mailing lists (general and ticket mailing list) are hosted by Codespeak. However, Codespeak is about to shut down its hosting services. Therefore, the ftputil lists have to be migrated to another provider. Steps: * Find an alternative provider; wishlist, ''ideally'': * free of charge for open source projects * no advertisements (a little advertisement for the service itself is ok) * reliable * can migrate list archives * search function for archives * Open new list accounts. * Configure lists. * If possible, provide forwarding from old to new lists. * Migrate archives. * Migrate list subscribers. * Announce switch on old list so that readers of the old archive are informed. * Announce switch on new list so that subscribers migrated by me are informed. -- -- Ticket URL: ftputil Python FTP client library