From h Sun Apr 13 23:18:02 2008 From: h (Terris) Date: Sun, 13 Apr 2008 23:18:02 +0200 Subject: [ftputil] People will strew you over with complements about how you look like. Message-ID: <78AB8335.9E%h]kssrof1972@CHAMBLEEPD.COM> Make your life safe and pure. Do it now. Don?t hesitate. Everything is in your own hands! http://www.grentiile.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/ftputil/attachments/20080413/f70d9cec/attachment.htm From Roger Thu May 1 16:41:22 2008 From: Roger (Roger) Date: Thu, 01 May 2008 17:41:22 +0300 Subject: [ftputil] Greetings from Roger J. Garang, Message-ID: Greetings from Roger J. Garang, Due to security reasons and the kind of place I am in BENIN REPUBLIC at themoment, I have to contact you in this manner with apology. I am Roger J. Garang, Son of late Dr. General John Garang the former rebel leader of Sudan who letter became the vice president of Sudan,before he was killed by a planned deal through a plane crash, which took place on Monday August 1st, 2005 in mountains located in southern Sudan. http://news.bbc.co.uk/2/hi/in_pictures/4735725.stm He was buried on Saturday August 6th, 2005. http://news.bbc.co.uk/2/hi/africa/4126370.stm After the death of my mother Mrs. Rebecca Garang on the 15th December 2005, the president of Sudan placed embargo on my late father's assets during his speech on January 2nd, 2006, which made me to flee from my country through road to another country where the United Nations granted me refugee under political asylum camp in Republic of BENIN Meanwhile, I am currently seeking to invest in small and large business enterprises in your country under partnership but I am preferred to deal directly with legitimate owners who can be able to handle my project rather than brokers or middlemen. Therefore, I would like to request full contact details where I can reach the Chief Executive Officer of your organization to enable us discuss further details on this project. However, I have a stake of USD$107,000.000.00 One hundred and fifty-Seven Million Dollars available for investment. So I would be honored if this is treated with Utmost Urgency and do not fail to include your direct telephone number for me to reach you easily for more imformation regarding this project. You can contact me only on my altanative email address (garangroger at yahoo.it) where i can be reach any time in the asylum camp 00229 97 98 99 52. Sincerely Yours, Roger Garang, (On behalf of the Family) From yvan.strahm at bccs.uib.no Tue May 13 14:39:13 2008 From: yvan.strahm at bccs.uib.no (Yvan Strahm) Date: Tue, 13 May 2008 14:39:13 +0200 Subject: [ftputil] host.download_if_newer problems Message-ID: <48298BF1.3040502@bccs.uib.no> Hello, I try to use ftputil to mirror biological databases. On the host the files are compressed (either .gz or .Z) on the local server they are uncompress. Is there a way of having uncompress file on the local server and using host.download_if_newer()? Thanks for your help And thanks a lot for ftputil. yvan From sschwarzer at sschwarzer.net Tue May 13 18:13:14 2008 From: sschwarzer at sschwarzer.net (Stefan Schwarzer) Date: Tue, 13 May 2008 18:13:14 +0200 Subject: [ftputil] host.download_if_newer problems In-Reply-To: <48298BF1.3040502@bccs.uib.no> References: <48298BF1.3040502@bccs.uib.no> Message-ID: <4829BE1A.2060704@sschwarzer.net> Hello Yvan, On 2008-05-13 14:39, Yvan Strahm wrote: > I try to use ftputil to mirror biological databases. On the host the > files are compressed (either .gz or .Z) on the local server they are > uncompress. Is there a way of having uncompress file on the local server > and using host.download_if_newer()? download_if_newer (and more generally, ftputil) has no built-in support for compressing or uncompressing files. You write "host" and "local server" which makes me wonder which computer is the client (i. e. the computer on which ftputil runs) and which the server (i. e. the argument to FTPHost's constructor) from ftputil's point of view. Below, I assume you have the compressed files on a server and want to download them to a client on which ftputil runs. Probably the simplest way is to download the files with download_if_newer and uncompress them locally, either with Python libraries or with the subprocess module and command line tools. Python supports uncompressing .gz files with the gzip and/or zlib modules, but not, as far as I understand, uncompressing .Z files. So for .Z files you will need the appropriate command line tools or libraries which aren't part of Python. To avoid the download followed by the local uncompressing, it should in principle be possible to construct a file-like object which supports uncompression on the fly, use ftputil to determine modification timestamps (possibly considering time shift between server and client) and use FTPHost.copyfileobj for the transfer. However, probably the local decompression is so fast compared to a download that the complex approach doesn't make sense. > Thanks for your help I don't know if you'll call that "help". ;-) > And thanks a lot for ftputil. Thank you! You're welcome. :-) Please ask if you have more questions. Best regards, Stefan From yvan.strahm at bccs.uib.no Thu May 22 08:54:17 2008 From: yvan.strahm at bccs.uib.no (Yvan Strahm) Date: Thu, 22 May 2008 08:54:17 +0200 Subject: [ftputil] host.download_if_newer problems In-Reply-To: <48331D75.1070203@sschwarzer.net> References: <48298BF1.3040502@bccs.uib.no> <4829BE1A.2060704@sschwarzer.net> <48327F59.3020502@bccs.uib.no> <48331D75.1070203@sschwarzer.net> Message-ID: <48351899.5060800@bccs.uib.no> Stefan Schwarzer wrote: > Hi Yvan, > > On 2008-05-20 09:35, Yvan Strahm wrote: >> Thanks for the reply. > > You're welcome :-) > >> yes the compressed files are on the server(host) and the client would >> have the uncompressed ones. >> At the moment I am downloading the compressed files and uncompress them >> on the client. My main problem id hard drive space... because to avoid >> unnecessary downloads I am keeping on the client both the compressed and >> the uncompressed files. I tried to use host.path.getmtime() to get the >> time form the host and compare to the time given by stat on the client >> for the same file, but as soon as the file are uncompressed the >> timestamp change. > > If you show me your code, I may be able to suggest > improvements. > >> Or am I completely wrong or on the wrong path here? > > If you uncompress from a program, you could test and > remember the timestamp of last modification, uncompress > the file and setting the modification timestamp to the > remembered value. So the uncompressed file would have the > same timestamp as the source file. (Or _almost_ the same > timestamp because the timestamps fetched from the server > can only be exact to a minute or even only a day, > depending on the remote directory listing.) > > If the server and client are in different timezones, you > will have to account for time shift. See the time shift- > related methods of FTPHost and for examples how they are > used in download_if_newer and upload_if_newer. > > In case you have trouble modifying the timestamps, you > could also store them as text in a file, e. g. in the > format > > 2008-05-20 17:00 my_file.gz > >> I don't really understand the file-like object term, > > You can use FTPHost.file to construct a file(-like) object, > just as Python's "open" function does. However, while Python's > "open" call refers to local files, ftputil's files are opened > on the remote host. > > For example, you could do with ftputil: > > host = ftputil.FTPHost(server, userid, password) > # write access and binary transfer are supported, too > f = host.file("remote_file.txt") > # gets the lines of the remote file one by one > for line in f: > print f, > f.close() > > As you see, just like in Python, you can iterate over the > file's lines. You can also read a number of bytes with > f.read(number). The remote file is _not_ copied to the > file system of the client unless to put it there yourself. > >> does it imply to >> uncompressed to a tmp folder, compare the date, delete if not newer or >> download if newer? > > The idea was to open a file-like object similar to the > code above and "pipe" it into a generator which uncompresses > the stream's data and writes the uncompressed data to the > disk. > > Would you mind me sending your mail and my reply to the > ftputil list? The information may be useful to others. > > Best regards, > Stefan Yes, not problem for sending the mails to the list I thought the reply went to the mailing list, sorry. I guess i will just compare the date of the compressed file on the server with the date of the uncompressed file on the client, then if the server file is newer then it will be downloaded. This is how I thought it can be done 1. getting a dictionary (key:filename ,value:date) from the server: host=ftputil.FTPHost(ftp,user,password) host.chdir(ftp_dir) files=host.listdir(host.curdir) for f in files: mtime=host.path.getmtime(f) dict={f:mtime} close(ftp) 2. getting a equivalent dictionary from the client 3. comparing both list if not present on the client, add the file to a list if server date > client date, add the file to a list 4. reopen ftp and download the list to_be_download but it doesn't work at all! Problems are size of the dictionaries, more than 50'000 files and how to compare efficiently files. And the main problem is the ftp connection dying. Best regards yvan From sschwarzer at sschwarzer.net Sat May 24 23:20:29 2008 From: sschwarzer at sschwarzer.net (Stefan Schwarzer) Date: Sat, 24 May 2008 23:20:29 +0200 Subject: [ftputil] host.download_if_newer problems In-Reply-To: <48351899.5060800@bccs.uib.no> References: <48298BF1.3040502@bccs.uib.no> <4829BE1A.2060704@sschwarzer.net> <48327F59.3020502@bccs.uib.no> <48331D75.1070203@sschwarzer.net> <48351899.5060800@bccs.uib.no> Message-ID: <4838869D.6030809@sschwarzer.net> Hi Yvan, On 2008-05-22 08:54, Yvan Strahm wrote: > Yes, not problem for sending the mails to the list I thought the reply > went to the mailing list, sorry. No problem, this has happened to almost anybody, I guess. :) > I guess i will just compare the date of the compressed file on the > server with the date of the uncompressed file on the client, then if the > server file is newer then it will be downloaded. Which timestamp will the uncompressed file have after uncompressing? The timestamp of the original file before it was compressed or the date and time of the uncompression process (i. e. when the uncompressed file was generated)? Probably none of these timestamps is that of the compressed file on the server (or is it?). Be careful not to compare apples and oranges. You may set the timestamp of the uncompressed local file to the timestamp of the corresponding compressed file on the server though. > This is how I thought it can be done > > 1. getting a dictionary (key:filename ,value:date) from the server: > > host=ftputil.FTPHost(ftp,user,password) > host.chdir(ftp_dir) > files=host.listdir(host.curdir) > for f in files: > mtime=host.path.getmtime(f) > dict={f:mtime} Here, you overwrite the dict every time. Did you actually write "dict[f] = mtime"? By the way, it's error-prone to introduce a name which is the same as a Python builtin (in this case, "dict"). Unless you use some trick, this will prevent you from using the builtin dict object until your object's name goes out of scope or the local name is deleted. > close(ftp) > > 2. getting a equivalent dictionary from the client Again, think of which timestamps you store and compare. > 3. comparing both list > if not present on the client, add the file to a list > if server date > client date, add the file to a list Seems sensible. > 4. reopen ftp and download the list to_be_download > > but it doesn't work at all! Can you show some output to demonstrate the problem(s)? Does your code work for small directories and becomes only slow for large directories? Or does it even fail when small directories are involved? > Problems are size of the dictionaries, more than 50'000 files How much RAM can you use on the client? Even if you use 100 bytes per dictionary entry, this will amount to 5 MB which is not much nowadays. Perhaps the bottleneck is elsewhere. If you have many files in the directory it may be sensible to increase ftputil's stat cache size (see the documentation), so ftputil will have to fetch the directory listing from the server only once. > and how to compare efficiently files. And the main problem is the ftp > connection dying. Do you mean dying in the sense that the server encounters a timeout and closes the connection "consciously" or do you speak of an instable connection where the time until it breaks varies rather randomly? Please explain a bit more in which way your code fails (see above), and we might be able to get to a working solution. :-) Best regards, Stefan From Feed Tue May 27 01:41:17 2008 From: Feed (Feed) Date: 26 May 2008 16:41:17 -0700 Subject: [ftputil] Feed Blaster puts your ad right to the screens of millions in 15 Minutes ! Message-ID: <20080526164116.FC37DA664D4ADDF0@from.header.has.no.domain> More and more people are subscribing to feeds every day and there are millions who are already subscribed. Thus, your ad will reach a very broad range of potential customers with each use of Feed Blaster! Feed Blaster is the first & only submitter that can submit your ads to thousands of feeds within a few minutes! Post your ads where people read them! - What if you could place your ad into all these feeds ? Right, that would mean you would have millions of sites linking to your ad - and millions of users reading your message within minutes - and my idea actually works For Full details please read the attached .html file Unsubscribe: please read the attached .html file -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/ftputil/attachments/20080526/663cc414/attachment-0002.htm -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/ftputil/attachments/20080526/663cc414/attachment-0003.htm From BlogBlaster Sun Jun 1 10:01:55 2008 From: BlogBlaster (BlogBlaster) Date: 01 Jun 2008 01:01:55 -0700 Subject: [ftputil] "How would you like to have your ad on 2 Million Websites ?" Message-ID: <20080601010154.17854834EED503C8@from.header.has.no.domain> How would you like 2 Million Sites linking to your ad ? Weblog or blog population is exploding around the world, resembling the growth of e-mail users in the 1990s. Post your ads where people read them! - What if you could place your ad on all these sites ? Right, that would mean you would have millions of sites linking to your ad. For Full details please read the attached .html file Unsubscribe please read the attached .html file, click on contact form -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/ftputil/attachments/20080601/e09e0852/attachment.htm From Feed Mon Jun 2 10:32:27 2008 From: Feed (Feed) Date: 02 Jun 2008 01:32:27 -0700 Subject: [ftputil] Feed Blaster puts your ad right to the screens of millions in 15 Minutes ! Message-ID: <20080602013225.14A3C85E5C18956D@from.header.has.no.domain> More and more people are subscribing to feeds every day and there are millions who are already subscribed. Thus, your ad will reach a very broad range of potential customers with each use of Feed Blaster! Feed Blaster is the first & only submitter that can submit your ads to thousands of feeds within a few minutes! Post your ads where people read them! - What if you could place your ad into all these feeds ? Right, that would mean you would have millions of sites linking to your ad - and millions of users reading your message within minutes - and my idea actually works For Full details please read the attached .html file Unsubscribe: On Full details page click on contact form -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/ftputil/attachments/20080602/c33941ec/attachment.htm From BlogBlaster Sun Jun 8 08:57:08 2008 From: BlogBlaster (BlogBlaster) Date: 07 Jun 2008 23:57:08 -0700 Subject: [ftputil] "How would you like to have your ad on 2 Million Websites ?" Message-ID: <20080607235708.DFED734C3D805CD1@from.header.has.no.domain> How would you like 2 Million Sites linking to your ad ? Weblog or blog population is exploding around the world, resembling the growth of e-mail users in the 1990s. Post your ads where people read them! - What if you could place your ad on all these sites ? Right, that would mean you would have millions of sites linking to your ad. For Full details please read the attached .html file Unsubscribe please read the attached .html file, click on contact form -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/ftputil/attachments/20080607/bc57b655/attachment.htm From Feed Mon Jun 9 03:29:51 2008 From: Feed (Feed) Date: 08 Jun 2008 18:29:51 -0700 Subject: [ftputil] Feed Blaster puts your ad right to the screens of millions in 15 Minutes ! Message-ID: <20080608182951.970A3F5EB0F9BD16@from.header.has.no.domain> More and more people are subscribing to feeds every day and there are millions who are already subscribed. Thus, your ad will reach a very broad range of potential customers with each use of Feed Blaster! Feed Blaster is the first & only submitter that can submit your ads to thousands of feeds within a few minutes! Post your ads where people read them! - What if you could place your ad into all these feeds ? Right, that would mean you would have millions of sites linking to your ad - and millions of users reading your message within minutes - and my idea actually works For Full details please read the attached .html file Unsubscribe: On Full details page click on contact form -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/ftputil/attachments/20080608/eb6e423f/attachment.htm From Instant Tue Jun 10 04:06:51 2008 From: Instant (Instant) Date: 09 Jun 2008 19:06:51 -0700 Subject: [ftputil] Can you afford to lose 300, 000 potential customers per year ? Message-ID: <20080609190651.8F7315297CB7A0DF@from.header.has.no.domain> Can you afford to lose 300,000 potential customers per year ? How would You like to divert 1000s of fresh, new visitors daily to Your web site or affiliate web site from Google, Yahoo, MSN and others At $0 cost to you...? ...iNSTANT BOOSTER diverts 1000s of fresh, new visitors daily to Your web site or affiliate web site from Google, Yahoo, MSN and others at $0 cost to you! ...No matter what you are selling or offering - INTSANT BOOSTER will pull in hordes of potential customers to your website - instantly! For Full Details Please read the attached .html file Unsubscribe: Please read the attached .html file and click contact form -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/ftputil/attachments/20080609/4ab54329/attachment.htm From \ Fri Jun 13 23:54:44 2008 From: \ (\) Date: Fri, 13 Jun 2008 18:54:44 -0300 Subject: [ftputil] Today get 30-70% off all watches. Message-ID: 1464a01c8cd6d$f350b060$4001a8c0@intelb87f10665 Watches Now on Sale Today get 30-70% off all watches. http://drabsmangyheh.com/ From \ Sun Jun 15 03:26:52 2008 From: \ (\) Date: Sat, 14 Jun 2008 18:26:52 -0700 Subject: [ftputil] Rolex Timepieces for Men and Women. Message-ID: c550601c8ce11$95308510$4001a8c0@ampa Watches Now on Sale Today get 30-70% off all watches. http://comalbid.com/