From georges at racinet.fr Wed Oct 1 11:26:47 2008 From: georges at racinet.fr (Racinet Georges) Date: Wed, 1 Oct 2008 11:26:47 +0200 Subject: [Tramline-dev] List of enhancement proposals In-Reply-To: References: Message-ID: On Sep 24, 2008, at 7:02 PM, Racinet Georges wrote: >> Hi all, sorry, I've been busier than expected. >> Here are my proposals for Tramline enhancements. > > Hello all, I'm replying to myself to keep the list posted about this. > > I started again from the current head of tramline trunk, ported back > some of the previous enhancements I'd done on tramline 0.5.1 and just > implemented Range requests (currently in testing, first positive > results are there). There are some more details below. After some bugfixing, the Range requests now work, including multiple ones. All of this should enter production status for a client of mine in the forthcoming days. > All of this is still living in my own repository, at viral-prod.com. > I'd still be glad to contribute it to tramline trunk. > I will of course open read access to this part of my repo if someone's > interested by the reading. The patched Tramline can be seen here: http://svn.viral-prod.com/tramline/trunk For those interested in svn log, note that the trunk and a branch have just been swapped The corresponding CPSTramline is at http://svn.viral-prod.com/CPSTramline It should shortly move to Nuxeo's subversion repository and enter the CPS-3-full distribution for CPS 3.4.7, that is currently scheduled for release in a few weeks. There are again some details below. Cheers, >> >> >> I try to keep in mind the principle that tramline is supposed to be (snip) >> >> >> IN PROGRESS: >> - support for Range queries and appropriate (416 and 206) responses. >> Things like Adobe plugin on windows, and, as it seems, >> Firefox 3 perform range queries, that don't behave well, because they >> pass through to the application server. We already discussed possible >> quick fixes (bypass actually) with Sylvain. > > Done, with unit tests and currently in functional testing. > > Simple range requests (for instance 'bytes = 13231-') work. More > specifically, Firefox 3 for MacOS X can now resume mp3 downloads from > my testing site, which issues such range requests. Haven't seen more > complex ones yet. The Adobe PDF plugin for Firefox 2 issues multiple range requests all the time. Same for IE 6 I heard. > > > Some implementation details: > - it is still the app server's business to choose between 200 and > 206 (or 304) responses But the app server has to understand that the requested range has nothing to do with what the it stores (ie just the tramline id). In CPSTramline, the range is replaced by "Range: bytes=0-" on the fly. I'm not very satisfied with this: changing the Request's incoming range reader is not part of Zope's public API. I'll try to do it at the tramline level (actually not so straightforward) so that it'll become transparent to the app server, too and other client code (like attramline) can benefit it. > - I used twisted.web2 to parse the 'Range' header. This can cause > PYTHONPATH issues on debian Etch if you want to run a self-compiled > installation of Zope. Re-implemented this simple parsing to drop the annoying dependency. > > >> PLANNED: >> - simple repository hierarchical structure (similar to Apache >> caches). >> ext3 does not deal happily with 30000+ files flat in a directory. >> Implementation would be mostly trivial, but would require migration >> of >> existing data. Implementation not breaking existing repositories >> would >> be less straightforward but nevertheless doable. Or we could have the >> tree structure optionall (and disabled by default). While we're at >> it, >> the id should have a fixed length (hexa dump instead of decimal would >> do). > > Not done yet. > >> >> - Slight refactoring to single out a method to get the filesystem >> path from the tramline id >> (if possible) The backend server needs to access the files >> anyway (indexing, various metadata extraction, basic management). I'd >> prefer the backend server to import code from tramline rather than >> re- >> implementing it. Not really necessary without previous point, the >> path >> being so simple. > > Done like this: > > def id_to_path(tramline_path, id, upload=False, > create_intermediate=False): > """Compute the path on filesystem of the file with given id. > > tramline_path: the tramline base path, as extracted from request > options. > id: the tramline id, as a string. > upload: if True, the computed path will be from the "upload" > directory > instead of the "repository" > create_intermediate: if True, all necessary objects will be > created, but > the target file won't.""" > > > -- > Georges Racinet > Zope/CPS/Plone expertise & assistance > > > > > > _______________________________________________ > Tramline-dev mailing list > Tramline-dev at codespeak.net > http://codespeak.net/mailman/listinfo/tramline-dev -- Georges Racinet Zope/CPS/Plone expertise & assistance