From kitblake at gmail.com  Tue Dec  5 17:35:59 2006
From: kitblake at gmail.com (kit BLAKE)
Date: Tue, 5 Dec 2006 17:35:59 +0100
Subject: [Tramline-dev] Further development of Tramline
In-Reply-To: <449921B7.2090603@infrae.com>
References: <200606131930.00519.jeroen.vloothuis@pareto.nl>
	<44980EF3.4070503@infrae.com>
	<200606210808.43683.jeroen.vloothuis@pareto.nl>
	<449921B7.2090603@infrae.com>
Message-ID: <32a4a00c0612050835q64ee94f8g2e968b246e3e577a@mail.gmail.com>

2006/6/21, Martijn Faassen <faassen at infrae.com>:
> Jeroen Vloothuis wrote:
> > On Tuesday 20 June 2006 17:06, Martijn Faassen wrote:
> >> I patched tramline and tried to run the tests, but unfortunately I got a
> >> test failure:
> [snip]
> >>
> >> Is the patch missing something?
> >
> > I am attaching the complete diff (I removed my path settings previously). This
> > code works for me with Python 2.4
>
> I just applied it, and still get the same test failure.
>
> How do you run the tests? I'm running them as follows:
>
> $ cd tramline
> $ python2.4 test.py

Did this go any further? We're doing a Tramline release, and I didn't
see Jeroen's name in the credits. Seems like the trail stops here....

We can do another release as needed.

Kit

-- 
Kit BLAKE ? Infrae ? http://infrae.com/ + 31 10 243 7051
Hoevestraat 10 ? 3033 GC ? Rotterdam + The Netherlands
Contact = http://xri.net/=kitblake

From chad.maine at gmail.com  Thu Dec 14 21:44:06 2006
From: chad.maine at gmail.com (Chad Maine)
Date: Thu, 14 Dec 2006 15:44:06 -0500
Subject: [Tramline-dev] AttributeError
Message-ID: <179e72170612141244m623a5c7cu1e9ae7161cdfd88d@mail.gmail.com>

I'm new to this list, but I couldn't find a reference to this in the
archives.  There appears to be a typo in core.py:

312c312
<             self._f_close()
---
>             self._f.close()

Also, is there a theoretical limit to the number of files that tramline can
handle in a single POST?  I'm having trouble right around 300 or so files.

Thanks.

Chad Maine
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/tramline-dev/attachments/20061214/e7d31db7/attachment.html 

From kitblake at gmail.com  Fri Dec 22 16:53:59 2006
From: kitblake at gmail.com (kit BLAKE)
Date: Fri, 22 Dec 2006 16:53:59 +0100
Subject: [Tramline-dev] AttributeError
In-Reply-To: <179e72170612141244m623a5c7cu1e9ae7161cdfd88d@mail.gmail.com>
References: <179e72170612141244m623a5c7cu1e9ae7161cdfd88d@mail.gmail.com>
Message-ID: <32a4a00c0612220753v20884e6ao3625cc7561f4c0eb@mail.gmail.com>

> I'm new to this list, but I couldn't find a reference to this in the
> archives.  There appears to be a typo in core.py:
>
> 312c312
> <             self._f_close()
> ---
>>             self._f.close()

Thanks for reporting this.  This has been fixed in
http://cheeseshop.python.org/pypi/tramline/0.5.1

> Also, is there a theoretical limit to the number of files that
> tramline can handle in a single POST?  I'm having trouble right around
> 300 or so files.

We'll investigate this.  Do you have an error message?

Daniel

From chad.maine at gmail.com  Tue Dec 26 15:40:08 2006
From: chad.maine at gmail.com (Chad Maine)
Date: Tue, 26 Dec 2006 09:40:08 -0500
Subject: [Tramline-dev] Django & Tramline
Message-ID: <179e72170612260640v2025763fxe76f069eb57d4465@mail.gmail.com>

I wanted to share some information with the list here re: my tests of
tramline with django.  Here's some information about my setup:

Front end apache server with patched mod_python proxy-ing a
django/mod_python instance on another server.  Here's the relavant tramline
httpd.conf:

LoadModule python_module modules/mod_python.so

PythonInputFilter tramline.core::inputfilter TRAMLINE_INPUT
PythonOutputFilter tramline.core::outputfilter TRAMLINE_OUTPUT
SetInputFilter TRAMLINE_INPUT
SetOutputFilter TRAMLINE_OUTPUT
PythonOption tramline_path /opt/tramline/var/data

RewriteEngine On

## serve some static media from this apache server ##
RewriteRule     ^/static/(.*)   -       [L]

## pass all other requests to django server ##
RewriteRule     ^/(.*)  http://django-server/$1 [P,L]


Here's my django model for referencing a tramline file:

class TramlineFile(models.Model):
    name = models.CharField(maxlength=200)
    type = models.ForeignKey(FileType)
    tramline_id = models.CharField(maxlength=20)

Here's my django 'view' code for uploading a file:

def upload(request):
    '''
    Handle file upload
    '''
    ## placeholder hack until file types are in db ##
    file_type = FileType.objects.get(pk=1)

    ## save each tramline file object to the db ##
    for file in request.FILES.values():
        tlf = TramlineFile(name=file["filename"], type=file_type,
tramline_id=file["content"].strip())
        tlf.save()

    ## build redirect response ##
    response = HttpResponseRedirect("/files/")

    ## set special tramline header ##
    response["tramline_ok"] = "True"

    return response

Here's my django 'view' code for downloading a file:

def get_file(request, file_id):
    '''
    Handle file download
    '''

    ## get tramline file object from db ##
    file = get_object_or_404(TramlineFile, pk=file_id)

    ## placeholder hack until we are can get content-type from the db ##
    ext = file.name.rsplit('.', 1)[-1].lower()
    if ext in EXTS.keys():
        contentType = EXTS[ext]
    else:
        contentType = "text/plain"

    ## send file id as the response body, and use appropriate content-type
##
    response = HttpResponse(file.tramline_id, contentType)

    ## set header for tramline ##
    response["tramline_file"] = "True"

    return response


Tramline's performance appears to be very good.  I am having trouble,
though, with django's parsing performance of the POST data when uploading
very large (600+) numbers of files in one post with jupload (
http://www.jupload.biz/).  (Even with tramline, the size of the request
django gets with these large uploads is ~3MB.  Part of that could be the
extra attributes that jupload sends along with each file as form data.)  I'm
going to look at handling the raw post data with some custom code and I'll
post my results here.

The end result of this so far:  Tramline works well with django and was very
simple to integrate.

Also, maybe I missed it, but does tramline include the file size metadata in
the post?  Is anyone else interested in a patch to provide this if it
doesn't already?

Chad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/tramline-dev/attachments/20061226/85f836c9/attachment-0001.htm 

From chad.maine at gmail.com  Tue Dec 26 16:35:00 2006
From: chad.maine at gmail.com (Chad Maine)
Date: Tue, 26 Dec 2006 10:35:00 -0500
Subject: [Tramline-dev] File size
Message-ID: <179e72170612260735h18614f86h34cf943198fec693@mail.gmail.com>

Nevermind my question about file size, I see that content-length is
untouched by tramline.

Chad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/tramline-dev/attachments/20061226/f6cf00fd/attachment.htm 

From chad.maine at gmail.com  Wed Dec 27 14:29:25 2006
From: chad.maine at gmail.com (Chad Maine)
Date: Wed, 27 Dec 2006 08:29:25 -0500
Subject: [Tramline-dev] django update
Message-ID: <179e72170612270529ja6b0a26i8540edd1642398a@mail.gmail.com>

I was able to code a more efficient parsing method and now I'm able to
upload over 1000 files in a single POST.  I think that should be a good
upper limit for my application.

My only chore now is to code a mod_python RPC delete method on the tramline
media server, which should be straightforward enough.

Chad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/tramline-dev/attachments/20061227/4ecbaaef/attachment.htm