[lxml-dev] lxml + mod_python: cannot unmarshal code objects in restricted execution mode

[Lee Brown]

Greetings!

Sorry, I should have stated my first question more clearly.  Are you calling
your routines from within a Mod Python requestHandler object or an outputFilter
object? 

> -----Original Message-----
> From: lxml-dev-bounces at codespeak.net 
> [mailto:lxml-dev-bounces at codespeak.net] On Behalf Of David Danier
> Sent: Thursday, September 13, 2007 12:02 PM
> To: lxml-dev at codespeak.net
> Subject: Re: [lxml-dev] lxml + mod_python: cannot unmarshal 
> code objects in restricted execution mode
> 
> > Everything became much easier and 10 times faster, but I've 
> > encountered the subject problem.
> 
> Same problem here, but with different code and versions:
>  * Django as webframework
>  * Apache 2.0.59 and 2.2.4
>  * lxml 1.3.x (all versions)
>  * mod_python 3.2.10 and 3.3.1
>  * libxml2 2.6.28 / libxslt 1.1.20
> 
> I think this might have something to do with mod_python 
> fiddling with __builtins__, at least googling for the error 
> message told me, that Python switches to restricted mode when 
> doing so (but this might one trigger of many). lxml seems to 
> have callbacks run in its own "sandbox"
> (or something like this, at least it seems to be a different 
> environment as the outer code had), which works fine unless 
> the restricted mode is triggered.
> 
> Somehow restricted mode is only mentioned in the docs for 
> RExec (http://docs.python.org/lib/module-rexec.html), but 
> should not be available any more, to I don't know what lxml 
> exactly does to use callbacks.
> 
> Some further bug-finding I did revealed, that the 
> "unmarshaling"-error only occured if all modules I used in 
> the callback are loaded before the callback runs. If I load 
> them inside the callback the error differs.
> Example:
> ------------8<----------------------------------------------------
> # unmarshaling error
> from foo import bar
> def callback(ctx, ...):
>     return bar()
> ---------------------------------------------------->8------------
> ------------8<----------------------------------------------------
> # other error
> def callback(ctx, ...):
>     from foo import bar
>     return bar()
> ---------------------------------------------------->8------------
> As I have the needed mod_python-configuration not done here I 
> can't tell the other error, but I will add this later. (And I 
> think it was some
> ImportError)
> 
> I did not report this problem, because I was not sure which 
> part in the chain to produce webpages was responsible. Django 
> does fiddle with __builtins__, too (but removing it didn't 
> help). And perhaps this is simply a mod_python-bug. So I used 
> FastCGI, which works well.
> But I'm very interested in a better solution. ;-)
> 
> For the questions raised by Lee Brown:
> > Are you trying to execute this code in a Handler or in a Filter?  
> > There's world of hidden trouble lurking in Filters because 
> of their re-entrant nature.
> 
> I use normal XSLT-callbacks. Tried different methods to tell 
> lxml which callbacks I have, none worked.
> (global namespace, callbacks as "extensions"-parameter for etree.XSLT)
> 
> XSLT-sample-snippet:
> <xsl:value-of select="py:highlightInline(string(.))"
> disable-output-escaping="yes"/>
> (Namespace is defined, callback gets called and works 
> fine...until I try to use the code with mod_python)
> 
> > Which Apache MPM are you using?  If you're using a multiple-process 
> > module, you might try swithing to a single-process-multiple-thread 
> > module to see if this behavior changes.
> 
> Using prefork here, as all threaded modules have problems 
> with mod_php.
> mod_php might be another error-source. Read something about 
> failing DB-connections when using mod_php and mod_python. But 
> I don't really think disabling mod_php will make a difference here.
> 
> Greetings, David Danier
> _______________________________________________
> lxml-dev mailing list
> lxml-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/lxml-dev
>