[lxml-dev] lxml + mod_python: cannot unmarshal code objects in restricted execution mode

[Stefan Behnel]

Hi,

David Danier wrote:
>> Somehow restricted mode is only mentioned in the docs for RExec
>> (http://docs.python.org/lib/module-rexec.html), but should not be
>> available any more, to I don't know what lxml exactly does to use callbacks.
> 
> Found another place that mentions restricted mode by accident:
> http://www.modpython.org/live/current/doc-html/pyapi-interps.html
> 
> I think this paragraph describes the problem pretty well:
> ------------8<----------------------------------------------------
> Note that if any third party module is being used which has a C code
> component that uses the simplified API for access to the Global
> Interpreter Lock (GIL) for Python extension modules, then the
> interpreter name must be forcibly set to be "main_interpreter". This is
> necessary as such a module will only work correctly if run within the
> context of the first Python interpreter created by the process. If not
> forced to run under the "main_interpreter", a range of Python errors can
> arise, each typically referring to code being run in restricted mode.
> ---------------------------------------------------->8------------
> (thanks to Lee Brown for asking about where lxml is called, it made me
> read the mod_python-docs again)

thanks for the infos, that's good to know.


> I'll try to setup my site on mod_python and using "PythonInterpreter
> main_interpreter" in the config. According to the docs this might
> help...but if I read this right might produce namespace-problems or at
> least pollute some global namespace. As this takes some time I will post
> the result later.

Please do.


> Perhaps it can be fixed in lxml by not using the "simplified API for
> access to the Global Interpreter Lock (GIL) for Python extension modules"?

No way. There's a reason why it is there which is the same why we use it: it's
simple and usable. Using anything else would mean a lot of rewriting.

You might want to try compiling lxml with "--without-threading", though, which
disables concurrency support completely (i.e. not more GIL freeing).

Stefan