[lxml-dev] lxml \ libxslt \ libxml2 leads to apache 2 crash on freebsd/amd64

[Stefan Behnel]

Hi,

Dmitri Fedoruk wrote:
> So, just to me more precise - iit is truly a deallocation problem of
> libxml2 inside of Apache.
[example code stripped]
> Is it worth of reporting this crash to libxml2 / apache mailing lists,
> what would you say?

I'm sure it's not a problem in libxml2. Since I do not have enough
information, I do not know if the following explanation fits here, but I'll
give it anyway.

The way XSLT is implemented in lxml is a bit tricky, as libxslt makes some
things hard to control that lxml uses in libxml2 for performance reasons. In
particular, lxml uses a thread-local hash table for constant strings, which is
much faster than a malloc() for each string that occurs in a document.
However, libxslt doesn't honour this dictionary and creates its own one based
on the stylesheet dictionary. The result is that the stylesheet can leak into
the result document through string references that now point into the hash
table of the stylesheet.

There isn't a way in libxslt that would allow us to prevent this or to control
the allocation. That's why I decided to restrict the execution of XSL
transformations to threads that inherit the same hash table as the stylesheet,
this should normally prevent any problems.

As I said, this might or might not be the source of this particular problem.
Threading is always hard to get right, so maybe there are constellations where
the current restrictions are not enough. So far, I'm not aware of any.
Redesigning the way XSLT interacts with threads is not a small change and
quite risky, so I'd prefer considering that the last resort...

Stefan