[lxml-dev] etree.clear_error_log() causes segfaults

[Stefan Behnel]

Hi,

Christian Heimes wrote:
> This is a friendly word of warning! Don't call etree.clear_error_log()
> from multiple threads.
> 
> We found this issue during stress tests of our CherryPy based
> application. Every worker thread was calling etree.clear_error_log()
> after the page was rendered. Apparently we hit some sort of race condition.
> 
> Backtrace:
> 
> C  [etree.so+0x21977]
> C  [etree.so+0x22d48]
> C  [libxslt.so.1+0xdc17]  xsltTransformError+0xf7
> C  [libxslt.so.1+0x839d]
> C  [libxslt.so.1+0xa5da]  xsltParseStylesheetProcess+0x80a
> C  [libxslt.so.1+0x1efbc]  xsltParseStylesheetInclude+0x1ac
> C  [libxslt.so.1+0xa0b4]  xsltParseStylesheetProcess+0x2e4
> C  [libxslt.so.1+0xb4b9]  xsltParseStylesheetImportedDoc+0x1e9
> C  [libxslt.so.1+0xb5b8]  xsltParseStylesheetDoc+0x28
> C  [etree.so+0xe640c]
> C  [python2.5+0x4ff83]
> C  [python2.5+0x11e67]  PyObject_Call+0x27

I think what happened here is that one thread cleared the log (my guess)
while another one was just appending an error (as the stacktrace shows).
The (somewhat radical) way to fix this is to make the global error log
thread local. I think that's the way it should work anyway, especially
since the error log is copied into exceptions. Otherwise, errors would leak
into exceptions of other threads.

I committed this change to SVN, it will be part of lxml 2.2. This means
that it will be safe to call clear_error_log() from a thread in 2.2.

Stefan