[lxml-dev] python crashes in xmlDictFree inside Zope

Sun Jun 3 18:35:12 CEST 2007

OK, I think I've sorted this out to a certain degree.

Using lxml 1.3 beta (including threading support) works to parse and
validate my XMLSchema stuff using the builtin MacOS libxml2 libs (2.6.16).
I think the failures I was seeing earlier had to do with my cluelessness
about how Zope coughs up file data from the database.

I will next try using with the MacPorts version of libxml2 2.6.28, but I'll
turn on the threading switches in the configuration (or, maybe, turn them
off).  I suspect that my earlier problems with the lxml - libxml2
configurations were due to mismatched threading expectations.

Anyway, thanks for the excellent product, and I'll report back on any
further insights.

ET

On 5/31/07 9:29 AM, "Stefan Behnel" <stefan_ml at behnel.de> wrote:

> Hi Eric,
> 
> Eric Tiffany wrote:
>>  OK, some more info.  If I use the builtin libxml2 (2.6.16) libs from
>> Apple, rather than the 2.6.28 version from MacPorts, then I don¹t get
>> these errors crashing Python/Zope.
> 
> That's expected, lxml switches off threading support for this version. It's
> the same as doing "--without-threading". That's why I was puzzled when you
> said --without-threading doesn't help you.
> 
> 
>>  However, the lxml parsing/validation
>> doesn¹t seem to work correctly.
> 
> That's expected, too, XMLSchema ist still under development and definitely was
> at the time.
> 
> 
>> So, it seems that Apple has built their libs in a more friendly way, but
>> I¹m now wondering whether there is some known issue with using lxml with
>> libxml2 2.6.16.
>> 
>> I¹m continuing to investigate.
> 
> Please do. It hard for me to come up with a solution without being able to
> reproduce the problem.
> 
> Stefan
> 
> 
>> On 5/30/07 2:14 PM, "Eric Tiffany" <etiffany at alum.mit.edu> wrote:
>> 
>>     Sorry for the delay in responding -- been on vacation in Italy.
>>      Responses inline.  I am quite mystified at this point.
>> 
>>     On 5/21/07 2:21 AM, "Stefan Behnel" <stefan_ml at behnel.de> wrote:
>> 
>>> Hi,
>>> 
>>> Eric Tiffany wrote:
>>>> I have been prototyping some XMLSchema parsing/validating using lxml
>>>> 1.3beta.
>>>>  
>>>> Everthing works great from python 2.4.4 started from the command
>>     line, or
>>>> running from inside Eclipse.
>>>>  
>>>> However, when I moved my code over to my Plone product, python
>>     crashes when
>>>> Zope is initializing the product. I am creating my XMLSchema
>>     object there.
>>>>  [...]
>>> 
>>> Is this the Python version?
>> 
>>     For some reason, the python reports its version incorrectly in the
>>     crashdump.  It is actually 2.4.4.
>> 
>>> 
>>> Is there any way to detect MacOS-X at the C level? In that case, we
>>     could try
>>> to disable thread concurrency support completely for this platform
>>     - in case
>>> that's the source of the segfault. You can try to see if this would
>>     fix the
>>> problem by passing the option "--without-threading" to setup.py
>>     when building
>>> lxml. Could you please try that with your current setup and report
>>     back to the
>>> list?
>> 
>>     There are certainly ways to detect MacOS at compile-time, though I'm
>>     not sure of the details.  I get this from the shell:
>> 
>>     $ uname -a
>>     Darwin etmac.local 8.9.1 Darwin Kernel Version 8.9.1: Thu Feb 22
>>     20:55:00 PST 2007; root:xnu-792.18.15~1/RELEASE_I386 i386 i386
>> 
>>     I have attempted to build from SVN using --without-threading using
>> 
>>     $ make clean
>>     $ python setup.py build --without-threading
>>     $ sudo python setup.py install
>> 
>>     (interestingly, it makes an egg called
>>     lxml-1.3beta-py2.4-macosx-10.3-i386.egg even though the OS version
>>     10.4.9 not 10.3, but whatever).
>> 
>>     Also, I see a problem with the self tests regarding
>>     test_module_HTML_unicode, but I'll report that elsewhere.
>> 
>>> 
>>> Another question: are you using a custom parser (i.e. passing a second
>>> argument to the parse() function) here or is it the default parser that
>>> crashes here?
>> 
>>     It is the default parser.  And it is still crashing inside Zope even
>>     with the --without-threading.  Here is my code:
>> 
>>     schemaPath =
>>     "/Applications/Plone-2.5.2/Instance/Products/xtend/xtend/schedules.xsd"
>>     print >> sys.stderr, "Loading schema doc from ", schemaPath
>>     schemaDoc = etree.parse(schemaPath) print >> sys.stderr, "creating
>>     XMLSchema ..." schemaTree = etree.XMLSchema(schemaDoc) print >>
>>     sys.stderr, "Trying validation"
>> 
>>     And here is the output (when running inside Zope):
>> 
>>     Loading schema doc from
>>      /Applications/Plone-2.5.2/Instance/Products/xtend/xtend/schedules.xsd
>>     creating XMLSchema ...
>>     Bus error
>> 
>>     So it seems pretty clear that it is croaking while trying to do the
>>     XMLSchema construction.
>> 
>>     Inside a python shell, that code runs fine.  In both environments
>>     (zope and shell) I have
>> 
>>     lxml.etree:        (1, 3, -1, 43887)
>>     libxml used:       (2, 6, 28)
>>     libxml compiled:   (2, 6, 28)
>>     libxslt used:      (1, 1, 20)
>>     libxslt compiled:  (1, 1, 20)
>> 
>> 
>>     Here is the thread backtrace for the thread that crashed:
>> 
>>     Thread 1 Crashed:
>>     0   <<00000000>>     0xffff07c7 __memcpy + 39 (cpu_capabilities.h:228)
>>     1   libSystem.B.dylib     0x9000b569 __sfvwrite + 409
>>     2   libSystem.B.dylib     0x9001063d __vfprintf + 19692
>>     3   libSystem.B.dylib     0x90011428 vfprintf + 91
>>     4   libxml2.2.dylib       0x91befd3b xmlGenericErrorDefaultFunc + 75
>>     5   libxml2.2.dylib       0x0354bad1 xmlSchemaCheckFacet + 709
>>     6   libxml2.2.dylib       0x0354c020
>>     xmlSchemaFixupSimpleTypeStageTwo + 927
>>     7   libxml2.2.dylib       0x0355175b xmlSchemaFixupComponents + 4054
>>     8   libxml2.2.dylib       0x03552207 xmlSchemaParse + 290
>>     9   etree.so              0x06464039
>>     __pyx_f_5etree_9XMLSchema___init__ + 980 (etree.c:38191)
>>     10  org.python.python     0x0025283e type_call + 166 (typeobject.c:435)
>>     11  org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     12  org.python.python     0x0027e397 PyEval_EvalFrame + 16838
>>     (ceval.c:3776)
>>     13  org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     14  org.python.python     0x00228063 function_call + 320
>>     (funcobject.c:548)
>>     15  org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     16  org.python.python     0x00215667 instancemethod_call + 401
>>     (classobject.c:2532)
>>     17  org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     18  org.python.python     0x00214c90 instance_call + 90
>>     (classobject.c:2087)
>>     19  org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     20  org.python.python     0x0027e69c PyEval_EvalFrame + 17611
>>     (ceval.c:3845)
>>     21  org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     22  org.python.python     0x00228063 function_call + 320
>>     (funcobject.c:548)
>>     23  org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     24  org.python.python     0x00215667 instancemethod_call + 401
>>     (classobject.c:2532)
>>     25  org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     26  org.python.python     0x00214c90 instance_call + 90
>>     (classobject.c:2087)
>>     27  org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     28  org.python.python     0x0027e69c PyEval_EvalFrame + 17611
>>     (ceval.c:3845)
>>     29  org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     30  org.python.python     0x00228063 function_call + 320
>>     (funcobject.c:548)
>>     31  org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     32  org.python.python     0x0027e69c PyEval_EvalFrame + 17611
>>     (ceval.c:3845)
>>     33  org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     34  org.python.python     0x0027e49f PyEval_EvalFrame + 17102
>>     (ceval.c:3661)
>>     35  org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     36  org.python.python     0x0027e49f PyEval_EvalFrame + 17102
>>     (ceval.c:3661)
>>     37  org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     38  org.python.python     0x0027e49f PyEval_EvalFrame + 17102
>>     (ceval.c:3661)
>>     39  org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     40  org.python.python     0x00228063 function_call + 320
>>     (funcobject.c:548)
>>     41  org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     42  org.python.python     0x0027e69c PyEval_EvalFrame + 17611
>>     (ceval.c:3845)
>>     43  org.python.python     0x0027ebaa PyEval_EvalFrame + 18905
>>     (ceval.c:3651)
>>     44  org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     45  org.python.python     0x0027e49f PyEval_EvalFrame + 17102
>>     (ceval.c:3661)
>>     46  org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     47  org.python.python     0x00228063 function_call + 320
>>     (funcobject.c:548)
>>     48  org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     49  org.python.python     0x00215667 instancemethod_call + 401
>>     (classobject.c:2532)
>>     50  org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     51  org.python.python     0x0027e69c PyEval_EvalFrame + 17611
>>     (ceval.c:3845)
>>     52  org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     53  org.python.python     0x00228063 function_call + 320
>>     (funcobject.c:548)
>>     54  org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     55  org.python.python     0x00215667 instancemethod_call + 401
>>     (classobject.c:2532)
>>     56  org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     57  org.python.python     0x0027e69c PyEval_EvalFrame + 17611
>>     (ceval.c:3845)
>>     58  org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     59  org.python.python     0x00228063 function_call + 320
>>     (funcobject.c:548)
>>     60  org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     61  org.python.python     0x00215667 instancemethod_call + 401
>>     (classobject.c:2532)
>>     62  org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     63  org.python.python     0x0027e69c PyEval_EvalFrame + 17611
>>     (ceval.c:3845)
>>     64  org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     65  org.python.python     0x00228063 function_call + 320
>>     (funcobject.c:548)
>>     66  org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     67  org.python.python     0x00215667 instancemethod_call + 401
>>     (classobject.c:2532)
>>     68  org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     69  org.python.python     0x0027944a PyEval_CallObjectWithKeywords +
>>     112 (ceval.c:3435)
>>     70  _Acquisition.so       0x0151e9ac CallMethodO + 60
>>     (_Acquisition.c:97)
>>     71  org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     72  org.python.python     0x0027944a PyEval_CallObjectWithKeywords +
>>     112 (ceval.c:3435)
>>     73  org.python.python     0x002715a1 builtin_apply + 201
>>     (bltinmodule.c:100)
>>     74  org.python.python     0x0027faca PyEval_EvalFrame + 22777
>>     (ceval.c:3568)
>>     75  org.python.python     0x0027ebaa PyEval_EvalFrame + 18905
>>     (ceval.c:3651)
>>     76  org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     77  org.python.python     0x0027e49f PyEval_EvalFrame + 17102
>>     (ceval.c:3661)
>>     78  org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     79  org.python.python     0x0027e49f PyEval_EvalFrame + 17102
>>     (ceval.c:3661)
>>     80  org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     81  org.python.python     0x00228063 function_call + 320
>>     (funcobject.c:548)
>>     82  org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     83  org.python.python     0x0027e69c PyEval_EvalFrame + 17611
>>     (ceval.c:3845)
>>     84  org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     85  org.python.python     0x00228063 function_call + 320
>>     (funcobject.c:548)
>>     86  org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     87  org.python.python     0x00215667 instancemethod_call + 401
>>     (classobject.c:2532)
>>     88  org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     89  org.python.python     0x0027944a PyEval_CallObjectWithKeywords +
>>     112 (ceval.c:3435)
>>     90  _Acquisition.so       0x0151e9ac CallMethodO + 60
>>     (_Acquisition.c:97)
>>     91  org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     92  org.python.python     0x0027944a PyEval_CallObjectWithKeywords +
>>     112 (ceval.c:3435)
>>     93  org.python.python     0x002715a1 builtin_apply + 201
>>     (bltinmodule.c:100)
>>     94  org.python.python     0x0027faca PyEval_EvalFrame + 22777
>>     (ceval.c:3568)
>>     95  org.python.python     0x0027ebaa PyEval_EvalFrame + 18905
>>     (ceval.c:3651)
>>     96  org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     97  org.python.python     0x0027e49f PyEval_EvalFrame + 17102
>>     (ceval.c:3661)
>>     98  org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     99  org.python.python     0x00228063 function_call + 320
>>     (funcobject.c:548)
>>     100 org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     101 org.python.python     0x00215667 instancemethod_call + 401
>>     (classobject.c:2532)
>>     102 org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     103 org.python.python     0x0027944a PyEval_CallObjectWithKeywords +
>>     112 (ceval.c:3435)
>>     104 _Acquisition.so       0x0151e9ac CallMethodO + 60
>>     (_Acquisition.c:97)
>>     105 org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     106 org.python.python     0x0027944a PyEval_CallObjectWithKeywords +
>>     112 (ceval.c:3435)
>>     107 org.python.python     0x002715a1 builtin_apply + 201
>>     (bltinmodule.c:100)
>>     108 org.python.python     0x0027faca PyEval_EvalFrame + 22777
>>     (ceval.c:3568)
>>     109 org.python.python     0x0027ebaa PyEval_EvalFrame + 18905
>>     (ceval.c:3651)
>>     110 org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     111 org.python.python     0x0027e49f PyEval_EvalFrame + 17102
>>     (ceval.c:3661)
>>     112 org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     113 org.python.python     0x0027e49f PyEval_EvalFrame + 17102
>>     (ceval.c:3661)
>>     114 org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     115 org.python.python     0x0027e49f PyEval_EvalFrame + 17102
>>     (ceval.c:3661)
>>     116 org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     117 org.python.python     0x0027e49f PyEval_EvalFrame + 17102
>>     (ceval.c:3661)
>>     118 org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     119 org.python.python     0x0027e49f PyEval_EvalFrame + 17102
>>     (ceval.c:3661)
>>     120 org.python.python     0x00280665 PyEval_EvalCodeEx + 1774
>>     (ceval.c:2741)
>>     121 org.python.python     0x00228063 function_call + 320
>>     (funcobject.c:548)
>>     122 org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     123 org.python.python     0x00215667 instancemethod_call + 401
>>     (classobject.c:2532)
>>     124 org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     125 org.python.python     0x0027944a PyEval_CallObjectWithKeywords +
>>     112 (ceval.c:3435)
>>     126 org.python.python     0x00217aaa PyInstance_New + 114
>>     (classobject.c:588)
>>     127 org.python.python     0x0020d87f PyObject_Call + 45
>>     (abstract.c:1795)
>>     128 org.python.python     0x0027944a PyEval_CallObjectWithKeywords +
>>     112 (ceval.c:3435)
>>     129 org.python.python     0x002b3126 t_bootstrap + 62
>>     (threadmodule.c:434)
>>     130 libSystem.B.dylib     0x90024987 _pthread_body + 84
>> 
>>     _______________________________________________
>>     lxml-dev mailing list
>>     lxml-dev at codespeak.net
>>     http://codespeak.net/mailman/listinfo/lxml-dev
>> 
>> 
>> 
>> 
>> ------------------------------------------------------------------------
>> 
>> _______________________________________________
>> lxml-dev mailing list
>> lxml-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/lxml-dev