[lxml-dev] Debugging tools?

[Stefan Behnel]

Hi Ian,

Ian Bicking wrote:
> Given sufficient load, I've been able to fairly consistently cause a
> segfault in Deliverance, which I suspect is due to lxml (it's on the one
> segfaulty part of the stack).  How should I go about debugging this?

You should make sure the system is not just running out of memory. There
are many cases where lxml can't handle malloc() failures in libxml2 gracefully.


> I also suspect there's a memory leak, probably in lxml (there's not many
> other persistent structures).  The segfault kind of trumps that for the
> moment.

Are you using 2.1.5? There is a memory leak in (at least) the 2.1.4 release
that is related to exception cleanup in Cython 0.10.x, x<3. lxml 2.2beta1
is also affected.


> So... if I figure out the segfault (or at least how to repeat it),
> how might I go about finding a memory leak?

Dag Seljebotn is currently working on a "ref-count nanny" for Cython, a
special debug mode that checks for ref-count leaks. Sadly, it's not a
trivial thing and it still has bugs in its own right. I don't know how
usable it currently is for that purpose.

Stefan