[lxml-dev] Debugging tools?

[Marius Gedminas]

On Wed, Jan 14, 2009 at 12:38:36PM -0600, Ian Bicking wrote:
> Given sufficient load, I've been able to fairly consistently cause a
> segfault in Deliverance, which I suspect is due to lxml (it's on the one
> segfaulty part of the stack).  How should I go about debugging this?

Create an automated reproducible test case, if you can.

Get a stack trace with debugging symbols for python, lxml, libxml2 and
any other relevant libraries.

Poke around with gdb at the point of the segfault, looking for NULL
pointers or whatnot.

> I also suspect there's a memory leak, probably in lxml (there's not many
> other persistent structures).  The segfault kind of trumps that for the
> moment.  So... if I figure out the segfault (or at least how to repeat it),
> how might I go about finding a memory leak?

Create an automated reproducible test case, if you can.

See if http://mg.pov.lt/blog/hunting-python-memleaks gives you any
ideas about extracting information from the 'gc' module's introspection
capabilities.  If you know that objects of some class should be all
garbage-collected at the end of your program, but you find that some of
them are still alive, you can do a search through the graph defined by
gc.get_referrers() to find which module (or stack frame) is holding that
reference.  Or you can look for an evidence of a refcount bug by
searching for objects with have a sys.getrefcount(obj) !=
len(gc.get_referrers(obj)).

Some people praise Dowser (http://www.aminus.net/wiki/Dowser).  I
haven't played with it, so I cannot describe it.

Then there's heapy and guppy.  I tried playing with them a while ago and
was scared away by the complexity of the setup.

Marius Gedminas
-- 
Never trust a computer you can't repair yourself.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://codespeak.net/pipermail/lxml-dev/attachments/20090117/f3bcdf97/attachment.pgp