[lxml-dev] Ref-counting bug returns in 1.3.1

[Grimes, David]

If there's anything else I can do to help test/diagnose, let me know ...
 
 --Dave
 

________________________________

From: Stefan Behnel [mailto:stefan_ml at behnel.de]
Sent: Tue 7/3/2007 4:41 PM
To: Grimes, David
Cc: lxml-dev at codespeak.net
Subject: Re: [lxml-dev] Ref-counting bug returns in 1.3.1




David M. Grimes wrote:
>   I posted a patch for what I believed to be a reference-counting bug in
> Attrib.pop() based on the 1.3 release.  The patch was accepted, and is
> present in 1.3.1.  The patch is included at the end of this message.
> Looking through the generated C code, I'm no longer sure my patch was
> correct - perhaps just masking the underlying problem in 1.3.  I'm not
> fluent in Pyrex, so not sure if the python.Py_INCREF is really necessary
> for something which would be a "borrwed reference" in the C-API
> (PyTuple_GET_ITEM result).  It looks like the Pyrex "return" is
> generating it's own INCREF ...

You can debug this kind of problem with

    print sys.getrefcount(None)

When I run the following on 1.3.1:

    et = etree.fromstring(xml)
    for i in range(10000):
        print i
        print sys.getrefcount(None)
        et.attrib.pop('x', None)
        print sys.getrefcount(None)

instead of your test, it shows me that the problem is not "pop()", as the
ref-count is constant at each iteration. Trying to remove the Py_INCREF() from
your patch makes it crash with a continuously decreasing ref-count. However,
when I run your test:

    for i in range(10000):
        print i
        et = etree.fromstring(xml)
        print sys.getrefcount(None)
        et.attrib.pop('x', None)
        print sys.getrefcount(None)

the ref-count keeps increasing until the garbage collector hits and then drops
below the start value and finally crashes on the second GC run. So the problem
is somewhere else. I'll investigate.

Thanks for the report,
Stefan



This e-mail is the property of NaviSite, Inc. It is intended only
for the person or entity to which it is addressed and may contain
information that is privileged, confidential, or otherwise protected
from disclosure. Distribution or copying of this e-mail, or the
information contained herein, to anyone other than the intended
recipient is prohibited.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/lxml-dev/attachments/20070703/e5e8ccbc/attachment.htm