[lxml-dev] Leaking tracebacks

[Christian Heimes]

Stefan Behnel wrote:

> There was a change in Cython regarding exception handling not so long ago.
> Cython now mimics more the behaviour of Py3, i.e. exceptions disappear once
> they were caught. If the problem was introduced at this point, it would
> explain why this appears with later lxml releases only, as older releases
> are based on older Cython versions.

I was able to come up with a minimal test case. It shows that lxml 2.0.7 
is fine but 2.0.9 and 2.1.2 are leaking references somewhere.

> I'm not sure what the problem is here, though, so this needs further
> investigation. From a couple of quick tests, I can't seem to reproduce this
> problem. Could you try to come up with some test code that shows this
> behaviour? Is it only a problem with functionality that interacts with
> libxml2 in both directions (such as XPath/XSLT, which uses function
> callbacks back into lxml), or does it also happen in functions like
> etree.iselement()?

---
import sys
import pkg_resources
pkg_resources.require("lxml==%s" % sys.argv[1])

from lxml import etree

class SampleException(Exception):
     pass

def test():
     print "lxml: %s, %s, %s" % (etree.LXML_VERSION,
                                 etree.LIBXML_VERSION,
                                 etree.LIBXSLT_VERSION)
     print "SampleException start: %i" % sys.getrefcount(SampleException)
     for i in range(1000):
         try:
             raise SampleException
         except Exception, err:
             el = etree.Element("test")
             if i == 999:
                 print "exception: %i" % sys.getrefcount(err)
     print "SampleException stop: %i" % sys.getrefcount(SampleException)

if __name__ == "__main__":
     test()

---

Output:
$ python2.5 lxml_bug.py 2.0.7
lxml: (2, 0, 7, 0), (2, 6, 31), (1, 1, 24)
SampleException start: 4
exception: 4
SampleException stop: 7

$ python2.5 lxml_bug.py 2.0.9
lxml: (2, 0, 9, 0), (2, 6, 31), (1, 1, 24)
SampleException start: 4
exception: 5
SampleException stop: 2006

$ python2.5 lxml_bug.py 2.1.2
lxml: (2, 1, 2, 0), (2, 6, 31), (1, 1, 24)
SampleException start: 4
exception: 5
SampleException stop: 2006


Every loop leaks 2 references to the exeption class.

Christian