[lxml-dev] [PATCH] 1.3 Fatal Python error: deallocating None

[David M. Grimes]

All -

I have identified a bug with the new attrib.pop() method in lxml-1.3  
There is a piece of code in etree.pyx (line 1483) related to using the 
provided default if the key lookup fails.  The issue is that the 
returned value (from PyTuple_GET_ITEM) is a borrowed reference.

I was using it as such:

  v = elt.attrib.get(k, None)

And eventually (after several iterations of my parsing loop while 
processing various documents) I would get this error:

 Fatal Python error: deallocating None


I was able to create a trivial test case to reproduce:

import lxml.etree as etree

xml = '''\
<?xml version="1.0"?>
<xml/>'''

for i in range(10000):
    et = etree.fromstring(xml)
    et.attrib.pop('x', None)



And the patched code replaces the pop() method in etree.pyx with:

    def pop(self, key, *default):
        if python.PyTuple_GET_SIZE(default) > 1:
            raise TypeError, "pop expected at most 2 arguments, got %d" % \
                  (python.PyTuple_GET_SIZE(default)+1)
        result = _getAttributeValue(self._element, key, None)
        if result is None:
            if python.PyTuple_GET_SIZE(default) == 0:
                raise KeyError, key
            else:
                result = python.PyTuple_GET_ITEM(default, 0)
                python.Py_INCREF(result)
        else:
            _delAttribute(self._element, key)

        return result


This solves the issue for me, and should not introduce any memory leaks, 
as the returned object in the defaulted path should be INCREF'd.

Thanks,

 --Dave

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/lxml-dev/attachments/20070627/0139c2a1/attachment.htm