[lxml-dev] pickling ElementStringResult

[Martijn Faassen]

Hi there,

I just ran into a problem when upgrading an application to use lxml 
2.0.x instead of 1.3.6. xpath now returns a special 
lxml.etree._ElementStringResult object, a smart string. A bit too smart 
for my situation...

Previously, I'd get a string back and I could stuff that into the ZODB 
just fine - it was picklable. Now I get this and it can't be pickled, so 
the same code fails. Is it only xpath that is so affected or are there 
other operations where these things can be returned?

It's quite frustrating to have to worry whether a string came from an 
lxml xpath (and possibly operations?) and therefore is broken in a 
subtle way when you want to store them somewhere. You always worry 
whether your test coverage is complete enough to detect all cases. It's 
also relatively hard to track this kind of bug down even if you have 
good test coverage, as the error only occurs later on when the ZODB 
tries to pickle the object with this kind of special string somewhere in it.

Another potential problem is that a smart string when held in an object 
database might keep the underlying XML document alive long past its 
intended lifetime.

I was originally thinking about coming up with a special way to pickle 
these smart strings, but this actually leads me to prefer another 
solution: a flag that can be passed to xpath() that turns off the 
returning of smart strings at all. That would fit my use case quite 
well, though I'd have to remember to use the flag all the time. It might 
be nice if the flag could instead be passed to the parser, but I'm not 
sure whether that is implementable.

Regards,

Martijn