[lxml-dev] Help getting lxml to work reliably on MacOS-X

[Stefan Behnel]

Hi,

it looks like many Mac users have problems with lxml on their platform. This
usually involves installing up-to-date dependencies (libxml2/libxslt) in
addition to the system libraries. I would like to get these problems resolved.

To get a start on this, we must collect some information. We had a few
reports, but I need to know in more detail what people did, what they tried,
and to what avail. So here is a list of questions for Mac users. Please help
us by answering them. Some instructions follow at the end.

When building lxml, please move any installed Cython versions out of the way
and run the build on the unpacked lxml-2.0.tar.gz release sources. It must say
"trying to build without Cython" at the beginning.

Please provide the following information:

- what package management system (fink/macports) do you use?

- are you using the stock Python or one that is installed separately?

- what library versions are you using of libxml2, libxslt, zlib, libiconv?

- which library versions are preinstalled on your platform?
  (I do not know how to find that out, can anyone provide instructions here?)

- what library versions does lxml.etree find? (see below)


Just in case there are people who actually have a working installation,

- has anyone successfully built lxml statically against libxml2/libxslt?

  * does it work reliably? (see "Testing" below)
  * did you build with the --without-threads option?
  * does it work with *and* without that option?

- has anyone managed to get lxml working reliably (see "Testing" below) with
  a dynamic build?

  * did you set DYLD_LIBRARY_PATH?
  * is DYLD_LIBRARY_PATH required for you or does it work without?
  * is there anything special you did to make it work?
  * if there are crashes, is the install unusable or are there things you can
    still do reliably?


If lxml crashes for you,

- does it work if you set DYLD_LIBRARY_PATH at runtime? (dynamic builds only)

- does it work when building with the --without-threads option?

- does it crash when running the normal tests?

- if the tests pass fine, does it crash with the "ot-test" script below?


Every reply is appreciated. You can reply in private e-mail, if you prefer,
although it might be helpful to others to see your answer.

Thanks in advance,
Stefan


Here are some instructions:

* Checking library versions

Please report the library versions that lxml.etree thinks it is using:

http://codespeak.net/lxml/FAQ.html#i-think-i-have-found-a-bug-in-lxml-what-should-i-do

* Setting DYLD_LIBRARY_PATH

When you run tests or an application with lxml, you can pass the environment
variable DYLD_LIBRARY_PATH to your program. It needs to be set to the
directory (or directories) where lxml's library dependencies are installed,
i.e. libxml2 and libxslt.

http://codespeak.net/lxml/build.html#providing-newer-library-versions-on-mac-os-x

* Testing

When I say "works reliably", I mean, without crashes. The first thing to
verify that is to run "make test". If it already crashes here, there is no
need to try the script below. Please report anything you can find out about
the crash in this case. A debugger run and/or a stack trace might give us some
useful hints here.

If the normal tests pass, please try another test. Here is an XML document and
a script that fires off a bunch of threads to parse the document and run
multiple XSLTs over it. The archive is about 1MB.

http://codespeak.net/lxml/ot-test.zip

It's kind of a worst-case scenario that I use to find problems, so if you have
a working installation of lxml 2.0, please run it to see if it crashes for
you. Parsing may fail (usually for threads 7-9), that's fine, but it must not
crash with your otherwise working setup. It normally starts up 16 threads,
which should require a couple of hundred MB of memory. If you find that it
runs out of memory, you can try running less threads instead by passing the
number to the script at the command line.