[lxml-dev] lxml Mac installation idea

[Ian Bicking]

Stefan Behnel wrote:
> Hi Ian,
> 
> Ian Bicking wrote:
>> So... I hear that lxml installs better on a Mac if it's built along with 
>> libxml2/libxslt.  That's not what everyone would do, so I was unclear 
>> how to enable something like that, and if setup.py would be the right 
>> place.  A number of attempts to get stuff setup have been tried before 
>> external to lxml (like buildouts and now staticlxml).
> 
> I hadn't heard of static lxml yet, so I'll have to check what it's doing.
> 
> Anyway, is it still that hard to install lxml on a Mac? Later 2.0 versions and
> 2.1 should behave much better here, given that they use "-flat_namespace" now.

Yeah, I'm getting reports of problems.  Some class of people have 
figured out the way to get it installed, but it's still a big problem, 
and I hear from lots of people who won't use lxml because of it.


>> While it's kind of lame, I wonder if enabling this static installation 
>> via an environmental variable would be reasonable?  It would be easier 
>> to apply in a number of circumstances.  I imagine it would mean 
>> something like, on installation, if a variable like LXML_INSTALL_STATIC 
>> (or INSTALL_LIBXML2 or something) was set, it'd download the libxml2 and 
>> libxslt libraries, run configure/make/make install with a prefix inside 
>> the lxml source directory itself, then build lxml using that library.
> 
> But isn't that what a buildout does best?

Yes, but I'd like it to work without buildout, and there's also several 
buildout recipes and configurations out there and not one clear 
canonical way to build lxml.


>> Another option would be simply a different tarball that contains the 
>> libxml2/libxslt source, and its setup.py would always build those.  It 
>> could be versioned like 2.1static or something, which should keep it 
>> from being implicitly used by easy_install, etc. (since 2.1static is 
>> considered an earlier version than 2.1).  This might be more reasonable?
> 
> The problem with this (and with the static Windows builds) is that
> libxml2/libxslt both have their release cycles, which are independent of
> lxml's releases. If you want to upgrade your libxml2 in a static build, you'll
> have to copy it to the right place anyway.

Another option is yet another environmental variable to set the 
libxml2/libxslt versions, which are set to defaults.  If you chose a 
version that didn't exist in the tarball it'd download that version.


>>   staticlxml is kind of weird, because installing staticlxml installs 
>> lxml, which can confuse tools.
> 
> Yep, I agree that that's not the way to go.
> 
> 
>> Maybe the two versions could be arranged with some svn:externals,
> 
> You mean lxml and libxml2? That would mean you either build libxml2 from a tag
> (implying the same problems as with a shipped, ready-to-get-outdated version),
> or from the trunk, which is definitely not suitable for most users.

No, I was just thinking about ways of structuring a full build (that 
includes the libxml2 library) and the current build.


>> or just as a build script of some sort (e.g., 
>> drop a marker file in the source to make it "static", and have setup.py 
>> look for that file and change the sdist/install commands appropriately).
> 
> That's just another way of triggering it, in which case I prefer the env var way.
> 
> 
>> One nuisance with any attempt to fix this is that I don't think either 
>> myself nor Stefan have ready access to a Mac to test this stuff...
> 
> Yes, that's part of the problem. Another problem is the way this problem pops
> up. From time to time, Mac users complain on the list that it doesn't work for
> them to build lxml. Some can provide helpful hints, debugging time or patches,
> others cannot. It's impossible for me to find out if things are really
> settled, or if there are still Mac users out there who just do not feel like
> investing any work to at least complain.
> 
> From what I witness, there haven't been any complains for a while, so I
> considered this problem settled since the late days of 2.0. If you bring it
> back up now (and if people feel urged to do things like staticlxml), it sounds
> to me like it's not.

Yeah... there seem to be a few problems:

* Macs come with bad versions of libxml2 and libxslt (depending on the 
version of the OS, you get either very bad, or not as bad, but bad 
enough that you'll eventually get a segfault but not immediately, which 
is actually much worse)

* There's several kinds of Python that people use on a Mac: at least the 
system python, macports python, and fink python.  I think there might be 
another.  They are all somewhat different.

* People keep getting the wrong runtime linking, and DYLD_LIBRARY_PATH 
seems to be necessary.

* It's not that obvious how to build libxml2, unless you are using 
macports (which has a port for it).

* Once you do build it, you have to be sure to get the right 
xml2-config, which doesn't happen by default.

These are the problems I've heard about at least.

I do now have ssh access to a Mac.  Unfortunately it's behind our VPN, 
so I'm not sure if I can get you access to it too, but I'll ask about that.


-- 
Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org