[lxml-dev] Using, or building, lxml in Windows with Cygwin

[Stefan Behnel]

Hi,

David Antliff wrote:
> I am trying a slightly different approach - compiling entirely within
> Cygwin, using Cygwin's gcc.

Right, that should be best anyway.


> What I have done is unpacked lxml-2.2.1.tgz.gz into lxml-2.2.1, then
> inside that directory I try:
> 
> $ python setup.py build --static-deps
> 
> This proceeds to download libxml2 and libxslt, unpack them, and build
> them. But it runs into numerous problems related to include/library
> paths.

... which you may be able to fix using appropriate CFLAGS/LDFLAGS.


> Here are my 'fixes' as they pop up. I run the correct command
> manually, then type 'make' to continue the process. Unfortunately if I
> run that setup.py command again, it clears out the progress-so-far, so
> I'm not sure what I'm going to do once Make completes successfully...
> 
> running build
> running build_py
> running build_ext
> building 'lxml.objectify' extension
> gcc -shared -Wl,--enable-auto-image-base
> build/temp.cygwin-1.5.25-i686-2.5/src/lxml/lxml.objectify.o
> /cygdrive/d/git/work/lxml-exp/lxml-2.2.1/build/tmp/libxml2/lib/libexslt.a
> /cygdrive/d/git/work/lxml-exp/lxml-2.2.1/build/tmp/libxml2/lib/libxml2.a
> /cygdrive/d/git/work/lxml-exp/lxml-2.2.1/build/tmp/libxml2/lib/libxslt.a
> -L/cygdrive/d/git/work/lxml-exp/lxml-2.2.1/build/tmp/libxml2/lib
> -L/usr/lib/python2.5/config -lz -lm -lpython2.5 -o
> build/lib.cygwin-1.5.25-i686-2.5/lxml/objectify.dll
> /cygdrive/d/git/work/lxml-exp/lxml-2.2.1/build/tmp/libxml2/lib/libxml2.a(encoding.o):
> In function `xmlFindCharEncodingHandler':
> /cygdrive/d/git/work/lxml-exp/lxml-2.2.1/build/tmp/libxml2-2.7.3/encoding.c:1614:
> undefined reference to `_libiconv_open'
> ...more libiconv errors...
> 
> I fix this by adding "/usr/lib/libiconv.a":
> $ gcc -shared -Wl,--enable-auto-image-base
> build/temp.cygwin-1.5.25-i686-2.5/src/lxml/lxml.objectify.o
> /cygdrive/d/git/work/lxml-exp/lxml-2.2.1/build/tmp/libxml2/lib/libexslt.a
> /cygdrive/d/git/work/lxml-exp/lxml-2.2.1/build/tmp/libxml2/lib/libxml2.a
> /cygdrive/d/git/work/lxml-exp/lxml-2.2.1/build/tmp/libxml2/lib/libxslt.a
> -L/cygdrive/d/git/work/lxml-exp/lxml-2.2.1/build/tmp/libxml2/lib
> -L/usr/lib/python2.5/config -lz -lm -lpython2.5 -o
> build/lib.cygwin-1.5.25-i686-2.5/lxml/objectify.dll
> /usr/lib/libiconv.a

I don't know if you install the libiconv developer package on your machine
(I hope that exists in Cygwin), because building against the shared lib
should work just fine here. Maybe you need to point gcc to the right
include and/or lib directory. I wonder why it didn't add "-liconv"
automatically...

You can also try to go the ugly route and add "/usr/lib/libiconv.a" to your
CFLAGS, but I think you'll be happier with the shared lib.


> I wonder why setup.py didn't automatically download libiconv when it
> downloaded libxml2 and libxslt... hmm

Because the static build was designed for MacOS-X, where only those two
libraries are a problem. The libiconv is binary compatible enough across
versions not to pose any major problems. So it's best to build dynamically
against libiconv.

That said, it shouldn't be too hard to add code to also download libiconv
and build it. I would be happy to receive a patch that accepts an optional
list of library names for the --static-deps option, as in
"--static-deps=libxml2,libxslt,iconv", and would then download and build
all requested libraries. Although I doubt that it would make sense (or even
work) to pass only "--static-deps=iconv", so maybe a new option
"--build-iconv" would be better.


> Now I type 'make' to continue to the next problem:
> Using build configuration of libxslt
> running build_ext
> building 'lxml.etree' extension
> gcc -shared -Wl,--enable-auto-image-base
> build/temp.cygwin-1.5.25-i686-2.5/src/lxml/lxml.etree.o
> -L/usr/lib/python2.5/config -lxslt -lexslt -lxml2 -lz -lm -lpython2.5
> -o build/lib.cygwin-1.5.25-i686-2.5/lxml/etree.dll
> /usr/lib/gcc/i686-pc-cygwin/3.4.4/../../../../i686-pc-cygwin/bin/ld:
> cannot find -lxslt
> collect2: ld returned 1 exit status
> 
> But at this point I'm not convinced 'make' is doing the same thing
> that the --static-deps would cause. So I'm not sure this is a valid
> path to take.

Yes, this is linking dynamically against libxslt.


> Is there a way to have 'setup.py build' continue from where it last
> reached, rather than starting everything again from scratch?

No. The goal is a completely automated build, and there is no option for
manual interaction. (Well, you can always start the debugger from the setup
script and run your own code, but I doubt that that's something that's
worth maintaining in the long term).

Stefan