URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: piet at vanoostrum.org
From lei at ipac.caltech.edu Tue May 19 23:40:13 2009
From: lei at ipac.caltech.edu (Mary Lei)
Date: Tue, 19 May 2009 14:40:13 -0700
Subject: [lxml-dev] could not find doctest
Message-ID: <4A13273D.6020100@ipac.caltech.edu>
I copied the following from the lxml document on the web:
#! /stage/irsa-sw-dev/cm/env_1.4_mnt/python/bin/python
import urllib
import urllib2
import urlparse
import os
import popen2
import StringIO
import sys, getopt
import re
import string
import doctest
version = sys.version_info
print "python version: ", version
print os.environ['PYTHONPATH']
## test with lxml
import lxml.html
from lxml.html import builder as E
from lxml.html import usedoctest
html = E.HTML(
E.HEAD(
E.LINK(rel="stylesheet", href="great.css", type="text/css"),
E.TITLE("Best Page Ever")
),
E.BODY(
E.H1(E.CLASS("heading"), "Top News"),
E.P("World News only on this page", style="font-size: 200%"),
"Ah, and here's some more text, by the way.",
lxml.html.fromstring(" and this is a parsed fragment
")
)
)
print lxml.html.tostring(html)
And got the following error:
python version: (2, 5, 1, 'final', 0)
.:/home/lei/python-stuff/BeautifulSoup-3.1.0.1:./lib:/home/lei/lxml-2.2/src
Traceback (most recent call last):
File "/home/lei/python-stuff/test_lxml_htmlout.py", line 22, in
from lxml.html import usedoctest
File "/home/lei/lxml-2.2/src/lxml/html/usedoctest.py", line 13, in
doctestcompare.temp_install(html=True, del_module=__name__)
File "/home/lei/lxml-2.2/src/lxml/doctestcompare.py", line 395, in
temp_install
frame = _find_doctest_frame()
File "/home/lei/lxml-2.2/src/lxml/doctestcompare.py", line 486, in
_find_doctest_frame
"Could not find doctest (only use this function *inside* a doctest)")
LookupError: Could not find doctest (only use this function *inside* a
doctest)
Any clues to this problem ?
--
Mary Lei
Software Testing
IPAC-NExScl
Rm: KS-233
MS: 220-6
Phone: 395-1998
From lei at ipac.caltech.edu Tue May 19 23:44:19 2009
From: lei at ipac.caltech.edu (Mary Lei)
Date: Tue, 19 May 2009 14:44:19 -0700
Subject: [lxml-dev] parse unable to load url
Message-ID: <4A132833.8050009@ipac.caltech.edu>
I copied sample code from lxml documentation
#! /stage/irsa-sw-dev/cm/env_1.4_mnt/python/bin/python
import urllib
import urllib2
import urlparse
import os
import popen2
import StringIO
import sys, getopt
import re
import string
version = sys.version_info
print "python version: ", version
print os.environ['PYTHONPATH']
## test with lxml
from lxml import etree
from lxml.html import fromstring, tostring, parse, submit_form
#page = parse('http://tinyurl.com').getroot()
page = parse('/home/lei/python-stuff/tmp_CoRoT_exo_index.html').getroot()
print page
page = parse('http://tinyurl.com').getroot()
And got errors when parsing url, local file is ok
dodo:lei > test_lxml.py
python version: (2, 5, 1, 'final', 0)
.:/home/lei/python-stuff/BeautifulSoup-3.1.0.1:./lib:/home/lei/lxml-2.2/src
Traceback (most recent call last):
File "test_lxml.py", line 27, in
page = parse('http://tinyurl.com').getroot()
File "/home/lei/lxml-2.2/src/lxml/html/__init__.py", line 661, in parse
return etree.parse(filename_or_url, parser, base_url=base_url, **kw)
File "lxml.etree.pyx", line 2693, in lxml.etree.parse
(src/lxml/lxml.etree.c:52591)
File "parser.pxi", line 1478, in lxml.etree._parseDocument
(src/lxml/lxml.etree.c:75665)
File "parser.pxi", line 1507, in lxml.etree._parseDocumentFromURL
(src/lxml/lxml.etree.c:75993)
File "parser.pxi", line 1407, in lxml.etree._parseDocFromFile
(src/lxml/lxml.etree.c:75002)
File "parser.pxi", line 965, in
lxml.etree._BaseParser._parseDocFromFile (src/lxml/lxml.etree.c:72023)
File "parser.pxi", line 539, in
lxml.etree._ParserContext._handleParseResultDoc
(src/lxml/lxml.etree.c:67830)
File "parser.pxi", line 625, in lxml.etree._handleParseResult
(src/lxml/lxml.etree.c:68877)
File "parser.pxi", line 563, in lxml.etree._raiseParseError
(src/lxml/lxml.etree.c:68093)
IOError: Error reading file 'http://tinyurl.com': failed to load HTTP
resource
--
Mary Lei
Software Testing
IPAC-NExScl
Rm: KS-233
MS: 220-6
Phone: 395-1998
From sergio at sergiomb.no-ip.org Wed May 20 04:17:13 2009
From: sergio at sergiomb.no-ip.org (Sergio Monteiro Basto)
Date: Wed, 20 May 2009 03:17:13 +0100
Subject: [lxml-dev] parse unable to load url
In-Reply-To: <4A132833.8050009@ipac.caltech.edu>
References: <4A132833.8050009@ipac.caltech.edu>
Message-ID: <1242785833.2437.9.camel@monteirov>
Forgot cc to list !
On Tue, 2009-05-19 at 14:44 -0700, Mary Lei wrote:
> page = parse('http://tinyurl.com').getroot()
http://tinyurl.com redirect to http://tinyurl.com/
page = parse('http://tinyurl.com/').getroot()
works
best regards
--
S?rgio M.B.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 2192 bytes
Desc: not available
Url : http://codespeak.net/pipermail/lxml-dev/attachments/20090520/9647cc28/attachment.bin
From stefan_ml at behnel.de Wed May 20 09:35:03 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 20 May 2009 09:35:03 +0200 (CEST)
Subject: [lxml-dev] lxml 2.2 validation question
In-Reply-To: <18963.2625.405755.476698@cochabamba.local>
References: <88bf15530905181921v334ad86dyb098871925983b0f@mail.gmail.com>
<20090519090643.288760@gmx.net>
<18963.2625.405755.476698@cochabamba.local>
Message-ID: <4c23cbfec2ca32a0e50b2fb5ed508646.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Piet van Oostrum wrote:
>> jholg at gmx.de wrote:
> >>> nsmap={None: 'http://example.com', 'foo': 'http://example.com'}
> >>> rootElem = etree.Element('Foo', {}, nsmap)
>
>> Note that this does not put Foo into the http://example.com NS. It
>> creates an element Foo wit no namespace. The nsmap is rather a collection
>> of known namespace prefixes in the context of an element.
>
> But that means that the serialization:
>
>
> Contents
>
>
> that etree.tostring produces is wrong.
Correct. This is actually an error case that we could catch at the API
level: the new element has no namespace *and* the nsmap defines a default
namespace, i.e. this should fail:
el = etree.Element('thetag', nsmap={None : 'uri:some-namespace'})
We'd then need to make sure that you can write
el = etree.Element('{}thetag', nsmap={None : 'uri:some-namespace'})
which would result in something like
Similarly, adding a namespace-less subelement to a tree that defines a
default namespace will not work 'as expected' (whatever a user may expect
in doing that). In this case, we'd have to cut the default namespace
definition by inserting a xmlns="" on the new element, so this case would
not be an error. The same applies in the general case where you create a
tree without namespaces and one that uses a default namespace, and then
insert the namespace-less tree into the other one at some place.
Another sick case:
el1 = etree.fromstring(
'')
el2 = etree.Element("{uri:tns}thetag", nsmap={None: "uri:otherns"})
el2.append(el1)
So it looks like we'd have to integrate something similar into the
namespace fixing mechanism...
Ugly, ugly...
Stefan
From stefan_ml at behnel.de Wed May 20 09:47:23 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 20 May 2009 09:47:23 +0200 (CEST)
Subject: [lxml-dev] lxml 2.2 validation question
In-Reply-To: <88bf15530905191118u6bf40e09gf65869e69e82b16d@mail.gmail.com>
References: <88bf15530905181921v334ad86dyb098871925983b0f@mail.gmail.com>
<4A12F00E.6040009@behnel.de>
<88bf15530905191118u6bf40e09gf65869e69e82b16d@mail.gmail.com>
Message-ID:
James Slagle wrote:
> My main issue is with an external tool I'm passing my generated xml
> to. This tool expects no prefixes on the elements, but prefixes on
> the attributes, and thus needs the namespace declared with a prefix
> and as the default. Yes, I know this is broken, and the tool needs to
> be fixed to be more flexible :).
... "to support XML namespaces", you mean. ;-)
> I was mainly wanting to know if it was possible to use lxml to
> generate xml output in this manner.
I recall adding a namespace setup rule that explicitly prefers the
prefixed namespace over an equivalent default namespace when you define
both on the same node. This is because otherwise you end up with a similar
problem with unnamespaced attributes on namespaced elements.
The output you get is a side effect of that fix, so, no, there isn't a way
to define a namespace as both the default namespace and a prefixed
namespace, and basically let lxml ignore the prefixed namespace in favour
of the default.
That said, was there actually a reason why you defined the namespace
prefix in the first place? Why isn't the default namespace enough to do
what you want? (Note that the prefix used in the schema is independent of
the one used in the document, that's what I meant with prefixes being
'overrated'.)
Stefan
From stefan_ml at behnel.de Wed May 20 09:52:46 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 20 May 2009 09:52:46 +0200 (CEST)
Subject: [lxml-dev] could not find doctest
In-Reply-To: <4A13273D.6020100@ipac.caltech.edu>
References: <4A13273D.6020100@ipac.caltech.edu>
Message-ID:
Mary Lei wrote:
> I copied the following from the lxml document on the web:
... and you copied a bit too much.
> from lxml.html import usedoctest
>
> Traceback (most recent call last):
> File "/home/lei/lxml-2.2/src/lxml/doctestcompare.py", line 486, in
> _find_doctest_frame
> "Could not find doctest (only use this function *inside* a doctest)")
> LookupError: Could not find doctest (only use this function *inside* a
> doctest)
As the error message tells you, importing "usedoctest" will only work from
a doctest. Unless you are writing a doctest, there is no reason to use
that module.
Stefan
From lei at ipac.caltech.edu Wed May 20 19:14:00 2009
From: lei at ipac.caltech.edu (Mary Lei)
Date: Wed, 20 May 2009 10:14:00 -0700
Subject: [lxml-dev] parse unable to load url
In-Reply-To: <1242785833.2437.9.camel@monteirov>
References: <4A132833.8050009@ipac.caltech.edu>
<1242785833.2437.9.camel@monteirov>
Message-ID: <4A143A58.3020301@ipac.caltech.edu>
This works great for me:
tiny
fd >
['http://tinyurl.com/2xae8s', 'http://preview.tinyurl.com/2xae8s']
Thanks so much.
I am beginning to believe in lxml.
Sorry, Stefan :-)
Sergio Monteiro Basto wrote:
> Forgot cc to list !
>
> On Tue, 2009-05-19 at 14:44 -0700, Mary Lei wrote:
>> page = parse('http://tinyurl.com').getroot()
> http://tinyurl.com redirect to http://tinyurl.com/
>
> page = parse('http://tinyurl.com/').getroot()
> works
>
> best regards
>
--
Mary Lei
Software Testing
IPAC-NExScl
Rm: KS-233
MS: 220-6
Phone: 395-1998
From lei at ipac.caltech.edu Wed May 20 20:06:36 2009
From: lei at ipac.caltech.edu (Mary Lei)
Date: Wed, 20 May 2009 11:06:36 -0700
Subject: [lxml-dev] could not find doctest
In-Reply-To:
References: <4A13273D.6020100@ipac.caltech.edu>
Message-ID: <4A1446AC.9060406@ipac.caltech.edu>
I discovered the following for my purpose:
---------------------------------------
import lxml.html
## html output
html = lxml.html.fromstring('''\
Creating HTML with the E-factory
Hi !
''')
print lxml.html.tostring(html)
#Hi !
print lxml.html.tostring(html)
from lxml.html import builder as E
html = E.HTML(
E.HEAD(
# E.LINK(rel="stylesheet", href="great.css", type="text/css"),
E.TITLE("Best Page Ever")
),
E.BODY(
E.H1(E.CLASS("heading"), "Top News"),
E.P("World News only on this page", style="font-size: 200%"),
"Ah, and here's some more text, by the way.",
lxml.html.fromstring(" and this is a parsed fragment
")
)
)
print lxml.html.tostring(html,pretty_print=True) <-- is what I want
------------------------------------------
but got confused by doctest yesterday, being new to python.
Stefan Behnel wrote:
> Mary Lei wrote:
>> I copied the following from the lxml document on the web:
>
> ... and you copied a bit too much.
>
>> from lxml.html import usedoctest
>>
>> Traceback (most recent call last):
>> File "/home/lei/lxml-2.2/src/lxml/doctestcompare.py", line 486, in
>> _find_doctest_frame
>> "Could not find doctest (only use this function *inside* a doctest)")
>> LookupError: Could not find doctest (only use this function *inside* a
>> doctest)
>
> As the error message tells you, importing "usedoctest" will only work from
> a doctest. Unless you are writing a doctest, there is no reason to use
> that module.
>
> Stefan
--
Mary Lei
Software Testing
IPAC-NExScl
Rm: KS-233
MS: 220-6
Phone: 395-1998
From james.slagle at gmail.com Wed May 20 21:27:41 2009
From: james.slagle at gmail.com (James Slagle)
Date: Wed, 20 May 2009 15:27:41 -0400
Subject: [lxml-dev] lxml 2.2 validation question
In-Reply-To:
References: <88bf15530905181921v334ad86dyb098871925983b0f@mail.gmail.com>
<4A12F00E.6040009@behnel.de>
<88bf15530905191118u6bf40e09gf65869e69e82b16d@mail.gmail.com>
Message-ID: <88bf15530905201227t6a43e8a1na22880945d7af6d8@mail.gmail.com>
On Wed, May 20, 2009 at 3:47 AM, Stefan Behnel wrote:
> That said, was there actually a reason why you defined the namespace
> prefix in the first place? Why isn't the default namespace enough to do
> what you want? (Note that the prefix used in the schema is independent of
> the one used in the document, that's what I meant with prefixes being
> 'overrated'.)
There was no good reason. I was merely trying to match the example
xml that I have for working with this particular tool.
Thanks for the helpful information, it has certainly helped clear
things up for me.
--
-- James Slagle
--
From ovnicraft at gmail.com Mon May 25 17:54:17 2009
From: ovnicraft at gmail.com (Ovnicraft)
Date: Mon, 25 May 2009 10:54:17 -0500
Subject: [lxml-dev] UTF-8 not supported
Message-ID:
Hi folks, when do this, encoding='iso-8859-1' write xml enconding ok, but
when try the same thing with 'UTF-8', not appears in my file, i have 2.2
version.
How i can encoding my file with utf-8?
regards,
--
http://twitter.com/ovnicraft
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/lxml-dev/attachments/20090525/ec9f7e64/attachment.htm
From shigin at rambler-co.ru Mon May 25 18:01:00 2009
From: shigin at rambler-co.ru (Alexander Shigin)
Date: Mon, 25 May 2009 20:01:00 +0400
Subject: [lxml-dev] UTF-8 not supported
In-Reply-To:
References:
Message-ID: <1243267260.6964.4414.camel@atlas>
? ???, 25/05/2009 ? 10:54 -0500, Ovnicraft ?????:
> Hi folks, when do this, encoding='iso-8859-1' write xml enconding ok,
> but when try the same thing with 'UTF-8', not appears in my file, i
> have 2.2 version.
> How i can encoding my file with utf-8?
Can you give your code snippet? UTF-8 works fine for me.
Here is an example:
In [6]: print etree.tostring(etree.Element(unicode('???', 'utf-8')), encoding='utf-8')
If you haven't got cyrillic letters:
In [9]: etree.tostring(etree.Element(u'\u044b\u044a\u044a'), encoding='utf-8')
Out[9]: '<\xd1\x8b\xd1\x8a\xd1\x8a/>'
From sergio at sergiomb.no-ip.org Mon May 25 18:11:45 2009
From: sergio at sergiomb.no-ip.org (Sergio Monteiro Basto)
Date: Mon, 25 May 2009 17:11:45 +0100
Subject: [lxml-dev] UTF-8 not supported
In-Reply-To: <1243267260.6964.4414.camel@atlas>
References:
<1243267260.6964.4414.camel@atlas>
Message-ID: <1243267905.4887.2.camel@segulix>
Hi,
I use to set the enconding to parser like this:
hparser = etree.HTMLParser(encoding='utf-8', remove_comments=True)
etree_document = etree.HTML(f, parser=hparser)
On Mon, 2009-05-25 at 20:01 +0400, Alexander Shigin wrote:
> ? ???, 25/05/2009 ? 10:54 -0500, Ovnicraft ?????:
> > Hi folks, when do this, encoding='iso-8859-1' write xml enconding ok,
> > but when try the same thing with 'UTF-8', not appears in my file, i
> > have 2.2 version.
> > How i can encoding my file with utf-8?
>
> Can you give your code snippet? UTF-8 works fine for me.
>
> Here is an example:
> In [6]: print etree.tostring(etree.Element(unicode('???', 'utf-8')), encoding='utf-8')
>
>
> If you haven't got cyrillic letters:
> In [9]: etree.tostring(etree.Element(u'\u044b\u044a\u044a'), encoding='utf-8')
> Out[9]: '<\xd1\x8b\xd1\x8a\xd1\x8a/>'
>
>
>
> _______________________________________________
> lxml-dev mailing list
> lxml-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/lxml-dev
--
S?rgio M. B.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 2192 bytes
Desc: not available
Url : http://codespeak.net/pipermail/lxml-dev/attachments/20090525/596f8e46/attachment.bin
From ovnicraft at gmail.com Mon May 25 18:20:28 2009
From: ovnicraft at gmail.com (Ovnicraft)
Date: Mon, 25 May 2009 11:20:28 -0500
Subject: [lxml-dev] UTF-8 not supported
In-Reply-To: <1243267260.6964.4414.camel@atlas>
References:
<1243267260.6964.4414.camel@atlas>
Message-ID:
2009/5/25 Alexander Shigin
> ? ???, 25/05/2009 ? 10:54 -0500, Ovnicraft ?????:
> > Hi folks, when do this, encoding='iso-8859-1' write xml enconding ok,
> > but when try the same thing with 'UTF-8', not appears in my file, i
> > have 2.2 version.
> > How i can encoding my file with utf-8?
>
> Can you give your code snippet? UTF-8 works fine for me.
>
> Here is an example:
> In [6]: print etree.tostring(etree.Element(unicode('???', 'utf-8')),
> encoding='utf-8')
>
>
> If you haven't got cyrillic letters:
> In [9]: etree.tostring(etree.Element(u'\u044b\u044a\u044a'),
> encoding='utf-8')
> Out[9]: '<\xd1\x8b\xd1\x8a\xd1\x8a/>'
>
>
>
>
I write something very simple, http://lxml.pastebin.com/m3f8c5b9c
--
Cristian Salamea
CEO GnuThink Software Labs
Software Libre / Open Source
(+593-8) 4-36-44-48
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/lxml-dev/attachments/20090525/70c5a1ab/attachment.htm
From ovnicraft at gmail.com Mon May 25 18:23:46 2009
From: ovnicraft at gmail.com (Ovnicraft)
Date: Mon, 25 May 2009 11:23:46 -0500
Subject: [lxml-dev] UTF-8 not supported
In-Reply-To:
References:
<1243267260.6964.4414.camel@atlas>
Message-ID:
2009/5/25 Ovnicraft
>
>
> 2009/5/25 Alexander Shigin
>
> ? ???, 25/05/2009 ? 10:54 -0500, Ovnicraft ?????:
>> > Hi folks, when do this, encoding='iso-8859-1' write xml enconding ok,
>> > but when try the same thing with 'UTF-8', not appears in my file, i
>> > have 2.2 version.
>> > How i can encoding my file with utf-8?
>>
>> Can you give your code snippet? UTF-8 works fine for me.
>>
>> Here is an example:
>> In [6]: print etree.tostring(etree.Element(unicode('???', 'utf-8')),
>> encoding='utf-8')
>>
>>
>> If you haven't got cyrillic letters:
>> In [9]: etree.tostring(etree.Element(u'\u044b\u044a\u044a'),
>> encoding='utf-8')
>> Out[9]: '<\xd1\x8b\xd1\x8a\xd1\x8a/>'
>>
>>
>>
>>
> I write something very simple, http://lxml.pastebin.com/m3f8c5b9c
With utf-8, http://lxml.pastebin.com/m72a14d.
>
>
>
>
> --
> Cristian Salamea
> CEO GnuThink Software Labs
> Software Libre / Open Source
> (+593-8) 4-36-44-48
>
--
Cristian Salamea
CEO GnuThink Software Labs
Software Libre / Open Source
(+593-8) 4-36-44-48
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/lxml-dev/attachments/20090525/e353e9e9/attachment-0001.htm
From shigin at rambler-co.ru Mon May 25 18:28:55 2009
From: shigin at rambler-co.ru (Alexander Shigin)
Date: Mon, 25 May 2009 20:28:55 +0400
Subject: [lxml-dev] UTF-8 not supported
In-Reply-To:
References:
<1243267260.6964.4414.camel@atlas>
Message-ID: <1243268936.6964.4425.camel@atlas>
? ???, 25/05/2009 ? 11:23 -0500, Ovnicraft ?????:
> I write something very simple, http://lxml.pastebin.com/m3f8c5b9c
>
> With utf-8, http://lxml.pastebin.com/m72a14d.
In [33]: etree.tostring(openerp, pretty_print=True, encoding='utf-8') == \
....: etree.tostring(openerp, pretty_print=True,
encoding='utf-8')
Out[33]: True
Is it possible that you paste the wrong code snippet? I don't see any
non-ascii chars and the output is the same for me.
From ovnicraft at gmail.com Mon May 25 21:41:58 2009
From: ovnicraft at gmail.com (Ovnicraft)
Date: Mon, 25 May 2009 14:41:58 -0500
Subject: [lxml-dev] UTF-8 not supported
In-Reply-To: <1243268936.6964.4425.camel@atlas>
References:
<1243267260.6964.4414.camel@atlas>
<1243268936.6964.4425.camel@atlas>
Message-ID:
2009/5/25 Alexander Shigin
> ? ???, 25/05/2009 ? 11:23 -0500, Ovnicraft ?????:
> > I write something very simple, http://lxml.pastebin.com/m3f8c5b9c
> >
> > With utf-8, http://lxml.pastebin.com/m72a14d.
>
>
> In [33]: etree.tostring(openerp, pretty_print=True, encoding='utf-8') == \
> ....: etree.tostring(openerp, pretty_print=True,
> encoding='utf-8')
> Out[33]: True
>
> Is it possible that you paste the wrong code snippet? I don't see any
> non-ascii chars and the output is the same for me.
>
>
http://lxml.pastebin.com/m5d4a419,in this, the hightlighted line dont
appears in the output when use utf-8
--
Cristian Salamea
CEO GnuThink Software Labs
Software Libre / Open Source
(+593-8) 4-36-44-48
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/lxml-dev/attachments/20090525/d9da1f81/attachment.htm
From limi at plone.org Tue May 26 00:03:35 2009
From: limi at plone.org (Alexander Limi)
Date: Mon, 25 May 2009 15:03:35 -0700
Subject: [lxml-dev] Binary egg for Mac OS X
Message-ID:
Hi,
I'm working on documenting Deliverance / xdv for use with Plone. It has
lxml as a dependency, and I have run into a serious issue:
On Mac OS X, we can't assume that people have Xcode (ie. gcc and friends)
installed, thus we can't really compile lxml on those computers, not even
using the staticlxml[1] recipe.
I see that there are binary eggs for Windows, is there a special reason
why there are no binary eggs for OS X, or is it just a matter of not
having the infrastructure to make it available?
Happy to help find a solution if it's just a matter of locating a reliable
way to get it compiled every time there is a new release.
[1] http://pypi.python.org/pypi/z3c.recipe.staticlxml/
--
Alexander Limi ? http://limi.net
From shigin at rambler-co.ru Tue May 26 11:04:25 2009
From: shigin at rambler-co.ru (Alexander Shigin)
Date: Tue, 26 May 2009 13:04:25 +0400
Subject: [lxml-dev] UTF-8 not supported
In-Reply-To:
References:
<1243267260.6964.4414.camel@atlas>
<1243268936.6964.4425.camel@atlas>
Message-ID: <1243328665.6964.4441.camel@atlas>
? ???, 25/05/2009 ? 14:41 -0500, Ovnicraft ?????:
> http://lxml.pastebin.com/m5d4a419,in this, the hightlighted line dont
> appears in the output when use utf-8
Oh, I get it. The XML specification says:
"""
In the absence of information provided by an external transport protocol
(e.g. HTTP or MIME), it is a fatal error for an entity including an
encoding declaration to be presented to the XML processor in an encoding
other than that named in the declaration, or for an entity which begins
with neither a Byte Order Mark nor an encoding declaration to use an
encoding other than UTF-8.
"""
i.e. utf-8 is default encoding for xml and the xml library can omit
encoding declaration if encoding is utf-8.
The tostring routine has an option xml_declaration. The option force
lxml to write or omit xml encoding declaration.
In [37]: print etree.tostring(openerp, pretty_print=True, encoding='utf-8', xml_declaration=True)
field
From avleen at gmail.com Wed May 27 17:16:17 2009
From: avleen at gmail.com (Avleen Vig)
Date: Wed, 27 May 2009 16:16:17 +0100
Subject: [lxml-dev] Possible bug, libxml segfault?
Message-ID: <33c66c80905270816le007e19ua8844d5a5d8b8c28@mail.gmail.com>
Hi folks :)
I sent this message to the xml at gnome list, but that place is seems
very quiet lately. I'm hoping someone here might have some insight.
Indeed this might be a bug in lxml, we're not sure.
Background:
We use libxml and libxslt in one of our applications (specifically, in
Python via lxml).
Recently we've seen our application dying at strange times for no
apparent reason.
We managed to get a core file out of one crash, and the results of
some of our debugging are here:
http://xml.pastebin.com/m70c259d6
(I'd be happy to poke more in a particular direction on there, I'm a
bit new to gdb :)
To me, it seems the parser is complaining while trying to parse the
namespaces in the node in transforms/_base.xslt
The node for that opens like this:
I dug a little deeper and found a bunch of the "address out of bounds"
errors and thought I should ask here as I'm drawing a blank on where
to go next.
The problem happens intermittently, but usually several times a day. I
could probably reproduce it.
I also see the 'exclude-result-prefixes' mentioned in the backtrace,
could that be involved here?
Any suggestions you have would be much appreciated!
From stefan_ml at behnel.de Sat May 30 13:04:14 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 30 May 2009 13:04:14 +0200
Subject: [lxml-dev] Binary egg for Mac OS X
In-Reply-To:
References:
Message-ID: <4A2112AE.8040903@behnel.de>
Hi,
Alexander Limi wrote:
> I'm working on documenting Deliverance / xdv for use with Plone. It has
> lxml as a dependency, and I have run into a serious issue:
>
> On Mac OS X, we can't assume that people have Xcode (ie. gcc and friends)
> installed, thus we can't really compile lxml on those computers, not even
> using the staticlxml[1] recipe.
>
> I see that there are binary eggs for Windows, is there a special reason
> why there are no binary eggs for OS X, or is it just a matter of not
> having the infrastructure to make it available?
The main problem is that many MacOS-X users have some kind of package
distribution like macports installed, which usually has some distribution
specific setup/dependencies/paths/whatever. OTOH, those users won't be the
target for a binary distribution of lxml anyway.
> Happy to help find a solution if it's just a matter of locating a
> reliable way to get it compiled every time there is a new release.
Yes, I'd be happy if we could get a static binary egg for each release. I
don't have a Mac myself (and I'm definitely not a Mac user), so
contributions are welcome.
http://codespeak.net/lxml/build.html#building-lxml-on-macos-x
Stefan
From stefan_ml at behnel.de Sat May 30 13:25:34 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 30 May 2009 13:25:34 +0200
Subject: [lxml-dev] Possible bug, libxml segfault?
In-Reply-To: <33c66c80905270816le007e19ua8844d5a5d8b8c28@mail.gmail.com>
References: <33c66c80905270816le007e19ua8844d5a5d8b8c28@mail.gmail.com>
Message-ID: <4A2117AE.7010908@behnel.de>
Hi,
Avleen Vig wrote:
> Background:
> We use libxml and libxslt in one of our applications (specifically, in
> Python via lxml).
>
> Recently we've seen our application dying at strange times for no
> apparent reason.
> We managed to get a core file out of one crash, and the results of
> some of our debugging are here:
> http://xml.pastebin.com/m70c259d6
> (I'd be happy to poke more in a particular direction on there, I'm a
> bit new to gdb :)
The information you give is pretty informative. It just lacks a hint which
versions of lxml, libxml2 and libxslt you are using. Is this reproducible
with the latest release versions of all three?
> To me, it seems the parser is complaining while trying to parse the
> namespaces in the node in transforms/_base.xslt
> The node for that opens like this:
> xmlns="http://www.w3.org/1999/xhtml";
> xmlns:xsl="http://www.w3.org/1999/XSL/Transform";
> xmlns:tfxslt="http://tfnet.co.uk/ns/tfxslt";
> xmlns:fb="http://www.facebook.com/2008/fbml";
> extension-element-prefixes="tfxslt str exsl"
> exclude-result-prefixes="str exsl tfxslt fb"
> xmlns:error="http://www.woome.com/error/";>
The triggering problem is that you try to exclude the namespace prefixes
"str" and "exsl" which are not defined in your stylesheet. This seems to
induce a problem in lxml's error reporting mechanism in your specific setup.
Is your application threaded? I do remember a couple of problems with
threaded error reporting in the past, although all of those were solved
back then.
Stefan
From qhlonline at 163.com Sun May 31 04:29:42 2009
From: qhlonline at 163.com (qhlonline)
Date: Sun, 31 May 2009 10:29:42 +0800 (CST)
Subject: [lxml-dev] problem about lxml encoding
Message-ID: <28194717.92641243736982945.JavaMail.coremail@bj163app25.163.com>
Hi, all
I have a question about lxml encoding and html encoding. What is the relationship between the encoding which lxml parser will be using and the encoding of an HTML file? if the HTML file encoding is not the same with it's , What the choice of lxml's parser will be? the 'charset' encoding or the HTML file coding?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/lxml-dev/attachments/20090531/1357b75d/attachment.htm
From stefan_ml at behnel.de Sun May 31 08:10:16 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 31 May 2009 08:10:16 +0200
Subject: [lxml-dev] problem about lxml encoding
In-Reply-To: <28194717.92641243736982945.JavaMail.coremail@bj163app25.163.com>
References: <28194717.92641243736982945.JavaMail.coremail@bj163app25.163.com>
Message-ID: <4A221F48.7080503@behnel.de>
qhlonline wrote:
> Hi, all I have a question about lxml encoding and html encoding. What is
> the relationship between the encoding which lxml parser will be using
> and the encoding of an HTML file? if the HTML file encoding is not the
> same with it's , What the choice of lxml's
> parser will be?
The HTML parser will use the encoding specified by the tag if it's
present, otherwise it will expect the document to be Latin-1. It will not
magically guess the right encoding if the tag happens to be
incorrect. If you somehow know the encoding better, you can override the
behaviour by passing the "encoding" option when instantiating the parser.
Does that help?
Stefan
From jamie at artefact.org.nz Sun May 31 08:40:33 2009
From: jamie at artefact.org.nz (Jamie Norrish)
Date: Sun, 31 May 2009 18:40:33 +1200
Subject: [lxml-dev] xpath on text nodes
In-Reply-To: <4A09BE54.1030501@behnel.de>
References: <1240994671.8989.9.camel@atman.artefact.org.nz>
<1241065853.5570.4.camel@atman.artefact.org.nz>
<1241122046.5549.19.camel@atman.artefact.org.nz>
<4A053F34.9040809@behnel.de>
<1241861247.5355.41.camel@atman.artefact.org.nz>
<4A09BE54.1030501@behnel.de>
Message-ID: <1243752033.6621.21.camel@atman.artefact.org.nz>
Stefan Behnel wrote:
> This sounds like your algorithm is already more complex than a simple "any
> text node preceding the one that matches". That convinces me that an API
> based solution will be a lot more flexible than anything you could scratch
> out of XPath. It would allow you to special case certain tag types, for
> example, or to notice when you cross parent boundaries.
Well, the original plan didn't really call for much special casing of
particular elements, but now that things are working I'll likely add in
such as I think of them. I've changed the approach completely, to use
XSLT to transform the entire document into something that has the
context handled appropriately (and using XPath on text nodes :). It
takes two transformations (the second one to handle ordering issues with
the preceding context, and to do a little cleanup of whitespace, but it
is more than an order of magnitude faster than what I had before.
I'm not sure why I didn't go down that route in the first place, but now
that I have I'm very happy. And of course it's great that XSLT is so
easy to use with lxml.
Oh, I also tried using .getparent() and some logic to get the equivalent
of preceding::text()[1] and following::text()[1], but it turned out (not
surprisingly, given the complexities of that approach) to be marginally
slower than what I had.
Jamie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url : http://codespeak.net/pipermail/lxml-dev/attachments/20090531/b28709dd/attachment-0001.pgp
From stefan_ml at behnel.de Sun May 31 11:29:05 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 31 May 2009 11:29:05 +0200
Subject: [lxml-dev] problem about lxml encoding
In-Reply-To: <26911870.306181243760617643.JavaMail.coremail@bj163app60.163.com>
References: <4A221F48.7080503@behnel.de>
<28194717.92641243736982945.JavaMail.coremail@bj163app25.163.com>
<26911870.306181243760617643.JavaMail.coremail@bj163app60.163.com>
Message-ID: <4A224DE1.5000907@behnel.de>
Hi,
please keep the list involved. Thanks.
qhlonline wrote:
> Since I am processing Chinese Webs. There are instances that some Webs
> are not regular. When it said , we can't decode
> the HTML string with GB2312 decoder, we found that this content is
> encoded with GBK or GB18030 in fact. But the lxml parser will process it
> according to the meta declaration. I have read the source of libxml2,
> there are no "encoding" string match "GB2312","GBK" or "GB18030", but
> some other encodings like ENCODING_2022_JP which may be a super set of
> GB2312 I just don't know where in libxml2 or in lxml the declared
> GB2312 encoding is converted to some other encodings that apparently
> supported by libxml2's XML's xmlCharEncoding structure, and How?
Most encodings are handled by libiconv. libxml2 only handles the 'normal'
(or most common) XML encodings in its own code.
> I think
> since "GB18030" is a super set of "GB2312", if we change the lxml source
> to let all strings were decoded with GB18030
> codec, then there will be no error even if the HTML file is not regular.
No need to play with lxml's sources here. What should work is to read the
HTML page into a byte string, decode it manually into a unicode string, and
then let lxml parse that. That way, you also have full control over the
decoding and can handle any decoding errors yourself.
Stefan
From foolistbar at googlemail.com Sun May 31 14:28:31 2009
From: foolistbar at googlemail.com (Geoffrey Sneddon)
Date: Sun, 31 May 2009 13:28:31 +0100
Subject: [lxml-dev] problem about lxml encoding
In-Reply-To: <4A224DE1.5000907@behnel.de>
References: <4A221F48.7080503@behnel.de>
<28194717.92641243736982945.JavaMail.coremail@bj163app25.163.com>
<26911870.306181243760617643.JavaMail.coremail@bj163app60.163.com>
<4A224DE1.5000907@behnel.de>
Message-ID: <4BF7C14A-DE65-492D-8C69-EB5B04F213F7@googlemail.com>
On 31 May 2009, at 10:29, Stefan Behnel wrote:
> qhlonline wrote:
>> Since I am processing Chinese Webs. There are instances that some
>> Webs
>> are not regular.
If you need support for deployed web content, you'll probably have
better success with html5lib than you will with libxml2's HTML parser.
--
Geoffrey Sneddon