[Lxml-checkins] r51361 - in lxml/trunk: . doc
scoder at codespeak.net
scoder at codespeak.net
Sat Feb 9 18:31:28 CET 2008
Author: scoder
Date: Sat Feb 9 18:31:26 2008
New Revision: 51361
Modified:
lxml/trunk/ (props changed)
lxml/trunk/doc/performance.txt
Log:
r3457 at delle: sbehnel | 2008-02-09 18:30:53 +0100
updated benchmark results
Modified: lxml/trunk/doc/performance.txt
==============================================================================
--- lxml/trunk/doc/performance.txt (original)
+++ lxml/trunk/doc/performance.txt Sat Feb 9 18:31:26 2008
@@ -71,8 +71,8 @@
a specific part of the API yourself, please consider sending it to the lxml
mailing list.
-The timings cited below compare lxml 2.0alpha (with libxml2 2.6.30) to
-the December 2007 SVN trunk versions of ElementTree (1.3) and
+The timings cited below compare lxml 2.0 final (with libxml2 2.6.31)
+to the January 2008 SVN trunk versions of ElementTree (1.3) and
cElementTree (1.2.7). They were run single-threaded on a 1.8GHz Intel
Core Duo machine under Ubuntu Linux 7.10 (Gutsy). The C libraries
were compiled with the same platform specific optimisation flags. The
@@ -117,23 +117,23 @@
1.2, lxml is still more than 5 times as fast as the much improved
ElementTree 1.3::
- lxe: tostring_utf16 (SATR T1) 23.4821 msec/pass
+ lxe: tostring_utf16 (SATR T1) 19.0921 msec/pass
cET: tostring_utf16 (SATR T1) 129.8430 msec/pass
ET : tostring_utf16 (SATR T1) 136.1301 msec/pass
- lxe: tostring_utf16 (UATR T1) 23.4859 msec/pass
+ lxe: tostring_utf16 (UATR T1) 20.4630 msec/pass
cET: tostring_utf16 (UATR T1) 130.1570 msec/pass
ET : tostring_utf16 (UATR T1) 136.3101 msec/pass
- lxe: tostring_utf16 (S-TR T2) 24.2729 msec/pass
+ lxe: tostring_utf16 (S-TR T2) 18.8632 msec/pass
cET: tostring_utf16 (S-TR T2) 136.9388 msec/pass
ET : tostring_utf16 (S-TR T2) 143.9550 msec/pass
- lxe: tostring_utf8 (S-TR T2) 18.4860 msec/pass
+ lxe: tostring_utf8 (S-TR T2) 14.4310 msec/pass
cET: tostring_utf8 (S-TR T2) 137.0859 msec/pass
ET : tostring_utf8 (S-TR T2) 144.3110 msec/pass
- lxe: tostring_utf8 (U-TR T3) 2.7399 msec/pass
+ lxe: tostring_utf8 (U-TR T3) 2.6381 msec/pass
cET: tostring_utf8 (U-TR T3) 52.1040 msec/pass
ET : tostring_utf8 (U-TR T3) 53.1070 msec/pass
@@ -205,10 +205,10 @@
(given in seconds)::
lxe: -- S- U- -A SA UA
- T1: 0.0914 0.0875 0.0872 0.0892 0.0882 0.0900
- T2: 0.0894 0.0897 0.0892 0.0988 0.0978 0.0974
- T3: 0.0219 0.0194 0.0189 0.0570 0.0570 0.0573
- T4: 0.0004 0.0003 0.0003 0.0012 0.0012 0.0012
+ T1: 0.0783 0.0777 0.0774 0.0787 0.0781 0.0783
+ T2: 0.0799 0.0796 0.0799 0.0879 0.0882 0.0886
+ T3: 0.0245 0.0216 0.0217 0.0577 0.0575 0.0572
+ T4: 0.0003 0.0003 0.0003 0.0011 0.0011 0.0011
cET: -- S- U- -A SA UA
T1: 0.0272 0.0264 0.0267 0.0268 0.0261 0.0265
T2: 0.0280 0.0274 0.0273 0.0273 0.0276 0.0275
@@ -235,21 +235,21 @@
create a shallow copy of their list of children, lxml has to create a
Python object for each child and collect them in a list::
- lxe: root_list_children (--TR T1) 0.0169 msec/pass
+ lxe: root_list_children (--TR T1) 0.0160 msec/pass
cET: root_list_children (--TR T1) 0.0081 msec/pass
ET : root_list_children (--TR T1) 0.0541 msec/pass
- lxe: root_list_children (--TR T2) 0.2339 msec/pass
+ lxe: root_list_children (--TR T2) 0.2100 msec/pass
cET: root_list_children (--TR T2) 0.0319 msec/pass
ET : root_list_children (--TR T2) 0.4420 msec/pass
This handicap is also visible when accessing single children::
- lxe: first_child (--TR T2) 0.2470 msec/pass
+ lxe: first_child (--TR T2) 0.2429 msec/pass
cET: first_child (--TR T2) 0.2170 msec/pass
ET : first_child (--TR T2) 0.9968 msec/pass
- lxe: last_child (--TR T1) 0.2482 msec/pass
+ lxe: last_child (--TR T1) 0.2470 msec/pass
cET: last_child (--TR T1) 0.2291 msec/pass
ET : last_child (--TR T1) 0.9830 msec/pass
@@ -258,11 +258,11 @@
The data structure used by libxml2 is a linked tree, and thus, a
linked list of children::
- lxe: middle_child (--TR T1) 0.2789 msec/pass
+ lxe: middle_child (--TR T1) 0.2759 msec/pass
cET: middle_child (--TR T1) 0.2229 msec/pass
ET : middle_child (--TR T1) 1.0030 msec/pass
- lxe: middle_child (--TR T2) 1.9610 msec/pass
+ lxe: middle_child (--TR T2) 1.7071 msec/pass
cET: middle_child (--TR T2) 0.2229 msec/pass
ET : middle_child (--TR T2) 0.9930 msec/pass
@@ -274,7 +274,7 @@
in. This results in a major performance difference for creating independent
Elements that end up in independently created documents::
- lxe: create_elements (--TC T2) 3.1691 msec/pass
+ lxe: create_elements (--TC T2) 2.8961 msec/pass
cET: create_elements (--TC T2) 0.1929 msec/pass
ET : create_elements (--TC T2) 1.3590 msec/pass
@@ -282,11 +282,11 @@
are supposed to end up in, either as SubElements of an Element or using the
explicit ``Element.makeelement()`` call::
- lxe: makeelement (--TC T2) 2.2650 msec/pass
+ lxe: makeelement (--TC T2) 1.9000 msec/pass
cET: makeelement (--TC T2) 0.3211 msec/pass
ET : makeelement (--TC T2) 1.6358 msec/pass
- lxe: create_subelements (--TC T2) 1.9531 msec/pass
+ lxe: create_subelements (--TC T2) 1.7891 msec/pass
cET: create_subelements (--TC T2) 0.2351 msec/pass
ET : create_subelements (--TC T2) 3.2270 msec/pass
@@ -305,11 +305,11 @@
The following benchmark appends all root children of the second tree to the
root of the first tree::
- lxe: append_from_document (--TR T1,T2) 3.8681 msec/pass
+ lxe: append_from_document (--TR T1,T2) 3.3841 msec/pass
cET: append_from_document (--TR T1,T2) 0.2699 msec/pass
ET : append_from_document (--TR T1,T2) 1.2650 msec/pass
- lxe: append_from_document (--TR T3,T4) 0.0570 msec/pass
+ lxe: append_from_document (--TR T3,T4) 0.0441 msec/pass
cET: append_from_document (--TR T3,T4) 0.0169 msec/pass
ET : append_from_document (--TR T3,T4) 0.0820 msec/pass
@@ -322,20 +322,20 @@
This difference is not always as visible, but applies to most parts of the
API, like inserting newly created elements::
- lxe: insert_from_document (--TR T1,T2) 5.8019 msec/pass
+ lxe: insert_from_document (--TR T1,T2) 5.7020 msec/pass
cET: insert_from_document (--TR T1,T2) 0.4041 msec/pass
ET : insert_from_document (--TR T1,T2) 1.4789 msec/pass
or replacing the child slice by a newly created element::
- lxe: replace_children_element (--TC T1) 0.2480 msec/pass
+ lxe: replace_children_element (--TC T1) 0.2210 msec/pass
cET: replace_children_element (--TC T1) 0.0238 msec/pass
ET : replace_children_element (--TC T1) 0.1600 msec/pass
as opposed to replacing the slice with an existing element from the
same document::
- lxe: replace_children (--TC T1) 0.0188 msec/pass
+ lxe: replace_children (--TC T1) 0.0179 msec/pass
cET: replace_children (--TC T1) 0.0119 msec/pass
ET : replace_children (--TC T1) 0.0739 msec/pass
@@ -347,16 +347,16 @@
Deep copying a tree is fast in lxml::
- lxe: deepcopy_all (--TR T1) 10.9420 msec/pass
+ lxe: deepcopy_all (--TR T1) 9.7558 msec/pass
cET: deepcopy_all (--TR T1) 120.6188 msec/pass
ET : deepcopy_all (--TR T1) 902.6880 msec/pass
- lxe: deepcopy_all (-ATR T2) 12.5830 msec/pass
+ lxe: deepcopy_all (-ATR T2) 12.3210 msec/pass
cET: deepcopy_all (-ATR T2) 136.9810 msec/pass
ET : deepcopy_all (-ATR T2) 944.2801 msec/pass
- lxe: deepcopy_all (S-TR T3) 4.1170 msec/pass
- cET: deepcopy_all (S-TR T3) 36.1221 msec/pass
+ lxe: deepcopy_all (S-TR T3) 8.3981 msec/pass
+ cET: deepcopy_all (S-TR T3) 35.6541 msec/pass
ET : deepcopy_all (S-TR T3) 221.6041 msec/pass
So, for example, if you have a database-like scenario where you parse in a
@@ -372,37 +372,37 @@
especially if few elements are of interest or the target element tag name is
known, lxml is a good choice::
- lxe: getiterator_all (--TR T1) 5.8582 msec/pass
+ lxe: getiterator_all (--TR T1) 5.7251 msec/pass
cET: getiterator_all (--TR T1) 39.9489 msec/pass
ET : getiterator_all (--TR T1) 23.0000 msec/pass
- lxe: getiterator_islice (--TR T2) 0.0780 msec/pass
+ lxe: getiterator_islice (--TR T2) 0.0830 msec/pass
cET: getiterator_islice (--TR T2) 0.3440 msec/pass
ET : getiterator_islice (--TR T2) 0.2429 msec/pass
- lxe: getiterator_tag (--TR T2) 0.3119 msec/pass
+ lxe: getiterator_tag (--TR T2) 0.3011 msec/pass
cET: getiterator_tag (--TR T2) 14.1001 msec/pass
ET : getiterator_tag (--TR T2) 7.4241 msec/pass
- lxe: getiterator_tag_all (--TR T2) 0.6540 msec/pass
+ lxe: getiterator_tag_all (--TR T2) 0.6340 msec/pass
cET: getiterator_tag_all (--TR T2) 40.7901 msec/pass
ET : getiterator_tag_all (--TR T2) 21.0390 msec/pass
This translates directly into similar timings for ``Element.findall()``::
- lxe: findall (--TR T2) 8.1239 msec/pass
+ lxe: findall (--TR T2) 7.8950 msec/pass
cET: findall (--TR T2) 44.5340 msec/pass
ET : findall (--TR T2) 27.1149 msec/pass
- lxe: findall (--TR T3) 1.6870 msec/pass
+ lxe: findall (--TR T3) 1.7281 msec/pass
cET: findall (--TR T3) 12.9611 msec/pass
ET : findall (--TR T3) 8.6131 msec/pass
- lxe: findall_tag (--TR T2) 0.7660 msec/pass
+ lxe: findall_tag (--TR T2) 0.7720 msec/pass
cET: findall_tag (--TR T2) 40.6358 msec/pass
ET : findall_tag (--TR T2) 21.4581 msec/pass
- lxe: findall_tag (--TR T3) 0.2160 msec/pass
+ lxe: findall_tag (--TR T3) 0.2050 msec/pass
cET: findall_tag (--TR T3) 9.6831 msec/pass
ET : findall_tag (--TR T3) 5.2109 msec/pass
@@ -420,38 +420,38 @@
of the lxml API you use. The most straight forward way is to call the
``xpath()`` method on an Element or ElementTree::
- lxe: xpath_method (--TC T1) 1.8251 msec/pass
- lxe: xpath_method (--TC T2) 23.3159 msec/pass
- lxe: xpath_method (--TC T3) 0.1378 msec/pass
- lxe: xpath_method (--TC T4) 1.1270 msec/pass
+ lxe: xpath_method (--TC T1) 1.7459 msec/pass
+ lxe: xpath_method (--TC T2) 22.0850 msec/pass
+ lxe: xpath_method (--TC T3) 0.1309 msec/pass
+ lxe: xpath_method (--TC T4) 1.0772 msec/pass
This is well suited for testing and when the XPath expressions are as diverse
as the trees they are called on. However, if you have a single XPath
expression that you want to apply to a larger number of different elements,
the ``XPath`` class is the most efficient way to do it::
- lxe: xpath_class (--TC T1) 0.6981 msec/pass
- lxe: xpath_class (--TC T2) 3.6111 msec/pass
- lxe: xpath_class (--TC T3) 0.0591 msec/pass
- lxe: xpath_class (--TC T4) 0.1979 msec/pass
+ lxe: xpath_class (--TC T1) 0.6740 msec/pass
+ lxe: xpath_class (--TC T2) 3.1760 msec/pass
+ lxe: xpath_class (--TC T3) 0.0548 msec/pass
+ lxe: xpath_class (--TC T4) 0.1700 msec/pass
Note that this still allows you to use variables in the expression, so you can
parse it once and then adapt it through variables at call time. In other
cases, where you have a fixed Element or ElementTree and want to run different
expressions on it, you should consider the ``XPathEvaluator``::
- lxe: xpath_element (--TR T1) 0.4342 msec/pass
- lxe: xpath_element (--TR T2) 11.9958 msec/pass
- lxe: xpath_element (--TR T3) 0.1690 msec/pass
- lxe: xpath_element (--TR T4) 0.3510 msec/pass
+ lxe: xpath_element (--TR T1) 0.4151 msec/pass
+ lxe: xpath_element (--TR T2) 11.6129 msec/pass
+ lxe: xpath_element (--TR T3) 0.1299 msec/pass
+ lxe: xpath_element (--TR T4) 0.3409 msec/pass
While it looks slightly slower, creating an XPath object for each of the
expressions generates a much higher overhead here::
- lxe: xpath_class_repeat (--TC T1) 1.7619 msec/pass
- lxe: xpath_class_repeat (--TC T2) 21.9102 msec/pass
- lxe: xpath_class_repeat (--TC T3) 0.1330 msec/pass
- lxe: xpath_class_repeat (--TC T4) 1.0631 msec/pass
+ lxe: xpath_class_repeat (--TC T1) 1.6699 msec/pass
+ lxe: xpath_class_repeat (--TC T2) 20.4420 msec/pass
+ lxe: xpath_class_repeat (--TC T3) 0.1230 msec/pass
+ lxe: xpath_class_repeat (--TC T4) 0.9859 msec/pass
A longer example
@@ -608,21 +608,21 @@
tree. It avoids step-by-step Python element instantiations along the path,
which can substantially improve the access time::
- lxe: attribute (--TR T1) 9.8128 msec/pass
- lxe: attribute (--TR T2) 53.2899 msec/pass
- lxe: attribute (--TR T4) 9.6800 msec/pass
-
- lxe: objectpath (--TR T1) 5.4898 msec/pass
- lxe: objectpath (--TR T2) 48.4819 msec/pass
- lxe: objectpath (--TR T4) 5.3761 msec/pass
-
- lxe: attributes_deep (--TR T1) 56.3290 msec/pass
- lxe: attributes_deep (--TR T2) 62.4361 msec/pass
- lxe: attributes_deep (--TR T4) 15.8000 msec/pass
-
- lxe: objectpath_deep (--TR T1) 49.0060 msec/pass
- lxe: objectpath_deep (--TR T2) 52.5169 msec/pass
- lxe: objectpath_deep (--TR T4) 7.1371 msec/pass
+ lxe: attribute (--TR T1) 9.4581 msec/pass
+ lxe: attribute (--TR T2) 52.5560 msec/pass
+ lxe: attribute (--TR T4) 9.1729 msec/pass
+
+ lxe: objectpath (--TR T1) 4.8690 msec/pass
+ lxe: objectpath (--TR T2) 47.8780 msec/pass
+ lxe: objectpath (--TR T4) 4.7870 msec/pass
+
+ lxe: attributes_deep (--TR T1) 54.7471 msec/pass
+ lxe: attributes_deep (--TR T2) 62.7451 msec/pass
+ lxe: attributes_deep (--TR T4) 15.1050 msec/pass
+
+ lxe: objectpath_deep (--TR T1) 48.2810 msec/pass
+ lxe: objectpath_deep (--TR T2) 51.3949 msec/pass
+ lxe: objectpath_deep (--TR T4) 6.1419 msec/pass
Note, however, that parsing ObjectPath expressions is not for free either, so
this is most effective for frequently accessing the same element.
@@ -648,17 +648,17 @@
subtrees and elements) to cache, you can trade memory usage against access
speed::
- lxe: attribute_cached (--TR T1) 7.6170 msec/pass
- lxe: attribute_cached (--TR T2) 50.7941 msec/pass
- lxe: attribute_cached (--TR T4) 7.4880 msec/pass
-
- lxe: attributes_deep_cached (--TR T1) 49.9220 msec/pass
- lxe: attributes_deep_cached (--TR T2) 55.9340 msec/pass
- lxe: attributes_deep_cached (--TR T4) 10.0131 msec/pass
-
- lxe: objectpath_deep_cached (--TR T1) 44.9121 msec/pass
- lxe: objectpath_deep_cached (--TR T2) 48.2371 msec/pass
- lxe: objectpath_deep_cached (--TR T4) 3.9630 msec/pass
+ lxe: attribute_cached (--TR T1) 7.5061 msec/pass
+ lxe: attribute_cached (--TR T2) 50.1881 msec/pass
+ lxe: attribute_cached (--TR T4) 7.4170 msec/pass
+
+ lxe: attributes_deep_cached (--TR T1) 48.7239 msec/pass
+ lxe: attributes_deep_cached (--TR T2) 55.2199 msec/pass
+ lxe: attributes_deep_cached (--TR T4) 9.9740 msec/pass
+
+ lxe: objectpath_deep_cached (--TR T1) 43.4160 msec/pass
+ lxe: objectpath_deep_cached (--TR T2) 47.6480 msec/pass
+ lxe: objectpath_deep_cached (--TR T4) 3.4680 msec/pass
Things to note: you cannot currently use ``weakref.WeakKeyDictionary`` objects
for this as lxml's element objects do not support weak references (which are
More information about the lxml-checkins
mailing list