[Lxml-checkins] r38837 - in lxml/pyrex: . Doc Pyrex/Compiler Pyrex/Mac
scoder at codespeak.net
scoder at codespeak.net
Wed Feb 14 15:20:33 CET 2007
Author: scoder
Date: Wed Feb 14 15:20:32 2007
New Revision: 38837
Added:
lxml/pyrex/Makefile
lxml/pyrex/Pyrex/Compiler/ModuleNode.py
Modified:
lxml/pyrex/CHANGES.txt
lxml/pyrex/Doc/extension_types.html
lxml/pyrex/Doc/overview.html
lxml/pyrex/Doc/sharing.html
lxml/pyrex/Doc/special_methods.html
lxml/pyrex/Pyrex/Compiler/Code.py
lxml/pyrex/Pyrex/Compiler/ExprNodes.py
lxml/pyrex/Pyrex/Compiler/Nodes.py
lxml/pyrex/Pyrex/Compiler/Parsing.py
lxml/pyrex/Pyrex/Compiler/PyrexTypes.py
lxml/pyrex/Pyrex/Compiler/Symtab.py
lxml/pyrex/Pyrex/Compiler/Version.py
lxml/pyrex/Pyrex/Mac/DarwinSystem.py
lxml/pyrex/ToDo.txt
lxml/pyrex/setup.py
Log:
updated to 0.9.5.1a (+ enum work around)
Modified: lxml/pyrex/CHANGES.txt
==============================================================================
--- lxml/pyrex/CHANGES.txt (original)
+++ lxml/pyrex/CHANGES.txt Wed Feb 14 15:20:32 2007
@@ -1,3 +1,139 @@
+0.9.5.1a
+--------
+
+Bug fixes:
+
+ - Package list now calculated dynamically in setup.py so that
+ it will work with or without the testing framework installed.
+
+
+0.9.5.1
+-------
+
+Bug fixes:
+
+ - Comparing two values of the same enum type incorrectly
+ produced an error. [Anders Gustafsson <andersg at 0x63.nu>]
+
+ - Compiler crash caused by assigning a Python value to
+ a variable of an enum type. [Peter Johnson <peter at tortall.net>]
+
+ - Comparison between pointer and array incorrectly produced
+ a type mismatch error.
+ [Helmut Jarausch <jarausch at igpm.rwth-aachen.de>]
+
+ - Unused local Python variable had spurious init/cleanup code
+ generated for it, causing C compilation errors.
+ [Helmut Jarausch <jarausch at igpm.rwth-aachen.de>]
+
+ - Updated list of packages in setup.py.
+
+Modifications:
+
+ - NULL in Pyrex source now translated into NULL instead of 0
+ in C code, to allow for the possibility of calling something
+ not defined with a prototype in an external header.
+ [Adapted Cat <a.c.junker at gmail.com>]
+
+
+0.9.5
+-----
+
+Enhancements:
+
+ - Exception return values may now be specified by arbitrary
+ constant expressions of appropriate type, not just literals.
+ [Stefan Behnel <behnel_ml at gkec.informatik.tu-darmstadt.de>]
+
+ - Redundant type check now omitted when passing a literal None
+ to a function expecting an extension type.
+ [Patch by Sam Rushing <sam-pyrex at rushing.nightmare.com>]
+
+ - New-style classes now allowed as exceptions for compatibility
+ with Python 2.5 (inheritance from BaseException not currently
+ checked).
+ [Stefan Behnel <behnel_ml at gkec.informatik.tu-darmstadt.de>]
+
+ - Sequence unpacking is now done using the iterator protocol
+ instead of indexing.
+
+ - Allocation of an empty tuple is avoided when making a
+ Python call with no arguments.
+ [Stefan Behnel <behnel_ml at gkec.informatik.tu-darmstadt.de>]
+
+ - Most warnings about unused variables and labels have been
+ eliminated.
+
+ - Support for running the test suite on Linux added but not
+ yet fully tested. [Based in part on patch by Eric Wald
+ <eswald at gmail.com>].
+
+ - Makefile included for compiling the patched Carbon File module
+ used by the MacOSX test code.
+
+Modifications:
+
+ - Type rules for enums tightened for compatibility with C++.
+
+ - Direct assignment from float to int disallowed to prevent
+ C++ compilation warnings.
+
+ - Hex literals left as hex in C code to avoid warnings from
+ the C compiler about decimal constants becoming unsigned.
+
+Bug fixes:
+
+ - Exception raised during argument conversion could cause crash
+ due to uninitialised local variables.
+ [Konrad Hinsen <konrad.hinsen at laposte.net>]
+
+ - Assignment to a C attribute of an extension type from a
+ different type could generate C code with a pointer type
+ mismatch. [Atsuo Ishimoto <ishimoto at gembook.org>]
+
+ - Backslash in a string literal before a non-special character
+ was not handled correctly. [Yuan Mang <mangyuan at gmail.com>]
+
+ - Temporary vars used by del statement not being properly
+ released, sometimes leading to double decrefs.
+ [Jiba <jibalamy at free.fr>]
+
+ - A return statement whose expression raises an exception
+ inside a try-except that catches the exception could cause
+ a crash. [Anders Gustafsson <andersg at 0x63.nu>]
+
+ - Fixed type compatibility checking problem between pointers
+ and arrays. [Lenard Lindstrom <len-l at telus.net>]
+
+ - Circular imports between modules defining extension types
+ caused unresolvable import order conflicts.
+ [Mike Wyatt <mwyatt at wi.rr.com>]
+
+ - Cimporting multiple submodules from the same package caused
+ a redefined name error for the top level name.
+ [Martin Albrecht <malb at informatik.uni-bremen.de>]
+
+ - Incorrect reference counting when assigning to an element of an
+ array that is a C attribute of an extension type.
+ [Igor Khavkine <igor.kh at gmail.com>]
+
+ - Weak-referenceable extension types were not implemented
+ properly. [Chris Perkins <chrisperkins99 at gmail.com>,
+ Peter Johnson <pete at cvs.tortall.net>]
+
+ - Crash if C variable declared readonly outside an extension
+ type definition. [Eric Huss <e-huss at netmeridian.com>]
+
+Doc updates:
+
+ - Expanded discussion of the need for type declarations to enable
+ access to attributes of extension types.
+
+ - Added a section "Source Files and Compilation" explaining the
+ rules for naming of source files of modules residing in packages,
+ and instructions for using the compiler and distutils extension.
+
+
0.9.4.1
-------
Modified: lxml/pyrex/Doc/extension_types.html
==============================================================================
--- lxml/pyrex/Doc/extension_types.html (original)
+++ lxml/pyrex/Doc/extension_types.html Wed Feb 14 15:20:32 2007
@@ -1,48 +1,148 @@
<!DOCTYPE doctype PUBLIC "-//w3c//dtd html 4.0 transitional//en">
-<html><head>
- <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
- <meta name="GENERATOR" content="Mozilla/4.61 (Macintosh; I; PPC) [Netscape]"><title>Extension Types</title></head>
+<html>
+<head>
+
+
+
+
+ <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
+
+
+
+
+ <meta name="GENERATOR" content="Mozilla/4.61 (Macintosh; I; PPC) [Netscape]">
+
+
+ <title>Extension Types</title>
+</head>
+
+
<body>
- <h1> <hr width="100%">Extension Types
+
+
+
+<h1>
+<hr width="100%">Extension Types
<hr width="100%"></h1>
- <h2> Contents</h2>
- <ul>
+
+
+
+<h2> Contents</h2>
+
+
+
+<ul>
+
+
<li> <a href="#Introduction">Introduction</a></li>
+
+
<li> <a href="#ExtTypeAttrs">Attributes</a></li>
+ <li><a href="#TypeDeclarations">Type declarations</a></li>
+
+
<li> <a href="#NotNone">Extension types and None</a></li>
+
+
<li> <a href="special_methods.html">Special methods</a></li>
+
+
<li> <a href="#Properties">Properties</a> <font style="color: rgb(0, 153, 0);" color="#ed181e">(NEW in
0.9)</font></li>
+
+
<li><a href="#SubclassingExtTypes">Subclassing</a></li>
+
+
<li> <a href="#CMethods">C Methods</a> <font style="color: rgb(0, 153, 0);" color="#ff0000">(NEW in 0.9)</font><br>
- <a href="#ForwardDeclaringExtTypes">Forward-declaring extension types</a></li><li><a href="#WeakRefs">Making extension types weak-referenceable</a> <span style="color: rgb(255, 0, 0);">(NEW in 0.9.4)</span><br>
+
+
+ <a href="#ForwardDeclaringExtTypes">Forward-declaring extension types</a></li>
+
+ <li><a href="#WeakRefs">Making extension types weak-referenceable</a> <span style="color: rgb(255, 0, 0);">(NEW in 0.9.4)</span><br>
+
+
</li>
+
+
<li> <a href="#PublicAndExtern">Public and external extension types</a><font color="#2f8b20"><br>
-</font></li>
- <ul>
+
+
+ </font></li>
+
+
+
+
+ <ul>
+
+
<li> <a href="#ExternalExtTypes">External extension types</a></li>
+
+
<li> <a href="#ImplicitImport">Implicit importing</a><font color="#2f8b20"><br>
-</font></li>
+
+
+ </font></li>
+
+
<li> <a href="#TypeVsConstructor">Type names vs. constructor names</a></li>
+
+
<li> <a href="#PublicExtensionTypes">Public extension types</a></li>
+
+
<li> <a href="#NameSpecClause">Name specification clause</a></li>
- </ul>
+
+
+
+
</ul>
- <h2> <a name="Introduction"></a>Introduction</h2>
+
+
+
+</ul>
+
+
+
+<h2> <a name="Introduction"></a>Introduction</h2>
+
+
As well as creating normal user-defined classes with the Python <b>class</b>
statement, Pyrex also lets you create new built-in Python types, known as
<i>extension types</i>. You define an extension type using the <b>cdef class</b> statement. Here's an example:
-<blockquote><tt>cdef class Shrubbery:</tt> <p><tt> cdef int width, height</tt> </p>
- <p><tt> def __init__(self, w, h):</tt> <br>
+<blockquote><tt>cdef class Shrubbery:</tt>
+
+ <p><tt> cdef int width, height</tt> </p>
+
+
+
+
+ <p><tt> def __init__(self, w, h):</tt> <br>
+
+
<tt> self.width = w</tt> <br>
+
+
<tt> self.height = h</tt> </p>
- <p><tt> def describe(self):</tt> <br>
+
+
+
+
+ <p><tt> def describe(self):</tt> <br>
+
+
<tt> print "This shrubbery is",
self.width, \</tt> <br>
+
+
<tt>
"by", self.height, "cubits."</tt></p>
+
+
</blockquote>
+
+
As you can see, a Pyrex extension type definition looks a lot like a Python
class definition. Within it, you use the <b>def</b> statement to define
methods that can be called from Python code. You can even define many of
@@ -52,7 +152,12 @@
extension type), or they may be of any C data type. So you can use extension
types to wrap arbitrary C data structures and provide a Python-like interface
to them. </p>
- <h2> <a name="ExtTypeAttrs"></a>Attributes</h2>
+
+
+
+<h2> <a name="ExtTypeAttrs"></a>Attributes</h2>
+
+
Attributes of an extension type are stored directly in the object's C struct.
The set of attributes is fixed at compile time; you can't add attributes
to an extension type instance at run time simply by assigning to them, as
@@ -62,12 +167,24 @@
by Python attribute lookup, or by direct access to the C struct from Pyrex
code. Python code is only able to access attributes of an extension type
by the first method, but Pyrex code can use either method. </p>
- <p>By default, extension type attributes are only accessible by direct access,
+
+
+
+<p>By default, extension type attributes are only accessible by direct access,
not Python access, which means that they are not accessible from Python code.
To make them accessible from Python code, you need to declare them as <tt>public</tt> or <tt>readonly</tt>. For example, </p>
- <blockquote><tt>cdef class Shrubbery:</tt> <br>
+
+
+
+<blockquote><tt>cdef class Shrubbery:</tt> <br>
+
+
<tt> cdef public int width, height</tt> <br>
+
+
<tt> cdef readonly float depth</tt></blockquote>
+
+
makes the <tt>width</tt> and <tt>height</tt> attributes readable and writable
from Python code, and the <tt>depth</tt> attribute readable but not writable.
@@ -76,21 +193,64 @@
although read-write exposure is only possible for generic Python attributes
(of type <tt>object</tt>). If the attribute is declared to be of an extension
type, it must be exposed <tt>readonly</tt>. </p>
- <p>Note also that the <tt>public</tt> and <tt>readonly</tt> options apply
+
+
+
+<p>Note also that the <tt>public</tt> and <tt>readonly</tt> options apply
only to <i>Python</i> access, not direct access. All the attributes of an
-extension type are always readable and writable by direct access. </p>
- <p>Howerver, for direct access to be possible, the Pyrex compiler must know
+extension type are always readable and writable by direct access.</p>
+
+<h2><a name="TypeDeclarations"></a>Type declarations </h2>
+
+
+
+<p>Before you can directly access the attributes of an extension type, the Pyrex compiler must know
that you have an instance of that type, and not just a generic Python object.
It knows this already in the case of the "self" parameter of the methods of
-that type, but in other cases you will have to tell it by means of a declaration.
-For example, </p>
- <blockquote><tt>cdef widen_shrubbery(Shrubbery sh, extra_width):</tt> <br>
+that type, but in other cases you will have to use a type declaration.</p>
+
+<p>For example, in the following function,</p>
+
+<blockquote><tt>cdef widen_shrubbery(sh, extra_width): # </tt><span style="font-family: monospace;"><span style="color: rgb(255, 0, 0);">BAD</span></span><br>
+
+
<tt> sh.width = sh.width + extra_width</tt></blockquote>
- If you attempt to access an extension type attribute through a generic
-object reference, Pyrex will use a Python attribute lookup. If the attribute
-is exposed for Python access (using <tt>public</tt> or <tt>readonly</tt>)
-then this will work, but it will be much slower than direct access.
+
+<p> because the <span style="font-family: monospace;">sh</span> parameter hasn't been given a type, the <span style="font-family: monospace;">width</span>
+attribute will be accessed by a Python attribute lookup. If the
+attribute has been declared <span style="font-style: italic;">public</span> or <span style="font-style: italic;">readonly</span> then this will work, but
+it will be very inefficient. If the attribute is private, it will not work at all -- the
+code will compile, but an attribute error will be raised at run time.</p>
+
+<p>The solution is to declare sh as being of type <span style="font-family: monospace;">Shrubbery</span>, as follows:</p>
+
+
+
+<blockquote><tt>cdef widen_shrubbery(Shrubbery sh, extra_width):</tt> <br>
+
+
+ <tt> sh.width = sh.width + extra_width</tt></blockquote>
+
+Now the Pyrex compiler knows that <span style="font-family: monospace;">sh</span> has a C attribute called <span style="font-family: monospace;">width</span> and will generate code to access it directly and efficiently. The same consideration applies to local variables, for example,<br>
+
+<br>
+
+<div style="margin-left: 40px;"><code>cdef Shrubbery another_shrubbery(Shrubbery sh1):<br>
+
+ cdef Shrubbery sh2<br>
+
+ sh2 = Shrubbery()<br>
+
+ sh2.width = sh1.width<br>
+
+ sh2.height = sh1.height<br>
+
+ return sh2</code></div>
+
+
<h2> <a name="NotNone"></a>Extension types and None</h2>
+
+
When you declare a parameter or C variable as being of an extension type,
Pyrex will allow it to take on the value None as well as values of its declared
type. This is analogous to the way a C pointer can take on the value NULL,
@@ -103,223 +263,589 @@
<p>You need to be particularly careful when exposing Python functions which
take extension types as arguments. If we wanted to make <tt>widen_shrubbery</tt>
a Python function, for example, if we simply wrote </p>
- <blockquote><tt>def widen_shrubbery(Shrubbery sh, extra_width): # <font color="#ed181e">This is</font></tt> <br>
+
+
+
+<blockquote><tt>def widen_shrubbery(Shrubbery sh, extra_width): # <font color="#ed181e">This is</font></tt> <br>
+
+
<tt> sh.width = sh.width + extra_width
# <font color="#ed181e">dangerous!</font></tt></blockquote>
+
+
then users of our module could crash it by passing None for the <tt>sh</tt>
parameter.
<p>One way to fix this would be </p>
- <blockquote><tt>def widen_shrubbery(Shrubbery sh, extra_width):</tt> <br>
+
+
+
+<blockquote><tt>def widen_shrubbery(Shrubbery sh, extra_width):</tt> <br>
+
+
<tt> if sh is None:</tt> <br>
+
+
<tt> raise TypeError</tt> <br>
+
+
<tt> sh.width = sh.width + extra_width</tt></blockquote>
+
+
but since this is anticipated to be such a frequent requirement, Pyrex
provides a more convenient way. Parameters of a Python function declared
as an extension type can have a <b><tt>not None</tt></b> clause:
<blockquote><tt>def widen_shrubbery(Shrubbery sh not None, extra_width):</tt>
<br>
+
+
<tt> sh.width = sh.width + extra_width</tt></blockquote>
+
+
Now the function will automatically check that <tt>sh</tt> is not None
along with checking that it has the right type.
<p>Note, however that the <tt>not None</tt> clause can <i>only</i> be used
in Python functions (defined with <tt>def</tt>) and not C functions (defined
with <tt>cdef</tt>). If you need to check whether a parameter to a C function
is None, you will need to do it yourself. </p>
- <p>Some more things to note: </p>
- <ul>
+
+
+
+<p>Some more things to note: </p>
+
+
+
+<ul>
+
+
<li> The <b>self</b> parameter of a method of an extension type is guaranteed
never to be None.</li>
- </ul>
- <ul>
+
+
+
+</ul>
+
+
+
+<ul>
+
+
<li> When comparing a value with None, keep in mind that, if <tt>x</tt> is a Python object, <tt>x is None</tt> and <tt>x is not None</tt> are very
efficient because they translate directly to C pointer comparisons, whereas
<tt>x == None</tt> and <tt>x != None</tt>, or simply using <tt>x</tt> as a boolean value (as in <tt>if x: ...</tt>) will invoke Python operations
and therefore be much slower.</li>
- </ul>
- <h2> <a name="ExtTypeSpecialMethods"></a>Special methods</h2>
+
+
+
+</ul>
+
+
+
+<h2> <a name="ExtTypeSpecialMethods"></a>Special methods</h2>
+
+
Although the principles are similar, there are substantial differences
between many of the <span style="font-family: monospace;">__xxx__</span> special methods of extension types and their
Python counterparts. There is a <a href="special_methods.html">separate page</a> devoted to this subject, and you should read it carefully before attempting
to use any special methods in your extension types.
<h2> <a name="Properties"></a>Properties</h2>
+
+
There is a special syntax for defining <b>properties</b> in an extension
class:
-<blockquote><tt>cdef class Spam:</tt> <p><tt> property cheese:</tt> </p>
- <p><tt> "A doc string can go
+<blockquote><tt>cdef class Spam:</tt>
+
+ <p><tt> property cheese:</tt> </p>
+
+
+
+
+ <p><tt> "A doc string can go
here."</tt> </p>
- <p><tt> def __get__(self):</tt>
+
+
+
+
+ <p><tt> def __get__(self):</tt>
<br>
+
+
<tt>
# This is called when the property is read.</tt> <br>
+
+
<tt>
...</tt> </p>
- <p><tt> def __set__(self, value):</tt>
+
+
+
+
+ <p><tt> def __set__(self, value):</tt>
<br>
+
+
<tt>
# This is called when the property is written.</tt> <br>
+
+
<tt>
...</tt> </p>
- <p><tt> def __del__(self):</tt>
+
+
+
+
+ <p><tt> def __del__(self):</tt>
<br>
+
+
<tt>
# This is called when the property is deleted.</tt> <br>
+
+
</p>
+
+
</blockquote>
+
+
The <tt>__get__</tt>, <tt>__set__</tt> and <tt>__del__</tt> methods are
all optional; if they are omitted, an exception will be raised when the corresponding
operation is attempted.
<p>Here's a complete example. It defines a property which adds to a list
each time it is written to, returns the list when it is read, and empties
the list when it is deleted. <br>
+
+
</p>
- <center> <table align="center" cellpadding="5">
+
+
+
+<center>
+<table align="center" cellpadding="5">
+
+
<tbody>
+
+
<tr>
+
+
<td bgcolor="#ffaf18"><b><tt>cheesy.pyx</tt></b></td>
+
+
<td bgcolor="#5dbaca"><b>Test input</b></td>
+
+
</tr>
+
+
<tr>
+
+
<td rowspan="3" bgcolor="#ffaf18" valign="top"><tt>cdef class CheeseShop:</tt>
- <p><tt> cdef object cheeses</tt> </p>
- <p><tt> def __new__(self):</tt> <br>
+
+
+ <p><tt> cdef object cheeses</tt> </p>
+
+
+
+
+ <p><tt> def __new__(self):</tt> <br>
+
+
<tt> self.cheeses = []</tt> </p>
- <p><tt> property cheese:</tt> </p>
- <p><tt> def __get__(self):</tt> <br>
+
+
+
+
+ <p><tt> property cheese:</tt> </p>
+
+
+
+
+ <p><tt> def __get__(self):</tt> <br>
+
+
<tt> return "We don't have: %s" % self.cheeses</tt>
</p>
- <p><tt> def __set__(self, value):</tt> <br>
+
+
+
+
+ <p><tt> def __set__(self, value):</tt> <br>
+
+
<tt> self.cheeses.append(value)</tt>
</p>
- <p><tt> def __del__(self):</tt> <br>
+
+
+
+
+ <p><tt> def __del__(self):</tt> <br>
+
+
<tt> del self.cheeses[:]</tt></p>
+
+
</td>
+
+
<td bgcolor="#5dbaca" valign="top"><tt>from cheesy import CheeseShop</tt>
- <p><tt>shop = CheeseShop()</tt> <br>
+
+
+ <p><tt>shop = CheeseShop()</tt> <br>
+
+
<tt>print shop.cheese</tt> </p>
- <p><tt>shop.cheese = "camembert"</tt> <br>
+
+
+
+
+ <p><tt>shop.cheese = "camembert"</tt> <br>
+
+
<tt>print shop.cheese</tt> </p>
- <p><tt>shop.cheese = "cheddar"</tt> <br>
+
+
+
+
+ <p><tt>shop.cheese = "cheddar"</tt> <br>
+
+
<tt>print shop.cheese</tt> </p>
- <p><tt>del shop.cheese</tt> <br>
+
+
+
+
+ <p><tt>del shop.cheese</tt> <br>
+
+
<tt>print shop.cheese</tt></p>
+
+
</td>
+
+
</tr>
+
+
<tr>
+
+
<td bgcolor="#8cbc1c"><b>Test output</b></td>
+
+
</tr>
+
+
<tr>
+
+
<td bgcolor="#8cbc1c"><tt>We don't have: []</tt> <br>
+
+
<tt>We don't have: ['camembert']</tt> <br>
+
+
<tt>We don't have: ['camembert', 'cheddar']</tt> <br>
+
+
<tt>We don't have: []</tt></td>
+
+
</tr>
- </tbody> </table>
+
+
+
+
+ </tbody>
+</table>
+
+
</center>
- <h2> <a name="SubclassingExtTypes"></a>Subclassing</h2>
+
+
+
+<h2> <a name="SubclassingExtTypes"></a>Subclassing</h2>
+
+
An extension type may inherit from a built-in type or another extension
type:
<blockquote><tt>cdef class Parrot:</tt> <br>
- <tt> ...</tt><tt></tt> <p><tt>cdef class Norwegian(Parrot):</tt> <br>
+
+
+ <tt> ...</tt><tt></tt>
+
+ <p><tt>cdef class Norwegian(Parrot):</tt> <br>
+
+
<tt> ...</tt></p>
+
+
</blockquote>
- <p><br>
+
+
+
+<p><br>
+
+
A complete definition of the base type must be available to Pyrex, so if
the base type is a built-in type, it must have been previously declared as
an <b>extern</b> extension type. If the base type is defined in another Pyrex
module, it must either be declared as an extern extension type or imported
using the <b><a href="sharing.html">cimport</a></b> statement. </p>
- <p>An extension type can only have one base class (no multiple inheritance).
+
+
+
+<p>An extension type can only have one base class (no multiple inheritance).
</p>
- <p>Pyrex extension types can also be subclassed in Python. A Python class
+
+
+
+<p>Pyrex extension types can also be subclassed in Python. A Python class
can inherit from multiple extension types provided that the usual Python
rules for multiple inheritance are followed (i.e. the C layouts of all the
base classes must be compatible).<br>
+
+
</p>
- <h2><a name="CMethods"></a>C methods</h2>
+
+
+
+<h2><a name="CMethods"></a>C methods</h2>
+
+
Extension types can have C methods as well as Python methods. Like C functions,
C methods are declared using <tt>cdef</tt> instead of <tt>def</tt>. C methods
are "virtual", and may be overridden in derived extension types.<br>
+
+
<br>
- <table align="center" cellpadding="5">
+
+
+
+<table align="center" cellpadding="5">
+
+
<tbody>
+
+
<tr>
+
+
<td bgcolor="#ffaf18" valign="top" width="50%"><b><tt>pets.pyx</tt></b><br>
+
+
</td>
+
+
<td bgcolor="#8cbc1c" valign="top" width="30%"><b>Output</b><br>
+
+
</td>
+
+
</tr>
+
+
<tr>
+
+
<td bgcolor="#ffaf18" valign="top" width="50%"><tt>cdef class Parrot:<br>
+
+
<br>
+
+
cdef void describe(self):<br>
+
+
print "This parrot is resting."<br>
+
+
<br>
+
+
cdef class Norwegian(Parrot):<br>
+
+
<br>
+
+
cdef void describe(self):<br>
+
+
Parrot.describe(self)<br>
+
+
print "Lovely plumage!"<br>
+
+
<br>
+
+
<br>
+
+
cdef Parrot p1, p2<br>
+
+
p1 = Parrot()<br>
+
+
p2 = Norwegian()<br>
+
+
print "p1:"<br>
+
+
p1.describe()<br>
+
+
print "p2:"<br>
+
+
p2.describe()</tt> <br>
+
+
</td>
+
+
<td bgcolor="#8cbc1c" valign="top" width="30%"><tt>p1:<br>
+
+
This parrot is resting.<br>
+
+
p2:<br>
+
+
</tt><tt>This parrot is resting.<br>
+
+
</tt><tt> Lovely plumage!</tt><br>
+
+
</td>
+
+
</tr>
- </tbody> </table>
+
+
+
+
+ </tbody>
+</table>
+
+
<br>
+
+
The above example also illustrates that a C method can call an inherited
C method using the usual Python technique, i.e.<br>
+
+
<blockquote><tt>Parrot.describe(self)</tt><br>
+
+
</blockquote>
- <h2><a name="ForwardDeclaringExtTypes"></a>Forward-declaring extension types</h2>
+
+
+
+<h2><a name="ForwardDeclaringExtTypes"></a>Forward-declaring extension types</h2>
+
+
Extension types can be forward-declared, like struct and union types. This
will be necessary if you have two extension types that need to refer to
each other, e.g.
-<blockquote><tt>cdef class Shrubbery # forward declaration</tt> <p><tt>cdef class Shrubber:</tt> <br>
+<blockquote><tt>cdef class Shrubbery # forward declaration</tt>
+
+ <p><tt>cdef class Shrubber:</tt> <br>
+
+
<tt> cdef Shrubbery work_in_progress</tt> </p>
- <p><tt>cdef class Shrubbery:</tt> <br>
+
+
+
+
+ <p><tt>cdef class Shrubbery:</tt> <br>
+
+
<tt> cdef Shrubber creator</tt></p>
+
+
</blockquote>
+
+
If you are forward-declaring an exension type that has a base class, you
must specify the base class in both the forward declaration and its subsequent
definition, for example,<br>
+
+
<blockquote><tt>cdef class A(B)<br>
+
+
<br>
+
+
...<br>
+
+
<br>
+
+
cdef class A(B):<br>
+
+
# attributes and methods</tt><br>
+
+
</blockquote>
- <h2><a name="WeakRefs"></a>Making extension types weak-referenceable</h2>By
+
+
+
+<h2><a name="WeakRefs"></a>Making extension types weak-referenceable</h2>
+
+By
default, extension types do not support having weak references made to
them. You can enable weak referencing by declaring a C attribute of
type <span style="font-family: monospace;">object</span> called <span style="font-family: monospace; font-weight: bold;">__weakref__</span>. For example,<br>
+
+
<br>
+
+
<div style="margin-left: 40px;"><span style="font-family: monospace;">cdef class ExplodingAnimal:</span><br style="font-family: monospace;">
+
+
<span style="font-family: monospace;"> """This animal will self-destruct when it is</span><br>
+
+
<span style="font-family: monospace;"> no longer strongly referenced."""</span><br>
+
+
<span style="font-family: monospace;"> </span><br style="font-family: monospace;">
+
+
<span style="font-family: monospace;"></span><span style="font-family: monospace;"> cdef object __weakref__</span><br>
+
+
</div>
+
+
<br>
+
+
<h2><a name="PublicAndExtern"></a>Public and external extension types</h2>
+
+
Extension types can be declared <b>extern</b> or <b>public</b>. An <a href="#ExternalExtTypes"><b>extern</b> extension type declaration</a> makes
an extension type defined in external C code available to a Pyrex module.
A <a href="#PublicExtensionTypes"><b>public</b> extension type declaration</a> makes an extension type defined in a Pyrex module available to external C
code.
<h3> <a name="ExternalExtTypes"></a>External extension types</h3>
+
+
An <b>extern</b> extension type allows you to gain access to the internals
of Python objects defined in the Python core or in a non-Pyrex extension
module.
@@ -328,80 +854,191 @@
module. While you can still do that, Pyrex 0.8 and later provides a better
mechanism for this. See <a href="sharing.html">Sharing C Declarations Between
Pyrex Modules</a>.</blockquote>
+
+
Here is an example which will let you get at the C-level members of the
built-in <i>complex</i> object.
-<blockquote><tt>cdef extern from "complexobject.h":</tt> <p><tt> struct Py_complex:</tt> <br>
+<blockquote><tt>cdef extern from "complexobject.h":</tt>
+
+ <p><tt> struct Py_complex:</tt> <br>
+
+
<tt> double real</tt> <br>
+
+
<tt> double imag</tt> </p>
- <p><tt> ctypedef class __builtin__.complex [object PyComplexObject]:</tt>
+
+
+
+
+ <p><tt> ctypedef class __builtin__.complex [object PyComplexObject]:</tt>
<br>
+
+
<tt> cdef Py_complex cval</tt>
</p>
- <p><tt># A function which uses the above type</tt> <br>
+
+
+
+
+ <p><tt># A function which uses the above type</tt> <br>
+
+
<tt>def spam(complex c):</tt> <br>
+
+
<tt> print "Real:", c.cval.real</tt> <br>
+
+
<tt> print "Imag:", c.cval.imag</tt></p>
+
+
</blockquote>
+
+
Some important things to note are:
<ol>
+
+
<li> In this example, <b>ctypedef class</b> has been used. This is because,
in the Python header files, the <tt>PyComplexObject</tt> struct is declared
with<br>
+
+
<br>
+
+
+
+
<div style="margin-left: 40px;"><tt>ctypedef struct {</tt> <br>
+
+
<tt> ...</tt> <br>
+
+
<tt>} PyComplexObject;<br>
+
+
<br>
+
+
</tt></div>
-</li><li>As well as the name of the extension type, the <i>module</i> in which
+
+
+ </li>
+
+ <li>As well as the name of the extension type, the <i>module</i> in which
its type object can be found is also specified. See the <a href="#ImplicitImport">implicit importing</a> section below. <br>
+
+
<br>
+
+
</li>
-<li> When declaring an external extension type, you don't declare
+
+
+ <li> When declaring an external extension type, you don't declare
any methods. Declaration of methods is not required in order to call them,
because the calls are Python method calls. Also, as with structs and unions,
if your extension class declaration is inside a <i>cdef extern from</i> block,
you only need to declare those C members which you wish to access.</li>
- </ol>
- <h3> <a name="ImplicitImport"></a>Implicit importing</h3>
- <blockquote><font color="#ef1f1d">Backwards Incompatibility Note</font>:
+
+
+
+</ol>
+
+
+
+<h3> <a name="ImplicitImport"></a>Implicit importing</h3>
+
+
+
+<blockquote><font color="#ef1f1d">Backwards Incompatibility Note</font>:
You will have to update any pre-0.8 Pyrex modules you have which use <b>extern</b>
extension types. I apologise for this, but for complicated reasons it proved
to be too difficult to continue supporting the old way of doing these while
introducing the new features that I wanted.</blockquote>
+
+
Pyrex 0.8 and later requires you to include a module name in an extern
extension class declaration, for example,
<blockquote><tt>cdef extern class MyModule.Spam:</tt> <br>
+
+
<tt> ...</tt></blockquote>
+
+
The type object will be implicitly imported from the specified module and
bound to the corresponding name in this module. In other words, in this
example an implicit
<ol>
- <pre>from <tt>MyModule</tt> import Spam</pre>
- </ol>
+
+
+
+
+ <pre>from <tt>MyModule</tt> import Spam</pre>
+
+
+
+</ol>
+
+
statement will be executed at module load time.
<p>The module name can be a dotted name to refer to a module inside a package
hierarchy, for example, </p>
- <blockquote><tt>cdef extern class My.Nested.Package.Spam:</tt> <br>
+
+
+
+<blockquote><tt>cdef extern class My.Nested.Package.Spam:</tt> <br>
+
+
<tt> ...</tt></blockquote>
+
+
You can also specify an alternative name under which to import the type
using an <b>as</b> clause, for example,
<ol>
+
+
<tt>cdef extern class My.Nested.Package.Spam as Yummy:</tt> <br>
- <tt> ...</tt> </ol>
+
+
+ <tt> ...</tt>
+</ol>
+
+
which corresponds to the implicit import statement
<ol>
- <pre>from <tt>My.Nested.Package</tt> import <tt>Spam</tt> as <tt>Yummy</tt></pre>
- </ol>
- <h3> <a name="TypeVsConstructor"></a>Type names vs. constructor names</h3>
+
+
+
+
+ <pre>from <tt>My.Nested.Package</tt> import <tt>Spam</tt> as <tt>Yummy</tt></pre>
+
+
+
+</ol>
+
+
+
+<h3> <a name="TypeVsConstructor"></a>Type names vs. constructor names</h3>
+
+
Inside a Pyrex module, the name of an extension type serves two distinct
purposes. When used in an expression, it refers to a module-level global
variable holding the type's constructor (i.e. its type-object). However,
it can also be used as a C type name to declare variables, arguments and
return values of that type.
<p>When you declare </p>
- <blockquote><tt>cdef extern class MyModule.Spam:</tt> <br>
+
+
+
+<blockquote><tt>cdef extern class MyModule.Spam:</tt> <br>
+
+
<tt> ...</tt></blockquote>
+
+
the name <tt>Spam</tt> serves both these roles. There may be other names
by which you can refer to the constructor, but only <tt>Spam</tt> can be
used as a type name. For example, if you were to explicity <tt>import MyModule</tt>,
@@ -409,21 +1046,34 @@
wouldn't be able to use <tt>MyModule.Spam</tt> as a type name.
<p>When an <b>as</b> clause is used, the name specified in the <b>as</b>
clause also takes over both roles. So if you declare </p>
- <blockquote><tt>cdef extern class MyModule.Spam as Yummy:</tt> <br>
+
+
+
+<blockquote><tt>cdef extern class MyModule.Spam as Yummy:</tt> <br>
+
+
<tt> ...</tt></blockquote>
+
+
then <tt>Yummy</tt> becomes both the type name and a name for the constructor.
Again, there are other ways that you could get hold of the constructor,
but only <tt>Yummy</tt> is usable as a type name.
<h3> <a name="PublicExtensionTypes"></a>Public extension types</h3>
+
+
An extension type can be declared <b>public</b>, in which case a <b>.h</b>
file is generated containing declarations for its object struct and type
object. By including the <b>.h</b> file in external C code that you write,
that code can access the attributes of the extension type.
<h3> <a name="NameSpecClause"></a>Name specification clause</h3>
+
+
The part of the class declaration in square brackets is a special feature
only available for <b>extern</b> or <b>public</b> extension types. The full
form of this clause is
<blockquote><tt>[object </tt><i>object_struct_name</i><tt>, type </tt><i>type_object_name</i><span style="font-family: monospace;"> ]</span></blockquote>
+
+
where <i>object_struct_name</i> is the name to assume for the type's C
struct, and <i>type_object_name</i> is the name to assume for the type's
statically declared type object. (The object and type clauses can be written
@@ -433,12 +1083,30 @@
generate code that is compatible with the declarations in the header file.
Otherwise, for <b>extern</b> extension types, the <b>object</b> clause is
optional. </p>
- <p>For <b>public</b> extension types, the <b>object</b> and <b>type</b> clauses
+
+
+
+<p>For <b>public</b> extension types, the <b>object</b> and <b>type</b> clauses
are both required, because Pyrex must be able to generate code that is compatible
with external C code. </p>
- <p> </p>
- <hr width="100%"> <br>
+
+
+
+<p> </p>
+
+
+
+<hr width="100%"> <br>
+
+
Back to the <a href="overview.html">Language Overview</a> <br>
+
+
<br>
+
+
<br>
-</body></html>
\ No newline at end of file
+
+
+</body>
+</html>
Modified: lxml/pyrex/Doc/overview.html
==============================================================================
--- lxml/pyrex/Doc/overview.html (original)
+++ lxml/pyrex/Doc/overview.html Wed Feb 14 15:20:32 2007
@@ -1,108 +1,216 @@
<!DOCTYPE doctype PUBLIC "-//w3c//dtd html 4.0 transitional//en">
<html><head>
+
+
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
+
- <meta name="GENERATOR" content="Mozilla/4.61 (Macintosh; I; PPC) [Netscape]">
- <title>Pyrex Language Overview</title>
-</head>
+
+ <meta name="GENERATOR" content="Mozilla/4.61 (Macintosh; I; PPC) [Netscape]"><title>Pyrex Language Overview</title></head>
+
<body>
+
-<h1> <hr width="100%">Overview of the Pyrex Language <hr width="100%"></h1>
+<h1>
+<hr width="100%">Overview of the Pyrex Language
+<hr width="100%"></h1>
+
This document informally describes the extensions to the Python language
made by Pyrex. Some day there will be a reference manual covering everything
- in more detail. <br>
+ in more detail. <h2> Contents</h2>
-
-<h2> Contents</h2>
<ul>
- <li> <a href="#Basics">Basics</a></li>
- <ul>
+
+ <li><a href="#SourceFiles">Source Files and Compilation</a> <span style="color: rgb(255, 0, 0);">(Section added in 0.9.5)</span><br>
+ </li><li><a href="#Basics">Basics</a></li>
+
+
+
+ <ul>
+
<li> <a href="#PyFuncsVsCFuncs">Python functions vs. C functions</a></li>
+
<li> <a href="#PyObjParams">Python objects as parameters</a></li>
- <li> <a href="#CVarAndTypeDecls">C variable and type definitions</a></li><li><a href="#AutomaticTypeConversions">Automatic type conversions</a></li>
+
+ <li> <a href="#CVarAndTypeDecls">C variable and type definitions</a></li>
+ <li><a href="#AutomaticTypeConversions">Automatic type conversions</a></li>
+
+
<ul>
+
<li><a href="#PyToCStringCaveats">Caveats when using a Python string in a C context</a></li>
+
+
</ul>
+
<li> <a href="#ScopeRules">Scope rules</a></li>
+
<li> <a href="#StatsAndExprs">Statements and expressions</a></li>
- <ul>
+
+
+ <ul>
+
<li> <a href="#ExprSyntaxDifferences">Differences between C and Pyrex
expressions<br>
-</a></li>
+
+ </a></li>
+
<li> <a href="#ForFromLoop">Integer for-loops</a></li>
- </ul>
+
+
+ </ul>
+
<li> <a href="#ExceptionValues">Error return values</a></li>
- <ul>
+
+
+ <ul>
+
<li> <a href="#CheckingReturnValues">Checking return values of non-Pyrex
functions</a></li>
- </ul>
+
+
+ </ul>
+
<li> <a href="#IncludeStatement">The <tt>include</tt> statement</a></li>
- </ul>
+
+
+ </ul>
+
<li> <a href="#InterfacingWithExternal">Interfacing with External C Code</a></li>
- <ul>
+
+
+ <ul>
+
<li> <a href="#ExternDecls">External declarations</a></li>
- <ul>
+
+
+ <ul>
+
<li> <a href="#ReferencingHeaders">Referencing C header files</a></li>
+
<li> <a href="#StructDeclStyles">Styles of struct/union/enum declaration</a></li>
+
<li> <a href="#AccessingAPI">Accessing Python/C API routines</a></li>
+
<li> <a href="#CNameSpecs">Resolving naming conflicts - C name specifications</a></li>
- </ul>
+
+
+ </ul>
+
<li> <a href="#PublicDecls">Public declarations</a></li>
- </ul>
- <li> <a href="extension_types.html">Extension Types</a> <font color="#006600">(Section revised in 0.9)</font></li>
- <li> <a href="sharing.html">Sharing Declarations Between Pyrex Modules</a>
- <font color="#006600">(NEW in 0.8)</font></li>
+
+
+ </ul>
+
+ <li> <a href="extension_types.html">Extension Types</a></li>
+
+ <li> <a href="sharing.html">Sharing Declarations Between Pyrex Modules</a></li>
+
<li> <a href="#Limitations">Limitations</a></li>
- <ul>
+
+
+ <ul>
+
<li> <a href="#Unsupported">Unsupported Python features</a></li>
+
<li> <a href="#SemanticDifferences">Semantic differences between Python
and Pyrex</a></li>
- </ul>
+
+
+ </ul>
+
</ul>
+
-<h2> <hr width="100%"><a name="Basics"></a>Basics
+<h2>
<hr width="100%"></h2>
+<h2><a name="SourceFiles"></a>Source Files and Compilation<br>
+</h2>
+<hr style="width: 100%; height: 2px;"><br>
+Pyrex source file names consist of the name of the module followed by a <span style="font-family: monospace;">.pyx</span> extension, for example a module called <span style="font-family: monospace;">primes</span> would have a source file named <span style="font-family: monospace;">primes.pyx</span>.<br>
+
+<br>
- This section describes the basic features of the Pyrex language. The facilities
+If your module is destined to live in a package, the source file name should include the <span style="font-style: italic;">full dotted name</span> that the module will eventually have. For example, a module called <span style="font-family: monospace;">primes</span> that will be installed in a package called <span style="font-family: monospace;">numbers</span> should have a source file called <span style="font-family: monospace;">numbers.primes.pyx</span>.
+This will ensure that the __name__ properties of the module and any
+classes defined in it are set correctly. If you don't do this, you may
+find that pickling doesn't work, among other problems. It also ensures
+that the Pyrex compiler has the right idea about the layout of the
+module namespace, which can be important when accessing extension types
+defined in other modules.<br>
+<br>
+Once you have written your .pyx file, there are a couple of ways of
+turning it into an extension module. One way is to compile it manually
+with the Pyrex compiler, e.g.<br>
+<br>
+<div style="margin-left: 40px;"><span style="font-family: monospace;">pyrexc primes.pyx</span><br>
+</div>
+<br>
+This will produce a file called <span style="font-family: monospace;">primes.c</span>,
+which then needs to be compiled with the C compiler using whatever
+options are appropriate on your platform for generating an extension
+module. There's a Makefile in the Demos directory (called <span style="font-family: monospace;">Makefile.nodistutils</span>) that shows how to do this for Linux.<br>
+<br>
+The other, and probably better, way is to use the distutils extension provided with Pyrex. See the <span style="font-family: monospace;">Setup.py</span>
+file in the Demos directory for an example of how to use it. This
+method has the advantage of being cross-platform -- the same setup file
+should work on any platform where distutils can compile an extension
+module.<br>
+<br>
+<hr style="width: 100%; height: 2px;">
+<h2><a name="Basics"></a>Language Basics
+<hr width="100%"></h2>
+This section describes the basic features of the Pyrex language. The facilities
covered in this section allow you to create Python-callable functions that
manipulate C data structures and convert between Python and C data types.
- Later sections will cover facilities for <a href="#InterfacingWithExternal">wrapping external C code</a>, <a href="extension_types.html">creating new Python types</a> and <a href="sharing.html">cooperation between Pyrex modules</a>.
+ Later sections will cover facilities for <a href="#InterfacingWithExternal">wrapping external C code</a>, <a href="extension_types.html">creating new Python types</a> and <a href="sharing.html">cooperation between Pyrex modules</a>.<br>
+
<h3> <a name="PyFuncsVsCFuncs"></a>Python functions vs. C functions</h3>
+
There are two kinds of function definition in Pyrex:
<p><b>Python functions</b> are defined using the <b>def</b> statement, as
in Python. They take Python objects as parameters and return Python objects.
</p>
+
<p><b>C functions</b> are defined using the new <b>cdef</b> statement. They
take either Python objects or C values as parameters, and can return either
Python objects or C values. </p>
+
<p>Within a Pyrex module, Python functions and C functions can call each other
freely, but only Python functions can be called from outside the module by
interpreted Python code. So, any functions that you want to "export" from
your Pyrex module must be declared as Python functions using <span style="font-weight: bold;">def</span>. </p>
+
<p>Parameters of either type of function can be declared to have C data types,
using normal C declaration syntax. For example, </p>
+
-<blockquote> <pre>def spam(int i, char *s):<br> ...</pre>
- <pre>cdef int eggs(unsigned long l, float f):<br> ...</pre>
+<blockquote>
+ <pre>def spam(int i, char *s):<br> ...</pre>
+
+
+ <pre>cdef int eggs(unsigned long l, float f):<br> ...</pre>
+
</blockquote>
+
When a parameter of a Python function is declared to have a C data type,
it is passed in as a Python object and automatically converted to a C value,
if possible. Automatic conversion is currently only possible for numeric
@@ -111,17 +219,22 @@
<p>C functions, on the other hand, can have parameters of any type, since
they're passed in directly using a normal C function call. </p>
+
<h3> <a name="PyObjParams"></a>Python objects as parameters and return values</h3>
+
If no type is specified for a parameter or return value, <i>it is assumed
to be a Python object.</i> (Note that this is different from the C convention,
where it would default to <tt>int</tt>.) For example, the following defines
a C function that takes two Python objects as parameters and returns a Python
object:
-<blockquote> <pre>cdef spamobjs(x, y):<br> ...</pre>
+<blockquote>
+ <pre>cdef spamobjs(x, y):<br> ...</pre>
+
</blockquote>
+
Reference counting for these objects is performed automatically according
to the standard Python/C API rules (i.e. borrowed references are taken as
parameters and a new reference is returned).
@@ -129,126 +242,217 @@
as a Python object. This can be useful if the name being declared would otherwise
be taken as the name of a type, for example, </p>
+
-<blockquote> <pre>cdef ftang(object int):<br> ...</pre>
+<blockquote>
+ <pre>cdef ftang(object int):<br> ...</pre>
+
</blockquote>
+
declares a parameter called <tt>int</tt> which is a Python object. You
can also use <b>object </b>as the explicit return type of a function, e.g.
-<blockquote> <pre>cdef object ftang(object int):<br> ...</pre>
+<blockquote>
+ <pre>cdef object ftang(object int):<br> ...</pre>
+
</blockquote>
+
In the interests of clarity, it is probably a good idea to always be explicit
about <b>object </b>parameters in C functions.
<h3> <a name="CVarAndTypeDecls"></a>C variable and type definitions</h3>
+
The <b>cdef</b> statement is also used to declare C variables, either
local or module-level:
-<blockquote> <pre>cdef int i, j, k<br>cdef float f, g[42], *h</pre>
+<blockquote>
+ <pre>cdef int i, j, k<br>cdef float f, g[42], *h</pre>
+
</blockquote>
+
and C struct, union or enum types:
-<blockquote> <pre>cdef struct Grail:<br> int age<br> float volume</pre>
- <pre>cdef union Food:<br> char *spam<br> float *eggs</pre>
- <pre>cdef enum CheeseType:<br> cheddar, edam, <br> camembert</pre>
- <pre>cdef enum CheeseState:<br> hard = 1<br> soft = 2<br> runny = 3</pre>
+<blockquote>
+ <pre>cdef struct Grail:<br> int age<br> float volume</pre>
+
+
+ <pre>cdef union Food:<br> char *spam<br> float *eggs</pre>
+
+
+ <pre>cdef enum CheeseType:<br> cheddar, edam, <br> camembert</pre>
+
+
+ <pre>cdef enum CheeseState:<br> hard = 1<br> soft = 2<br> runny = 3</pre>
+
</blockquote>
+
There is currently no special syntax for defining a constant, but you
can use an anonymous enum declaration for this purpose, for example,
<blockquote><tt>cdef enum:</tt> <br>
+
<tt> tons_of_spam = 3</tt></blockquote>
+
Note that the words <span style="font-family: monospace;">struct</span>, <span style="font-family: monospace;">union</span> and <span style="font-family: monospace;">enum</span> are used only when <i>defining</i> a type, not when referring to it. For example, to declare a variable pointing
to a Grail you would write
-<blockquote> <pre>cdef Grail *gp</pre>
+<blockquote>
+ <pre>cdef Grail *gp</pre>
+
</blockquote>
+
and <i>not</i>
-<blockquote> <pre>cdef struct Grail *gp <font color="#ed181e"># WRONG</font></pre>
+<blockquote>
+ <pre>cdef struct Grail *gp <font color="#ed181e"># WRONG</font></pre>
+
</blockquote>
+
There is also a <b>ctypedef</b> statement for giving names to types, e.g.
-<blockquote> <pre>ctypedef unsigned long ULong</pre>
- <pre>ctypedef int *IntPtr<br></pre></blockquote>
+<blockquote>
+ <pre>ctypedef unsigned long ULong</pre>
+
+
+ <pre>ctypedef int *IntPtr<br></pre>
+</blockquote>
+
<h3><a name="AutomaticTypeConversions"></a>Automatic type conversions</h3>
+
In most situations, automatic conversions will be performed for the
basic numeric and string types when a Python object is used in a
context requiring a C value, or vice versa. The following table
summarises the conversion possibilities.<br>
+
<br>
+
<table style="margin-left: auto; margin-right: auto; width: 10%; text-align: left;" border="1" cellpadding="4" cellspacing="0">
+
<tbody>
+
<tr>
+
<th style="vertical-align: top; width: 40%; white-space: nowrap;">C types<br>
+
</th>
+
<th style="vertical-align: top; width: 150px; white-space: nowrap;">From Python types<br>
+
</th>
+
<th style="vertical-align: top; width: 150px; white-space: nowrap;">To Python types<br>
+
</th>
+
</tr>
+
<tr>
+
<td colspan="1" rowspan="1" style="vertical-align: top; width: 40%; white-space: nowrap;">[unsigned] char<br>
+
[unsigned] short<br>
+
int, long</td>
+
<td colspan="1" rowspan="1" style="vertical-align: top; width: 150px; white-space: nowrap;">int, long<br>
+
</td>
+
<td colspan="1" rowspan="1" style="vertical-align: top; width: 150px; white-space: nowrap;">int<br>
+
</td>
+
</tr>
+
<tr>
+
</tr>
+
<tr>
+
<td colspan="1" rowspan="1" style="vertical-align: top; width: 40%; white-space: nowrap;">unsigned int<br>
+
unsigned long<br>
+
[unsigned] long long<br>
+
</td>
+
<td colspan="1" rowspan="1" style="vertical-align: top; white-space: nowrap;">int, long<br>
+
<br>
+
</td>
+
<td colspan="1" rowspan="1" style="vertical-align: top; white-space: nowrap;">long<br>
+
<br>
+
</td>
+
</tr>
+
<tr>
+
</tr>
+
<tr>
+
<td style="vertical-align: top; width: 40%; white-space: nowrap;">float, double, long double<br>
+
</td>
+
<td style="vertical-align: top; width: 150px; white-space: nowrap;">int, long, float<br>
+
</td>
+
<td style="vertical-align: top; width: 150px; white-space: nowrap;">float<br>
+
</td>
+
</tr>
+
<tr>
+
<td style="vertical-align: top; width: 40%; white-space: nowrap;">char *<br>
+
</td>
+
<td style="vertical-align: top; width: 150px; white-space: nowrap;">str<span style="font-style: italic;"></span><br>
+
</td>
+
<td style="vertical-align: top; width: 150px; white-space: nowrap;">str<br>
+
</td>
+
</tr>
+
+
</tbody>
</table>
+
<br>
+
<h4><a name="PyToCStringCaveats"></a>Caveats when using a Python string in a C context</h4>
+
You need to be careful when using a Python string in a context expecting a <span style="font-family: monospace;">char *</span>.
In this situation, a pointer to the contents of the Python string is
used, which is only valid as long as the Python string exists. So you
@@ -257,12 +461,16 @@
the Python string will live long enough, you will need to copy the C
string.<br>
+
<br>
+
Pyrex detects and prevents <span style="font-style: italic;">some</span> mistakes of this kind. For instance, if you attempt something like<br>
+
<pre style="margin-left: 40px;">cdef char *s<br>s = pystring1 + pystring2</pre>
+
then Pyrex will produce the error message "<span style="font-weight: bold;">Obtaining char * from temporary Python value</span>".
The reason is that concatenating the two Python strings produces a new
Python string object that is referenced only by a temporary internal
@@ -270,28 +478,37 @@
the temporary variable will be decrefed and the Python string
deallocated, leaving <span style="font-family: monospace;">s</span> dangling. Since this code could not possibly work, Pyrex refuses to compile it.<br>
+
<br>
+
The solution is to assign the result of the concatenation to a Python variable, and then obtain the char * from that, i.e.<br>
+
<pre style="margin-left: 40px;">cdef char *s<br>p = pystring1 + pystring2<br>s = p<br></pre>
+
It is then your responsibility to hold the reference <span style="font-family: monospace;">p</span> for as long as necessary.<br>
+
<br>
+
Keep in mind that the rules used to detect such errors are only
heuristics. Sometimes Pyrex will complain unnecessarily, and sometimes
it will fail to detect a problem that exists. Ultimately, you need to
understand the issue and be careful what you do.<br>
+
<blockquote>
</blockquote>
+
<h3> <a name="ScopeRules"></a>Scope rules</h3>
+
Pyrex determines whether a variable belongs to a local scope, the module
scope, or the built-in scope <i>completely statically.</i> As with Python,
assigning to a variable which is not otherwise declared implicitly declares
@@ -304,35 +521,49 @@
declare a name to be a module-level name when there would otherwise not be
any indication of this, for example, </p>
+
<blockquote><tt>global __name__</tt> <br>
+
<tt>print __name__</tt></blockquote>
+
Without the <b>global</b> statement, the above would print the name of
the builtins module.<br>
+
<br>
+
Note: A consequence of these rules is that the module-level scope behaves
the same way as a Python local scope if you refer to a variable before assigning
to it. In particular, tricks such as the following will <i>not</i> work
in Pyrex:<br>
+
-<blockquote> <pre>try:<br> x = True<br>except NameError:<br> True = 1<br></pre>
+<blockquote>
+ <pre>try:<br> x = True<br>except NameError:<br> True = 1<br></pre>
+
</blockquote>
+
because, due to the assignment, the True will always be looked up in the
module-level scope. You would have to do something like this instead:<br>
+
-<blockquote> <pre>import __builtin__<br>try:<br> True = __builtin__.True<br>except AttributeError:<br> True = 1<br></pre>
+<blockquote>
+ <pre>import __builtin__<br>try:<br> True = __builtin__.True<br>except AttributeError:<br> True = 1<br></pre>
+
</blockquote>
+
<hr width="100%">
<h3> <a name="StatsAndExprs"></a>Statements and expressions</h3>
+
Control structures and expressions follow Python syntax for the most part.
When applied to Python objects, they have the same semantics as in Python
(unless otherwise noted). Most of the Python operators can also be applied
@@ -341,87 +572,128 @@
are performed automatically between Python objects and C numeric or string
types. </p>
+
<p>Reference counts are maintained automatically for all Python objects, and
all Python operations are automatically checked for errors, with appropriate
action taken. </p>
+
<h4> <a name="ExprSyntaxDifferences"></a>Differences between C and Pyrex
expressions</h4>
+
There
are some differences in syntax and semantics between C expressions and
Pyrex expressions, particularly in the area of C constructs which have
no direct equivalent in Python.<br>
+
<ul>
-<li>An integer literal without an <span style="font-family: monospace; font-weight: bold;">L</span> suffix is treated as a C constant, and will be truncated to whatever size your C compiler thinks appropriate. With an <span style="font-family: monospace; font-weight: bold;">L</span> suffix, it will be converted to Python long integer (even if it would be small enough to fit into a C int).<br>
+
+ <li>An integer literal without an <span style="font-family: monospace; font-weight: bold;">L</span> suffix is treated as a C constant, and will be truncated to whatever size your C compiler thinks appropriate. With an <span style="font-family: monospace; font-weight: bold;">L</span> suffix, it will be converted to Python long integer (even if it would be small enough to fit into a C int).<br>
+
<br>
+
</li>
-<li> There is no <b><tt>-></tt></b> operator in Pyrex. Instead of <tt>p->x</tt>,
+
+ <li> There is no <b><tt>-></tt></b> operator in Pyrex. Instead of <tt>p->x</tt>,
use <tt>p.x</tt></li>
+
<li> There is no <b><tt>*</tt></b> operator in Pyrex. Instead of
<tt>*p</tt>, use <tt>p[0]</tt></li>
+
<li> There is an <b><tt>&</tt></b> operator, with the same semantics
as in C.</li>
+
<li> The null C pointer is called <b><tt>NULL</tt></b>, not 0 (and
<tt>NULL</tt> is a reserved word).</li>
+
<li> Character literals are written with a <b>c</b> prefix, for
example:</li>
- <ul>
- <pre>c'X'</pre>
- </ul>
+
+
+ <ul>
+
+
+ <pre>c'X'</pre>
+
+
+ </ul>
+
<li> Type casts are written <b><tt><type>value</tt></b> , for example:</li>
- <ul>
- <pre>cdef char *p, float *q<br>p = <char*>q</pre>
- </ul>
+
+
+ <ul>
+
+
+ <pre>cdef char *p, float *q<br>p = <char*>q</pre>
+
+
+ </ul>
+
<i><b>Warning</b>: Don't attempt to use a typecast to convert between
Python and C data types -- it won't do the right thing. Leave Pyrex to perform
the conversion automatically.</i>
</ul>
+
<h4> <a name="ForFromLoop"></a>Integer for-loops</h4>
+
You should be aware that a for-loop such as
<blockquote><tt>for i in range(n):</tt> <br>
+
<tt> ...</tt></blockquote>
+
won't be very fast, even if <tt>i</tt> and <tt>n</tt> are declared as
C integers, because <tt>range</tt> is a Python function. For iterating over
ranges of integers, Pyrex has another form of for-loop:
<blockquote><tt>for i from 0 <= i < n:</tt> <br>
+
<tt> ...</tt></blockquote>
+
If the loop variable and the lower and upper bounds are all C integers,
this form of loop will be much faster, because Pyrex will translate it into
pure C code.
<p>Some things to note about the <tt>for-from</tt> loop: </p>
+
<ul>
+
<li> The target expression must be a variable name.</li>
+
<li> The name between the lower and upper bounds must be the same as
the target name.</li>
+
<li> The direction of iteration is determined by the relations. If they
are both from the set {<tt><</tt>, <tt><=</tt>} then it is upwards;
if they are both from the set {<tt>></tt>, <tt>>=</tt>} then it is
downwards. (Any other combination is disallowed.)</li>
+
</ul>
+
Like other Python looping statements, <tt>break</tt> and <tt>continue</tt> may be used in the body, and the loop may have an <tt>else</tt> clause.
-<h2> <hr width="100%"></h2>
+<h2>
+<hr width="100%"></h2>
+
<h3> <a name="ExceptionValues"></a>Error return values</h3>
+
If you don't do anything special, a function declared with <b>cdef</b> that does not return a Python object has no way of reporting Python exceptions
to its caller. If an exception is detected in such a function, a warning
message is printed and the exception is ignored.
@@ -429,10 +701,13 @@
to propagate exceptions to its caller, you need to declare an <b>exception
value</b> for it. Here is an example: </p>
+
<blockquote><tt>cdef int spam() except -1:</tt> <br>
+
<tt> ...</tt></blockquote>
+
With this declaration, whenever an exception occurs inside <tt>spam</tt>,
it will immediately return with the value <tt>-1</tt>. Furthermore, whenever
a call to <tt>spam</tt> returns <tt>-1</tt>, an exception will be assumed
@@ -442,77 +717,107 @@
reserve one entirely for signalling errors, you can use an alternative form
of exception value declaration: </p>
+
<blockquote><tt>cdef int spam() except? -1:</tt> <br>
+
<tt> ...</tt></blockquote>
- The "?" indicates that the value <tt>-1</tt> only indicates a <i>possible</i> error. In this case, Pyrex generates a call to <tt>PyErr_Occurred</tt>if the
+
+ The "?" indicates that the value <tt>-1</tt> only indicates a <i>possible</i> error. In this case, Pyrex generates a call to <tt>PyErr_Occurred</tt> if the
exception value is returned, to make sure it really is an error.
<p>There is also a third form of exception value declaration: </p>
+
<blockquote><tt>cdef int spam() except *:</tt> <br>
+
<tt> ...</tt></blockquote>
+
This form causes Pyrex to generate a call to <tt>PyErr_Occurred</tt> after
- <i>every</i> call to spam, regardless of what value it returns. If you have
+ <i>every</i> call to <code>spam</code>, regardless of what value it returns. If you have
a function returning <tt>void</tt> that needs to propagate errors, you will
have to use this form, since there isn't any return value to test.
<p>Some things to note: </p>
+
<ul>
- <li> Currently, exception values can only declared for functions returning
- an integer, float or pointer type, and the value must be a <i>literal</i>,
- not an expression (although it can be negative). The only possible pointer
+
+ <li>Exception values can only declared for functions returning
+ an integer, enum, float or pointer type, and the value must be a constant expression. The only possible pointer
exception value is <tt>NULL</tt>. Void functions can only use the <tt>except
*</tt> form.</li>
- <br>
+
+
<li> The exception value specification is part of the signature
of the function. If you're passing a pointer to a function as a parameter
or assigning it to a variable, the declared type of the parameter or variable
must have the same exception value specification (or lack thereof). Here
is an example of a pointer-to-function declaration with an exception value:</li>
- <ul>
- <pre><tt>int (*grail)(int, char *) except -1</tt></pre>
- </ul>
+
+
+ <ul>
+
+
+ <pre><tt>int (*grail)(int, char *) except -1</tt></pre>
+
+
+ </ul>
+
<li> You don't need to (and shouldn't) declare exception values for functions
which return Python objects. Remember that a function with no declared return
type implicitly returns a Python object.</li>
+
</ul>
+
<h4> <a name="CheckingReturnValues"></a>Checking return values of non-Pyrex
functions</h4>
+
It's important to understand that the <tt>except</tt> clause does <i>not</i> cause an error to be <i>raised</i> when the specified value is returned. For
example, you can't write something like
-<blockquote> <pre>cdef extern FILE *fopen(char *filename, char *mode) except NULL <font color="#ed181e"># WRONG!</font></pre>
+<blockquote>
+ <pre>cdef extern FILE *fopen(char *filename, char *mode) except NULL <font color="#ed181e"># WRONG!</font></pre>
+
</blockquote>
+
and expect an exception to be automatically raised if a call to fopen
returns NULL. The except clause doesn't work that way; its only purpose
is for <i>propagating</i> exceptions that have already been raised, either
by a Pyrex function or a C function that calls Python/C API routines. To
get an exception from a non-Python-aware function such as fopen, you will
have to check the return value and raise it yourself, for example,
-<blockquote> <pre>cdef FILE *p<br>p = fopen("spam.txt", "r")<br>if p == NULL:<br> raise SpamError("Couldn't open the spam file")</pre>
+<blockquote>
+ <pre>cdef FILE *p<br>p = fopen("spam.txt", "r")<br>if p == NULL:<br> raise SpamError("Couldn't open the spam file")</pre>
+
</blockquote>
+
-<h4> <hr width="100%"></h4>
+<h4>
+<hr width="100%"></h4>
+
<h4> <a name="IncludeStatement"></a>The <tt>include</tt> statement</h4>
+
For convenience, a large Pyrex module can be split up into a number of
files which are put together using the <b>include</b> statement, for example
-<blockquote> <pre>include "spamstuff.pxi"</pre>
+<blockquote>
+ <pre>include "spamstuff.pxi"</pre>
+
</blockquote>
+
The contents of the named file are textually included at that point. The
included file can contain any complete top-level Pyrex statements, including
other <b>include</b> statements. The <b>include</b> statement itself can
@@ -524,11 +829,14 @@
and it is expected that use of the <b>include</b> statement for this purpose
will be phased out altogether in future versions. </p>
+
-<h2> <hr width="100%"><a name="InterfacingWithExternal"></a>Interfacing with External
+<h2>
+<hr width="100%"><a name="InterfacingWithExternal"></a>Interfacing with External
C Code
<hr width="100%"></h2>
+
One of the main uses of Pyrex is wrapping existing libraries of C code.
This is achieved by using <a href="#ExternDecls">external declarations</a> to declare the C functions and variables from the library that you want to
use.
@@ -540,23 +848,32 @@
allow Python code to call C code, it can also be used to allow C code to
call Python code. </p>
+
<h3> <a name="ExternDecls"></a>External declarations</h3>
+
By default, C functions and variables declared at the module level are
local to the module (i.e. they have the C <b>static</b> storage class). They
can also be declared <b>extern</b> to specify that they are defined elsewhere,
for example:
-<blockquote> <pre>cdef extern int spam_counter</pre>
- <pre>cdef extern void order_spam(int tons)</pre>
+<blockquote>
+ <pre>cdef extern int spam_counter</pre>
+
+
+ <pre>cdef extern void order_spam(int tons)</pre>
+
</blockquote>
+
<blockquote> </blockquote>
+
<h4> <a name="ReferencingHeaders"></a>Referencing C header files</h4>
+
When you use an extern definition on its own as in the examples above,
Pyrex includes a declaration for it in the generated C file. This can cause
problems if the declaration doesn't exactly match the declaration that will
@@ -566,26 +883,41 @@
<p>To achieve this, you can tell Pyrex that the declarations are to be found
in a C header file, like this: </p>
+
-<blockquote> <pre>cdef extern from "spam.h":</pre>
- <pre> int spam_counter</pre>
- <pre> void order_spam(int tons)</pre>
+<blockquote>
+ <pre>cdef extern from "spam.h":</pre>
+
+
+ <pre> int spam_counter</pre>
+
+
+ <pre> void order_spam(int tons)</pre>
+
</blockquote>
+
The <b>cdef extern from</b> clause does three things:
<ol>
+
<li> It directs Pyrex to place a <b>#include</b> statement for the named
header file in the generated C code.<br>
+
</li>
+
<li> It prevents Pyrex from generating any C code for the declarations
found in the associated block.<br>
+
</li>
+
<li> It treats all declarations within the block as though they
started with <b>cdef extern</b>.</li>
+
</ol>
+
It's important to understand that Pyrex does <i>not</i> itself read the
C header file, so you still need to provide Pyrex versions of any declarations
from it that you use. However, the Pyrex declarations don't always have to
@@ -593,83 +925,129 @@
<ol>
+
<li> Don't use <b>const</b>. Pyrex doesn't know anything about const,
so just leave it out. Most of the time this shouldn't cause any problem,
although on rare occasions you might have to use a cast.<sup><a href="#Footnote1"> 1</a></sup><br>
+
</li>
+
<li> Leave out any platform-specific extensions to C declarations
such as <b>__declspec()</b>.<br>
+
</li>
+
<li> If the header file declares a big struct and you only want
to use a few members, you only need to declare the members you're interested
in. Leaving the rest out doesn't do any harm, because the C compiler will
use the full definition from the header file.<br>
+
<br>
+
In some cases, you might not need <i>any</i> of the struct's members, in
which case you can just put <tt>pass</tt> in the body of the struct declaration,
e.g.<br>
+
<br>
+
<tt> cdef extern from "foo.h":<br>
+
struct spam:<br>
+
pass</tt><br>
+
<br>
+
Note that you can only do this inside a <b>cdef extern from</b> block; struct
declarations anywhere else must be non-empty.<br>
+
<br>
+
</li>
+
<li> If the header file uses typedef names such as <b>size_t </b>to refer
to platform-dependent flavours of numeric types, you will need a corresponding
<b>ctypedef</b> statement, but you don't need to match the type exactly,
just use something of the right general kind (int, float, etc). For example,</li>
- <ol>
- <pre>ctypedef int size_t</pre>
- </ol>
+
+
+ <ol>
+
+
+ <pre>ctypedef int size_t</pre>
+
+
+ </ol>
+
will work okay whatever the actual size of a size_t is (provided the header
file defines it correctly). <br>
+
<li> If the header file uses macros to define constants, translate
them into a dummy <b>enum</b> declaration.<br>
+
</li>
+
<li> If the header file defines a function using a macro, declare
it as though it were an ordinary function, with appropriate argument and
result types.</li>
+
</ol>
+
A few more tricks and tips:
<ul>
+
<li> If you want to include a C header because it's needed by another
header, but don't want to use any declarations from it, put <tt><font size="+1">pass</font></tt> in the extern-from block:</li>
+
</ul>
+
<ul>
- <ul>
+
+
+ <ul>
+
<tt>cdef extern from "spam.h":</tt> <br>
- <tt> pass</tt> </ul>
+
+ <tt> pass</tt>
+ </ul>
+
</ul>
+
<ul>
+
<li> If you want to include some external declarations, but don't want
to specify a header file (because it's included by some other header that
you've already included) you can put <tt>*</tt> in place of the header file
name:</li>
+
</ul>
+
-<blockquote> <blockquote><tt>cdef extern from *:</tt> <br>
+<blockquote>
+ <blockquote><tt>cdef extern from *:</tt> <br>
+
<tt> ...</tt></blockquote>
+
</blockquote>
+
<h4> <a name="StructDeclStyles"></a>Styles of struct, union and enum declaration</h4>
+
There are two main ways that structs, unions and enums can be declared
in C header files: using a tag name, or using a typedef. There are also some
variations based on various combinations of these.
@@ -680,93 +1058,158 @@
above corresponds to the use of a tag name. To get the other style, you prefix
the declaration with <b>ctypedef</b>, as illustrated below. </p>
+
<p>The following table shows the various possible styles that can be found
in a header file, and the corresponding Pyrex declaration that you should
put in the <b>cdef exern from </b>block. Struct declarations are used as
an example; the same applies equally to union and enum declarations. </p>
+
<p>Note that in all the cases below, you refer to the type in Pyrex code simply
as <tt><font size="+1">Foo</font></tt>, not <tt><font size="+1">struct Foo</font></tt>.
<br>
- <table cellpadding="5">
+
+
+<table cellpadding="5">
+
<tbody>
+
<tr bgcolor="#8cbc1c" valign="top">
+
<td bgcolor="#8cbc1c"> </td>
+
<td bgcolor="#ff9933" nowrap="nowrap"><b>C code</b></td>
+
<td bgcolor="#66cccc" valign="top"><b>Possibilities for corresponding
Pyrex code</b></td>
+
<td bgcolor="#99cc33" valign="top"><b>Comments</b></td>
+
</tr>
+
<tr bgcolor="#8cbc1c" valign="top">
+
<td>1</td>
+
<td bgcolor="#ff9900"><tt>struct Foo {</tt> <br>
+
<tt> ...</tt> <br>
+
<tt>};</tt></td>
+
<td bgcolor="#66cccc"><tt>cdef struct Foo:</tt> <br>
+
<tt> ...</tt></td>
+
<td>Pyrex will refer to the type as <tt>struct Foo </tt>in the generated
C code<tt>.</tt></td>
+
</tr>
+
<tr bgcolor="#8cbc1c" valign="top">
+
<td valign="top">2</td>
+
<td bgcolor="#ff9900" nowrap="nowrap"><tt>typedef struct {</tt> <br>
+
<tt> ...</tt> <br>
+
<tt>} Foo;</tt></td>
+
<td bgcolor="#66cccc" valign="top"><tt>ctypedef struct Foo:</tt> <br>
+
<tt> ...</tt></td>
+
<td valign="top">Pyrex will refer to the type simply as <tt>Foo</tt>
in the generated C code.</td>
+
</tr>
+
<tr bgcolor="#8cbc1c" valign="top">
+
<td rowspan="2">3</td>
+
<td rowspan="2" bgcolor="#ff9900" nowrap="nowrap"><tt>typedef struct
foo {</tt> <br>
+
<tt> ...</tt> <br>
+
<tt>} Foo;</tt></td>
+
<td bgcolor="#66cccc" nowrap="nowrap" valign="top"><tt>cdef struct foo:</tt> <br>
+
<tt> ...</tt> <br>
+
<tt>ctypedef foo Foo #optional</tt></td>
+
<td rowspan="2" valign="top">If the C header uses both a tag and a typedef
with <i>different</i> names, you can use either form of declaration in Pyrex
(although if you need to forward reference the type, you'll have to use
the first form).</td>
+
</tr>
+
<tr>
+
<td bgcolor="#66cccc"><tt>ctypedef struct Foo:</tt> <br>
+
<tt> ...</tt></td>
+
</tr>
+
<tr bgcolor="#8cbc1c" valign="top">
+
<td>4</td>
+
<td bgcolor="#ff9900" nowrap="nowrap"><tt>typedef struct Foo {</tt> <br>
+
<tt> ...</tt> <br>
+
<tt>} Foo;</tt></td>
+
<td bgcolor="#66cccc" valign="top"><tt>cdef struct Foo:</tt> <br>
+
<tt> ...</tt></td>
+
<td>If the header uses the <i>same</i> name for the tag and the typedef,
you won't be able to include a <b>ctypedef</b> for it -- but then, it's not
necessary.</td>
+
</tr>
- </tbody> </table>
+
+
+ </tbody>
+</table>
+
</p>
+
<h4> <a name="AccessingAPI"></a>Accessing Python/C API routines</h4>
+
One particular use of the <b>cdef extern from</b> statement is for gaining
access to routines in the Python/C API. For example,
-<blockquote> <pre>cdef extern from "Python.h":</pre>
- <pre> object PyString_FromStringAndSize(char *s, int len)</pre>
+<blockquote>
+ <pre>cdef extern from "Python.h":</pre>
+
+
+ <pre> object PyString_FromStringAndSize(char *s, int len)</pre>
+
</blockquote>
+
will allow you to create Python strings containing null bytes.
<p> </p>
+
<hr width="100%">
<h3> <a name="CNameSpecs"></a>Resolving naming conflicts - C name specifications</h3>
+
Each Pyrex module has a single module-level namespace for both Python
and C names. This can be inconvenient if you want to wrap some external
C functions and provide the Python user with Python functions of the same
@@ -777,22 +1220,30 @@
the facilities described in the section on <a href="sharing.html">sharing
declarations between Pyrex modules</a>. </p>
+
<p>The other way is to use a <b>c name specification</b> to give different
Pyrex and C names to the C function. Suppose, for example, that you want
to wrap an external function called <tt>eject_tomato</tt>. If you declare
it as </p>
+
-<blockquote> <pre>cdef extern void c_eject_tomato "eject_tomato" (float speed)</pre>
+<blockquote>
+ <pre>cdef extern void c_eject_tomato "eject_tomato" (float speed)</pre>
+
</blockquote>
+
then its name inside the Pyrex module will be <tt>c_eject_tomato</tt>,
whereas its name in C will be <tt>eject_tomato</tt>. You can then wrap it
with
-<blockquote> <pre>def eject_tomato(speed):<br> c_eject_tomato(speed)</pre>
+<blockquote>
+ <pre>def eject_tomato(speed):<br> c_eject_tomato(speed)</pre>
+
</blockquote>
+
so that users of your module can refer to it as <tt>eject_tomato</tt>.
<p>Another use for this feature is referring to external names that happen
@@ -800,26 +1251,38 @@
called <tt>print</tt>, you can rename it to something else in your Pyrex
module. </p>
+
<p>As well as functions, C names can be specified for variables, structs,
unions, enums, struct and union members, and enum values. For example, </p>
+
-<blockquote> <pre>cdef extern int one "ein", two "zwei"<br>cdef extern float three "drei"<br><br>cdef struct spam "SPAM":<br> int i "eye"</pre>
+<blockquote>
+ <pre>cdef extern int one "ein", two "zwei"<br>cdef extern float three "drei"<br><br>cdef struct spam "SPAM":<br> int i "eye"</pre>
+
<tt>cdef enum surprise "inquisition":</tt> <br>
+
<tt> first "alpha"</tt> <br>
+
<tt> second "beta" = 3</tt></blockquote>
+
<hr width="100%">
<h3> <a name="PublicDecls"></a>Public Declarations</h3>
+
You can make C variables and functions defined in a Pyrex module accessible
to external C code (or another Pyrex module) using the <b><tt>public</tt></b> keyword, as follows:
-<blockquote><tt>cdef public int spam # public variable declaration</tt> <p><tt>cdef public void grail(int num_nuns): # public function declaration</tt> <br>
+<blockquote><tt>cdef public int spam # public variable declaration</tt>
+ <p><tt>cdef public void grail(int num_nuns): # public function declaration</tt> <br>
+
<tt> ...</tt></p>
+
</blockquote>
+
If there are any <tt>public</tt> declarations in a Pyrex module, a <b>.h</b> file is generated containing equivalent C declarations for inclusion in other
C code.
<p>Pyrex also generates a <b>.pxi</b> file containing Pyrex versions of the
@@ -829,122 +1292,191 @@
the dynamic linker at run time. I haven't tested this, so I can't say how
well it will work on the various platforms. </p>
+
<blockquote>NOTE: If all you want to export is an extension type, there is
now a better way -- see <a href="sharing.html">Sharing Declarations Between
Pyrex Modules</a>.</blockquote>
+
-<h2> <hr width="100%">Extension Types
+<h2>
+<hr width="100%">Extension Types
<hr width="100%"></h2>
+
One of the most powerful features of Pyrex is the ability to easily create
new built-in Python types, called <b>extension types</b>. This is a major
topic in itself, so there is a <a href="extension_types.html">separate
page</a> devoted to it.
-<h2> <hr width="100%">Sharing Declarations Between Pyrex Modules
+<h2>
+<hr width="100%">Sharing Declarations Between Pyrex Modules
<hr width="100%"></h2>
+
Pyrex 0.8 introduces a substantial new set of facilities allowing a Pyrex
module to easily import and use C declarations and extension types from another
Pyrex module. You can now create a set of co-operating Pyrex modules just
as easily as you can create a set of co-operating Python modules. There is
a <a href="sharing.html">separate page</a> devoted to this topic.
-<h2> <hr width="100%"><a name="Limitations"></a>Limitations
+<h2>
+<hr width="100%"><a name="Limitations"></a>Limitations
<hr width="100%"></h2>
+
<h3> <a name="Unsupported"></a>Unsupported Python features</h3>
+
Pyrex is not quite a full superset of Python. The following restrictions
apply:
<blockquote> <li> Function definitions (whether using <b>def</b> or <b>cdef</b>)
cannot be nested within other function definitions.<br>
+
</li>
+
<li> Class definitions can only appear at the top level of a module,
not inside a function.<br>
+
</li>
+
<li> The<tt> import *</tt> form of import is not allowed anywhere
(other forms of the import statement are fine, though).<br>
+
</li>
+
<li> Generators cannot be defined in Pyrex.<br>
+
<br>
+
</li>
+
<li> The <tt>globals()</tt> and <tt>locals()</tt> functions cannot be
used.</li>
+
</blockquote>
+
The above restrictions will most likely remain, since removing them would
be difficult and they're not really needed for Pyrex's intended applications.
<p>There are also some temporary limitations, which may eventually be lifted, including:
</p>
+
<blockquote> <li> Class and function definitions cannot be placed inside
control structures.<br>
+
</li>
+
<li> In-place arithmetic operators (+=, etc) are not yet supported.<br>
+
</li>
+
<li> List comprehensions are not yet supported.<br>
+
</li>
+
<li> There is no support for Unicode.<br>
+
</li>
+
<li> Special methods of extension types cannot have functioning
docstrings.<br>
+
<br>
+
</li>
+
<li> The use of string literals as comments is not recommended at present,
because Pyrex doesn't optimize them away, and won't even accept them in
places where executable statements are not allowed.</li>
+
</blockquote>
+
<h3> <a name="SemanticDifferences"></a>Semantic differences between Python
and Pyrex</h3>
+
<h4> Behaviour of class scopes</h4>
+
In Python, referring to a method of a class inside the class definition,
i.e. while the class is being defined, yields a plain function object, but
in Pyrex it yields an unbound method<sup><font size="-2"><a href="#Footnote2">2</a></font></sup>. A consequence of this is that the
usual idiom for using the classmethod and staticmethod functions, e.g.
-<blockquote> <pre>class Spam:</pre>
- <pre> def method(cls):<br> ...</pre>
- <pre> method = classmethod(method)</pre>
+<blockquote>
+ <pre>class Spam:</pre>
+
+
+ <pre> def method(cls):<br> ...</pre>
+
+
+ <pre> method = classmethod(method)</pre>
+
</blockquote>
+
will not work in Pyrex. This can be worked around by defining the function
<i>outside</i> the class, and then assigning the result of classmethod or
staticmethod inside the class, i.e.
-<blockquote> <pre>def Spam_method(cls):<br> ...</pre>
- <pre>class Spam:</pre>
- <pre> method = classmethod(Spam_method)</pre>
+<blockquote>
+ <pre>def Spam_method(cls):<br> ...</pre>
+
+
+ <pre>class Spam:</pre>
+
+
+ <pre> method = classmethod(Spam_method)</pre>
+
</blockquote>
+
-<h1> <hr width="100%"><font size="+0">Footnotes</font> <hr width="100%"></h1>
+<h1>
+<hr width="100%"><font size="+0">Footnotes</font>
+<hr width="100%"></h1>
+
<a name="Footnote1"></a>1. A problem with const could arise if you have
something like
-<blockquote> <pre>cdef extern from "grail.h":<br> char *nun</pre>
+<blockquote>
+ <pre>cdef extern from "grail.h":<br> char *nun</pre>
+
</blockquote>
+
where grail.h actually contains
-<blockquote> <pre>extern const char *nun;</pre>
+<blockquote>
+ <pre>extern const char *nun;</pre>
+
</blockquote>
+
and you do
-<blockquote> <pre>cdef void languissement(char *s):<br> #something that doesn't change s</pre>
- <pre>...</pre>
- <pre>languissement(nun)</pre>
+<blockquote>
+ <pre>cdef void languissement(char *s):<br> #something that doesn't change s</pre>
+
+
+ <pre>...</pre>
+
+
+ <pre>languissement(nun)</pre>
+
</blockquote>
+
which will cause the C compiler to complain. You can work around it by
casting away the constness:
-<blockquote> <pre>languissement(<char *>nun)</pre>
+<blockquote>
+ <pre>languissement(<char *>nun)</pre>
+
</blockquote>
+
<hr width="100%"><a name="Footnote2"></a>2. The reason for the different behaviour
of class scopes is that Pyrex-defined Python functions are PyCFunction objects,
@@ -953,8 +1485,12 @@
around this, Pyrex wraps each method in an unbound method object itself before
storing it in the class's dictionary. <br>
+
<br>
+
<br>
- </body></html>
\ No newline at end of file
+
+
+</body></html>
\ No newline at end of file
Modified: lxml/pyrex/Doc/sharing.html
==============================================================================
--- lxml/pyrex/Doc/sharing.html (original)
+++ lxml/pyrex/Doc/sharing.html Wed Feb 14 15:20:32 2007
@@ -1,106 +1,192 @@
<!DOCTYPE doctype PUBLIC "-//w3c//dtd html 4.0 transitional//en">
-<html><head>
- <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
- <meta name="GENERATOR" content="Mozilla/4.61 (Macintosh; I; PPC) [Netscape]"><title>Sharing Declarations Between Pyrex Modules</title></head>
+<html>
+<head>
+
+
+ <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
+
+
+ <meta name="GENERATOR" content="Mozilla/4.61 (Macintosh; I; PPC) [Netscape]">
+ <title>Sharing Declarations Between Pyrex Modules</title>
+</head>
<body>
- <h1> <hr width="100%">Sharing Declarations Between Pyrex Modules
+
+
+<h1>
+<hr width="100%">Sharing Declarations Between Pyrex Modules
<hr width="100%"></h1>
+
This section describes a new set of facilities introduced in Pyrex 0.8
for making C declarations and extension types in one Pyrex module available
for use in another Pyrex module. These facilities are closely modelled on
the Python import mechanism, and can be thought of as a compile-time version
of it.
<h2> Contents</h2>
- <ul>
+
+
+<ul>
+
<li> <a href="#DefAndImpFiles">Definition and Implementation files</a></li>
- <ul>
+
+
+ <ul>
+
<li> <a href="#WhatDefFileContains">What a Definition File contains</a></li>
+
<li> <a href="#WhatImpFileContains">What an Implementation File contains</a></li>
- </ul>
+
+
+ </ul>
+
<li> <a href="#CImportStatement">The <tt>cimport</tt> statement</a></li>
- <ul>
+
+
+ <ul>
+
<li> <a href="#SearchPaths">Search paths for definition files</a></li>
+
<li> <a href="#ResolvingNamingConflicts">Using <tt>cimport</tt> to resolve
naming conflicts</a></li>
- </ul>
- <li> <a href="#SharingExtensionTypes">Sharing extension types</a></li>
+
+
</ul>
- <h2> <a name="DefAndImpFiles"></a>Definition and Implementation files</h2>
+
+ <li> <a href="#SharingExtensionTypes">Sharing extension types</a></li>
+
+
+</ul>
+
+
+<h2> <a name="DefAndImpFiles"></a>Definition and Implementation files</h2>
+
A Pyrex module can be split into two parts: a <i>definition file</i> with
a <tt>.pxd</tt> suffix, containing C declarations that are to be available
to other Pyrex modules, and an <i>implementation file</i> with a <tt>.pyx</tt>
suffix, containing everything else. When a module wants to use something
declared in another module's definition file, it imports it using the <a href="#CImportStatement"><b>cimport</b> statement</a>.
<h3> <a name="WhatDefFileContains"></a>What a Definition File contains</h3>
+
A definition file can contain:
<ul>
+
<li> Any kind of C type declaration.</li>
+
<li> <b>extern</b> C function or variable declarations.</li>
+
<li> The definition part of an extension type (<a href="#SharingExtensionTypes">see below</a>).</li>
- </ul>
+
+
+</ul>
+
It cannot currently contain any non-extern C function or variable declarations
(although this may be possible in a future version).
<p>It cannot contain the implementations of any C or Python functions, or
any Python class definitions, or any executable statements. </p>
- <blockquote>NOTE: You don't need to (and shouldn't) declare anything in a
+
+
+<blockquote>NOTE: You don't need to (and shouldn't) declare anything in a
declaration file <b>public</b> in order to make it available to other Pyrex
modules; its mere presence in a definition file does that. You only need a
public declaration if you want to make something available to external C code.</blockquote>
- <h3> <a name="WhatImpFileContains"></a>What an Implementation File contains</h3>
+
+
+<h3> <a name="WhatImpFileContains"></a>What an Implementation File contains</h3>
+
An implementation file can contain any kind of Pyrex statement, although
there are some restrictions on the implementation part of an extension type
if the corresponding definition file also defines that type (see below).
<h2> <a name="CImportStatement"></a>The <tt>cimport</tt> statement</h2>
+
The <b>cimport</b> statement is used in a definition or implementation
file to gain access to names declared in another definition file. Its syntax
exactly parallels that of the normal Python import statement:
<blockquote><tt>cimport </tt><i>module</i><tt> [, </tt><i>module</i><tt>...]</tt></blockquote>
- <blockquote><tt>from </tt><i>module</i><tt> cimport </tt><i>name</i><tt>
+
+
+<blockquote><tt>from </tt><i>module</i><tt> cimport </tt><i>name</i><tt>
[as </tt><i>name</i><tt>] [, </tt><i>name</i><tt> [as </tt><i>name</i><tt>]
...]</tt></blockquote>
+
Here is an example. The file on the left is a definition file which exports
a C data type. The file on the right is an implementation file which imports
and uses it. <br>
- <table cellpadding="5" cols="2" width="100%">
+
+
+<table cellpadding="5" cols="2" width="100%">
+
<tbody>
+
<tr>
+
<td bgcolor="#ffcc00" width="40%"><b><tt>dishes.pxd</tt></b></td>
+
<td bgcolor="#5dbaca"><b><tt>restaurant.pyx</tt></b></td>
+
</tr>
+
<tr align="left" valign="top">
+
<td bgcolor="#ffcc18" width="40%"><tt>cdef enum otherstuff:</tt> <br>
- <tt> sausage, eggs, lettuce</tt> <p><tt>cdef struct spamdish:</tt> <br>
+
+ <tt> sausage, eggs, lettuce</tt>
+ <p><tt>cdef struct spamdish:</tt> <br>
+
<tt> int oz_of_spam</tt> <br>
+
<tt> otherstuff filler</tt></p>
+
</td>
+
<td bgcolor="#5dbaca"><tt>cimport dishes</tt> <br>
- <tt>from dishes cimport spamdish</tt> <p><tt>cdef void prepare(spamdish *d):</tt> <br>
+
+ <tt>from dishes cimport spamdish</tt>
+ <p><tt>cdef void prepare(spamdish *d):</tt> <br>
+
<tt> d.oz_of_spam = 42</tt> <br>
+
<tt> d.filler = dishes.sausage</tt> </p>
- <p><tt>def serve():</tt> <br>
+
+
+ <p><tt>def serve():</tt> <br>
+
<tt> spamdish d</tt> <br>
+
<tt> prepare(&d)</tt> <br>
+
<tt> print "%d oz spam, filler no. %d" % \</tt>
<br>
+
<tt> (d->oz_of_spam,
d->otherstuff)</tt></p>
+
</td>
+
</tr>
- </tbody> </table>
- <p>It is important to understand that the <b>cimport</b> statement can <i>only</i>
+
+
+ </tbody>
+</table>
+
+
+<p>It is important to understand that the <b>cimport</b> statement can <i>only</i>
be used to import C data types, external C functions and variables, and extension
types. It cannot be used to import any Python objects, and (with one exception)
it doesn't imply any Python import at run time. If you want to refer to any
Python names from a module that you have cimported, you will have to include
a regular <b>import</b> statement for it as well. </p>
- <p>The exception is that when you use <b>cimport</b> to import an extension
+
+
+<p>The exception is that when you use <b>cimport</b> to import an extension
type, its type object is imported at run time and made available by the
name under which you imported it. Using <b>cimport</b> to import extension
types is covered in more detail <a href="#SharingExtensionTypes">below</a>.
</p>
- <h3> <a name="SearchPaths"></a>Search paths for definition files</h3>
+
+
+<h3> <a name="SearchPaths"></a>Search paths for definition files</h3>
+
When you <b>cimport</b> a module called <tt>modulename</tt>, the Pyrex
compiler searches for a file called <tt>modulename.pxd</tt> along the search
path for include files, as specified by <b>-I</b> command line options.
@@ -108,94 +194,172 @@
definition file <tt>modulename.pxd</tt> is first searched for along the
same path, and if found, it is processed before processing the <tt>.pyx</tt>
file. </p>
- <h3> <a name="ResolvingNamingConflicts"></a>Using cimport to resolve naming
+
+
+<h3> <a name="ResolvingNamingConflicts"></a>Using cimport to resolve naming
conflicts</h3>
+
The cimport mechanism provides a clean and simple way to solve the problem
of wrapping external C functions with Python functions of the same name.
All you need to do is put the extern C declarations into a .pxd file for
an imaginary module, and cimport that module. You can then refer to the C
functions by qualifying them with the name of the module. Here's an example:
<br>
- <table cellpadding="5" cols="2" width="100%">
+
+
+<table cellpadding="5" cols="2" width="100%">
+
<tbody>
+
<tr>
+
<td bgcolor="#ffcc00" width="50%"><b><tt>c_lunch.pxd</tt></b></td>
+
<td bgcolor="#5dbaca"><b><tt>lunch.pyx</tt></b></td>
+
</tr>
+
<tr align="left" valign="top">
+
<td bgcolor="#ffcc18" width="50%"><tt>cdef extern from "lunch.h":</tt>
<br>
+
<tt> void eject_tomato(float)</tt></td>
- <td bgcolor="#5dbaca"><tt>cimport c_lunch</tt> <p><tt>def eject_tomato(float speed):</tt> <br>
+
+ <td bgcolor="#5dbaca"><tt>cimport c_lunch</tt>
+ <p><tt>def eject_tomato(float speed):</tt> <br>
+
<tt> c_lunch.eject_tomato(speed)</tt></p>
+
</td>
+
</tr>
- </tbody> </table>
- <p>You don't need any <tt>c_lunch.pyx</tt> file, because the only things
+
+
+ </tbody>
+</table>
+
+
+<p>You don't need any <tt>c_lunch.pyx</tt> file, because the only things
defined in <tt>c_lunch.pxd</tt> are extern C entities. There won't be any
actual <tt>c_lunch</tt> module at run time, but that doesn't matter -- <tt>c_lunch</tt>
has done its job of providing an additional namespace at compile time. </p>
- <h2> <a name="SharingExtensionTypes"></a>Sharing Extension Types</h2>
+
+
+<h2> <a name="SharingExtensionTypes"></a>Sharing Extension Types</h2>
+
An extension type declaration can also be split into two parts, one in
a definition file and the other in the corresponding implementation file.
<br>
+
<br>
+
The definition part of the extension type can only declare C attributes
and C methods, not Python methods, and it must declare <i>all</i> of that
type's C attributes and C methods.<br>
+
<br>
+
The implementation part must implement all of the C methods declared in
the definition part, and may not add any further C attributes. It may also
define Python methods.
<p>Here is an example of a module which defines and exports an extension
type, and another module which uses it. <br>
- <table cellpadding="5" cols="2" width="100%">
+
+
+<table cellpadding="5" cols="2" width="100%">
+
<tbody>
+
<tr>
+
<td bgcolor="#ffcc18" width="30%"><b><tt>Shrubbing.pxd</tt></b></td>
+
<td bgcolor="#5dbaca" width="50%"><b><tt>Shrubbing.pyx</tt></b></td>
+
</tr>
+
<tr align="left" valign="top">
+
<td bgcolor="#ffcc18" width="30%"><tt>cdef class Shrubbery:</tt> <br>
+
<tt> cdef int width</tt> <br>
+
<tt> cdef int length</tt></td>
+
<td bgcolor="#5dbaca" width="50%"><tt>cdef class Shrubbery:</tt> <br>
+
<tt> def __new__(self, int w, int l):</tt> <br>
+
<tt> self.width = w</tt>
<br>
+
<tt> self.length = l</tt>
- <p><tt>def standard_shrubbery():</tt> <br>
+
+ <p><tt>def standard_shrubbery():</tt> <br>
+
<tt> return Shrubbery(3, 7)</tt></p>
+
</td>
+
</tr>
+
<tr>
+
<td colspan="2" bgcolor="#8cbc1c" width="30%"><b><tt>Landscaping.pyx</tt></b></td>
+
</tr>
+
<tr>
+
<td colspan="2" bgcolor="#99cc00" width="30%"><tt>cimport Shrubbing</tt>
<br>
- <tt>import Shrubbing</tt> <p><tt>cdef Shrubbing.Shrubbery sh</tt> <br>
+
+ <tt>import Shrubbing</tt>
+ <p><tt>cdef Shrubbing.Shrubbery sh</tt> <br>
+
<tt>sh = Shrubbing.standard_shrubbery()</tt> <br>
+
<tt>print "Shrubbery size is %d x %d" % (sh.