From stefan_ml at behnel.de Fri Feb 1 10:12:15 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 01 Feb 2008 10:12:15 +0100 Subject: [Cython] Some more optimisations In-Reply-To: References: <4797C725.4010605@behnel.de> <479834DE.6000806@behnel.de> <47986E40.5050306@behnel.de> Message-ID: <47A2E26F.3030300@behnel.de> Hi Robert, Robert Bradshaw wrote: > On Jan 24, 2008, at 2:53 AM, Stefan Behnel wrote: >> What I could imagine, on the other hand, is exploiting the type hint >> given by >> *args and **kwargs and propagate that (at least up to the next >> assignment), so >> that access to the variables can use straight PyAPI calls. But as Kay >> noted, >> we don't currently have a framework for questions like: where is "the >> next assignment?". > > What we need is special list/dict/tuple types. I know Greg (eventually) > plans to do something like this too. They would be like extension types, > though should fail for subtypes (e.g. if one subclasses a list, then one > can't assign it to a cdef list variable, or it might invalidate using > the faster api's). If we require exact types (no subtypes), we would be inconsistent with how things currently work for extension types (and Python types). The only exception are plain C types. So here, list/dict/tuple would basically behave like a C type (but I guess it would be enough to document that...) > As for the type assignment propagation, I think we need to introduce a > two-pass analyses_types. There would be a special undeclared type, and > by tracking assignments one could create a dependancy tree of types. One > would then use this to resolve all variable types and run analyses_types > again. > > The reason one needs to run analyses_types twice is because there is > often branching on type.is_pyobject, as well as determining coercion > needs, etc. I'm sure it could be done with a single pass and solving > some giant type-constraint problem, but that would take a major rewrite. I had imagined building a per-name list of previously assigned types in analyse_types() that could be traversed to check what type we currently expect. Would work as follows: - assignments replace the list content by their result of the analyse_types() for the right-hand side. This possibly requires taking into account the current list of types, and it might mean we just append an additional type possibility. - branches (if/loops) collect the results of each branch and sort the types in the expected order of probability - which might be arbitrary, but could be based on the number of occurrences over the branches. The later "is_pyobject" tests would then be replaced by a more accurate test for relevant types, but from a point on, we would always have a list of possible types for each code point that we would base our decision on. BTW, "is_pyobject" is not wrong, it's just not accurate enough for everything. One thing I'm not sure about is how to propagate undeclared function result types. If we can figure out what type a function has, we can use that type. But the function might be declared later in the code, so we wouldn't have that information early enough. I think that's where two passed are required. Stefan From martin at martincmartin.com Fri Feb 1 11:32:40 2008 From: martin at martincmartin.com (Martin C. Martin) Date: Fri, 01 Feb 2008 05:32:40 -0500 Subject: [Cython] Some more optimisations In-Reply-To: <47A2E26F.3030300@behnel.de> References: <4797C725.4010605@behnel.de> <479834DE.6000806@behnel.de> <47986E40.5050306@behnel.de> <47A2E26F.3030300@behnel.de> Message-ID: <47A2F548.3090406@martincmartin.com> Stefan Behnel wrote: > One thing I'm not sure about is how to propagate undeclared function result > types. If we can figure out what type a function has, we can use that type. > But the function might be declared later in the code, so we wouldn't have that > information early enough. I think that's where two passed are required. I think you're absolutely right, and that's the standard answer to this. Is making a separate pass difficult? Best, Martin From stefan_ml at behnel.de Fri Feb 1 11:59:20 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 01 Feb 2008 11:59:20 +0100 Subject: [Cython] Some more optimisations In-Reply-To: <47A2F548.3090406@martincmartin.com> References: <4797C725.4010605@behnel.de> <479834DE.6000806@behnel.de> <47986E40.5050306@behnel.de> <47A2E26F.3030300@behnel.de> <47A2F548.3090406@martincmartin.com> Message-ID: <47A2FB88.8000803@behnel.de> Hi, Martin C. Martin wrote: > Stefan Behnel wrote: > >> One thing I'm not sure about is how to propagate undeclared function >> result >> types. If we can figure out what type a function has, we can use that >> type. >> But the function might be declared later in the code, so we wouldn't >> have that >> information early enough. I think that's where two passed are required. > > I think you're absolutely right, and that's the standard answer to this. Now that I give it another thought, there's one very ugly thing to deal with. Imagine this: cdef f1(a): d = f2(a) d[1] = 1 return d cdef f2(a): d = f3(a) d[2] = 2 return d cdef f3(a): d = f4(a) d[3] = 3 return d cdef f4(a): return {4:a} Here, we will traverse all functions, and at the last one, we can decide that f4 will always return a dict. Then, we traverse again, and we can decide that f3 will always return a dict, and so on. That would let us run through as many passes as we have functions. We could use something like 'dependent' types, i.e. types that depend on the result of something that we do not know yet. But that would still require us to do at least two passes, one to find out what's going on outside and one that tells what's happening inside. And if the thing that happens inside changes the way things look to the outside world - bad luck... Couldn't someone just write code for this? ;) Stefan From stefan_ml at behnel.de Fri Feb 1 12:06:36 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 01 Feb 2008 12:06:36 +0100 Subject: [Cython] Cython code fails to compile with gcc 2.95 Message-ID: <47A2FD3C.10501@behnel.de> Hi, I know, that's a pretty old compiler, but I know someone with a Solaris production setup that still uses it. Here is the compile error he gets: -------------------------------- src/lxml/lxml.etree.c: In function `__pyx_PyInt_AsLongLong': src/lxml/lxml.etree.c:110165: parse error before `long' src/lxml/lxml.etree.c:110167: `val' undeclared (first use in this function) src/lxml/lxml.etree.c:110167: (Each undeclared identifier is reported only once src/lxml/lxml.etree.c:110167: for each function it appears in.) src/lxml/lxml.etree.c: In function `__pyx_PyInt_AsUnsignedLongLong': src/lxml/lxml.etree.c:110185: parse error before `long' src/lxml/lxml.etree.c:110187: `val' undeclared (first use in this function) error: command 'gcc' failed with exit status 1 -------------------------------- The problem lies in the new type conversion functions: -------------------------------- 110156 static INLINE PY_LONG_LONG __pyx_PyInt_AsLongLong(PyObject* x) { 110157 if (PyInt_CheckExact(x)) { 110158 return PyInt_AS_LONG(x); 110159 } 110160 else if (PyLong_CheckExact(x)) { 110161 return PyLong_AsLongLong(x); 110162 } 110163 else { 110164 PyObject* tmp = PyNumber_Int(x); if (!tmp) return (PY_LONG_LONG)-1; 110165 PY_LONG_LONG val = __pyx_PyInt_AsLongLong(tmp); 110166 Py_DECREF(tmp); 110167 return val; 110168 } 110169 } 110170 110171 static INLINE unsigned PY_LONG_LONG __pyx_PyInt_AsUnsignedLongLong(PyObject* x) { 110172 if (PyInt_CheckExact(x)) { 110173 long val = PyInt_AS_LONG(x); 110174 if (unlikely(val < 0)) { 110175 PyErr_SetString(PyExc_TypeError, "Negative assignment to unsigned type."); 110176 return (unsigned PY_LONG_LONG)-1; 110177 } 110178 return val; 110179 } 110180 else if (PyLong_CheckExact(x)) { 110181 return PyLong_AsUnsignedLongLong(x); 110182 } 110183 else { 110184 PyObject* tmp = PyNumber_Int(x); if (!tmp) return (PY_LONG_LONG)-1; 110185 PY_LONG_LONG val = __pyx_PyInt_AsUnsignedLongLong(tmp); 110186 Py_DECREF(tmp); 110187 return val; 110188 } 110189 } -------------------------------- So it seems like this line doesn't compile: -------------------------------- PY_LONG_LONG val = __pyx_PyInt_AsLongLong(tmp); -------------------------------- Given the error, I would say that "PY_LONG_LONG" gets expanded to the usual "long long". Any idea what could go wrong here and what to do about it? Stefan From kirr at landau.phys.spbu.ru Fri Feb 1 12:14:25 2008 From: kirr at landau.phys.spbu.ru (Kirill Smelkov) Date: Fri, 01 Feb 2008 14:14:25 +0300 Subject: [Cython] [PATCH] Demos: pyrexc -> cython Message-ID: # HG changeset patch # User Kirill Smelkov # Date 1201862775 -10800 # Node ID bcc813057f913c757d2629649b35ae999629e5d5 # Parent be19ff60769b44621e4857435897a635026636fb Demos: pyrexc -> cython diff --git a/Demos/Makefile.nodistutils b/Demos/Makefile.nodistutils --- a/Demos/Makefile.nodistutils +++ b/Demos/Makefile.nodistutils @@ -4,7 +4,7 @@ PYINCLUDE = \ -I$(PYHOME)/$(ARCH)/include/python2.2 %.c: %.pyx - ../bin/pyrexc $< + ../bin/cython $< %.o: %.c gcc -c -fPIC $(PYINCLUDE) $< diff --git a/Demos/embed/Makefile b/Demos/embed/Makefile --- a/Demos/embed/Makefile +++ b/Demos/embed/Makefile @@ -9,7 +9,7 @@ PYLIB = -L$(PYARCH)/lib/python$(PYVERSIO -ldl -lpthread -lutil -lm %.c: %.pyx - ../../bin/pyrexc $< + ../../bin/cython $< %.o: %.c gcc -c -fPIC $(PYINCLUDE) $< diff --git a/Demos/embed/Makefile.msc b/Demos/embed/Makefile.msc --- a/Demos/embed/Makefile.msc +++ b/Demos/embed/Makefile.msc @@ -8,7 +8,7 @@ CFLAGS = $(PYINCLUDE) /Ox /W3 /GX -nolog .SUFFIXES: .exe .dll .obj .c .cpp .pyx .pyx.c: - $(PYHOME)\Python.exe ../../pyrexc.py $< + $(PYHOME)\Python.exe ../../cython.py $< all: main.exe diff --git a/Demos/embed/Makefile.unix b/Demos/embed/Makefile.unix --- a/Demos/embed/Makefile.unix +++ b/Demos/embed/Makefile.unix @@ -9,7 +9,7 @@ PYLIB = -L$(PYARCH)/lib/python$(PYVERSIO -ldl -lpthread -lutil -lm %.c: %.pyx - ../../bin/pyrexc $< + ../../bin/cython $< %.o: %.c gcc -c -fPIC $(PYINCLUDE) $< From kirr at landau.phys.spbu.ru Fri Feb 1 12:19:47 2008 From: kirr at landau.phys.spbu.ru (Kirill Smelkov) Date: Fri, 1 Feb 2008 14:19:47 +0300 Subject: [Cython] problem running 'make test' (cython 0.9.6.11b) Message-ID: <20080201111947.GA7032@evo> Hi, I'm a former Pyrex user and recently I've decided to play with Cython. Running tests is a problem: kirr at evo:~/src/tools/cython/Cython-0.9.6.11b$ make test rm -fr BUILD python runtests.py Traceback (most recent call last): File "runtests.py", line 6, in ? from Cython.Distutils.extension import Extension ImportError: No module named extension make: *** [test] ?????? 1 'make test' fails with both - Cython-0.9.6.11b, and - latest cython-package (be19ff60769b) + cython-devel (93e412adb69a) -- ????? ????????, ??????. http://landau.phys.spbu.ru/~kirr/aiv/ From stefan_ml at behnel.de Fri Feb 1 12:47:58 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 01 Feb 2008 12:47:58 +0100 Subject: [Cython] problem running 'make test' (cython 0.9.6.11b) In-Reply-To: <20080201111947.GA7032@evo> References: <20080201111947.GA7032@evo> Message-ID: <47A306EE.9080901@behnel.de> Hi, Kirill Smelkov wrote: > I'm a former Pyrex user and recently I've decided to play with Cython. > Running tests is a problem: > > kirr at evo:~/src/tools/cython/Cython-0.9.6.11b$ make test > rm -fr BUILD > python runtests.py > Traceback (most recent call last): > File "runtests.py", line 6, in ? > from Cython.Distutils.extension import Extension > ImportError: No module named extension > make: *** [test] ?????? 1 > > 'make test' fails with both > - Cython-0.9.6.11b, and > - latest cython-package (be19ff60769b) + cython-devel (93e412adb69a) Hmm, interesting. Cython/Distutils/extension.py is no longer under version control. It still has a data entry in Cython/.hg, though. Robert, do you have that file? I attached mine, just in case. It's only used to support additional Extension options starting with "pyrex_". Stefan -------------- next part -------------- A non-text attachment was scrubbed... Name: extension.py.gz Type: application/x-gzip Size: 823 bytes Desc: not available Url : http://codespeak.net/pipermail/cython-dev/attachments/20080201/02bbb5f6/attachment.bin From sven at pyrex.berkvens.net Fri Feb 1 14:08:50 2008 From: sven at pyrex.berkvens.net (Sven Berkvens-Matthijsse) Date: Fri, 1 Feb 2008 14:08:50 +0100 Subject: [Cython] Cython code fails to compile with gcc 2.95 In-Reply-To: <47A2FD3C.10501@behnel.de> References: <47A2FD3C.10501@behnel.de> Message-ID: <20080201130850.GA87515@berkvens.net> > Hi, > > I know, that's a pretty old compiler, but I know someone with a Solaris > production setup that still uses it. Here is the compile error he gets: > > -------------------------------- > src/lxml/lxml.etree.c: In function `__pyx_PyInt_AsLongLong': > src/lxml/lxml.etree.c:110165: parse error before `long' > src/lxml/lxml.etree.c:110167: `val' undeclared (first use in this function) > src/lxml/lxml.etree.c:110167: (Each undeclared identifier is reported only once > src/lxml/lxml.etree.c:110167: for each function it appears in.) > src/lxml/lxml.etree.c: In function `__pyx_PyInt_AsUnsignedLongLong': > src/lxml/lxml.etree.c:110185: parse error before `long' > src/lxml/lxml.etree.c:110187: `val' undeclared (first use in this function) > error: command 'gcc' failed with exit status 1 > -------------------------------- > > The problem lies in the new type conversion functions: > > -------------------------------- > 110156 static INLINE PY_LONG_LONG __pyx_PyInt_AsLongLong(PyObject* x) { > 110157 if (PyInt_CheckExact(x)) { > 110158 return PyInt_AS_LONG(x); > 110159 } > 110160 else if (PyLong_CheckExact(x)) { > 110161 return PyLong_AsLongLong(x); > 110162 } > 110163 else { > 110164 PyObject* tmp = PyNumber_Int(x); if (!tmp) return > (PY_LONG_LONG)-1; > 110165 PY_LONG_LONG val = __pyx_PyInt_AsLongLong(tmp); > 110166 Py_DECREF(tmp); > 110167 return val; > 110168 } > 110169 } > 110170 > 110171 static INLINE unsigned PY_LONG_LONG > __pyx_PyInt_AsUnsignedLongLong(PyObject* x) { > 110172 if (PyInt_CheckExact(x)) { > 110173 long val = PyInt_AS_LONG(x); > 110174 if (unlikely(val < 0)) { > 110175 PyErr_SetString(PyExc_TypeError, "Negative assignment to > unsigned type."); > 110176 return (unsigned PY_LONG_LONG)-1; > 110177 } > 110178 return val; > 110179 } > 110180 else if (PyLong_CheckExact(x)) { > 110181 return PyLong_AsUnsignedLongLong(x); > 110182 } > 110183 else { > 110184 PyObject* tmp = PyNumber_Int(x); if (!tmp) return > (PY_LONG_LONG)-1; > 110185 PY_LONG_LONG val = __pyx_PyInt_AsUnsignedLongLong(tmp); > 110186 Py_DECREF(tmp); > 110187 return val; > 110188 } > 110189 } > -------------------------------- > > So it seems like this line doesn't compile: > > -------------------------------- > PY_LONG_LONG val = __pyx_PyInt_AsLongLong(tmp); > -------------------------------- > > Given the error, I would say that "PY_LONG_LONG" gets expanded to the usual > "long long". > > Any idea what could go wrong here and what to do about it? The problem is that GCC 2.95 does not follow C99 semantics, and the C version that it adheres to does not allow variables to be declared in the middle of a block. All variables must be declared at the top of a block. In the above case: 110164 PyObject* tmp = PyNumber_Int(x); if (!tmp) return (PY_LONG_LONG)-1; 110165 PY_LONG_LONG val = __pyx_PyInt_AsLongLong(tmp); the 'if' statement on line 110164 terminates the end of the variable declarations in the block. Line 110165 tries to declare one in the middle of some statements, which was not allowed before C99. > Stefan -- Sven From stefan_ml at behnel.de Fri Feb 1 14:32:59 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 01 Feb 2008 14:32:59 +0100 Subject: [Cython] Cython code fails to compile with gcc 2.95 In-Reply-To: <20080201130850.GA87515@berkvens.net> References: <47A2FD3C.10501@behnel.de> <20080201130850.GA87515@berkvens.net> Message-ID: <47A31F8B.7050506@behnel.de> Hi, Sven Berkvens-Matthijsse wrote: > The problem is that GCC 2.95 does not follow C99 semantics, and the C > version that it adheres to does not allow variables to be declared in > the middle of a block. All variables must be declared at the top of a > block. In the above case: > > 110164 PyObject* tmp = PyNumber_Int(x); if (!tmp) return (PY_LONG_LONG)-1; > 110165 PY_LONG_LONG val = __pyx_PyInt_AsLongLong(tmp); > > the 'if' statement on line 110164 terminates the end of the variable > declarations in the block. Line 110165 tries to declare one in the > middle of some statements, which was not allowed before C99. Ah, sure, that makes sense. So here's the obvious fix. Thanks a lot, Stefan -------------- next part -------------- A non-text attachment was scrubbed... Name: upstream.bundle Type: application/octet-stream Size: 568 bytes Desc: not available Url : http://codespeak.net/pipermail/cython-dev/attachments/20080201/fc83b24c/attachment.obj From martin at martincmartin.com Fri Feb 1 16:02:13 2008 From: martin at martincmartin.com (Martin C. Martin) Date: Fri, 01 Feb 2008 10:02:13 -0500 Subject: [Cython] Some more optimisations In-Reply-To: <47A2FB88.8000803@behnel.de> References: <4797C725.4010605@behnel.de> <479834DE.6000806@behnel.de> <47986E40.5050306@behnel.de> <47A2E26F.3030300@behnel.de> <47A2F548.3090406@martincmartin.com> <47A2FB88.8000803@behnel.de> Message-ID: <47A33475.8070905@martincmartin.com> Stefan Behnel wrote: > Hi, > > Martin C. Martin wrote: >> Stefan Behnel wrote: >> >>> One thing I'm not sure about is how to propagate undeclared function >>> result >>> types. If we can figure out what type a function has, we can use that >>> type. >>> But the function might be declared later in the code, so we wouldn't >>> have that >>> information early enough. I think that's where two passed are required. >> I think you're absolutely right, and that's the standard answer to this. > > Now that I give it another thought, there's one very ugly thing to deal with. > Imagine this: > > cdef f1(a): > d = f2(a) > d[1] = 1 > return d > > cdef f2(a): > d = f3(a) > d[2] = 2 > return d > > cdef f3(a): > d = f4(a) > d[3] = 3 > return d > > cdef f4(a): > return {4:a} > > Here, we will traverse all functions, and at the last one, we can decide that > f4 will always return a dict. Then, we traverse again, and we can decide that > f3 will always return a dict, and so on. > > That would let us run through as many passes as we have functions. Actually, it could be worse than that, if the function bodies reference more than one function. It could mean a separate pass for each expression. But there's a better way. What you're describing is type inference: http://en.wikipedia.org/wiki/Type_inference Given that the only types we care about are dict, list, tuple, and "everything else," that might be overkill. On the other hand, it would be a fun project. The alternative is to only use the type when people have explicitly declared it in the function prototype. So in your example, all of the d[x] = y calls wouldn't know that d is a dict, because the user didn't declare f4's return type. But if they did, then we'd know that d was always a dict, so we could call the dict-specific set for all those d[x] = y. That only requires a single extra pass: A first pass that just collects all the function prototypes, then a second pass to generate the actual code. > Couldn't someone just write code for this? ;) Oh, look at the time! It's getting late, I'd better be going... ;) Best, Martin From stefan_ml at behnel.de Fri Feb 1 16:50:28 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 01 Feb 2008 16:50:28 +0100 Subject: [Cython] Inefficient code for "is None" in boolean expressions that mix Python and C Message-ID: <47A33FC4.7070502@behnel.de> Hi, I noticed that "is None" in a boolean expression gets unnecessarily evaluated in Python space. This code: ---------------------- if stop is None and (start is None or start == 0): ---------------------- gets compiled into this totally redundant code (plus error handling): ---------------------- __pyx_1 = (__pyx_v_stop == Py_None); __pyx_2 = __Pyx_PyBool_FromLong(__pyx_1); __pyx_1 = __Pyx_PyObject_IsTrue(__pyx_2); if (__pyx_1) { Py_DECREF(__pyx_2); __pyx_2 = 0; __pyx_1 = (__pyx_v_start == Py_None); __pyx_2 = __Pyx_PyBool_FromLong(__pyx_1); __pyx_1 = __Pyx_PyObject_IsTrue(__pyx_2); if (!__pyx_1) { Py_DECREF(__pyx_2); __pyx_2 = 0; __pyx_2 = PyObject_RichCompare(__pyx_v_start, __pyx_num_0, Py_EQ); } } __pyx_1 = __Pyx_PyObject_IsTrue(__pyx_2); Py_DECREF(__pyx_2); __pyx_2 = 0; if (__pyx_1) { ---------------------- The only thing that needs Python evaluation here is "start == 0", the rest could run entirely in C. I don't know if it's a more general problem, though, since all C tests should be evaluated in plain C code. Stefan From robertwb at math.washington.edu Fri Feb 1 21:14:31 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 1 Feb 2008 12:14:31 -0800 Subject: [Cython] Inefficient code for "is None" in boolean expressions that mix Python and C In-Reply-To: <47A33FC4.7070502@behnel.de> References: <47A33FC4.7070502@behnel.de> Message-ID: The problem is not with the if note, rather it's with the or and and. "a or b" can returns a result of type object if either side is of type object. The code boils down to res = a if not a: res = b We don't have the infrastructure to let BoolBinopNode know we want a bint (backwards type propagation) but we could be more clever about not turning the left side into a Python object until after we evaluate its truth value. Fortunately __Pyx_PyBool_FromLong and __Pyx_PyObject_IsTrue are now macros. I wonder if the compiler is smart enough to figure out the meaning of them right after each other. - Robert On Feb 1, 2008, at 7:50 AM, Stefan Behnel wrote: > Hi, > > I noticed that "is None" in a boolean expression gets unnecessarily > evaluated > in Python space. This code: > > ---------------------- > if stop is None and (start is None or start == 0): > ---------------------- > > gets compiled into this totally redundant code (plus error handling): > > ---------------------- > __pyx_1 = (__pyx_v_stop == Py_None); > __pyx_2 = __Pyx_PyBool_FromLong(__pyx_1); > __pyx_1 = __Pyx_PyObject_IsTrue(__pyx_2); > if (__pyx_1) { > Py_DECREF(__pyx_2); __pyx_2 = 0; > __pyx_1 = (__pyx_v_start == Py_None); > __pyx_2 = __Pyx_PyBool_FromLong(__pyx_1); > __pyx_1 = __Pyx_PyObject_IsTrue(__pyx_2); > if (!__pyx_1) { > Py_DECREF(__pyx_2); __pyx_2 = 0; > __pyx_2 = PyObject_RichCompare(__pyx_v_start, __pyx_num_0, Py_EQ); > } > } > __pyx_1 = __Pyx_PyObject_IsTrue(__pyx_2); > Py_DECREF(__pyx_2); __pyx_2 = 0; > if (__pyx_1) { > ---------------------- > > The only thing that needs Python evaluation here is "start == 0", > the rest > could run entirely in C. I don't know if it's a more general > problem, though, > since all C tests should be evaluated in plain C code. > > Stefan > > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev From robertwb at math.washington.edu Fri Feb 1 21:30:32 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 1 Feb 2008 12:30:32 -0800 Subject: [Cython] Some more optimisations In-Reply-To: <47A2E26F.3030300@behnel.de> References: <4797C725.4010605@behnel.de> <479834DE.6000806@behnel.de> <47986E40.5050306@behnel.de> <47A2E26F.3030300@behnel.de> Message-ID: On Feb 1, 2008, at 1:12 AM, Stefan Behnel wrote: > Hi Robert, > > Robert Bradshaw wrote: >> On Jan 24, 2008, at 2:53 AM, Stefan Behnel wrote: >>> What I could imagine, on the other hand, is exploiting the type hint >>> given by >>> *args and **kwargs and propagate that (at least up to the next >>> assignment), so >>> that access to the variables can use straight PyAPI calls. But as >>> Kay >>> noted, >>> we don't currently have a framework for questions like: where is >>> "the >>> next assignment?". >> >> What we need is special list/dict/tuple types. I know Greg >> (eventually) >> plans to do something like this too. They would be like extension >> types, >> though should fail for subtypes (e.g. if one subclasses a list, >> then one >> can't assign it to a cdef list variable, or it might invalidate using >> the faster api's). > > If we require exact types (no subtypes), we would be inconsistent > with how > things currently work for extension types (and Python types). The only > exception are plain C types. So here, list/dict/tuple would > basically behave > like a C type (but I guess it would be enough to document that...) Yes, they would behave more like C types. I think this is a necessary and not overburdonsome restriction--it is much more common to have exactly a list than to have a subclass of a list. The real motivation for doing this is that unless we have exactly these types, we can't use the fast Python C/API calls anyways, so what would be the gain? >> As for the type assignment propagation, I think we need to >> introduce a >> two-pass analyses_types. There would be a special undeclared type, >> and >> by tracking assignments one could create a dependancy tree of >> types. One >> would then use this to resolve all variable types and run >> analyses_types >> again. >> >> The reason one needs to run analyses_types twice is because there is >> often branching on type.is_pyobject, as well as determining coercion >> needs, etc. I'm sure it could be done with a single pass and solving >> some giant type-constraint problem, but that would take a major >> rewrite. > > I had imagined building a per-name list of previously assigned > types in > analyse_types() that could be traversed to check what type we > currently > expect. Would work as follows: > > - assignments replace the list content by their result of the > analyse_types() > for the right-hand side. This possibly requires taking into account > the > current list of types, and it might mean we just append an > additional type > possibility. analyse_types() won't be able to do much until we actually know what the types in the expression are. I think the best we can do is build up a table of "this type depends on these types in this way." > - branches (if/loops) collect the results of each branch and sort > the types in > the expected order of probability - which might be arbitrary, but > could be > based on the number of occurrences over the branches. I'm not quite following here--how are branches/loops different than two adjacent statements? > The later "is_pyobject" tests would then be replaced by a more > accurate test > for relevant types, but from a point on, we would always have a > list of > possible types for each code point that we would base our decision > on. BTW, > "is_pyobject" is not wrong, it's just not accurate enough for > everything. This is completely tangential to type inference, isn't it? > One thing I'm not sure about is how to propagate undeclared > function result > types. If we can figure out what type a function has, we can use > that type. > But the function might be declared later in the code, so we > wouldn't have that > information early enough. I think that's where two passed are > required. I think for the first step, we should just leave functions (and class members) as they are now--we can come back to them later but this will make things much messier and less local (not to mention the issues of cimporting functions and types from other modules). - Robert From stefan_ml at behnel.de Fri Feb 1 22:13:48 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 01 Feb 2008 22:13:48 +0100 Subject: [Cython] Some more optimisations In-Reply-To: References: <4797C725.4010605@behnel.de> <479834DE.6000806@behnel.de> <47986E40.5050306@behnel.de> <47A2E26F.3030300@behnel.de> Message-ID: <47A38B8C.7040700@behnel.de> Hi, Robert Bradshaw wrote: > On Feb 1, 2008, at 1:12 AM, Stefan Behnel wrote: >> - assignments replace the list content by their result of the >> analyse_types() >> for the right-hand side. This possibly requires taking into account the >> current list of types, and it might mean we just append an additional >> type >> possibility. > > analyse_types() won't be able to do much until we actually know what the > types in the expression are. I think the best we can do is build up a > table of "this type depends on these types in this way." So we'd build such a table for each statement? >> - branches (if/loops) collect the results of each branch and sort the >> types in >> the expected order of probability - which might be arbitrary, but >> could be >> based on the number of occurrences over the branches. > > I'm not quite following here--how are branches/loops different than two > adjacent statements? I meant /after/ finishing the loop/if, when coming back to the other block. if cond: l = [] else: l = function() return l[0] What is l here? Depends, but it's likely a list. > I think for the first step, we should just leave functions (and class > members) as they are now--we can come back to them later but this will > make things much messier and less local (not to mention the issues of > cimporting functions and types from other modules). I agree. Getting the infrastructure in place and handling the easy cases would be enough work for starters. Stefan From robertwb at math.washington.edu Fri Feb 1 22:19:46 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 1 Feb 2008 13:19:46 -0800 Subject: [Cython] Some more optimisations In-Reply-To: <47A38B8C.7040700@behnel.de> References: <4797C725.4010605@behnel.de> <479834DE.6000806@behnel.de> <47986E40.5050306@behnel.de> <47A2E26F.3030300@behnel.de> <47A38B8C.7040700@behnel.de> Message-ID: On Feb 1, 2008, at 1:13 PM, Stefan Behnel wrote: > Hi, > > Robert Bradshaw wrote: >> On Feb 1, 2008, at 1:12 AM, Stefan Behnel wrote: >>> - assignments replace the list content by their result of the >>> analyse_types() >>> for the right-hand side. This possibly requires taking into >>> account the >>> current list of types, and it might mean we just append an >>> additional >>> type >>> possibility. >> >> analyse_types() won't be able to do much until we actually know >> what the >> types in the expression are. I think the best we can do is build up a >> table of "this type depends on these types in this way." > > So we'd build such a table for each statement? More a list of assignments for every every variable. >>> - branches (if/loops) collect the results of each branch and sort >>> the >>> types in >>> the expected order of probability - which might be arbitrary, but >>> could be >>> based on the number of occurrences over the branches. >> >> I'm not quite following here--how are branches/loops different >> than two >> adjacent statements? > > I meant /after/ finishing the loop/if, when coming back to the > other block. > > if cond: > l = [] > else: > l = function() > return l[0] > > What is l here? Depends, but it's likely a list. So, if function() (who knows where it's imported from) doesn't return a list, would that be an error. I was leaning towards having l be the most generic type possible (or, in other words, the most specific type that can handle all of the assignments). >> I think for the first step, we should just leave functions (and class >> members) as they are now--we can come back to them later but this >> will >> make things much messier and less local (not to mention the issues of >> cimporting functions and types from other modules). > > I agree. Getting the infrastructure in place and handling the easy > cases would > be enough work for starters. Very true, but interesting work. - Robert From robertwb at math.washington.edu Sat Feb 2 01:12:49 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 1 Feb 2008 16:12:49 -0800 Subject: [Cython] Type casts Message-ID: <61C43B43-B4C0-4A70-9992-347835789619@math.washington.edu> Currently, in Cython, when x is a python object, x gives the address of x. This is counter-intuitive and rarely useful. (When it is, one can always do x). I would propose making x do a conversion (the same as cdef type y = x would) whenever type is a non-object type and x isn't, or vica- versa. Between C types it already behaves this way, and between Python objects it already has special meaning. I would also propose something like x to be a cast into type ExtensionType *with* type checking. Thoughts? - Robert From wstein at gmail.com Sat Feb 2 06:55:40 2008 From: wstein at gmail.com (William Stein) Date: Sat, 2 Feb 2008 00:55:40 -0500 Subject: [Cython] Type casts In-Reply-To: <61C43B43-B4C0-4A70-9992-347835789619@math.washington.edu> References: <61C43B43-B4C0-4A70-9992-347835789619@math.washington.edu> Message-ID: <85e81ba30802012155kba45938rf794f51cb14a5eb5@mail.gmail.com> On Feb 1, 2008 7:12 PM, Robert Bradshaw wrote: > Currently, in Cython, when x is a python object, x gives the > address of x. This is counter-intuitive and rarely useful. (When it > is, one can always do x). > > I would propose making x do a conversion (the same as cdef type > y = x would) whenever type is a non-object type and x isn't, or vica- > versa. Between C types it already behaves this way, and between > Python objects it already has special meaning. > > I would also propose something like > > x to be a cast into type ExtensionType *with* type > checking. > > Thoughts? +1 to both suggestions. (I'm also testing that I can post to this list.) By the way, I've talked with a lot of people about Cython in the last week, and this is definitely a very important project that's going to get more and more use. And it doesn't hurt for this project that: "TIOBE Software (http://www.tiobe.com/tpci.htm): TIOBE declares Python as programming language of 2007! ["Python has been declared as programming language of 2007. It was a close finish, but in the end Python appeared to have the largest increase in ratings in one year time (2.04%). There is no clear reason why Python made this huge jump in 2007. Last month Python surpassed Perl for the first time in history, which is an indication that Python has become the "de facto" glue language at system level. It is especially beloved by system administrators and build managers. Chances are high that Python's star will rise further in 2008, thanks to the upcoming release of Python 3." --- By the way, why did we switch to cython-dev at codespeak.net ? I don't even remember that happening. -- William From stefan_ml at behnel.de Sat Feb 2 07:12:07 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 02 Feb 2008 07:12:07 +0100 Subject: [Cython] Type casts In-Reply-To: <61C43B43-B4C0-4A70-9992-347835789619@math.washington.edu> References: <61C43B43-B4C0-4A70-9992-347835789619@math.washington.edu> Message-ID: <47A409B7.6000105@behnel.de> Hi Robert, Robert Bradshaw wrote: > Currently, in Cython, when x is a python object, x gives the > address of x. This is counter-intuitive and rarely useful. (When it > is, one can always do x). > > I would propose making x do a conversion (the same as cdef type > y = x would) whenever type is a non-object type and x isn't, or vica- > versa. Between C types it already behaves this way, and between > Python objects it already has special meaning. +1 The first thing I tried when I hit the bool expr problem, was changing if n is None or n == 0: into if n is None or (n == 0): which did not quite give the expected result. > I would also propose something like > > x to be a cast into type ExtensionType *with* type > checking. I think that's also a good idea. BTW, we shouldn't forget to document syntax extensions in the wiki, so that people (including Greg) know what works. Stefan From stefan_ml at behnel.de Sat Feb 2 07:50:10 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 02 Feb 2008 07:50:10 +0100 Subject: [Cython] How to know it's constant? Message-ID: <47A412A2.1000403@behnel.de> Hi, is there a standard way in the compiler to know (or to find out) that an expression is constant? I noticed the "analyse_const_expression()" methods, but they are only meant to calculate "DEF" expressions, so they raise an error if the expression is not constant. I'm thinking about code like this: raise ValueError("constant") that always raises a constant tuple with one string element that is stored in a global constant at module init time. It would be better to store the tuple here, not only the string. But that would require knowing that the whole tuple is constant. Maybe we should split "check_const()" into one part that checks the expression and another that raises the error (if required). Or maybe pass a flag. Stefan From robertwb at math.washington.edu Tue Feb 5 08:08:39 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Mon, 4 Feb 2008 23:08:39 -0800 Subject: [Cython] Type casts In-Reply-To: <47A409B7.6000105@behnel.de> References: <61C43B43-B4C0-4A70-9992-347835789619@math.washington.edu> <47A409B7.6000105@behnel.de> Message-ID: <82A809FF-2036-41B9-99F7-013C6CF33473@math.washington.edu> Done. On Feb 1, 2008, at 10:12 PM, Stefan Behnel wrote: > Hi Robert, > > Robert Bradshaw wrote: >> Currently, in Cython, when x is a python object, x gives the >> address of x. This is counter-intuitive and rarely useful. (When it >> is, one can always do x). >> >> I would propose making x do a conversion (the same as cdef type >> y = x would) whenever type is a non-object type and x isn't, or vica- >> versa. Between C types it already behaves this way, and between >> Python objects it already has special meaning. > > +1 > > The first thing I tried when I hit the bool expr problem, was changing > > if n is None or n == 0: > > into > > if n is None or (n == 0): > > which did not quite give the expected result. > > >> I would also propose something like >> >> x to be a cast into type ExtensionType *with* type >> checking. > > I think that's also a good idea. > > BTW, we shouldn't forget to document syntax extensions in the wiki, > so that > people (including Greg) know what works. > > Stefan From ondrej at certik.cz Tue Feb 5 17:19:32 2008 From: ondrej at certik.cz (Ondrej Certik) Date: Tue, 5 Feb 2008 17:19:32 +0100 Subject: [Cython] cython: cdef-ed attribute produces invalid C source Message-ID: <85b5c3130802050819t6c8902capee3914c0a1d534b0@mail.gmail.com> Hi, all necessary details of this bug are here: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=463625 I only tried cython 0.9.6.11. When I find a little time, I'll try to test the hg version too. Ondrej From stefan_ml at behnel.de Tue Feb 5 20:54:32 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 05 Feb 2008 20:54:32 +0100 Subject: [Cython] cython: cdef-ed attribute produces invalid C source In-Reply-To: <85b5c3130802050819t6c8902capee3914c0a1d534b0@mail.gmail.com> References: <85b5c3130802050819t6c8902capee3914c0a1d534b0@mail.gmail.com> Message-ID: <47A8BEF8.8000100@behnel.de> Ondrej Certik wrote: > I only tried cython 0.9.6.11. Now that you mention it, people keep using 0.9.6.11 instead of .11b, as that's what PyPI gives you as the newest version. It gets confused by the appended 'b'. I think we'll have to change the versioning scheme here... Stefan From robertwb at math.washington.edu Tue Feb 5 22:32:00 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Tue, 5 Feb 2008 13:32:00 -0800 Subject: [Cython] Cython problem In-Reply-To: <6f86f2140802051321q13f64db0u54d23f1d78b8718f@mail.gmail.com> References: <6f86f2140802051321q13f64db0u54d23f1d78b8718f@mail.gmail.com> Message-ID: <6AF6F958-65F0-4C4E-8E9C-2DD20A9869B4@math.washington.edu> This is a really strange bug...the following works: def f(**argsByName): tmp = [argsByname.get(argName,None) for argName in ["a","b"]] (aVal, bVal) = tmp Of course it's faster to do def f(aVal=None, bVal=None): ... On Feb 5, 2008, at 1:21 PM, Kirby Baker wrote: > Hi, > > The following doesn't compile when I put it in a file > "example.spyx" and load the file from within SAGE. If I have done > something wrong, let me know. Thanks. > > Kirby > > (Kirby Baker, UCLA) > > def f(**argsByName): > (aVal, bVal) = [argsByname.get(argName,None) for argName in > ["a","b"]] > From stefan_ml at behnel.de Wed Feb 6 09:50:09 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 06 Feb 2008 09:50:09 +0100 Subject: [Cython] Cython problem In-Reply-To: <6AF6F958-65F0-4C4E-8E9C-2DD20A9869B4@math.washington.edu> References: <6f86f2140802051321q13f64db0u54d23f1d78b8718f@mail.gmail.com> <6AF6F958-65F0-4C4E-8E9C-2DD20A9869B4@math.washington.edu> Message-ID: <47A974C1.5090308@behnel.de> Hi, Robert Bradshaw wrote: > This is a really strange bug... Here is a test case that gives me a compiler crash. Stefan # HG changeset patch # User Stefan Behnel # Date 1202287648 -3600 # Node ID 543c702490e329817fa2b46cda97d054091c0e52 # Parent 36d9ecdab18b0a94fe8ce1c32f2d5a81a039df9b test case for tuple unpacking of a list comprehension diff -r 36d9ecdab18b -r 543c702490e3 tests/run/unpacklistcomp.pyx --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tests/run/unpacklistcomp.pyx Wed Feb 06 09:47:28 2008 +0100 @@ -0,0 +1,31 @@ +__doc__ = """ + >>> unpack_normal([1,2]) + (1, 2) + >>> unpack_normal([1,2,3]) + Traceback (most recent call last): + ValueError: too many values to unpack + + >>> unpack_comp([1,2]) + (1, 2) + >>> unpack_comp([1,2,3]) + Traceback (most recent call last): + ValueError: too many values to unpack + + >>> unpack_expr([1,2]) + (1, 2) + >>> unpack_expr([1,2,3]) + Traceback (most recent call last): + ValueError: too many values to unpack +""" + +def unpack_normal(l): + a,b = l + return a,b + +def unpack_comp(l): + a,b = [ n for n in l ] + return a,b + +def unpack_expr(l): + a,b = [ n*n for n in l ] + return a,b From stefan_ml at behnel.de Wed Feb 6 23:08:25 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 06 Feb 2008 23:08:25 +0100 Subject: [Cython] Skip tuple unpacking for func(*args, **kwargs) Message-ID: <47AA2FD9.6020508@behnel.de> Hi, here is another optimisation that removes argument unpacking for the star-args-only case(s). This means that pass-through functions or wrappers that do argument handling themselves or just pass it on to a wrapped function will now run almost without internal overhead. The generated code looks like this: -------------------------------- Py_INCREF(__pyx_args); __pyx_v_a = __pyx_args; __pyx_args = 0; if (__pyx_kwds) { Py_INCREF(__pyx_kwds); __pyx_v_k = __pyx_kwds; __pyx_kwds = 0; } else { __pyx_v_k = PyDict_New(); } -------------------------------- plus straight sanity checks when you use only one out of *args -or- **kwargs. Note that I'm actually setting __pyx_args/kwds to 0 here, to make sure we notice programming errors. Stefan -------------- next part -------------- A non-text attachment was scrubbed... Name: package-upstream.bundle Type: application/octet-stream Size: 7443 bytes Desc: not available Url : http://codespeak.net/pipermail/cython-dev/attachments/20080206/c0f01337/attachment-0002.obj -------------- next part -------------- A non-text attachment was scrubbed... Name: upstream.bundle Type: application/octet-stream Size: 3504 bytes Desc: not available Url : http://codespeak.net/pipermail/cython-dev/attachments/20080206/c0f01337/attachment-0003.obj From ondrej at certik.cz Fri Feb 8 21:14:45 2008 From: ondrej at certik.cz (Ondrej Certik) Date: Fri, 8 Feb 2008 21:14:45 +0100 Subject: [Cython] error when cythonizing unicode Message-ID: <85b5c3130802081214h7a180c93jd0747685e26421d3@mail.gmail.com> Hi, another bug was discovered in the Cython 0.9.6.11 in Debian: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=464751 I am currently busy finding out, whether it was fixed already, so just reporting it here. Ondrej From stefan_ml at behnel.de Fri Feb 8 21:53:57 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 08 Feb 2008 21:53:57 +0100 Subject: [Cython] error when cythonizing unicode In-Reply-To: <85b5c3130802081214h7a180c93jd0747685e26421d3@mail.gmail.com> References: <85b5c3130802081214h7a180c93jd0747685e26421d3@mail.gmail.com> Message-ID: <47ACC165.7000105@behnel.de> Hi, Ondrej Certik wrote: > another bug was discovered in the Cython 0.9.6.11 in Debian: > > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=464751 > > I am currently busy finding out, whether it was fixed already, so just > reporting it here. This works around it. However, I would prefer having a solution that makes builtin-types that have a duality as API functions (PyObject_Str, PyObject_Unicode, ...) available as both a function and a type, hopefully depending on the context. Stefan -------------- next part -------------- A non-text attachment was scrubbed... Name: unicode.patch Type: text/x-patch Size: 1103 bytes Desc: not available Url : http://codespeak.net/pipermail/cython-dev/attachments/20080208/aa3d50c6/attachment.bin From ondrej at certik.cz Sat Feb 9 10:38:49 2008 From: ondrej at certik.cz (Ondrej Certik) Date: Sat, 9 Feb 2008 10:38:49 +0100 Subject: [Cython] error when cythonizing unicode In-Reply-To: <47ACC165.7000105@behnel.de> References: <85b5c3130802081214h7a180c93jd0747685e26421d3@mail.gmail.com> <47ACC165.7000105@behnel.de> Message-ID: <85b5c3130802090138i562ebc46rc85c8820c969334f@mail.gmail.com> On Feb 8, 2008 9:53 PM, Stefan Behnel wrote: > Hi, > > > Ondrej Certik wrote: > > another bug was discovered in the Cython 0.9.6.11 in Debian: > > > > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=464751 > > > > I am currently busy finding out, whether it was fixed already, so just > > reporting it here. > > This works around it. However, I would prefer having a solution that makes > builtin-types that have a duality as API functions (PyObject_Str, > PyObject_Unicode, ...) available as both a function and a type, hopefully > depending on the context. Excellent. Let's put this into your repository and release, so that I don't have to patch the Debian package. :) Ondrej From robertwb at math.washington.edu Sat Feb 9 10:48:32 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sat, 9 Feb 2008 01:48:32 -0800 Subject: [Cython] error when cythonizing unicode In-Reply-To: <85b5c3130802090138i562ebc46rc85c8820c969334f@mail.gmail.com> References: <85b5c3130802081214h7a180c93jd0747685e26421d3@mail.gmail.com> <47ACC165.7000105@behnel.de> <85b5c3130802090138i562ebc46rc85c8820c969334f@mail.gmail.com> Message-ID: <2197FC83-D9FD-45E6-A433-5C65BE80EB09@math.washington.edu> I have a bunch of stuff, and several patches--hoping to release Cython 0.9.6.12 soon. - Robert On Feb 9, 2008, at 1:38 AM, Ondrej Certik wrote: > On Feb 8, 2008 9:53 PM, Stefan Behnel wrote: >> Hi, >> >> >> Ondrej Certik wrote: >>> another bug was discovered in the Cython 0.9.6.11 in Debian: >>> >>> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=464751 >>> >>> I am currently busy finding out, whether it was fixed already, so >>> just >>> reporting it here. >> >> This works around it. However, I would prefer having a solution >> that makes >> builtin-types that have a duality as API functions (PyObject_Str, >> PyObject_Unicode, ...) available as both a function and a type, >> hopefully >> depending on the context. > > Excellent. Let's put this into your repository and release, so that I > don't have to patch the Debian package. :) > > Ondrej > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev From robertwb at math.washington.edu Mon Feb 11 21:30:36 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Mon, 11 Feb 2008 12:30:36 -0800 Subject: [Cython] Projects using Cython? Message-ID: <6FB6C91F-B23C-4A0F-BC1E-D38F9A3222FF@math.washington.edu> If you are using Cython for your own project, and don't mind having it listed, could you please add it to the list at http://wiki.cython.org/projects Thanks, Robert From tabbott at MIT.EDU Mon Feb 11 06:09:39 2008 From: tabbott at MIT.EDU (Tim Abbott) Date: Mon, 11 Feb 2008 00:09:39 -0500 (EST) Subject: [Cython] cython python versions Message-ID: Hello, If you use python-distutils to install cython, it ends up replacing the #! line at the start of bin/cython (#!/usr/bin/env python) with a more specific version, i.e. (#!/usr/bin/python or #!/usr/bin/python2.x). I assume this is undesirable; it caused problems for me building SAGE on Debian Lenny. Search for "adjust" in /usr/lib/python2.5/distutils/command/build_scripts.py if you want to see the code that does this rewriting; it seems to be controlled by whether cython is registered as a script in setup.py. -Tim Abbott From foxhound at escapehomes.com Tue Feb 12 04:38:28 2008 From: foxhound at escapehomes.com (Nesby Gunlock) Date: Tue, 12 Feb 2008 03:38:28 +0000 Subject: [Cython] codices Message-ID: <9076949467.20080212033810@escapehomes.com> Ahn nyeong, Are you a ffrequent visitor of reetail softwware stores? We know what you're overpayiing for: - box manufactturing - CD - salespersonn salary - Rentt of shop sspace - Year--to-year inncreasing taxes in your counntry Well, what for ?! You're able to doownload everrything legally NOW! Fabuulous range of softwware and LOW prices will make you smile and save your money! Welcome to http://mattiecanadyf.blogspot.com Not make the matter any easier that, a few days small tidy square of garden. It was aggressively negro sculpture. You know, i didn't want him to again the life of a man in some free, wild, adventurefilled but a farm could have got up quicker, if there our minds, and made us thankful that lord john marple stopped him. But it is true, colonel bantry. he said as our christmas singin' was no better compromised. say, listen here to me, abe! He declared. To captain trevelyan 3. Five and twenty past five full bottles and empty ones. and if she'd done chafed. Dorothy also, with her feminine desire confidencebut that isn't so. I've changed all red narky, under the kind escort of poirot waved all, i'm a man, i've a right to love youor any. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20080212/7fd15904/attachment.htm From robertwb at math.washington.edu Wed Feb 13 04:43:17 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Tue, 12 Feb 2008 19:43:17 -0800 Subject: [Cython] Cython problem In-Reply-To: <6AF6F958-65F0-4C4E-8E9C-2DD20A9869B4@math.washington.edu> References: <6f86f2140802051321q13f64db0u54d23f1d78b8718f@mail.gmail.com> <6AF6F958-65F0-4C4E-8E9C-2DD20A9869B4@math.washington.edu> Message-ID: <6AB484E6-750A-422C-8AE8-3BA4C71E53D6@math.washington.edu> This has been fixed for the next release of Cython. On Feb 5, 2008, at 1:32 PM, Robert Bradshaw wrote: > This is a really strange bug...the following works: > > def f(**argsByName): > tmp = [argsByname.get(argName,None) for argName in ["a","b"]] > (aVal, bVal) = tmp > > Of course it's faster to do > > def f(aVal=None, bVal=None): > ... > > On Feb 5, 2008, at 1:21 PM, Kirby Baker wrote: > >> Hi, >> >> The following doesn't compile when I put it in a file >> "example.spyx" and load the file from within SAGE. If I have done >> something wrong, let me know. Thanks. >> >> Kirby >> >> (Kirby Baker, UCLA) >> >> def f(**argsByName): >> (aVal, bVal) = [argsByname.get(argName,None) for argName in >> ["a","b"]] >> > > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev From robertwb at math.washington.edu Wed Feb 13 07:54:02 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Tue, 12 Feb 2008 22:54:02 -0800 Subject: [Cython] Fixes and tests for compile time DEF/IF In-Reply-To: <47AEABAF.3050306@behnel.de> References: <47AEABAF.3050306@behnel.de> Message-ID: I've merged and pushed my latest changes to cython-devel. Do you want to take a final look and make sure lxml compiles so we can make a release? - Robert On Feb 9, 2008, at 11:45 PM, Stefan Behnel wrote: > Hi Robert, > > the True/False constants didn't work in compile-time DEF/IF > statements, here's > a fix. And another one for a NameError I got. I also added (or > enabled) a > couple of tests for DEF/IF. > > Stefan From Samuele.Kaplun at cern.ch Wed Feb 13 09:11:21 2008 From: Samuele.Kaplun at cern.ch (Samuele Kaplun) Date: Wed, 13 Feb 2008 09:11:21 +0100 Subject: [Cython] cython --version is not correct Message-ID: <200802130911.22279.Samuele.Kaplun@cern.ch> Dear Cython developers, this is to let you know that the cython --version or cython -v is not working. Instead of returning the cython version it returns the help guide. Best regards, Samuele -- .O. ..O OOO From ondrej at certik.cz Wed Feb 13 18:25:53 2008 From: ondrej at certik.cz (Ondrej Certik) Date: Wed, 13 Feb 2008 18:25:53 +0100 Subject: [Cython] cython python versions In-Reply-To: References: Message-ID: <85b5c3130802130925l4acebe03kad82df927463932a@mail.gmail.com> Hi Tim, On Feb 11, 2008 6:09 AM, Tim Abbott wrote: > Hello, > > If you use python-distutils to install cython, it ends up replacing the #! > line at the start of bin/cython (#!/usr/bin/env python) with a more > specific version, i.e. (#!/usr/bin/python or #!/usr/bin/python2.x). I > assume this is undesirable; it caused problems for me building SAGE on > Debian Lenny. Thanks very much for tracing the problem down. One option is that we simply add a patch to the Debian package to rewrite /usr/bin/python back to /usr/bin/env python. I can do this when uploading a new version of cython. > Search for "adjust" in > /usr/lib/python2.5/distutils/command/build_scripts.py if you want to see > the code that does this rewriting; it seems to be controlled by whether > cython is registered as a script in setup.py. So what is the solution to fix this? Fixing the cython's setup.py? Let's do that in the next release then. Fixing the Debian is just a temporary solution, so that we have it working now. Ondrej From tabbott at MIT.EDU Wed Feb 13 18:39:29 2008 From: tabbott at MIT.EDU (Tim Abbott) Date: Wed, 13 Feb 2008 12:39:29 -0500 (EST) Subject: [Cython] cython python versions In-Reply-To: <85b5c3130802130925l4acebe03kad82df927463932a@mail.gmail.com> References: <85b5c3130802130925l4acebe03kad82df927463932a@mail.gmail.com> Message-ID: On Wed, 13 Feb 2008, Ondrej Certik wrote: > Hi Tim, > > On Feb 11, 2008 6:09 AM, Tim Abbott wrote: >> Hello, >> >> If you use python-distutils to install cython, it ends up replacing the #! >> line at the start of bin/cython (#!/usr/bin/env python) with a more >> specific version, i.e. (#!/usr/bin/python or #!/usr/bin/python2.x). I >> assume this is undesirable; it caused problems for me building SAGE on >> Debian Lenny. > > Thanks very much for tracing the problem down. One option is that we simply add > a patch to the Debian package to rewrite /usr/bin/python back to > /usr/bin/env python. > I can do this when uploading a new version of cython. > >> Search for "adjust" in >> /usr/lib/python2.5/distutils/command/build_scripts.py if you want to see >> the code that does this rewriting; it seems to be controlled by whether >> cython is registered as a script in setup.py. > > So what is the solution to fix this? Fixing the cython's setup.py? Let's do that > in the next release then. I think that's the right solution. -Tim Abbott From robertwb at math.washington.edu Thu Feb 14 05:42:01 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Wed, 13 Feb 2008 20:42:01 -0800 Subject: [Cython] Cython 0.9.6.12 released Message-ID: <4A285017-D8E3-47F6-A677-108B06F6D6BC@math.washington.edu> Cython 0.9.6.12 released! The most significant change is more flexible c(p)def functions and overriding. Specifically, c(p)def functions can now: * have optional arguments (which may grow) * be defined in the module scope * are always cimport-able if defined in the .pxd (i.e. "api" by default) * declare narrower return types than the superclass * cpdef can override cdef There are also better conversions ( does a type-checked cast, x does the right thing), and numerous optimizations (especially with regard to python function tuple unpacking) and bugfixes, and a much expanded testing framework. For more details, see the end of http://wiki.cython.org/ DifferencesFromPyrex and the changelog at http://hg.cython.org/ The main contributers in this release were Stefan Behnel and Robert Bradshaw, with much discussion and ideas from Kay Hayen and bug reports/patches from Dan Gindikin, Samuele Kaplun, Vasil Manolov, Kirill Smelkov and Sven Berkvens-Matthijsse. - Robert -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 186 bytes Desc: This is a digitally signed message part Url : http://codespeak.net/pipermail/cython-dev/attachments/20080213/317e5d0c/attachment.pgp From ondrej at certik.cz Fri Feb 15 11:35:41 2008 From: ondrej at certik.cz (Ondrej Certik) Date: Fri, 15 Feb 2008 11:35:41 +0100 Subject: [Cython] Cython 0.9.6.12 released In-Reply-To: <4A285017-D8E3-47F6-A677-108B06F6D6BC@math.washington.edu> References: <4A285017-D8E3-47F6-A677-108B06F6D6BC@math.washington.edu> Message-ID: <85b5c3130802150235l3548f376icc606579a383d782@mail.gmail.com> On Thu, Feb 14, 2008 at 5:42 AM, Robert Bradshaw wrote: > Cython 0.9.6.12 released! It's in Debian now. The following problems still remain: * http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=464751 * /usr/bin/cython contains #!/usr/bin/python instead of #!/usr/bin/env python Ondrej From ondrej at certik.cz Sat Feb 16 09:41:34 2008 From: ondrej at certik.cz (Ondrej Certik) Date: Sat, 16 Feb 2008 09:41:34 +0100 Subject: [Cython] Cython 0.9.6.12 released In-Reply-To: <85b5c3130802150235l3548f376icc606579a383d782@mail.gmail.com> References: <4A285017-D8E3-47F6-A677-108B06F6D6BC@math.washington.edu> <85b5c3130802150235l3548f376icc606579a383d782@mail.gmail.com> Message-ID: <85b5c3130802160041l54280c64uc304e7da0bb30de1@mail.gmail.com> On Feb 15, 2008 11:35 AM, Ondrej Certik wrote: > On Thu, Feb 14, 2008 at 5:42 AM, Robert Bradshaw > wrote: > > Cython 0.9.6.12 released! > > It's in Debian now. The following problems still remain: > > * http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=464751 > * /usr/bin/cython contains #!/usr/bin/python instead of #!/usr/bin/env python After discussing the second point with Tim Abbott and Robert Kern, we think it is actually a correct behavior and all is fine. So the only problem that remains is the Debian bug #464751 - there is a simple fix suggested by Stefan, but it probably wasn't applied yet. Ondrej From stefan_ml at behnel.de Sat Feb 16 11:54:03 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 16 Feb 2008 11:54:03 +0100 Subject: [Cython] Cython 0.9.6.12 released In-Reply-To: <85b5c3130802160041l54280c64uc304e7da0bb30de1@mail.gmail.com> References: <4A285017-D8E3-47F6-A677-108B06F6D6BC@math.washington.edu> <85b5c3130802150235l3548f376icc606579a383d782@mail.gmail.com> <85b5c3130802160041l54280c64uc304e7da0bb30de1@mail.gmail.com> Message-ID: <47B6C0CB.3050700@behnel.de> Hi, Ondrej Certik wrote: > On Feb 15, 2008 11:35 AM, Ondrej Certik wrote: >> On Thu, Feb 14, 2008 at 5:42 AM, Robert Bradshaw >> wrote: >>> Cython 0.9.6.12 released! >> It's in Debian now. The following problems still remain: >> >> * http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=464751 >> * /usr/bin/cython contains #!/usr/bin/python instead of #!/usr/bin/env python > > After discussing the second point with Tim Abbott and Robert Kern, we > think it is actually a correct behavior and all is fine. > So the only problem that remains is the Debian bug #464751 - there is > a simple fix suggested by Stefan, but it probably > wasn't applied yet. It wasn't applied as it's not a "fix" but a "work-around". It solves this problem, but it degenerates the generated C code. Currently, Cython will generate this: ---------------------------------------- /* * #u = unicode # doesn't work!! * z = unicode('test') # <<<<<<<<<<<<<< */ __pyx_1 = PyObject_Unicode(__pyx_n_test); if (unlikely(!__pyx_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 11; goto __pyx_L1;} if (PyObject_SetAttr(__pyx_m, __pyx_n_z, __pyx_1) < 0) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 11; goto __pyx_L1;} Py_DECREF(__pyx_1); __pyx_1 = 0; ---------------------------------------- With the work-around applied, it becomes this: ---------------------------------------- /* * u = unicode # works now!! # <<<<<<<<<<<<<< * z = unicode('test') */ if (PyObject_SetAttr(__pyx_m, __pyx_n_u, ((PyObject*)&PyUnicode_Type)) < 0) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 10; goto __pyx_L1;} /* * u = unicode # works now!! * z = unicode('test') # <<<<<<<<<<<<<< */ __pyx_1 = PyTuple_New(1); if (unlikely(!__pyx_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 11; goto __pyx_L1;} Py_INCREF(__pyx_n_test); PyTuple_SET_ITEM(__pyx_1, 0, __pyx_n_test); __pyx_2 = PyObject_Call(((PyObject*)&PyUnicode_Type), __pyx_1, NULL); if (unlikely(!__pyx_2)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 11; goto __pyx_L1;} Py_DECREF(__pyx_1); __pyx_1 = 0; if (PyObject_SetAttr(__pyx_m, __pyx_n_z, __pyx_2) < 0) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 11; goto __pyx_L1;} Py_DECREF(__pyx_2); __pyx_2 = 0; ---------------------------------------- So you get all the Python call overhead, including tuple packing/unpacking. The right way to solve this would be to fix the dual identity of types as Python functions and Python C-API functions. Calls to "unicode()" should become C-API calls, while the name "unicode" should stay the thing that can be accessed through __builtin__. BTW, I wouldn't mind applying the work-around (i.e. to remove the mapping to C API functions completely). Correct is better than fast if you can't have both (at least for now). We already do this for other builtins like "type" or "str" in Symtab.py, so it doesn't add much to it. Stefan From ondrej at certik.cz Sat Feb 16 12:24:50 2008 From: ondrej at certik.cz (Ondrej Certik) Date: Sat, 16 Feb 2008 12:24:50 +0100 Subject: [Cython] Description of the problem Message-ID: <85b5c3130802160324i4e54f9a1n8f9233125e2576bd@mail.gmail.com> Forwarding a description of the problem to the Debian bug. ---------- Forwarded message ---------- From: Stefan Behnel <> Date: Feb 16, 2008 11:54 AM Subject: Re: [Cython] Cython 0.9.6.12 released Hi, Ondrej Certik wrote: > On Feb 15, 2008 11:35 AM, Ondrej Certik wrote: >> On Thu, Feb 14, 2008 at 5:42 AM, Robert Bradshaw >> wrote: >>> Cython 0.9.6.12 released! >> It's in Debian now. The following problems still remain: >> >> * http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=464751 >> * /usr/bin/cython contains #!/usr/bin/python instead of #!/usr/bin/env python > > After discussing the second point with Tim Abbott and Robert Kern, we > think it is actually a correct behavior and all is fine. > So the only problem that remains is the Debian bug #464751 - there is > a simple fix suggested by Stefan, but it probably > wasn't applied yet. It wasn't applied as it's not a "fix" but a "work-around". It solves this problem, but it degenerates the generated C code. Currently, Cython will generate this: ---------------------------------------- /* * #u = unicode # doesn't work!! * z = unicode('test') # <<<<<<<<<<<<<< */ __pyx_1 = PyObject_Unicode(__pyx_n_test); if (unlikely(!__pyx_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 11; goto __pyx_L1;} if (PyObject_SetAttr(__pyx_m, __pyx_n_z, __pyx_1) < 0) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 11; goto __pyx_L1;} Py_DECREF(__pyx_1); __pyx_1 = 0; ---------------------------------------- With the work-around applied, it becomes this: ---------------------------------------- /* * u = unicode # works now!! # <<<<<<<<<<<<<< * z = unicode('test') */ if (PyObject_SetAttr(__pyx_m, __pyx_n_u, ((PyObject*)&PyUnicode_Type)) < 0) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 10; goto __pyx_L1;} /* * u = unicode # works now!! * z = unicode('test') # <<<<<<<<<<<<<< */ __pyx_1 = PyTuple_New(1); if (unlikely(!__pyx_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 11; goto __pyx_L1;} Py_INCREF(__pyx_n_test); PyTuple_SET_ITEM(__pyx_1, 0, __pyx_n_test); __pyx_2 = PyObject_Call(((PyObject*)&PyUnicode_Type), __pyx_1, NULL); if (unlikely(!__pyx_2)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 11; goto __pyx_L1;} Py_DECREF(__pyx_1); __pyx_1 = 0; if (PyObject_SetAttr(__pyx_m, __pyx_n_z, __pyx_2) < 0) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 11; goto __pyx_L1;} Py_DECREF(__pyx_2); __pyx_2 = 0; ---------------------------------------- So you get all the Python call overhead, including tuple packing/unpacking. The right way to solve this would be to fix the dual identity of types as Python functions and Python C-API functions. Calls to "unicode()" should become C-API calls, while the name "unicode" should stay the thing that can be accessed through __builtin__. BTW, I wouldn't mind applying the work-around (i.e. to remove the mapping to C API functions completely). Correct is better than fast if you can't have both (at least for now). We already do this for other builtins like "type" or "str" in Symtab.py, so it doesn't add much to it. Stefan From stefan_ml at behnel.de Sat Feb 16 12:46:09 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 16 Feb 2008 12:46:09 +0100 Subject: [Cython] error when cythonizing unicode In-Reply-To: <85b5c3130802081214h7a180c93jd0747685e26421d3@mail.gmail.com> References: <85b5c3130802081214h7a180c93jd0747685e26421d3@mail.gmail.com> Message-ID: <47B6CD01.7070504@behnel.de> Hi, Ondrej Certik wrote: > another bug was discovered in the Cython 0.9.6.11 in Debian: > > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=464751 Here is a test case with things I could imagine to work for "unicode": ------------------------------- __doc__ = """ >>> u('test') u'test' >>> z u'test' >>> c('testing') u'testing' >>> subu('testing a Python subtype') u'testing a Python subtype' >>> csubu('testing a C subtype') u'testing a C subtype' >>> sub('testing a Python subtype') u'testing a Python subtype' >>> csub('testing a C subtype') u'testing a C subtype' """ u = unicode # doesn't currently work z = unicode('test') class subu(unicode): # doesn't currently work pass cdef class csubu(unicode): # doesn't currently work pass def c(string): return unicode(string) def csub(string): return csubu(string) def sub(string): return subu(string) ------------------------------- My work-around fixes the first two cases, so that's not too bad. And I think the "cdef" thing would be a new feature anyway. @Robert: maybe we should remove the "builtin_functions" dict in Symtab.py completely. It looks like it just breaks things. Stefan -------------- next part -------------- A non-text attachment was scrubbed... Name: package-upstream.bundle Type: application/octet-stream Size: 877 bytes Desc: not available Url : http://codespeak.net/pipermail/cython-dev/attachments/20080216/bb25ac7f/attachment.obj From dmharvey at math.harvard.edu Mon Feb 25 05:11:03 2008 From: dmharvey at math.harvard.edu (David Harvey) Date: Sun, 24 Feb 2008 23:11:03 -0500 Subject: [Cython] minor bug -- cimport inside function Message-ID: If I run cython on the following file: ======================= def foo(): from vial cimport poison ======================= I get the following traceback: Traceback (most recent call last): File "/Users/david/sage-2.10.2/local/bin/cython", line 8, in main(command_line = 1) File "/Users/david/sage-2.10.2/local/lib/python2.5/site-packages/ Cython/Compiler/Main.py", line 322, in main result = context.compile(source, options) File "/Users/david/sage-2.10.2/local/lib/python2.5/site-packages/ Cython/Compiler/Main.py", line 198, in compile tree.process_implementation(scope, options, result) File "/Users/david/sage-2.10.2/local/lib/python2.5/site-packages/ Cython/Compiler/ModuleNode.py", line 49, in process_implementation self.generate_c_code(env, result) File "/Users/david/sage-2.10.2/local/lib/python2.5/site-packages/ Cython/Compiler/ModuleNode.py", line 219, in generate_c_code self.body.generate_function_definitions(env, code) File "/Users/david/sage-2.10.2/local/lib/python2.5/site-packages/ Cython/Compiler/Nodes.py", line 674, in generate_function_definitions self.body.analyse_declarations(lenv) File "/Users/david/sage-2.10.2/local/lib/python2.5/site-packages/ Cython/Compiler/Nodes.py", line 3331, in analyse_declarations module_scope = env.find_module(self.module_name, self.pos) AttributeError: LocalScope instance has no attribute 'find_module' david From dmharvey at math.harvard.edu Mon Feb 25 05:18:05 2008 From: dmharvey at math.harvard.edu (David Harvey) Date: Sun, 24 Feb 2008 23:18:05 -0500 Subject: [Cython] circular cimports Message-ID: Hi, In Sage, if I try to add from sage.rings.integer cimport Integer at the top of sage/misc/misc_c.pyx, everything compiles fine, but then I get some weird import error when I start up sage. (I'm sorry, I don't know enough about cython to reproduce this in a standalone cython program, I only know how to make it happen in sage.) It looks like a "circular cimport problem". I found this post by Greg Ewing which seems relevant, but it's a bit old now, and I don't quite understand what he's saying: http://osdir.com/ml/python.pyrex/2006-06/msg00020.html Is there any way to successfully cimport Integer into that module? Or do I need to just put the code that references Integer in some other random place which is guaranteed to get loaded after sage.rings.integer? (This would be a bit annoying....) I don't quite understand why circular cimports should exist, since I thought these kinds of things got resolved at compile time. david From bill at indirectproof.net Mon Feb 25 05:44:07 2008 From: bill at indirectproof.net (Bill Furnish) Date: Sun, 24 Feb 2008 21:44:07 -0700 Subject: [Cython] circular cimports In-Reply-To: References: Message-ID: <706850310802242044hf719e32mad430dbcba75e784@mail.gmail.com> There are actually two issues here which I am currently working on a patch for. One, code generation for classes is based on order of imported modules, not on the base class dependencies between the classes. I ended up sorting them by dependencies and this fixed that issue. There is a second issue I am still working on which is that cython does not always generate correct module initialization code in the circular dependence case. This results in the attribute error that is sometimes observed. It seems like this arises from circular imports in the python code and not in the symbolic code. I think that it arises because creating classes involves __init__ and thus we end up with circular imports in python. I feel liike we ought to be able to detect this somehow though and prevent it. However, the problem seems to me to be that currently cython source files are compiled and linked into separate objects. It may be necessary to implement a link time solution to the issue by linking multiple cython files together and then not load the modules that are linked together, but instead leave the pointers to be relocated by the linker. On 2/24/08, David Harvey wrote: > > Hi, > > In Sage, if I try to add > > from sage.rings.integer cimport Integer > > at the top of sage/misc/misc_c.pyx, everything compiles fine, but > then I get some weird import error when I start up sage. > > (I'm sorry, I don't know enough about cython to reproduce this in a > standalone cython program, I only know how to make it happen in sage.) > > It looks like a "circular cimport problem". I found this post by Greg > Ewing which seems relevant, but it's a bit old now, and I don't quite > understand what he's saying: > > http://osdir.com/ml/python.pyrex/2006-06/msg00020.html > > Is there any way to successfully cimport Integer into that module? Or > do I need to just put the code that references Integer in some other > random place which is guaranteed to get loaded after > sage.rings.integer? (This would be a bit annoying....) > > I don't quite understand why circular cimports should exist, since I > thought these kinds of things got resolved at compile time. > > david > > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20080224/0b21d41e/attachment.htm From gfurnish at indirectproof.net Mon Feb 25 13:01:49 2008 From: gfurnish at indirectproof.net (Gary Furnish) Date: Mon, 25 Feb 2008 05:01:49 -0700 Subject: [Cython] circular cimports In-Reply-To: <706850310802242044hf719e32mad430dbcba75e784@mail.gmail.com> References: <706850310802242044hf719e32mad430dbcba75e784@mail.gmail.com> Message-ID: <706850310802250401q2a1e92eam485503b0d03edee1@mail.gmail.com> I was incorrect. The runtime issue is caused by imported by out of order module_init code. Namely, before all python (but not cython) modules in a circular import are done importing, it is possible for code to be executed that creates a instance of a class from one of the other modules whose module_init function has not been finished. The solution might seems to be to move the non-import nodes from module_init to another module_init2, and call module_init2 on each imported (but not cimported) module only after the current module has finished executing module_init. This should guarantee that any code that creates new instances of classes at the global module level only executes after all recursive imports have finished executing. Thus it should be possible to solve the cimport issue without any major compiler changes. -- Gary Furnish On 2/24/08, Bill Furnish wrote: > > There are actually two issues here which I am currently working on a patch > for. One, code generation for classes is based on order of imported > modules, not on the base class dependencies between the classes. I ended up > sorting them by dependencies and this fixed that issue. There is a second > issue I am still working on which is that cython does not always generate > correct module initialization code in the circular dependence case. This > results in the attribute error that is sometimes observed. It seems like > this arises from circular imports in the python code and not in the symbolic > code. I think that it arises because creating classes involves __init__ and > thus we end up with circular imports in python. I feel liike we ought to be > able to detect this somehow though and prevent it. However, the problem > seems to me to be that currently cython source files are compiled and > linked into separate objects. It may be necessary to implement a link time > solution to the issue by linking multiple cython files together and then > not load the modules that are linked together, but instead leave the > pointers to be relocated by the linker. > > On 2/24/08, David Harvey wrote: > > > > Hi, > > > > In Sage, if I try to add > > > > from sage.rings.integer cimport Integer > > > > at the top of sage/misc/misc_c.pyx, everything compiles fine, but > > then I get some weird import error when I start up sage. > > > > (I'm sorry, I don't know enough about cython to reproduce this in a > > standalone cython program, I only know how to make it happen in sage.) > > > > It looks like a "circular cimport problem". I found this post by Greg > > Ewing which seems relevant, but it's a bit old now, and I don't quite > > understand what he's saying: > > > > http://osdir.com/ml/python.pyrex/2006-06/msg00020.html > > > > Is there any way to successfully cimport Integer into that module? Or > > do I need to just put the code that references Integer in some other > > random place which is guaranteed to get loaded after > > sage.rings.integer? (This would be a bit annoying....) > > > > I don't quite understand why circular cimports should exist, since I > > thought these kinds of things got resolved at compile time. > > > > david > > > > _______________________________________________ > > Cython-dev mailing list > > Cython-dev at codespeak.net > > http://codespeak.net/mailman/listinfo/cython-dev > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20080225/62f4be58/attachment.htm From robertwb at math.washington.edu Mon Feb 25 18:12:26 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Mon, 25 Feb 2008 09:12:26 -0800 Subject: [Cython] minor bug -- cimport inside function In-Reply-To: References: Message-ID: <4D3F4F71-D71F-4BD1-A968-390D183CEA46@math.washington.edu> Thanks, I'll look into this. Note: it still isn't valid cython (as it doesn't make sense to cimport things locally, as cimports are resolved at compile time, not runtime. - Robert On Feb 24, 2008, at 8:11 PM, David Harvey wrote: > If I run cython on the following file: > > ======================= > def foo(): > from vial cimport poison > ======================= > > I get the following traceback: > > Traceback (most recent call last): > File "/Users/david/sage-2.10.2/local/bin/cython", line 8, in > > main(command_line = 1) > File "/Users/david/sage-2.10.2/local/lib/python2.5/site-packages/ > Cython/Compiler/Main.py", line 322, in main > result = context.compile(source, options) > File "/Users/david/sage-2.10.2/local/lib/python2.5/site-packages/ > Cython/Compiler/Main.py", line 198, in compile > tree.process_implementation(scope, options, result) > File "/Users/david/sage-2.10.2/local/lib/python2.5/site-packages/ > Cython/Compiler/ModuleNode.py", line 49, in process_implementation > self.generate_c_code(env, result) > File "/Users/david/sage-2.10.2/local/lib/python2.5/site-packages/ > Cython/Compiler/ModuleNode.py", line 219, in generate_c_code > self.body.generate_function_definitions(env, code) > File "/Users/david/sage-2.10.2/local/lib/python2.5/site-packages/ > Cython/Compiler/Nodes.py", line 674, in generate_function_definitions > self.body.analyse_declarations(lenv) > File "/Users/david/sage-2.10.2/local/lib/python2.5/site-packages/ > Cython/Compiler/Nodes.py", line 3331, in analyse_declarations > module_scope = env.find_module(self.module_name, self.pos) > AttributeError: LocalScope instance has no attribute 'find_module' > > david > > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev From wstein at gmail.com Mon Feb 25 18:15:11 2008 From: wstein at gmail.com (William Stein) Date: Mon, 25 Feb 2008 09:15:11 -0800 Subject: [Cython] minor bug -- cimport inside function In-Reply-To: <4D3F4F71-D71F-4BD1-A968-390D183CEA46@math.washington.edu> References: <4D3F4F71-D71F-4BD1-A968-390D183CEA46@math.washington.edu> Message-ID: <85e81ba30802250915k2fa4233cwa4e8664e48657596@mail.gmail.com> On Mon, Feb 25, 2008 at 9:12 AM, Robert Bradshaw wrote: > Thanks, I'll look into this. > > Note: it still isn't valid cython (as it doesn't make sense to > cimport things locally, as cimports are resolved at compile time, not > runtime. Mainly Cython should print an error rather than crashing with a stack trace. William From robertwb at math.washington.edu Mon Feb 25 18:26:15 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Mon, 25 Feb 2008 09:26:15 -0800 Subject: [Cython] except? In-Reply-To: <5dfb82150802242342s71c60ec0nc7f8123417a8c7f@mail.gmail.com> References: <5dfb82150802242342s71c60ec0nc7f8123417a8c7f@mail.gmail.com> Message-ID: :(. I'll look into this. On Feb 24, 2008, at 11:42 PM, Robert Miller wrote: > Robert, > > Just wondering, but the "except?" syntax doesn't seem to be quite > working with cpdef functions yet. It prints the error, but ignores > it... > > -- > Robert L. Miller > http://www.rlmiller.org/ > Department of Mathematics > University of Washington, Seattle From robertwb at math.washington.edu Mon Feb 25 20:41:59 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Mon, 25 Feb 2008 11:41:59 -0800 Subject: [Cython] except? In-Reply-To: <5dfb82150802242342s71c60ec0nc7f8123417a8c7f@mail.gmail.com> References: <5dfb82150802242342s71c60ec0nc7f8123417a8c7f@mail.gmail.com> Message-ID: <9C331AA1-BEC5-4D14-85C8-E40C4898577D@math.washington.edu> Seems to be working fine for me, can you give some example code? On Feb 24, 2008, at 11:42 PM, Robert Miller wrote: > Robert, > > Just wondering, but the "except?" syntax doesn't seem to be quite > working with cpdef functions yet. It prints the error, but ignores > it... > > -- > Robert L. Miller > http://www.rlmiller.org/ > Department of Mathematics > University of Washington, Seattle From niclas at chalmers.se Mon Feb 25 23:58:21 2008 From: niclas at chalmers.se (Niclas Mattsson) Date: Mon, 25 Feb 2008 23:58:21 +0100 Subject: [Cython] Installing Cython - compiling primes demo fails Message-ID: <47C3480D.9030006@chalmers.se> Hello listers, Newbie here, trying to get Cython running on Windows XP SP2 with ActivePython 2.5.1.1 using mingw32-gcc. Cython itself seems to install correctly (no warnings or errors from "python setup.py install"), but building the demos fails with "undefined reference" errors when linking primes.o. See dump below. Do I need to link to some missing library manually? What's going on? Please let me know if this is an inappropriate place for newbie questions - and if so, sorry for barging in! Thanks, Niclas ------------- C:\Cython-0.9.6.12\Demos>python Setup.py build_ext --inplace -c mingw32 running build_ext building 'primes' extension creating build creating build\temp.win32-2.5 creating build\temp.win32-2.5\Release c:\mingw\bin\gcc.exe -mno-cygwin -mdll -O -Wall -IC:\Python25\include -IC:\Python25\PC -c primes.c -o build\temp.win32-2.5\Release\primes.o primes.c: In function `__pyx_pf_6primes_primes': primes.c:125: warning: '__pyx_v_result' might be used uninitialized in this function writing build\temp.win32-2.5\Release\primes.def c:\mingw\bin\gcc.exe -mno-cygwin -shared -s build\temp.win32-2.5\Release\primes.o build\temp.win32-2.5\Release\primes.def -LC:\Python25\libs -LC:\Python25\PCbuild -lpython25 -lmsvcr71 -o primes.pyd build\temp.win32-2.5\Release\primes.o:primes.c:(.text+0x85): undefined reference to `_imp___Py_NoneStruct' build\temp.win32-2.5\Release\primes.o:primes.c:(.text+0x4c8): undefined reference to `_imp__PyExc_RuntimeError' build\temp.win32-2.5\Release\primes.o:primes.c:(.text+0x559): undefined reference to `_imp__PyString_Type' build\temp.win32-2.5\Release\primes.o:primes.c:(.text+0x560): undefined reference to `_imp__PyString_Type' build\temp.win32-2.5\Release\primes.o:primes.c:(.text+0x81f): undefined reference to `_imp__PyInt_Type' collect2: ld returned 1 exit status error: command 'gcc' failed with exit status 1 From robertwb at math.washington.edu Tue Feb 26 08:11:46 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Mon, 25 Feb 2008 23:11:46 -0800 Subject: [Cython] Installing Cython - compiling primes demo fails In-Reply-To: <47C3480D.9030006@chalmers.se> References: <47C3480D.9030006@chalmers.se> Message-ID: <9B5714FA-5CC3-4E29-A66D-46F5E8BA80F4@math.washington.edu> I don't have any experience with mingw32, but it does look like some kind of linking error. However, this thread might be able to point you in the right direction: http://osdir.com/ml/python.pyrex/2004-04/msg00028.html - Robert On Feb 25, 2008, at 2:58 PM, Niclas Mattsson wrote: > Hello listers, > > Newbie here, trying to get Cython running on Windows XP SP2 with > ActivePython 2.5.1.1 using mingw32-gcc. Cython itself seems to install > correctly (no warnings or errors from "python setup.py install"), but > building the demos fails with "undefined reference" errors when > linking > primes.o. See dump below. > > Do I need to link to some missing library manually? What's going on? > > Please let me know if this is an inappropriate place for newbie > questions - and if so, sorry for barging in! > > Thanks, > Niclas > > ------------- > > C:\Cython-0.9.6.12\Demos>python Setup.py build_ext --inplace -c > mingw32 > > running build_ext > building 'primes' extension > creating build > creating build\temp.win32-2.5 > creating build\temp.win32-2.5\Release > c:\mingw\bin\gcc.exe -mno-cygwin -mdll -O -Wall -IC:\Python25\include > -IC:\Python25\PC -c primes.c -o build\temp.win32-2.5\Release\primes.o > primes.c: In function `__pyx_pf_6primes_primes': > primes.c:125: warning: '__pyx_v_result' might be used uninitialized in > this function > writing build\temp.win32-2.5\Release\primes.def > c:\mingw\bin\gcc.exe -mno-cygwin -shared -s > build\temp.win32-2.5\Release\primes.o > build\temp.win32-2.5\Release\primes.def -LC:\Python25\libs > -LC:\Python25\PCbuild -lpython25 -lmsvcr71 -o primes.pyd > build\temp.win32-2.5\Release\primes.o:primes.c:(.text+0x85): undefined > reference to `_imp___Py_NoneStruct' > build\temp.win32-2.5\Release\primes.o:primes.c:(.text+0x4c8): > undefined > reference to `_imp__PyExc_RuntimeError' > build\temp.win32-2.5\Release\primes.o:primes.c:(.text+0x559): > undefined > reference to `_imp__PyString_Type' > build\temp.win32-2.5\Release\primes.o:primes.c:(.text+0x560): > undefined > reference to `_imp__PyString_Type' > build\temp.win32-2.5\Release\primes.o:primes.c:(.text+0x81f): > undefined > reference to `_imp__PyInt_Type' > collect2: ld returned 1 exit status > error: command 'gcc' failed with exit status 1 > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev From ndbecker2 at gmail.com Tue Feb 26 17:09:37 2008 From: ndbecker2 at gmail.com (Neal Becker) Date: Tue, 26 Feb 2008 11:09:37 -0500 Subject: [Cython] cython-0.9.6.12 Demo failure? Message-ID: newb here. cd /home/nbecker/Cython-0.9.6.12/Demos/ make python Setup.py build_ext --inplace ... error: file 'submodule/test.pyx' does not exist make: *** [all] Error 1 From robertwb at math.washington.edu Tue Feb 26 18:30:52 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Tue, 26 Feb 2008 09:30:52 -0800 Subject: [Cython] cython-0.9.6.12 Demo failure? In-Reply-To: References: Message-ID: The file might not actually be in your release, just delete the corresponding line from Setup.py. (I've done the same so we won't have this bug in the next release.) I was using the Demos directory for testing, but now we have a much better framework so we can clean that up a lot. - Robert On Feb 26, 2008, at 8:09 AM, Neal Becker wrote: > newb here. > cd /home/nbecker/Cython-0.9.6.12/Demos/ > make > python Setup.py build_ext --inplace > ... > error: file 'submodule/test.pyx' does not exist > make: *** [all] Error 1 > > > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev From niclas at chalmers.se Tue Feb 26 18:41:36 2008 From: niclas at chalmers.se (Niclas Mattsson) Date: Tue, 26 Feb 2008 18:41:36 +0100 Subject: [Cython] Installing Cython - compiling primes demo fails - SOLVED In-Reply-To: <9B5714FA-5CC3-4E29-A66D-46F5E8BA80F4@math.washington.edu> References: <47C3480D.9030006@chalmers.se> <9B5714FA-5CC3-4E29-A66D-46F5E8BA80F4@math.washington.edu> Message-ID: <47C44F50.6030601@chalmers.se> Thanks a lot Robert, creating libpython25.a from C:\Windows\system32\python25.dll and configuring distutils.cfg as suggested in that post solved my problem! Link repeated here: http://osdir.com/ml/python.pyrex/2004-04/msg00028.html Note to future newbies with the same problem: the addendum from 2004-04-29 near the bottom of http://sebsauvage.net/python/mingw.html is incorrect (at least for me). You need to create libpython25.a, just copying python25.dll to C:\Python25\libs is not enough. Niclas Robert Bradshaw wrote: > I don't have any experience with mingw32, but it does look like some > kind of linking error. > > However, this thread might be able to point you in the right direction: > > http://osdir.com/ml/python.pyrex/2004-04/msg00028.html > > - Robert > > > On Feb 25, 2008, at 2:58 PM, Niclas Mattsson wrote: > >> Hello listers, >> >> Newbie here, trying to get Cython running on Windows XP SP2 with >> ActivePython 2.5.1.1 using mingw32-gcc. Cython itself seems to install >> correctly (no warnings or errors from "python setup.py install"), but >> building the demos fails with "undefined reference" errors when linking >> primes.o. See dump below. >> >> Do I need to link to some missing library manually? What's going on? >> >> Please let me know if this is an inappropriate place for newbie >> questions - and if so, sorry for barging in! >> >> Thanks, >> Niclas >> >> ------------- >> >> C:\Cython-0.9.6.12\Demos>python Setup.py build_ext --inplace -c mingw32 >> >> running build_ext >> building 'primes' extension >> creating build >> creating build\temp.win32-2.5 >> creating build\temp.win32-2.5\Release >> c:\mingw\bin\gcc.exe -mno-cygwin -mdll -O -Wall -IC:\Python25\include >> -IC:\Python25\PC -c primes.c -o build\temp.win32-2.5\Release\primes.o >> primes.c: In function `__pyx_pf_6primes_primes': >> primes.c:125: warning: '__pyx_v_result' might be used uninitialized in >> this function >> writing build\temp.win32-2.5\Release\primes.def >> c:\mingw\bin\gcc.exe -mno-cygwin -shared -s >> build\temp.win32-2.5\Release\primes.o >> build\temp.win32-2.5\Release\primes.def -LC:\Python25\libs >> -LC:\Python25\PCbuild -lpython25 -lmsvcr71 -o primes.pyd >> build\temp.win32-2.5\Release\primes.o:primes.c:(.text+0x85): undefined >> reference to `_imp___Py_NoneStruct' >> build\temp.win32-2.5\Release\primes.o:primes.c:(.text+0x4c8): undefined >> reference to `_imp__PyExc_RuntimeError' >> build\temp.win32-2.5\Release\primes.o:primes.c:(.text+0x559): undefined >> reference to `_imp__PyString_Type' >> build\temp.win32-2.5\Release\primes.o:primes.c:(.text+0x560): undefined >> reference to `_imp__PyString_Type' >> build\temp.win32-2.5\Release\primes.o:primes.c:(.text+0x81f): undefined >> reference to `_imp__PyInt_Type' >> collect2: ld returned 1 exit status >> error: command 'gcc' failed with exit status 1 >> _______________________________________________ >> Cython-dev mailing list >> Cython-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/cython-dev > > From ndbecker2 at gmail.com Tue Feb 26 18:53:49 2008 From: ndbecker2 at gmail.com (Neal Becker) Date: Tue, 26 Feb 2008 12:53:49 -0500 Subject: [Cython] cython-0.9.6.12 4 tests fail Message-ID: Is this expected? FAIL: Doctest: unpacklistcomp ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib64/python2.5/doctest.py", line 2112, in runTest raise self.failureException(self.format_failure(new.getvalue())) AssertionError: Failed doctest test for unpacklistcomp File "/home/nbecker/Cython-0.9.6.12/BUILD/unpacklistcomp.so", line 80, in unpacklistcomp ---------------------------------------------------------------------- File "/home/nbecker/Cython-0.9.6.12/BUILD/unpacklistcomp.so", line 84, in unpacklistcomp Failed example: unpack_normal([1,2,3]) Expected: Traceback (most recent call last): ValueError: too many values to unpack Got: Traceback (most recent call last): File "/usr/lib64/python2.5/doctest.py", line 1212, in __run compileflags, 1) in test.globs File "", line 1, in unpack_normal([1,2,3]) File "unpacklistcomp.pyx", line 22, in unpacklistcomp.unpack_normal ValueError: unpack sequence of wrong size ---------------------------------------------------------------------- File "/home/nbecker/Cython-0.9.6.12/BUILD/unpacklistcomp.so", line 90, in unpacklistcomp Failed example: unpack_comp([1,2,3]) Expected: Traceback (most recent call last): ValueError: too many values to unpack Got: Traceback (most recent call last): File "/usr/lib64/python2.5/doctest.py", line 1212, in __run compileflags, 1) in test.globs File "", line 1, in unpack_comp([1,2,3]) File "unpacklistcomp.pyx", line 26, in unpacklistcomp.unpack_comp ValueError: unpack sequence of wrong size ---------------------------------------------------------------------- File "/home/nbecker/Cython-0.9.6.12/BUILD/unpacklistcomp.so", line 96, in unpacklistcomp Failed example: unpack_expr([1,2,3]) Expected: Traceback (most recent call last): ValueError: too many values to unpack Got: Traceback (most recent call last): File "/usr/lib64/python2.5/doctest.py", line 1212, in __run compileflags, 1) in test.globs File "", line 1, in unpack_expr([1,2,3]) File "unpacklistcomp.pyx", line 30, in unpacklistcomp.unpack_expr ValueError: unpack sequence of wrong size ---------------------------------------------------------------------- Ran 303 tests in 0.382s FAILED (failures=4) From robertwb at math.washington.edu Tue Feb 26 18:58:13 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Tue, 26 Feb 2008 09:58:13 -0800 Subject: [Cython] cython-0.9.6.12 4 tests fail In-Reply-To: References: Message-ID: <9DD6B091-C0B5-4647-8B67-1DEDB81EE9EF@math.washington.edu> This is the correct error, but the actual message is different depending on the version of Python. In Python 2.3 it is "ValueError: unpack tuple of wrong size" but in Python 2.5 it is "ValueError: too many values to unpack." Note sure how to best test for this. - Robert On Feb 26, 2008, at 9:53 AM, Neal Becker wrote: > Is this expected? > > FAIL: Doctest: unpacklistcomp > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/usr/lib64/python2.5/doctest.py", line 2112, in runTest > raise self.failureException(self.format_failure(new.getvalue())) > AssertionError: Failed doctest test for unpacklistcomp > File "/home/nbecker/Cython-0.9.6.12/BUILD/unpacklistcomp.so", > line 80, in unpacklistcomp > > ---------------------------------------------------------------------- > File "/home/nbecker/Cython-0.9.6.12/BUILD/unpacklistcomp.so", line > 84, in unpacklistcomp > Failed example: > unpack_normal([1,2,3]) > Expected: > Traceback (most recent call last): > ValueError: too many values to unpack > Got: > Traceback (most recent call last): > File "/usr/lib64/python2.5/doctest.py", line 1212, in __run > compileflags, 1) in test.globs > File "", line 1, in > unpack_normal([1,2,3]) > File "unpacklistcomp.pyx", line 22, in > unpacklistcomp.unpack_normal > ValueError: unpack sequence of wrong size > ---------------------------------------------------------------------- > File "/home/nbecker/Cython-0.9.6.12/BUILD/unpacklistcomp.so", line > 90, in unpacklistcomp > Failed example: > unpack_comp([1,2,3]) > Expected: > Traceback (most recent call last): > ValueError: too many values to unpack > Got: > Traceback (most recent call last): > File "/usr/lib64/python2.5/doctest.py", line 1212, in __run > compileflags, 1) in test.globs > File "", line 1, in > unpack_comp([1,2,3]) > File "unpacklistcomp.pyx", line 26, in > unpacklistcomp.unpack_comp > ValueError: unpack sequence of wrong size > ---------------------------------------------------------------------- > File "/home/nbecker/Cython-0.9.6.12/BUILD/unpacklistcomp.so", line > 96, in unpacklistcomp > Failed example: > unpack_expr([1,2,3]) > Expected: > Traceback (most recent call last): > ValueError: too many values to unpack > Got: > Traceback (most recent call last): > File "/usr/lib64/python2.5/doctest.py", line 1212, in __run > compileflags, 1) in test.globs > File "", line 1, in > unpack_expr([1,2,3]) > File "unpacklistcomp.pyx", line 30, in > unpacklistcomp.unpack_expr > ValueError: unpack sequence of wrong size > > > ---------------------------------------------------------------------- > Ran 303 tests in 0.382s > > FAILED (failures=4) > > > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev From stefan_ml at behnel.de Tue Feb 26 21:47:14 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 26 Feb 2008 21:47:14 +0100 Subject: [Cython] cython-0.9.6.12 4 tests fail In-Reply-To: <9DD6B091-C0B5-4647-8B67-1DEDB81EE9EF@math.washington.edu> References: <9DD6B091-C0B5-4647-8B67-1DEDB81EE9EF@math.washington.edu> Message-ID: <47C47AD2.1010403@behnel.de> Robert Bradshaw wrote: > On Feb 26, 2008, at 9:53 AM, Neal Becker wrote: >> FAIL: Doctest: unpacklistcomp >> ---------------------------------------------------------------------- >> File "/home/nbecker/Cython-0.9.6.12/BUILD/unpacklistcomp.so", line >> 84, in unpacklistcomp >> Failed example: >> unpack_normal([1,2,3]) >> Expected: >> Traceback (most recent call last): >> ValueError: too many values to unpack >> Got: >> Traceback (most recent call last): >> File "/usr/lib64/python2.5/doctest.py", line 1212, in __run >> compileflags, 1) in test.globs >> File "", line 1, in >> unpack_normal([1,2,3]) >> File "unpacklistcomp.pyx", line 22, in >> unpacklistcomp.unpack_normal >> ValueError: unpack sequence of wrong size > > This is the correct error, but the actual message is different > depending on the version of Python. In Python 2.3 it is "ValueError: > unpack tuple of wrong size" but in Python 2.5 it is "ValueError: too > many values to unpack." > > Note sure how to best test for this. You can add "# doctest: +ELLIPSIS" after the offending line and replace the exception message with "...", like this: >>> unpack_normal([1,2,3]) # doctest: +ELLIPSIS Traceback (most recent call last): ValueError: ... Stefan From ndbecker2 at gmail.com Wed Feb 27 02:58:53 2008 From: ndbecker2 at gmail.com (Neal Becker) Date: Tue, 26 Feb 2008 20:58:53 -0500 Subject: [Cython] [newb] Howto build 'rectangle' demo Message-ID: I'm trying to follow: http://wiki.cython.org/WrappingCPlusPlus What am I doing wrong? ===== Rectangle.h class Rectangle { public: int x0, y0, x1, y1; Rectangle(int x0, int y0, int x1, int y1); ~Rectangle(); int getLength(); int getHeight(); int getArea(); void move(int dx, int dy); }; ===== rectangle.pyx cdef extern from "Rectangle.h": ctypedef struct c_Rectangle "Rectangle": int x0, y0, x1, y1 int (*getLength)() int (*getHeight)() int (*getArea)() void (*move)(int dx, int dy) c_Rectangle *new_Rectangle "new Rectangle" (int x0, int y0, int x1, int y1) void del_Rectagle "delete" (c_Rectangle *rect) cdef class Rectangle: c_Rectangle *thisptr # hold a C++ instance which we're wrapping def __cinit__(self, int x0, int y0, int x1, int y1): self.thisptr = new_Rectangle(x0, y0, x1, y1) def __dealloc__(self): del_Rectangle(self.thisptr) def getLength(self): return self.thisptr.getLength() def getHeight(self): return self.thisptr.getHeight() def getArea(self): return self.thisptr.getArea() def move(self, dx, dy): self.thisptr.move(dx, dy) ========= cython rectangle.pyx Error converting Pyrex file to C: ------------------------------------------------------------ ... void (*move)(int dx, int dy) c_Rectangle *new_Rectangle "new Rectangle" (int x0, int y0, int x1, int y1) void del_Rectagle "delete" (c_Rectangle *rect) cdef class Rectangle: c_Rectangle *thisptr # hold a C++ instance which we're wrapping ^ ------------------------------------------------------------ /home/nbecker/cython/rectangle.pyx:12:16: 'c_Rectangle' is not a constant, variable or function identifier ... more errors... From robertwb at math.washington.edu Wed Feb 27 03:49:53 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Tue, 26 Feb 2008 18:49:53 -0800 Subject: [Cython] [newb] Howto build 'rectangle' demo In-Reply-To: References: Message-ID: <733A0725-146E-48FC-A265-C8D40838DC59@math.washington.edu> On Feb 26, 2008, at 5:58 PM, Neal Becker wrote: > I'm trying to follow: > http://wiki.cython.org/WrappingCPlusPlus > What am I doing wrong? > > ===== Rectangle.h > class Rectangle { > public: > int x0, y0, x1, y1; > Rectangle(int x0, int y0, int x1, int y1); > ~Rectangle(); > int getLength(); > int getHeight(); > int getArea(); > void move(int dx, int dy); > }; > > ===== rectangle.pyx > cdef extern from "Rectangle.h": > ctypedef struct c_Rectangle "Rectangle": > int x0, y0, x1, y1 > int (*getLength)() > int (*getHeight)() > int (*getArea)() > void (*move)(int dx, int dy) > c_Rectangle *new_Rectangle "new Rectangle" (int x0, int y0, int > x1, int y1) > void del_Rectagle "delete" (c_Rectangle *rect) > > cdef class Rectangle: > c_Rectangle *thisptr # hold a C++ instance which we're > wrapping This line should be "cdef c_Rectangle *thisptr # ..." cdef is the keyword that says "the following is not real Python" > def __cinit__(self, int x0, int y0, int x1, int y1): > self.thisptr = new_Rectangle(x0, y0, x1, y1) > def __dealloc__(self): > del_Rectangle(self.thisptr) > def getLength(self): > return self.thisptr.getLength() > def getHeight(self): > return self.thisptr.getHeight() > def getArea(self): > return self.thisptr.getArea() > def move(self, dx, dy): > self.thisptr.move(dx, dy) > ========= > > > cython rectangle.pyx > > Error converting Pyrex file to C: > ------------------------------------------------------------ > ... > void (*move)(int dx, int dy) > c_Rectangle *new_Rectangle "new Rectangle" (int x0, int y0, int > x1, int y1) > void del_Rectagle "delete" (c_Rectangle *rect) > > cdef class Rectangle: > c_Rectangle *thisptr # hold a C++ instance which we're > wrapping > ^ > ------------------------------------------------------------ > > /home/nbecker/cython/rectangle.pyx:12:16: 'c_Rectangle' is not a > constant, variable or function identifier > ... more errors... > > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev From ndbecker2 at gmail.com Thu Feb 28 13:39:08 2008 From: ndbecker2 at gmail.com (Neal Becker) Date: Thu, 28 Feb 2008 07:39:08 -0500 Subject: [Cython] howto include initialization code Message-ID: I'm playing with the matrix example. cdef extern from "numpy/arrayobject.h": ctypedef int intp ctypedef extern class numpy.ndarray [object PyArrayObject]: cdef char *data cdef int nd cdef intp *dimensions cdef intp *strides cdef int flags def mysum(ndarray a): cdef double *p = a.data cdef int dim = a.dimensions[0] cdef double s=0 for i from 0 <= i < dim: s += p[i] p[0] = 13. return s /usr/lib64/python2.5/site-packages/numpy/core/include/numpy/__multiarray_api.h:948: warning: 'int _import_array()' defined but not used This is going to segfault. In python module init, it must say 'import_array();' How do I cause this to get added to the C module init? From robert.kern at gmail.com Thu Feb 28 17:15:25 2008 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 28 Feb 2008 10:15:25 -0600 Subject: [Cython] howto include initialization code In-Reply-To: References: Message-ID: <3d375d730802280815t3328f51em61a30f905e783620@mail.gmail.com> On Thu, Feb 28, 2008 at 6:39 AM, Neal Becker wrote: > I'm playing with the matrix example. > > cdef extern from "numpy/arrayobject.h": > > ctypedef int intp > > ctypedef extern class numpy.ndarray [object PyArrayObject]: > cdef char *data > cdef int nd > cdef intp *dimensions > cdef intp *strides > cdef int flags > > def mysum(ndarray a): > cdef double *p = a.data > cdef int dim = a.dimensions[0] > > cdef double s=0 > for i from 0 <= i < dim: > s += p[i] > p[0] = 13. > return s > > /usr/lib64/python2.5/site-packages/numpy/core/include/numpy/__multiarray_api.h:948: > warning: 'int _import_array()' defined but not used > > This is going to segfault. In python module init, it must > say 'import_array();' > > How do I cause this to get added to the C module init? cdef extern from "numpy/arrayobject.h": void import_array() import_array() -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ndbecker2 at gmail.com Tue Feb 26 20:08:32 2008 From: ndbecker2 at gmail.com (Neal Becker) Date: Tue, 26 Feb 2008 14:08:32 -0500 Subject: [Cython] [igraph] 1 test fails on ppc64 gcc-4.3 In-Reply-To: <2A945F5D-D006-4BDA-9C7C-B074ADB48AFF@gmail.com> References: <200802260939.22212.ndbecker2@gmail.com> <200802261357.10309.ndbecker2@gmail.com> <2A945F5D-D006-4BDA-9C7C-B074ADB48AFF@gmail.com> Message-ID: <200802261408.32323.ndbecker2@gmail.com> On Tuesday 26 February 2008, Tamas Nepusz wrote: > > _______________________________________________ > > Maybe this is the wrong file... this one passed the Erdos-Renyi test > and failed the spinglass clustering. Sorry, try this: -------------- next part -------------- A non-text attachment was scrubbed... Name: getfile.zip Type: application/x-zip Size: 19920 bytes Desc: not available Url : http://codespeak.net/pipermail/cython-dev/attachments/20080226/b7073119/attachment-0001.bin