From robertwb at math.washington.edu Wed Jul 2 06:49:41 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Tue, 1 Jul 2008 21:49:41 -0700 Subject: [Cython] .pxd's that depend on other .pxd's In-Reply-To: <20080630151202.GA4534@tilt> References: <20080630151202.GA4534@tilt> Message-ID: Did this work in a former version of Cython? This may be an issue with the way the __init__ stuff was merge in. A short-term workaround would be to just use relative imports (i.e. "from translatable cimport Translatable" for now. - Robert On Jun 30, 2008, at 8:12 AM, Peter Todd wrote: > Using the attached tarball I get this error message with Cython 0.9.8: > > > pete at tilt:~/t/bug$ cython Tuke/id.pyx > > Error converting Pyrex file to C: > ------------------------------------------------------------ > ... > from Tuke.translatable cimport Translatable > ^ > ------------------------------------------------------------ > > /home/pete/t/bug/Tuke/id.pyx:1:0: 'Tuke.translatable.pxd' not found > > Error converting Pyrex file to C: > ------------------------------------------------------------ > ... > from Tuke.translatable cimport Translatable > ^ > ------------------------------------------------------------ > > /home/pete/t/bug/Tuke/id.pyx:1:31: Name 'Translatable' not declared > in module 'Tuke.translatable' > > Error converting Pyrex file to C: > ------------------------------------------------------------ > ... > from Tuke.translatable cimport Translatable > > cdef class Id(Translatable): > ^ > ------------------------------------------------------------ > > /home/pete/t/bug/Tuke/id.pyx:3:5: 'Translatable' is not declared > pete at tilt:~/t/bug$ > > Is this correct behavior? The files are pretty simple, I just want > the class in > id.pyx to be a sub-class of something in translatable.pxd > > -- > http://petertodd.org 'peter'[:-1]@petertodd.org > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 186 bytes Desc: This is a digitally signed message part Url : http://codespeak.net/pipermail/cython-dev/attachments/20080701/b5c81024/attachment.pgp From stefan_ml at behnel.de Wed Jul 2 08:03:18 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 02 Jul 2008 08:03:18 +0200 Subject: [Cython] .pxd's that depend on other .pxd's In-Reply-To: <20080630151202.GA4534@tilt> References: <20080630151202.GA4534@tilt> Message-ID: <486B1A26.401@behnel.de> Hi, Peter Todd wrote: > Using the attached tarball I get this error message with Cython 0.9.8: > > > pete at tilt:~/t/bug$ cython Tuke/id.pyx > /home/pete/t/bug/Tuke/id.pyx:1:0: 'Tuke.translatable.pxd' not found Do you have an __init__.py in the Tuke package? Stefan From dagss at student.matnat.uio.no Wed Jul 2 11:00:14 2008 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Wed, 02 Jul 2008 11:00:14 +0200 Subject: [Cython] Best way to fake buffers in Py2? Message-ID: <486B439E.7060805@student.matnat.uio.no> Part of my buffer proposal is to provide our own implementation of it for Python 2 (I'm unsure about the status for 2.6, it seems like not the whole buffer API might be backported). For now I'll just do whatever will get us something to play with, but what do people think is a long-term solution for this? What is needed is that for Python 2 is that we bundle something like this (psuedo-code): #ifdef Py2 PyAPI_FUNC(int) PyObject_GetBuffer(PyObject *obj, Py_buffer *view, int flags) { if (numpy is available && obj instanceof numpy.ndarray) { view->buf = obj->data; view->strides = obj->strides; view->shape = obj->shape; ... } else if (PIL is available && obj instanceof Imaging.Image) { ... } ... other buffers ... } #endif Now the question is, how do I best do this? The "X is available" (and if so, #include its headers...) is the tricky part. I'm thinking along the lines of having a new special attribute on the classes in question, something like this: cdef extern from "cythonbufferbackwardscompat.h": cdef int NumPyGetBuffer(PyObject*, Py_buffer*, int) cdef int NumPyReleaseBuffer(PyObject*, Py_buffer*) cdef class numpy.ndarray ... : __cython_py2_getbuffer__ = "NumPyGetBuffer" __cython_py2_releasebuffer__ = "NumPyReleaseBuffer" And then generate a pyx-file-specific local PyObject_GetBuffer depending on the __cython_py2_buffer__ we can find: #ifdef Py2 static int PyObject_GetBuffer(PyObject *, Py_buffer *, int); #endif ... code using PyObject_GetBuffer ... #ifdef Py2 PyAPI_FUNC(int) PyObject_GetBuffer(PyObject *obj, Py_buffer *view, int flags) { if (obj is numpy.ndarray) { return NumPyGetBuffer(obj, view, flags); } else if ... } else { raise not supported-exception } } #endif Thoughts? A disadvantage is that "cythonbufferbackwardscompat.h": (and .c?) must be provided, but inlineable-code-in-pxds can help with that on a later date (Robert: This is something I'll push past midterm-eval if needed; if so I'll just make it work by hard-coding NumPy-specific stuff directly). -- Dag Sverre From dagss at student.matnat.uio.no Wed Jul 2 11:02:03 2008 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Wed, 02 Jul 2008 11:02:03 +0200 Subject: [Cython] Best way to fake buffers in Py2? In-Reply-To: <486B439E.7060805@student.matnat.uio.no> References: <486B439E.7060805@student.matnat.uio.no> Message-ID: <486B440B.6090501@student.matnat.uio.no> Dag Sverre Seljebotn wrote: > And then generate a pyx-file-specific local PyObject_GetBuffer depending > on the __cython_py2_buffer__ we can find: > > #ifdef Py2 > static int PyObject_GetBuffer(PyObject *, Py_buffer *, int); > #endif > > ... code using PyObject_GetBuffer ... > > #ifdef Py2 > PyAPI_FUNC(int) PyObject_GetBuffer(PyObject *obj, Py_buffer *view, > int flags) { Sorry, this should have been declared static int as well. -- Dag Sverre From robertwb at math.washington.edu Wed Jul 2 11:18:45 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Wed, 2 Jul 2008 02:18:45 -0700 Subject: [Cython] Best way to fake buffers in Py2? In-Reply-To: <486B439E.7060805@student.matnat.uio.no> References: <486B439E.7060805@student.matnat.uio.no> Message-ID: <6AD216CF-BF97-42CF-AAFA-5E19CF77B2BD@math.washington.edu> On Jul 2, 2008, at 2:00 AM, Dag Sverre Seljebotn wrote: > Part of my buffer proposal is to provide our own implementation of it > for Python 2 (I'm unsure about the status for 2.6, it seems like > not the > whole buffer API might be backported). For now I'll just do whatever > will get us something to play with, but what do people think is a > long-term solution for this? > > What is needed is that for Python 2 is that we bundle something like > this (psuedo-code): > > #ifdef Py2 > PyAPI_FUNC(int) PyObject_GetBuffer(PyObject *obj, Py_buffer *view, > int flags) { > > if (numpy is available && obj instanceof numpy.ndarray) { > view->buf = obj->data; > view->strides = obj->strides; > view->shape = obj->shape; > ... > } else if (PIL is available && obj instanceof Imaging.Image) { > ... > } ... other buffers ... > > } > #endif That looks good. > Now the question is, how do I best do this? The "X is > available" (and if > so, #include its headers...) is the tricky part. I'm thinking along > the > lines of having a new special attribute on the classes in question, > something like this: > > cdef extern from "cythonbufferbackwardscompat.h": > cdef int NumPyGetBuffer(PyObject*, Py_buffer*, int) > cdef int NumPyReleaseBuffer(PyObject*, Py_buffer*) > > cdef class numpy.ndarray ... : > __cython_py2_getbuffer__ = "NumPyGetBuffer" > __cython_py2_releasebuffer__ = "NumPyReleaseBuffer" > > And then generate a pyx-file-specific local PyObject_GetBuffer > depending > on the __cython_py2_buffer__ we can find: > > #ifdef Py2 > static int PyObject_GetBuffer(PyObject *, Py_buffer *, int); > #endif > > ... code using PyObject_GetBuffer ... > > #ifdef Py2 > PyAPI_FUNC(int) PyObject_GetBuffer(PyObject *obj, Py_buffer *view, > int flags) { > > if (obj is numpy.ndarray) { > return NumPyGetBuffer(obj, view, flags); > } else if ... > > } else { raise not supported-exception } > > } > #endif > > > Thoughts? A disadvantage is that "cythonbufferbackwardscompat.h": (and > .c?) must be provided, but inlineable-code-in-pxds can help with > that on > a later date (Robert: This is something I'll push past midterm-eval if > needed; if so I'll just make it work by hard-coding NumPy-specific > stuff > directly). Yes, hard coding a NumPy-specific buffer function is fine at this point. Inlineable-code-in-pxds should be able to make this much cleaner: cdef class numpy.ndarray ... : cdef inline __cython_py2_getbuffer__(PyObject* self, Py_buffer* buf, int k): ... cdef inline __cython_py2_releasebuffer__(PyObject* self, Py_buffer* buf, int k): .... (though those particular names seem overly verbose...why not just __getbuffer__ and __releasebuffer__, maybe with a decorator to specify py2 only) - Robert From fperez.net at gmail.com Wed Jul 2 23:50:04 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 2 Jul 2008 14:50:04 -0700 Subject: [Cython] Is this a bug in ye'ole cython? Message-ID: Hi folks, I'm wondering if this is expected behavior. Consider some trivial little class like class Simpleton: def __str__(self): return "A simpleton" def incr(self,x): """Increment x by one. Examples: >>> s = Simpleton() >>> s.incr(1) 2 >>> s.incr(10) 12 """ return x+1 If I build this thing in cython, the Simpleton.incr.__module__ attribute is set to None, while if I make it a pure python module, it's correctly set to the module name: In [18]: p sprimes.primes.Simpleton.incr.__module__ None In [19]: p sprimes.pyprimes.Simpleton.incr.__module__ sprimes.pyprimes This difference causes the doctest to miss any doctest examples that might be included in the methods. Note that for *top-level* functions in the extension module, the __module__ attribute is correctly set: In [21]: p sprimes.primes.primes In [22]: p sprimes.primes.primes.__module__ sprimes.primes so the problem appears to be only for classes. This is running cython 0.9.8. Thanks for any feedback, f From robertwb at math.washington.edu Thu Jul 3 03:19:37 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Wed, 2 Jul 2008 18:19:37 -0700 Subject: [Cython] Is this a bug in ye'ole cython? In-Reply-To: References: Message-ID: No, this sounds like a bug. On Jul 2, 2008, at 2:50 PM, Fernando Perez wrote: > Hi folks, > > I'm wondering if this is expected behavior. Consider some trivial > little class like > > class Simpleton: > def __str__(self): > return "A simpleton" > > def incr(self,x): > """Increment x by one. > > Examples: > >>>> s = Simpleton() >>>> s.incr(1) > 2 >>>> s.incr(10) > 12 > """ > return x+1 > > If I build this thing in cython, the Simpleton.incr.__module__ > attribute is set to None, while if I make it a pure python module, > it's correctly set to the module name: > > > In [18]: p sprimes.primes.Simpleton.incr.__module__ > None > > In [19]: p sprimes.pyprimes.Simpleton.incr.__module__ > sprimes.pyprimes > > This difference causes the doctest to miss any doctest examples that > might be included in the methods. > > Note that for *top-level* functions in the extension module, the > __module__ attribute is correctly set: > > In [21]: p sprimes.primes.primes > > > In [22]: p sprimes.primes.primes.__module__ > sprimes.primes > > so the problem appears to be only for classes. > > This is running cython 0.9.8. > > Thanks for any feedback, > > f > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev From fperez.net at gmail.com Thu Jul 3 07:40:37 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 2 Jul 2008 22:40:37 -0700 Subject: [Cython] Is this a bug in ye'ole cython? In-Reply-To: References: Message-ID: On Wed, Jul 2, 2008 at 6:19 PM, Robert Bradshaw wrote: > No, this sounds like a bug. OK, thanks. I was able to work around it for my purposes (doctesting of extension code using nose) so it's no biggie for me, but it's still probably worth fixing. Cheers, f From dagss at student.matnat.uio.no Fri Jul 4 23:46:25 2008 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Fri, 04 Jul 2008 23:46:25 +0200 Subject: [Cython] Buffer must know exact C types Message-ID: <486E9A31.7040300@student.matnat.uio.no> Currently, wherever a C type is used in Cython, it is enough to specify sort-of what kind of class of type it is... the following will work all right: cdef extern ...: ctypedef int int8 ctypedef int int16 ctypedef int int32 ctypedef int int64 However, when types are used in buffers: cdef object[int16, 3] x one must translate the type (dtype) into a character representation (as listed in http://docs.python.org/lib/module-struct.html). For instance, signed short is "h" and signed int is "i"; and one can imagine int16 to be any of these (which one being decided by the included header-file; NumPy comes with a long list defining intXX for a lot of cases; you can choose whether to use a C-dependant type like "npy_longlong" or a fixed type like "npy_int64"). I see two options: a) Start requiring exact ctypedefs in Cython; at least to get correct buffer behaviour. However, for NumPy this would require a lot of "#ifdefs" Cython-side. (NumPy comes with two options for each type, b) "Somehow" get the C compiler to map the C-compiler-resolved type to the char. With C++ this would be easy: template struct TypeChar {}; template <> struct TypeChar { static const char value = 'h'; }; template <> struct TypeChar { static const char value = 'i'; }; int main() { int t; cout << TypeChar::value; } however I'm not sure if any tricks are possible in C to get something like this? -- Dag Sverre From dagss at student.matnat.uio.no Fri Jul 4 23:49:33 2008 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Fri, 04 Jul 2008 23:49:33 +0200 Subject: [Cython] Buffer must know exact C types In-Reply-To: <486E9A31.7040300@student.matnat.uio.no> References: <486E9A31.7040300@student.matnat.uio.no> Message-ID: <486E9AED.2000302@student.matnat.uio.no> Dag Sverre Seljebotn wrote: > int main() { > int t; > cout << TypeChar::value; > } > > however I'm not sure if any tricks are possible in C to get something > like this? > To be clear, C++ would then also allow this: typedef int foo; int main() { foo t; cout << TypeChar::value; } and so, one wouldn't need an "exact" foo ctypedef in Cython, one would just use the template to map the type to the character when needed... -- Dag Sverre From stefan_ml at behnel.de Sat Jul 5 07:11:34 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 05 Jul 2008 07:11:34 +0200 Subject: [Cython] Buffer must know exact C types In-Reply-To: <486E9A31.7040300@student.matnat.uio.no> References: <486E9A31.7040300@student.matnat.uio.no> Message-ID: <486F0286.7060105@behnel.de> Hi, Dag Sverre Seljebotn wrote: > Currently, wherever a C type is used in Cython, it is enough to specify > sort-of what kind of class of type it is... the following will work all > right: > > cdef extern ...: > ctypedef int int8 > ctypedef int int16 > ctypedef int int32 > ctypedef int int64 > > > However, when types are used in buffers: > > cdef object[int16, 3] x > > one must translate the type (dtype) into a character representation (as > listed in http://docs.python.org/lib/module-struct.html). What about going the opposite way and either a) replacing the type definition in buffers by the type character entirely (not sure, but this might not work if you require a specific type name to appear in the C code), or b) by requiring users to provide both the type and the character in a suitable buffer declaration syntax? Stefan From dagss at student.matnat.uio.no Sat Jul 5 11:09:46 2008 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Sat, 05 Jul 2008 11:09:46 +0200 Subject: [Cython] Buffer must know exact C types In-Reply-To: <486F0286.7060105@behnel.de> References: <486E9A31.7040300@student.matnat.uio.no> <486F0286.7060105@behnel.de> Message-ID: <486F3A5A.9050204@student.matnat.uio.no> Stefan Behnel wrote: > Hi, > > Dag Sverre Seljebotn wrote: >> Currently, wherever a C type is used in Cython, it is enough to specify >> sort-of what kind of class of type it is... the following will work all >> right: >> >> cdef extern ...: >> ctypedef int int8 >> ctypedef int int16 >> ctypedef int int32 >> ctypedef int int64 >> >> >> However, when types are used in buffers: >> >> cdef object[int16, 3] x >> >> one must translate the type (dtype) into a character representation (as >> listed in http://docs.python.org/lib/module-struct.html). > > What about going the opposite way and either a) replacing the type definition > in buffers by the type character entirely (not sure, but this might not work > if you require a specific type name to appear in the C code), or b) by > requiring users to provide both the type and the character in a suitable > buffer declaration syntax? Thanks, new ideas like that I was looking for! After thinking about it it turns out that it won't really help for the NumPy "int8/int16/..." types though; it will make the user code potentially unportable: cdef ndarray["i", 2] = zeros((10, 10), dtype=int32) Making it part of the "ctypedef" syntax: ctypedef int int32 [bufchar "i"] would still mean #ifdefs in pxd files. In fact, I think it turns out that the mapping to characters is not the real problem here; the mapping to exact C types is (since the buffer interface will only carry across native C types (the fact that they are encoded as characters is not crucial here)). I think my solution for now will be simply to drop the bit-size typedefs from numpy.pxd and require exact ctypedefs. So one can still do cdef numpy.ndarray[numpy.uint, 2] = ... but not use "uint8" or any other bit-size-specific types. I can also insert a simple check for sizeof(numpy.uint) == sizeof(unsigned int) in the C code at module loading time for all types which are used in buffers (it should be optimized away if things are ok, and will break things early if not). It looks like /usr/include/tgmath.h pulls what I want off; but the code gets very ugly and only works on some compilers; a warning sign that I perhaps don't want to do this but should go for the least obscure route of requiring exact ctypedefs. Though perhaps I can use the floating_type macro to add to the type check proposed above. Anyway, excerpt (I don't understand a tenth of it yet though :-) ): # define __floating_type(type) (((type) 0.25) && ((type) 0.25 - 1)) # define __tgmath_real_type_sub(T, E) \ __typeof__ (*(0 ? (__typeof__ (0 ? (double *) 0 : (void *) (E))) 0 \ : (__typeof__ (0 ? (T *) 0 : (void *) (!(E)))) 0)) ... # define __TGMATH_BINARY_REAL_IMAG(Val1, Val2, Fct, Cfct) \ (__extension__ (((sizeof (__real__ (Val1)) > sizeof (double) \ || sizeof (__real__ (Val2)) > sizeof (double)) \ && __builtin_classify_type (__real__ (Val1) \ + __real__ (Val2)) == 8) \ ? ((sizeof (__real__ (Val1)) == sizeof (Val1) \ && sizeof (__real__ (Val2)) == sizeof (Val2)) \ ? (__typeof ((__tgmath_real_type (Val1)) 0 \ + (__tgmath_real_type (Val2)) 0)) \ __tgml(Fct) (Val1, Val2) \ : (__typeof ((__tgmath_real_type (Val1)) 0 \ + (__tgmath_real_type (Val2)) 0)) \ ... #define cos(Val) __TGMATH_UNARY_REAL_IMAG (Val, cos, ccos) -- Dag Sverre From robertwb at math.washington.edu Sat Jul 5 20:02:36 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sat, 5 Jul 2008 11:02:36 -0700 Subject: [Cython] Buffer must know exact C types In-Reply-To: <486F3A5A.9050204@student.matnat.uio.no> References: <486E9A31.7040300@student.matnat.uio.no> <486F0286.7060105@behnel.de> <486F3A5A.9050204@student.matnat.uio.no> Message-ID: <5B222EAE-FC9E-4647-8BB3-C5E24E5CA40B@math.washington.edu> On Jul 5, 2008, at 2:09 AM, Dag Sverre Seljebotn wrote: > Stefan Behnel wrote: >> Hi, >> >> Dag Sverre Seljebotn wrote: >>> Currently, wherever a C type is used in Cython, it is enough to >>> specify >>> sort-of what kind of class of type it is... the following will >>> work all >>> right: >>> >>> cdef extern ...: >>> ctypedef int int8 >>> ctypedef int int16 >>> ctypedef int int32 >>> ctypedef int int64 >>> >>> >>> However, when types are used in buffers: >>> >>> cdef object[int16, 3] x >>> >>> one must translate the type (dtype) into a character >>> representation (as >>> listed in http://docs.python.org/lib/module-struct.html). >> >> What about going the opposite way and either a) replacing the type >> definition >> in buffers by the type character entirely (not sure, but this >> might not work >> if you require a specific type name to appear in the C code), or >> b) by >> requiring users to provide both the type and the character in a >> suitable >> buffer declaration syntax? > > Thanks, new ideas like that I was looking for! > > After thinking about it it turns out that it won't really help for the > NumPy "int8/int16/..." types though; it will make the user code > potentially unportable: > > cdef ndarray["i", 2] = zeros((10, 10), dtype=int32) > > Making it part of the "ctypedef" syntax: > > ctypedef int int32 [bufchar "i"] > > would still mean #ifdefs in pxd files. In fact, I think it turns out > that the mapping to characters is not the real problem here; the > mapping > to exact C types is (since the buffer interface will only carry across > native C types (the fact that they are encoded as characters is not > crucial here)). > > I think my solution for now will be simply to drop the bit-size > typedefs > from numpy.pxd and require exact ctypedefs. So one can still do > > cdef numpy.ndarray[numpy.uint, 2] = ... > > but not use "uint8" or any other bit-size-specific types. I can also > insert a simple check for sizeof(numpy.uint) == sizeof(unsigned > int) in > the C code at module loading time for all types which are used in > buffers (it should be optimized away if things are ok, and will break > things early if not). I think we want to have the actual type of the array (e.g. what is returned on arr[0]) in the type spec rather than forcing users to manually translate between "i" and it. If I understand correctly, the issue here is how to map cython/c types to the "format" string of the buffer. There are three things to match: Numeric type (int/float)--this can be detected at compile time by (1)/(2) == 0 vs the buffer format string Signed--this can be detected at compile time by (-1) < 0 vs the buffer format string Size--this can be detected at compile time by sizeof() vs the "itemsize" field of the buffer, or calcsize from the struct module if you need. All other types are required to match exactly (or something like that, some thought would have to be put into how to handle objects). For hybrid types I think it would be enough to just look at the total size after comparing the structs for (int/float/other and signed/ unsigned). Then you could use numpy.uint and numpy.unit32, etc. - Robert From dagss at student.matnat.uio.no Sat Jul 5 20:50:30 2008 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Sat, 05 Jul 2008 20:50:30 +0200 Subject: [Cython] Buffer must know exact C types In-Reply-To: <5B222EAE-FC9E-4647-8BB3-C5E24E5CA40B@math.washington.edu> References: <486E9A31.7040300@student.matnat.uio.no> <486F0286.7060105@behnel.de> <486F3A5A.9050204@student.matnat.uio.no> <5B222EAE-FC9E-4647-8BB3-C5E24E5CA40B@math.washington.edu> Message-ID: <486FC276.1010506@student.matnat.uio.no> Robert Bradshaw wrote: > If I understand correctly, the issue here is how to map cython/c > types to the "format" string of the buffer. There are three things to > match: > > Numeric type (int/float)--this can be detected at compile time by > (1)/(2) == 0 vs the buffer format string > Signed--this can be detected at compile time by (-1) < 0 vs the > buffer format string > Size--this can be detected at compile time by sizeof() vs the > "itemsize" field of the buffer, or calcsize from the struct module if > you need. Very nice! Or simply sizof(type) vs sizeof(int). > All other types are required to match exactly (or something like > that, some thought would have to be put into how to handle objects). > For hybrid types I think it would be enough to just look at the total > size after comparing the structs for (int/float/other and signed/ > unsigned). Then you could use numpy.uint and numpy.unit32, etc. OK, writeup for clarity in the discussion, this is what needs to happen: - A format string is recieved run-time when acquiring a buffer - We have a compile-time assumed type for the buffer - They need to be compared for sanity-checking And this is how I see it happen (which is quite different from what I thought a day ago): - For each type used in a buffer, Cython will output a string-validating inline function for that type. - For structs (and other complicated things) this essentially boils down to checking if a string corresponds to a known structure and is relatively straightforward if potentially messy/wordy. (These will not even be supported until working buffers for simple types hits cython-devel IMO, iterative programming and all that...) - When the exact C type is used directly, something like this may be used (if I bother to treat it specially): int Pyx_typestring_matches_unsigned_int(char* s) { /* skipping the one-char check for now*/ return *s == 'U'; } - When a typedef-ed type is output, something like this is used: int Pyx_typestring_matches_npy_uint8(char* s) { /* skipping the one-char check for now*/ switch (*s) { case 'i': return (((npy_uint8)1)/(npy_uint8)2) == 0) && ((npy_uint8)-1 < 0) && sizeof(npy_uint8) == sizeof(int); ... } } This should translate to a switch statement where every branch but one will return false, and I think it is reasonable to hope the compiler will make it as efficient as the former case. Thanks! :-) -- Dag Sverre From greg.ewing at canterbury.ac.nz Sun Jul 6 02:18:03 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 06 Jul 2008 12:18:03 +1200 Subject: [Cython] Buffer must know exact C types In-Reply-To: <486F3A5A.9050204@student.matnat.uio.no> References: <486E9A31.7040300@student.matnat.uio.no> <486F0286.7060105@behnel.de> <486F3A5A.9050204@student.matnat.uio.no> Message-ID: <48700F3B.7080109@canterbury.ac.nz> How does one write portable code in C to do this? If it's possible to write portably in C, then there must be a way to have Cython generate that portable C from some portable representation in Cython source. -- Greg From robertwb at math.washington.edu Sun Jul 6 02:55:24 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sat, 5 Jul 2008 17:55:24 -0700 Subject: [Cython] Buffer must know exact C types In-Reply-To: <486FC276.1010506@student.matnat.uio.no> References: <486E9A31.7040300@student.matnat.uio.no> <486F0286.7060105@behnel.de> <486F3A5A.9050204@student.matnat.uio.no> <5B222EAE-FC9E-4647-8BB3-C5E24E5CA40B@math.washington.edu> <486FC276.1010506@student.matnat.uio.no> Message-ID: On Jul 5, 2008, at 11:50 AM, Dag Sverre Seljebotn wrote: > Robert Bradshaw wrote: >> If I understand correctly, the issue here is how to map cython/c >> types to the "format" string of the buffer. There are three things to >> match: >> >> Numeric type (int/float)--this can be detected at compile time by >> (1)/(2) == 0 vs the buffer format string >> Signed--this can be detected at compile time by (-1) < 0 vs the >> buffer format string >> Size--this can be detected at compile time by sizeof() vs the >> "itemsize" field of the buffer, or calcsize from the struct module if >> you need. > > Very nice! > > Or simply sizof(type) vs sizeof(int). > >> All other types are required to match exactly (or something like >> that, some thought would have to be put into how to handle objects). >> For hybrid types I think it would be enough to just look at the total >> size after comparing the structs for (int/float/other and signed/ >> unsigned). Then you could use numpy.uint and numpy.unit32, etc. > > > OK, writeup for clarity in the discussion, this is what needs to > happen: > > - A format string is recieved run-time when acquiring a buffer > - We have a compile-time assumed type for the buffer > - They need to be compared for sanity-checking Exactly. > And this is how I see it happen (which is quite different from what I > thought a day ago): > > - For each type used in a buffer, Cython will output a string- > validating > inline function for that type. > > - For structs (and other complicated things) this essentially boils > down > to checking if a string corresponds to a known structure and is > relatively straightforward if potentially messy/wordy. (These will not > even be supported until working buffers for simple types hits > cython-devel IMO, iterative programming and all that...) > > - When the exact C type is used directly, something like this may be > used (if I bother to treat it specially): > > int Pyx_typestring_matches_unsigned_int(char* s) { > /* skipping the one-char check for now*/ > return *s == 'U'; > } > > - When a typedef-ed type is output, something like this is used: > > int Pyx_typestring_matches_npy_uint8(char* s) { > > /* skipping the one-char check for now*/ > > switch (*s) { > case 'i': > return (((npy_uint8)1)/(npy_uint8)2) == 0) && > ((npy_uint8)-1 < 0) && sizeof(npy_uint8) == sizeof(int); > ... > } > } > > This should translate to a switch statement where every branch but one > will return false, and I think it is reasonable to hope the compiler > will make it as efficient as the former case. Yep, looks good to me. The compiler will have no trouble evaluating these constant expressions and pruning the tree. In retrospect, I think the ((type)1)/((type)2) == 0 trick is probably overkill, the person who declares the type should ctypedef it as an int or float correctly, we don't need to be double-guessing that. > Thanks! :-) Any time :-) - Robert From dagss at student.matnat.uio.no Sun Jul 6 11:00:47 2008 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Sun, 6 Jul 2008 11:00:47 +0200 (CEST) Subject: [Cython] Buffer must know exact C types In-Reply-To: <48700F3B.7080109@canterbury.ac.nz> References: <486E9A31.7040300@student.matnat.uio.no> <486F0286.7060105@behnel.de> <486F3A5A.9050204@student.matnat.uio.no> <48700F3B.7080109@canterbury.ac.nz> Message-ID: <58465.193.157.229.67.1215334847.squirrel@webmail.uio.no> Greg wrote: > How does one write portable code in C to do this? > > If it's possible to write portably in C, then there > must be a way to have Cython generate that portable > C from some portable representation in Cython source. The problem (which is now solved) arises because Cython ctypedefs are not required to be exact, only approximate. When writing C code, your typedefs are exact and you can always know (at least in principle) the exact type you are dealing with. So if you have something like this: #ifdef A typedef short foo #else typedef int foo #endif ...then you can always throw in "#define FOO_LETTER 'h'" or "#define FOO_LETTER 'i'" inside those #ifdefs. However, if it says this in Cython: ctypedef int foo then you don't "really" know the exact type of foo (and this feature will be relied on for numpy.pxd to provide "int16", "int64" etc., and I wouldn't like to remove it and thus require duplication of all such #ifdefs in the pxd files). Dag Sverre From gael.varoquaux at normalesup.org Sun Jul 6 23:07:31 2008 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Sun, 6 Jul 2008 23:07:31 +0200 Subject: [Cython] Schedule for the SciPy08 conferencez Message-ID: <20080706210731.GC25810@phare.normalesup.org> We have received a large number of excellent contributions for papers for the SciPy 2008 conference. The program committee has had to make a difficult selection and we are happy to bring to you a preliminary schedule: Thursday ========= **8:00** Registration/Breakfast **8:55** Welcome (Travis Vaught) **9:10** Keynote (Alex Martelli) **10:00** State of SciPy (Travis Vaught, Jarrod Millman) **10:40** -- Break -- **11:00** Sympy - Python library for symbolic mathematics: introduction and applications (Ond?ej ?ertik) **11:40** Interval arithmetic: Python implementation and applications (Stefano Taschini) **12:00** Experiences Using Scipy for Computer Vision Research (Damian Eads) **12:20** -- Lunch -- **1:40** The new NumPy documentation framework (St?fan Van der Walt) **2:00** Matplotlib solves the riddle of the sphinx (Michael Droettboom) **2:40** The SciPy documentation project (Joe Harrington) **3:00** -- Break -- **3:40** Sage: creating a viable free Python-based open source alternatice to Magma, Maple, Mathematica and Matlab (William Stein) **4:20** Open space for lightning talks Friday ======== **8:30** Breakfast **9:00** Pysynphot: A Python Re-Implementation of a Legacy App in Astronomy (Perry Greenfield) **9:40** How the Large Synoptic Survey Telescope (LSST) is using Python (Robert Lupton) **10:00** Real-time Astronomical Time-series Classification and Broadcast Pipeline (Dan Starr) **10:20** Analysis and Visualization of Multi-Scale Astrophysical Simulations using Python and NumPy (Matthew Turk) **10:40** -- Break -- **11:00** Exploring network structure, dynamics, and function using NetworkX (Aric Hagberg) **11:40** Mayavi: Making 3D data visualization reusable (Prabhu Ramachandran, Ga?l Varoquaux) **12:00** Finite Element Modeling of Contact and Impact Problems Using Python (Ryan Krauss) **12:20** -- Lunch -- **2:00** PyCircuitScape: A Tool for Landscape Ecology (Viral Shah) **2:20** Summarizing Complexity in High Dimensional Spaces (Karl Young) **2:40** UFuncs: A generic function mechanism in Python (Travis Oliphant) **3:20** -- Break -- **3:40** NumPy Optimization: Manual tuning and automated approaches (Evan Patterson) **4:00** Converting Python functions to dynamically-compiled C (Ilan Schnell) **4:20** unPython: Converting Python numerical programs into C (Rahul Garg) **4:40** Implementing the Grammar of Graphics for Python (Robert Kern) **5:00** Ask the experts session. A more detailled booklet including the abstract text will be available soon. We are looking forward to seeing you in Caltech, Ga?l Varoquaux, on behalf of the program committee. -- SciPy2008 conference. Program committee Anne Archibald, McGill University Matthew Brett Perry Greenfield, Space Telescope Science Institute Charles Harris Ryan Krauss, Southern Illinois University Ga?l Varoquaux St?fan van der Walt, University of Stellenbosch From charlie at openmoko.org Tue Jul 8 05:37:10 2008 From: charlie at openmoko.org (Guillaume Chereau) Date: Tue, 08 Jul 2008 11:37:10 +0800 Subject: [Cython] Object not released when calling cpdef method from inherited class ? Message-ID: <1215488230.20373.14.camel@nikopol> Hello all, I am currently using cython for a project related to openmoko cell phone : http://charlie137-2.blogspot.com/2008/07/introducing-tichy.html I noticed that the memory is constantly increasing when I use cython. I tracked down the problem to the case where I create a subclass of a cython class and then redefine a cpdef method, asking it to call the parent method, but ONLY if the cpdef method then call an other cpdef method ! Here is the smallest example I could come with that fails : == test.pyx == cdef class A: cpdef func(self): return cpdef test(self): self.func() == main.py == import test import gc class B(test.A): def test(self): test.A.test(self) b = B() for i in range(10): b.test() gc.collect() print len(gc.get_objects()) The output will show that some objects are not released. Is it a known bug ? Is there a way to avoid it ? I tried with both Cython 0.9.6 and Cython 0.9.8 cheers, -Charlie -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://codespeak.net/pipermail/cython-dev/attachments/20080708/0fb6b6e8/attachment.pgp From robertwb at math.washington.edu Tue Jul 8 05:42:38 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Mon, 7 Jul 2008 20:42:38 -0700 Subject: [Cython] Object not released when calling cpdef method from inherited class ? In-Reply-To: <1215488230.20373.14.camel@nikopol> References: <1215488230.20373.14.camel@nikopol> Message-ID: On Jul 7, 2008, at 8:37 PM, Guillaume Chereau wrote: > Hello all, > > I am currently using cython for a project related to openmoko cell > phone : http://charlie137-2.blogspot.com/2008/07/introducing- > tichy.html > > I noticed that the memory is constantly increasing when I use > cython. I > tracked down the problem to the case where I create a subclass of a > cython class and then redefine a cpdef method, asking it to call the > parent method, but ONLY if the cpdef method then call an other cpdef > method ! > > Here is the smallest example I could come with that fails : > > > == test.pyx == > > cdef class A: > cpdef func(self): > return > > cpdef test(self): > self.func() > > > == main.py == > > import test > import gc > > class B(test.A): > def test(self): > test.A.test(self) > > b = B() > > for i in range(10): > b.test() > gc.collect() > print len(gc.get_objects()) > > > The output will show that some objects are not released. > > Is it a known bug ? Is there a way to avoid it ? This isn't a known bug, but does sound like something fishy is going on. Thanks for the report. The easiest fix I have for now is to make one or the other methods an ordinary def method (sacrificing speed for better memory handling), but I'll be looking into it. > I tried with both Cython 0.9.6 and Cython 0.9.8 > > cheers, > -Charlie > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 186 bytes Desc: This is a digitally signed message part Url : http://codespeak.net/pipermail/cython-dev/attachments/20080707/043950cb/attachment.pgp From languitar at semipol.de Tue Jul 8 18:49:53 2008 From: languitar at semipol.de (Johannes Wienke) Date: Tue, 08 Jul 2008 18:49:53 +0200 Subject: [Cython] Function signature does not match previous declaration Message-ID: <48739AB1.8080900@semipol.de> Hi, I've got one header file in my project, that raises a lot of "Function signature does not match previous declaration" warnings but I don't know why, especially because everything works. One warning e. g. is this one: warning: include/icewing/gui/Gimage.pxi:87:11: Function signature does not match previous declaration Line 87 in that file is: iwImage* iw_img_new_alloc(int width, int height, int planes, iwType type) iwImage and iwType are defined like this: ctypedef struct iwImage: guchar **data int planes iwType type int width, height int rowstride iwColtab ctab void *reserved ctypedef enum iwType: IW_8U IW_16U IW_32S IW_FLOAT IW_DOUBLE The original definitions in the header file look like this: typedef struct iwImage { guchar **data; int planes; iwType type; int width, height; int rowstride; iwColtab ctab; void *reserved; } iwImage; typedef enum { IW_8U, IW_16U, IW_32S, IW_FLOAT, IW_DOUBLE } iwType; iwImage* iw_img_new_alloc (int width, int height, int planes, iwType type); Has anyone got an idea why cython generates these warnings? Thanks for the help! Johannes -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature Url : http://codespeak.net/pipermail/cython-dev/attachments/20080708/75cc49af/attachment.pgp From cwitty at newtonlabs.com Tue Jul 8 19:55:31 2008 From: cwitty at newtonlabs.com (Carl Witty) Date: Tue, 8 Jul 2008 10:55:31 -0700 Subject: [Cython] Function signature does not match previous declaration In-Reply-To: <48739AB1.8080900@semipol.de> References: <48739AB1.8080900@semipol.de> Message-ID: On Tue, Jul 8, 2008 at 9:49 AM, Johannes Wienke wrote: > Hi, > > I've got one header file in my project, that raises a lot of "Function > signature does not match previous declaration" warnings but I don't know > why, especially because everything works. One warning e. g. is this one: > > warning: include/icewing/gui/Gimage.pxi:87:11: Function signature does > not match previous declaration > > Line 87 in that file is: > iwImage* iw_img_new_alloc(int width, int height, int planes, iwType type) ... This warning does not mean that the Cython declaration doesn't match the C declaration (Cython knows nothing about the C declaration). Instead, it means that there are two Cython declarations that don't match. Search all your .pxd and .pxi files for another declaration of iw_img_new_alloc; also, make sure that Gimage.pxi is not included multiple times. Carl From languitar at semipol.de Tue Jul 8 20:18:33 2008 From: languitar at semipol.de (Johannes Wienke) Date: Tue, 08 Jul 2008 20:18:33 +0200 Subject: [Cython] Function signature does not match previous declaration In-Reply-To: References: <48739AB1.8080900@semipol.de> Message-ID: <4873AF79.8000804@semipol.de> Am 07/08/2008 07:55 PM schrieb Carl Witty: > On Tue, Jul 8, 2008 at 9:49 AM, Johannes Wienke wrote: >> warning: include/icewing/gui/Gimage.pxi:87:11: Function signature does >> not match previous declaration >> >> Line 87 in that file is: >> iwImage* iw_img_new_alloc(int width, int height, int planes, iwType type) > ... > > This warning does not mean that the Cython declaration doesn't match > the C declaration (Cython knows nothing about the C declaration). > Instead, it means that there are two Cython declarations that don't > match. Search all your .pxd and .pxi files for another declaration of > iw_img_new_alloc; also, make sure that Gimage.pxi is not included > multiple times. I can't find any of those things in my project: This is what I can grab about thos things: languitar at bird ~/BA-workspace/shIP $ grep -in "Gimage.pxi" `find . -name '*.px*'` | grep -v svn ./src/ship/icewing/preview.pxd:2:include "icewing/gui/Gimage.pxi" ./src/ship/icewing/image_utils.pxd:1:include "icewing/gui/Gimage.pxi" ./include/icewing/gui/Grender.pxi:3:include "icewing/gui/Gimage.pxi" ./include/icewing/gui/Gpreview.pxi:4:include "icewing/gui/Gimage.pxi" languitar at bird ~/BA-workspace/shIP $ grep -in "Gimage.pxi" `find . -name '*.py*'` | grep -v svn ./src/ship/icewing/converters.pyx:9:include "icewing/gui/Gimage.pxi" languitar at bird ~/BA-workspace/shIP $ grep -in "iw_img_new_alloc" `find . -name '*.px*'` | grep -v svn ./include/icewing/gui/Gimage.pxi:87: iwImage* iw_img_new_alloc(int width, int height, int planes, iwType type) languitar at bird ~/BA-workspace/shIP $ grep -in "iw_img_new_alloc" `find . -name '*.py*'` | grep -v svn ./src/ship/icewing/converters.pyx:241: cdef iwImage *image = iw_img_new_alloc(data.size[0], - Johannes -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature Url : http://codespeak.net/pipermail/cython-dev/attachments/20080708/9d785303/attachment.pgp From dagss at student.matnat.uio.no Tue Jul 8 20:24:46 2008 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Tue, 08 Jul 2008 20:24:46 +0200 Subject: [Cython] Function signature does not match previous declaration In-Reply-To: <4873AF79.8000804@semipol.de> References: <48739AB1.8080900@semipol.de> <4873AF79.8000804@semipol.de> Message-ID: <4873B0EE.7030508@student.matnat.uio.no> Johannes Wienke wrote: > Am 07/08/2008 07:55 PM schrieb Carl Witty: >> On Tue, Jul 8, 2008 at 9:49 AM, Johannes Wienke wrote: >>> warning: include/icewing/gui/Gimage.pxi:87:11: Function signature does >>> not match previous declaration >>> >>> Line 87 in that file is: >>> iwImage* iw_img_new_alloc(int width, int height, int planes, iwType type) >> ... >> >> This warning does not mean that the Cython declaration doesn't match >> the C declaration (Cython knows nothing about the C declaration). >> Instead, it means that there are two Cython declarations that don't >> match. Search all your .pxd and .pxi files for another declaration of >> iw_img_new_alloc; also, make sure that Gimage.pxi is not included >> multiple times. > > I can't find any of those things in my project: This is what I can grab > about thos things: > > languitar at bird ~/BA-workspace/shIP $ grep -in "Gimage.pxi" `find . -name > '*.px*'` | grep -v svn > ./src/ship/icewing/preview.pxd:2:include "icewing/gui/Gimage.pxi" > ./src/ship/icewing/image_utils.pxd:1:include "icewing/gui/Gimage.pxi" > ./include/icewing/gui/Grender.pxi:3:include "icewing/gui/Gimage.pxi" > ./include/icewing/gui/Gpreview.pxi:4:include "icewing/gui/Gimage.pxi" Here it is included multiple times, each pxd file gets a copy of the function definitions in the pxi files. You're probably using pxi in the wrong place here, switch it to a pxd file and use cimport. -- Dag Sverre From dagss at student.matnat.uio.no Tue Jul 8 20:26:28 2008 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Tue, 08 Jul 2008 20:26:28 +0200 Subject: [Cython] Function signature does not match previous declaration In-Reply-To: <4873B0EE.7030508@student.matnat.uio.no> References: <48739AB1.8080900@semipol.de> <4873AF79.8000804@semipol.de> <4873B0EE.7030508@student.matnat.uio.no> Message-ID: <4873B154.2090303@student.matnat.uio.no> Dag Sverre Seljebotn wrote: > Johannes Wienke wrote: >> Am 07/08/2008 07:55 PM schrieb Carl Witty: >>> On Tue, Jul 8, 2008 at 9:49 AM, Johannes Wienke wrote: >>>> warning: include/icewing/gui/Gimage.pxi:87:11: Function signature does >>>> not match previous declaration >>>> >>>> Line 87 in that file is: >>>> iwImage* iw_img_new_alloc(int width, int height, int planes, iwType type) >>> ... >>> >>> This warning does not mean that the Cython declaration doesn't match >>> the C declaration (Cython knows nothing about the C declaration). >>> Instead, it means that there are two Cython declarations that don't >>> match. Search all your .pxd and .pxi files for another declaration of >>> iw_img_new_alloc; also, make sure that Gimage.pxi is not included >>> multiple times. >> I can't find any of those things in my project: This is what I can grab >> about thos things: >> >> languitar at bird ~/BA-workspace/shIP $ grep -in "Gimage.pxi" `find . -name >> '*.px*'` | grep -v svn >> ./src/ship/icewing/preview.pxd:2:include "icewing/gui/Gimage.pxi" >> ./src/ship/icewing/image_utils.pxd:1:include "icewing/gui/Gimage.pxi" >> ./include/icewing/gui/Grender.pxi:3:include "icewing/gui/Gimage.pxi" >> ./include/icewing/gui/Gpreview.pxi:4:include "icewing/gui/Gimage.pxi" > > Here it is included multiple times, each pxd file gets a copy of the > function definitions in the pxi files. You're probably using pxi in the > wrong place here, switch it to a pxd file and use cimport. Thinking about it this is likely not the cause of your problem though... -- Dag Sverre From languitar at semipol.de Tue Jul 8 20:26:10 2008 From: languitar at semipol.de (Johannes Wienke) Date: Tue, 08 Jul 2008 20:26:10 +0200 Subject: [Cython] Function signature does not match previous declaration In-Reply-To: <4873B0EE.7030508@student.matnat.uio.no> References: <48739AB1.8080900@semipol.de> <4873AF79.8000804@semipol.de> <4873B0EE.7030508@student.matnat.uio.no> Message-ID: <4873B142.2010802@semipol.de> Am 07/08/2008 08:24 PM schrieb Dag Sverre Seljebotn: > Johannes Wienke wrote: >> Am 07/08/2008 07:55 PM schrieb Carl Witty: >>> On Tue, Jul 8, 2008 at 9:49 AM, Johannes Wienke wrote: >>>> warning: include/icewing/gui/Gimage.pxi:87:11: Function signature does >>>> not match previous declaration >>>> >>>> Line 87 in that file is: >>>> iwImage* iw_img_new_alloc(int width, int height, int planes, iwType type) >>> ... [...] >> languitar at bird ~/BA-workspace/shIP $ grep -in "Gimage.pxi" `find . -name >> '*.px*'` | grep -v svn >> ./src/ship/icewing/preview.pxd:2:include "icewing/gui/Gimage.pxi" >> ./src/ship/icewing/image_utils.pxd:1:include "icewing/gui/Gimage.pxi" >> ./include/icewing/gui/Grender.pxi:3:include "icewing/gui/Gimage.pxi" >> ./include/icewing/gui/Gpreview.pxi:4:include "icewing/gui/Gimage.pxi" > > Here it is included multiple times, each pxd file gets a copy of the > function definitions in the pxi files. You're probably using pxi in the > wrong place here, switch it to a pxd file and use cimport. > Well, I thought pxi files are used for definitions from other header files and pxd files for definitions of your own cython code. Or is that wrong? Johannes -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature Url : http://codespeak.net/pipermail/cython-dev/attachments/20080708/b930ab3f/attachment.pgp From dagss at student.matnat.uio.no Tue Jul 8 20:35:21 2008 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Tue, 08 Jul 2008 20:35:21 +0200 Subject: [Cython] Function signature does not match previous declaration In-Reply-To: <4873B142.2010802@semipol.de> References: <48739AB1.8080900@semipol.de> <4873AF79.8000804@semipol.de> <4873B0EE.7030508@student.matnat.uio.no> <4873B142.2010802@semipol.de> Message-ID: <4873B369.8090703@student.matnat.uio.no> Johannes Wienke wrote: > Am 07/08/2008 08:24 PM schrieb Dag Sverre Seljebotn: >> Johannes Wienke wrote: >>> Am 07/08/2008 07:55 PM schrieb Carl Witty: >>>> On Tue, Jul 8, 2008 at 9:49 AM, Johannes Wienke wrote: >>>>> warning: include/icewing/gui/Gimage.pxi:87:11: Function signature does >>>>> not match previous declaration >>>>> >>>>> Line 87 in that file is: >>>>> iwImage* iw_img_new_alloc(int width, int height, int planes, iwType type) >>>> ... > [...] >>> languitar at bird ~/BA-workspace/shIP $ grep -in "Gimage.pxi" `find . -name >>> '*.px*'` | grep -v svn >>> ./src/ship/icewing/preview.pxd:2:include "icewing/gui/Gimage.pxi" >>> ./src/ship/icewing/image_utils.pxd:1:include "icewing/gui/Gimage.pxi" >>> ./include/icewing/gui/Grender.pxi:3:include "icewing/gui/Gimage.pxi" >>> ./include/icewing/gui/Gpreview.pxi:4:include "icewing/gui/Gimage.pxi" >> Here it is included multiple times, each pxd file gets a copy of the >> function definitions in the pxi files. You're probably using pxi in the >> wrong place here, switch it to a pxd file and use cimport. >> > > Well, I thought pxi files are used for definitions from other header > files and pxd files for definitions of your own cython code. Or is that > wrong? pxi files are not for any specific purpose; they simply insert the file verbatim into the spot of the include (and thus often can lead to defining things too many times). (There are ups and downs to this; I'd never use them myself for any purpose but that's only a subjective feeling). It is perfectly ok to cimport a pxd file into another pxd file. -- Dag Sverre From languitar at semipol.de Tue Jul 8 20:45:10 2008 From: languitar at semipol.de (Johannes Wienke) Date: Tue, 08 Jul 2008 20:45:10 +0200 Subject: [Cython] Function signature does not match previous declaration In-Reply-To: <4873B369.8090703@student.matnat.uio.no> References: <48739AB1.8080900@semipol.de> <4873AF79.8000804@semipol.de> <4873B0EE.7030508@student.matnat.uio.no> <4873B142.2010802@semipol.de> <4873B369.8090703@student.matnat.uio.no> Message-ID: <4873B5B6.1080905@semipol.de> Am 07/08/2008 08:35 PM schrieb Dag Sverre Seljebotn: > Johannes Wienke wrote: >> Am 07/08/2008 08:24 PM schrieb Dag Sverre Seljebotn: >>> Johannes Wienke wrote: >>>> Am 07/08/2008 07:55 PM schrieb Carl Witty: >>>>> On Tue, Jul 8, 2008 at 9:49 AM, Johannes Wienke wrote: >>>>>> warning: include/icewing/gui/Gimage.pxi:87:11: Function signature does >>>>>> not match previous declaration >>>>>> >>>>>> Line 87 in that file is: >>>>>> iwImage* iw_img_new_alloc(int width, int height, int planes, iwType type) >>>>> ... >> [...] >>>> languitar at bird ~/BA-workspace/shIP $ grep -in "Gimage.pxi" `find . -name >>>> '*.px*'` | grep -v svn >>>> ./src/ship/icewing/preview.pxd:2:include "icewing/gui/Gimage.pxi" >>>> ./src/ship/icewing/image_utils.pxd:1:include "icewing/gui/Gimage.pxi" >>>> ./include/icewing/gui/Grender.pxi:3:include "icewing/gui/Gimage.pxi" >>>> ./include/icewing/gui/Gpreview.pxi:4:include "icewing/gui/Gimage.pxi" >>> Here it is included multiple times, each pxd file gets a copy of the >>> function definitions in the pxi files. You're probably using pxi in the >>> wrong place here, switch it to a pxd file and use cimport. >>> >> Well, I thought pxi files are used for definitions from other header >> files and pxd files for definitions of your own cython code. Or is that >> wrong? > > pxi files are not for any specific purpose; they simply insert the file > verbatim into the spot of the include (and thus often can lead to > defining things too many times). (There are ups and downs to this; I'd > never use them myself for any purpose but that's only a subjective > feeling). It is perfectly ok to cimport a pxd file into another pxd file. Ok, nevertheless I use the approach with the pxi files for all other modules and definitions in my program and only Gimage.pxi generates these warnings. Furthermore only some of the functions have those warnings, not all in the file. - Johannes -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature Url : http://codespeak.net/pipermail/cython-dev/attachments/20080708/c11eca0b/attachment.pgp From cwitty at newtonlabs.com Tue Jul 8 21:18:38 2008 From: cwitty at newtonlabs.com (Carl Witty) Date: Tue, 8 Jul 2008 12:18:38 -0700 Subject: [Cython] Function signature does not match previous declaration In-Reply-To: <4873B5B6.1080905@semipol.de> References: <48739AB1.8080900@semipol.de> <4873AF79.8000804@semipol.de> <4873B0EE.7030508@student.matnat.uio.no> <4873B142.2010802@semipol.de> <4873B369.8090703@student.matnat.uio.no> <4873B5B6.1080905@semipol.de> Message-ID: On Tue, Jul 8, 2008 at 11:45 AM, Johannes Wienke wrote: > Ok, nevertheless I use the approach with the pxi files for all other > modules and definitions in my program and only Gimage.pxi generates > these warnings. Furthermore only some of the functions have those > warnings, not all in the file. I suspect that with multiple inclusions, either iwType or iwImage (or both) defines a "new" type each time it's loaded. So the first declaration doesn't match the second declaration, because it's got "different" argument types. If you look at which functions have warnings: is it, perhaps, only the functions that take an argument of type iwType? If not, is there some other pattern of that sort in which functions have warnings? Carl From languitar at semipol.de Tue Jul 8 21:41:08 2008 From: languitar at semipol.de (Johannes Wienke) Date: Tue, 08 Jul 2008 21:41:08 +0200 Subject: [Cython] Function signature does not match previous declaration In-Reply-To: References: <48739AB1.8080900@semipol.de> <4873AF79.8000804@semipol.de> <4873B0EE.7030508@student.matnat.uio.no> <4873B142.2010802@semipol.de> <4873B369.8090703@student.matnat.uio.no> <4873B5B6.1080905@semipol.de> Message-ID: <4873C2D4.3060706@semipol.de> Am 07/08/2008 09:18 PM schrieb Carl Witty: > On Tue, Jul 8, 2008 at 11:45 AM, Johannes Wienke wrote: >> Ok, nevertheless I use the approach with the pxi files for all other >> modules and definitions in my program and only Gimage.pxi generates >> these warnings. Furthermore only some of the functions have those >> warnings, not all in the file. > > I suspect that with multiple inclusions, either iwType or iwImage (or > both) defines a "new" type each time it's loaded. So the first > declaration doesn't match the second declaration, because it's got > "different" argument types. > > If you look at which functions have warnings: is it, perhaps, only the > functions that take an argument of type iwType? If not, is there some > other pattern of that sort in which functions have warnings? It seems that all functions, that use mapped enums are the problem. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature Url : http://codespeak.net/pipermail/cython-dev/attachments/20080708/b76c5dbb/attachment-0001.pgp From n.tautenhahn at gmx.de Wed Jul 9 19:30:50 2008 From: n.tautenhahn at gmx.de (Nikolas Tautenhahn) Date: Wed, 09 Jul 2008 19:30:50 +0200 Subject: [Cython] Pex / 64bit issues (?) Message-ID: <4874F5CA.2020100@gmx.de> Hi there, I have a small snippet of pex code (actually it is only one function, no cdef classes or stuff, but uses decorated arrays) which runs fine on my laptop (opensuse 10.3, 32bit linux) but won't run at my computer at university (64bit, ubuntu). Running pex on my .px file gets me a Traceback (most recent call last): File "/home/btmwtaut/bin/pex", line 221, in module=pimport(name,force_build=force_build) File "/home/btmwtaut/src/pex-0.8/pex/pimport.py", line 368, in pimport py_module_obj = do_python_import(pex_module_name,abspath,pkg_context) File "/home/btmwtaut/src/pex-0.8/pex/pimport.py", line 236, in do_python_import module_obj=my_import(pex_module_name,global_dict) File "/home/btmwtaut/src/pex-0.8/pex/pimport.py", line 224, in my_import mod = __import__(name,global_dict) File "c_heur_solve.pyx", line 46, in c_heur_solve assert sizeof(long)==4 # use these for our checksum AssertionError I already use %whencompiling: scope.pragma_gen_fastio=False %whencompiling: scope.pragma_gen_hashmeth=False I have numpy and python sources installed, I used a standalone cython on this account without problems. (Pex, however, uses its own cython for this (I think - at least it complained when it couldn't find cython.py in ~/src/pex-0.8/.../)) Any thoughts what the problem might be? Best wishes, Nikolas Tautenhahn uname -a: Linux btmwxi 2.6.22-15-generic #1 SMP Tue Jun 10 08:52:15 UTC 2008 x86_64 GNU/Linux Python version: 2.5.1 c_heur_solve.px: %whencompiling: scope.pragma_gen_fastio=False %whencompiling: scope.pragma_gen_hashmeth=False def rowcheck(g, evil_list): cdef lenn = len(g.n), lenM = len(g.M), lenl = len(g.losers), lene = len(evil_list), res rowset = set(g.rows) loserset = set(g.loserlist) for evil in evil_list: if evil[0].issubset(rowset) and evil[1].issubset(loserset): return False cdef ndarray W cdef ndarray L cdef ndarray R cdef bint done_something = 0 for i from 0 <= i < lenM: for j from 0 <= j < lenn: W{i,j} = g.M[i][j] for i from 0 <= i < lenl: for j from 0 <= j < lenn: L{i,j} = g.losers[i][j] for i from 0 <= i < lenM: for j from 0 <= j < lenl: for k from 0 <= k < lenn: R{i*lenl + j, k} = W{i,k} - L{j,k} for i from 0 <= i < lenM*lenl: for j from i <= j < lenM*lenl: found_m1 = -2 for k from 0 <= k < lenn: res = R{i,k} + R{j,k} if res != 0: if res == -1 and found_m1 < 0: found_m1 = k elif res == 1 and found_m1 == k-1: continue else: break else: wset = set() lset = set() wset.add(g.rows[i / lenl]) wset.add(g.rows[j / lenl]) lset.add(g.loserlist[i % lenl]) lset.add(g.loserlist[j % lenl]) evil_list.add((frozenset(wset), frozenset(lset))) return False return True From parlar at gmail.com Thu Jul 10 05:55:42 2008 From: parlar at gmail.com (Jay Parlar) Date: Wed, 9 Jul 2008 23:55:42 -0400 Subject: [Cython] cimport across packages Message-ID: I'm having trouble with .pxd files across packages. My main program directory has a few packages in it, one called 'sm' and one called 'filters'. In 'filters', I have a dc.pyx and a dc.pxd, implementing an extension class called DC. In 'sm', I have a sensor_manager.pyx that does the following: from filters.dc cimport DC I get the following error when running setup.py: /Users/parlarjb/src/gui/sm/sensor_manager.pyx:31:0: 'filters.dc.pxd' not found The important part of my setup.py is: setup( name = "Vibration Analysis", ext_modules=[ Extension("filters.dc", ["filters/dc.pyx"]), Extension("sm.sensor_manager", ["sm/sensor_manager.pyx"]), ], cmdclass = {'build_ext': build_ext} ) Any thoughts? Jay P. From jim-crow at rambler.ru Thu Jul 10 06:39:43 2008 From: jim-crow at rambler.ru (Anatoly A. Kazantsev) Date: Thu, 10 Jul 2008 11:39:43 +0700 Subject: [Cython] cimport across packages In-Reply-To: References: Message-ID: <20080710113943.2abd5584.jim-crow@rambler.ru> On Wed, 9 Jul 2008 23:55:42 -0400 "Jay Parlar" wrote: > I'm having trouble with .pxd files across packages. > > My main program directory has a few packages in it, one called 'sm' > and one called 'filters'. > > In 'filters', I have a dc.pyx and a dc.pxd, implementing an extension > class called DC. > > In 'sm', I have a sensor_manager.pyx that does the following: > > from filters.dc cimport DC > > I get the following error when running setup.py: > > /Users/parlarjb/src/gui/sm/sensor_manager.pyx:31:0: 'filters.dc.pxd' not found > > The important part of my setup.py is: > > setup( > name = "Vibration Analysis", > ext_modules=[ > Extension("filters.dc", ["filters/dc.pyx"]), > Extension("sm.sensor_manager", ["sm/sensor_manager.pyx"]), > ], > cmdclass = {'build_ext': build_ext} > ) > > > Any thoughts? > > Jay P. Rename 'dc.pxd' to 'filters.dc.pxd'. That's works for me. -- Anatoly A. Kazantsev Protect your digital freedom and privacy, eliminate DRM, learn more at http://www.defectivebydesign.org/what_is_drm -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://codespeak.net/pipermail/cython-dev/attachments/20080710/b4c6a0e6/attachment.pgp From parlar at gmail.com Thu Jul 10 07:02:48 2008 From: parlar at gmail.com (Jay Parlar) Date: Thu, 10 Jul 2008 01:02:48 -0400 Subject: [Cython] cimport across packages In-Reply-To: <20080710113943.2abd5584.jim-crow@rambler.ru> References: <20080710113943.2abd5584.jim-crow@rambler.ru> Message-ID: > Rename 'dc.pxd' to 'filters.dc.pxd'. > That's works for me. > > That doesn't seem to make a difference for me... I tried renaming it to filters.dc.pxd, and putting it in both the main directory, and the filters directory. Jay P. From stefan_ml at behnel.de Thu Jul 10 07:39:03 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 10 Jul 2008 07:39:03 +0200 Subject: [Cython] cimport across packages In-Reply-To: References: Message-ID: <4875A077.1030404@behnel.de> Hi, I assume you are using Cython 0.9.8? Jay Parlar wrote: > I'm having trouble with .pxd files across packages. > > My main program directory has a few packages in it, one called 'sm' > and one called 'filters'. Are these really packages? I.e. do they have an __init__.py file? Stefan From stefan_ml at behnel.de Thu Jul 10 08:40:08 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 10 Jul 2008 08:40:08 +0200 Subject: [Cython] generators Message-ID: <4875AEC8.9070006@behnel.de> Hi, since there isn't currently a dedicated CEP page on generator support, I'm wondering how they should be implemented. I kind of imagine that once closures are in place, it might be enough to put all local variables of a generator function into a closure to keep their state, and then put a label behind each yield statement and store the current label in the closure whenever we return from the function. That would allow us to just jump to the label on re-entry and continue the execution. I do see that there might be a problem with deallocation of the closure, though, as it's hard to tell when the generator is at an end. So an extended way of doing it would be a transformation of the function into an extension class, where local variables become class attributes. In order to avoid having to split up the function body, it could be transformed into a single __next__ method, and the execution could follow the goto-label scheme as described above. That would even be independent of the closure support, although we might want to avoid code duplication here. Would that make sense? Stefan From robertwb at math.washington.edu Thu Jul 10 09:17:34 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Thu, 10 Jul 2008 00:17:34 -0700 (PDT) Subject: [Cython] generators In-Reply-To: <4875AEC8.9070006@behnel.de> References: <4875AEC8.9070006@behnel.de> Message-ID: On Thu, 10 Jul 2008, Stefan Behnel wrote: > Hi, > > since there isn't currently a dedicated CEP page on generator support, I'm > wondering how they should be implemented. Sorry, I see them as natural (and relatively straightforward) extensions of closures. > I kind of imagine that once closures are in place, it might be enough to put > all local variables of a generator function into a closure to keep their > state, and then put a label behind each yield statement and store the current > label in the closure whenever we return from the function. That would allow us > to just jump to the label on re-entry and continue the execution. This is exactly how I was planning on implementing them. > I do see that there might be a problem with deallocation of the closure, > though, as it's hard to tell when the generator is at an end. So an extended > way of doing it would be a transformation of the function into an extension > class, where local variables become class attributes. In order to avoid having > to split up the function body, it could be transformed into a single __next__ > method, and the execution could follow the goto-label scheme as described > above. That would even be independent of the closure support, although we > might want to avoid code duplication here. > > Would that make sense? > > Stefan Yes. This is in fact very much in line with how closures would be implemented--there would be an extension class that would hold the state (and garbage collection would happen when this object went out of scope, so no issues for generators or closures). The difference would be that closures get the function body put into the __call__ method and generators get the function body put into the __next__ method (with the jump table of course). One hitch that we ran into with closures (and the main reason IMHO that we didn't have time to finish them up like planned) is that variables are not bound at creation time, rather the scope is bound. Thus a function containing a closure needs to have special code as well as the inner function itself. - Robert From stefan_ml at behnel.de Thu Jul 10 09:30:26 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 10 Jul 2008 09:30:26 +0200 Subject: [Cython] Pex / 64bit issues (?) In-Reply-To: <4874F5CA.2020100@gmx.de> References: <4874F5CA.2020100@gmx.de> Message-ID: <4875BA92.8030203@behnel.de> Hi, Nikolas Tautenhahn wrote: > File "/home/btmwtaut/src/pex-0.8/pex/pimport.py", line 224, in my_import > mod = __import__(name,global_dict) > File "c_heur_solve.pyx", line 46, in c_heur_solve > assert sizeof(long)==4 # use these for our checksum > AssertionError This obviously can't work on a 64 bit machine. Must be PEX generating it, and it looks like a bug to me, as your code doesn't suggest any such assertion. Stefan From dagss at student.matnat.uio.no Thu Jul 10 10:11:01 2008 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Thu, 10 Jul 2008 10:11:01 +0200 Subject: [Cython] generators In-Reply-To: References: <4875AEC8.9070006@behnel.de> Message-ID: <4875C415.6010706@student.matnat.uio.no> Robert Bradshaw wrote: > One hitch that we ran into with closures (and the main reason IMHO that we > didn't have time to finish them up like planned) is that variables are not > bound at creation time, rather the scope is bound. Thus a function > containing a closure needs to have special code as well as the inner > function itself. To be even more specific, it would help if self.result_code everywhere was calculated during code generation rather than type analysis (because then you could modify the scope after type analysis to move around the variables like you want to). This will clean up a lot of things IMO. I looked at it and it didn't look too difficult in principle to do that refactoring, perhaps a day or two. I suggest this as a dependency for closures (I think it was possible without but more of a mess code-wise that way...). -- Dag Sverre From parlar at gmail.com Thu Jul 10 14:49:15 2008 From: parlar at gmail.com (Jay Parlar) Date: Thu, 10 Jul 2008 08:49:15 -0400 Subject: [Cython] cimport across packages In-Reply-To: <4875A077.1030404@behnel.de> References: <4875A077.1030404@behnel.de> Message-ID: On 7/10/08, Stefan Behnel wrote: > Hi, > > I assume you are using Cython 0.9.8? Yes. I wasn't (I'd forgotten to upgrade this machine), but I'm getting the same behaviour with 0.9.8 > > > Jay Parlar wrote: > > I'm having trouble with .pxd files across packages. > > > > My main program directory has a few packages in it, one called 'sm' > > and one called 'filters'. > > > Are these really packages? I.e. do they have an __init__.py file? Yep. They've been performing fine as packages before, but now I'm trying to add the .pxd files so I can use cdef functions between modules, and thus the beginning of my pain. Jay P. From jim-crow at rambler.ru Thu Jul 10 16:40:47 2008 From: jim-crow at rambler.ru (Anatoly A. Kazantsev) Date: Thu, 10 Jul 2008 21:40:47 +0700 Subject: [Cython] cimport across packages In-Reply-To: References: <20080710113943.2abd5584.jim-crow@rambler.ru> Message-ID: <20080710214047.21eaf098.jim-crow@rambler.ru> On Thu, 10 Jul 2008 01:02:48 -0400 "Jay Parlar" wrote: > > Rename 'dc.pxd' to 'filters.dc.pxd'. > > That's works for me. > > > > > > That doesn't seem to make a difference for me... I tried renaming it > to filters.dc.pxd, and putting it in both the main directory, and the > filters directory. Oh, I don't split sources to different directories. -- Anatoly A. Kazantsev Protect your digital freedom and privacy, eliminate DRM, learn more at http://www.defectivebydesign.org/what_is_drm -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://codespeak.net/pipermail/cython-dev/attachments/20080710/df4672c6/attachment.pgp From parlar at gmail.com Thu Jul 10 16:49:53 2008 From: parlar at gmail.com (Jay Parlar) Date: Thu, 10 Jul 2008 10:49:53 -0400 Subject: [Cython] cimport across packages In-Reply-To: <20080710214047.21eaf098.jim-crow@rambler.ru> References: <20080710113943.2abd5584.jim-crow@rambler.ru> <20080710214047.21eaf098.jim-crow@rambler.ru> Message-ID: On 7/10/08, Anatoly A. Kazantsev wrote: > Oh, I don't split sources to different directories. Ok, so I *kind* of have it working, in that it works, but I'm not happy about it. It seems that if I put 'filters.dc.pxd' in *both* the 'filters' directory AND the 'sm' directory, then it happily compiles. But there must be a better way than copy-pasting. Jay P. From jim-crow at rambler.ru Thu Jul 10 17:24:15 2008 From: jim-crow at rambler.ru (Anatoly A. Kazantsev) Date: Thu, 10 Jul 2008 22:24:15 +0700 Subject: [Cython] cimport across packages In-Reply-To: References: <20080710113943.2abd5584.jim-crow@rambler.ru> <20080710214047.21eaf098.jim-crow@rambler.ru> Message-ID: <20080710222415.e38e9c49.jim-crow@rambler.ru> On Thu, 10 Jul 2008 10:49:53 -0400 "Jay Parlar" wrote: > On 7/10/08, Anatoly A. Kazantsev wrote: > > Oh, I don't split sources to different directories. > > Ok, so I *kind* of have it working, in that it works, but I'm not > happy about it. It seems that if I put 'filters.dc.pxd' in *both* the > 'filters' directory AND the 'sm' directory, then it happily compiles. > But there must be a better way than copy-pasting. Why do you want to split 2 files to different directories? Just put them in one root directory of your project sources. -- Anatoly A. Kazantsev Protect your digital freedom and privacy, eliminate DRM, learn more at http://www.defectivebydesign.org/what_is_drm -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://codespeak.net/pipermail/cython-dev/attachments/20080710/5c492e36/attachment.pgp From parlar at gmail.com Thu Jul 10 17:50:33 2008 From: parlar at gmail.com (Jay Parlar) Date: Thu, 10 Jul 2008 11:50:33 -0400 Subject: [Cython] cimport across packages In-Reply-To: <20080710222415.e38e9c49.jim-crow@rambler.ru> References: <20080710113943.2abd5584.jim-crow@rambler.ru> <20080710214047.21eaf098.jim-crow@rambler.ru> <20080710222415.e38e9c49.jim-crow@rambler.ru> Message-ID: On 7/10/08, Anatoly A. Kazantsev wrote: > Why do you want to split 2 files to different directories? > Just put them in one root directory of your project sources. Why do people ever separate their files into different directories? It's a better, more logical layout, given our project. There are a lot more files involved than the ones I'm talking about here, these just happen to be the Cython files. Jay P. From languitar at semipol.de Thu Jul 10 18:17:51 2008 From: languitar at semipol.de (Johannes Wienke) Date: Thu, 10 Jul 2008 18:17:51 +0200 Subject: [Cython] Function signature does not match previous declaration In-Reply-To: <4873C2D4.3060706@semipol.de> References: <48739AB1.8080900@semipol.de> <4873AF79.8000804@semipol.de> <4873B0EE.7030508@student.matnat.uio.no> <4873B142.2010802@semipol.de> <4873B369.8090703@student.matnat.uio.no> <4873B5B6.1080905@semipol.de> <4873C2D4.3060706@semipol.de> Message-ID: <4876362F.90802@semipol.de> Am 07/08/2008 09:41 PM schrieb Johannes Wienke: > Am 07/08/2008 09:18 PM schrieb Carl Witty: >> On Tue, Jul 8, 2008 at 11:45 AM, Johannes Wienke wrote: >>> Ok, nevertheless I use the approach with the pxi files for all other >>> modules and definitions in my program and only Gimage.pxi generates >>> these warnings. Furthermore only some of the functions have those >>> warnings, not all in the file. >> I suspect that with multiple inclusions, either iwType or iwImage (or >> both) defines a "new" type each time it's loaded. So the first >> declaration doesn't match the second declaration, because it's got >> "different" argument types. >> >> If you look at which functions have warnings: is it, perhaps, only the >> functions that take an argument of type iwType? If not, is there some >> other pattern of that sort in which functions have warnings? > > It seems that all functions, that use mapped enums are the problem. No one got an idea why enums are the problem? :( - Johannes -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature Url : http://codespeak.net/pipermail/cython-dev/attachments/20080710/216e1395/attachment.pgp From robertwb at math.washington.edu Thu Jul 10 20:30:31 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Thu, 10 Jul 2008 11:30:31 -0700 (PDT) Subject: [Cython] cimport across packages In-Reply-To: References: <20080710113943.2abd5584.jim-crow@rambler.ru> <20080710214047.21eaf098.jim-crow@rambler.ru> Message-ID: For the Sage project we have code split accross dozens of directories, with relative and local imports throughtout as if they were Python fils, so this does work. Perhaps you could post your code (or at least a non-working example) somewhere, and I could take a look later to see if you're configuring somethign wrong (or it's a bug in Cython). - Robert On Thu, 10 Jul 2008, Jay Parlar wrote: > On 7/10/08, Anatoly A. Kazantsev wrote: >> Oh, I don't split sources to different directories. > > Ok, so I *kind* of have it working, in that it works, but I'm not > happy about it. It seems that if I put 'filters.dc.pxd' in *both* the > 'filters' directory AND the 'sm' directory, then it happily compiles. > But there must be a better way than copy-pasting. > > Jay P. > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From parlar at gmail.com Fri Jul 11 00:12:36 2008 From: parlar at gmail.com (Jay Parlar) Date: Thu, 10 Jul 2008 18:12:36 -0400 Subject: [Cython] cimport across packages In-Reply-To: References: <20080710113943.2abd5584.jim-crow@rambler.ru> <20080710214047.21eaf098.jim-crow@rambler.ru> Message-ID: I've put up a minimal example at http://django.jayparlar.com/example.tar Unless I have filters.dc.pxd in both the 'filters' and 'sm' directories, it won't compile. If I place it in both directories, it compiles and runs fine, but I'd rather not copy-paste. Thanks, Jay P. On 7/10/08, Robert Bradshaw wrote: > For the Sage project we have code split accross dozens of directories, > with relative and local imports throughtout as if they were Python fils, > so this does work. > > Perhaps you could post your code (or at least a non-working example) > somewhere, and I could take a look later to see if you're configuring > somethign wrong (or it's a bug in Cython). > > > - Robert > > > On Thu, 10 Jul 2008, Jay Parlar wrote: > > > On 7/10/08, Anatoly A. Kazantsev wrote: > >> Oh, I don't split sources to different directories. > > > > Ok, so I *kind* of have it working, in that it works, but I'm not > > happy about it. It seems that if I put 'filters.dc.pxd' in *both* the > > 'filters' directory AND the 'sm' directory, then it happily compiles. > > But there must be a better way than copy-pasting. > > > > Jay P. > > > _______________________________________________ > > Cython-dev mailing list > > Cython-dev at codespeak.net > > http://codespeak.net/mailman/listinfo/cython-dev > > > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From stefan_ml at behnel.de Fri Jul 11 09:53:37 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 11 Jul 2008 09:53:37 +0200 Subject: [Cython] Running the CPython regression tests Message-ID: <48771181.90609@behnel.de> Hi, I extended the test runner script to let it run the CPython regression test suite. All you have to do is take a source distribution of CPython and copy the "Lib/test" directory over to Cython's "tests/" directory, naming it "pyregr". The test runner will compile each module and run all unit tests it finds. Currently, most of them fail, but I have about 160 out of 205 compiling from the CPython 2.5.1 test suite. Almost of all the failures are due to three things: - inner classes and functions - lambda - print >> I don't mind the latter since it's gone in Py3, and I'm happy to know that the rest is underways. There are also generator expressions, but they should be simple to implement once the function-to-class transformations work. Stefan From stefan_ml at behnel.de Fri Jul 11 17:04:16 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 11 Jul 2008 17:04:16 +0200 Subject: [Cython] generators In-Reply-To: References: <4875AEC8.9070006@behnel.de> Message-ID: <48777670.7030907@behnel.de> Hi, Robert Bradshaw wrote: > This is in fact very much in line with how closures would be > implemented--there would be an extension class that would hold the state > (and garbage collection would happen when this object went out of scope, > so no issues for generators or closures). Just coming up with a couple of corner cases to make sure they will work in this solution. Could you comment on how they are handled? 1) def f(): x = 1 def a(): print x def b(): print x return (a,b) 2) for i in range(10): def a(): print i a() (outputs 0-9 in Python) 3) for i in range(10): if i == 0: def a(): print i a() (outputs 0-9 in Python) Stefan From dalcinl at gmail.com Fri Jul 11 17:55:01 2008 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Fri, 11 Jul 2008 12:55:01 -0300 Subject: [Cython] problems with current cython-devel repo Message-ID: Hi all, after some time of inactivity, I'm back. I've just pulled cython-devel repo and I'm having some trouble. 1) It seems that the transforms code does not play nicely with the compiled Scanners.so. After installing Cython, I have to manually remove Scanners.so to get Cython running. 2) There is a typo, trivial to fix (just change a 'p' to a 's'), the diff pasted below: diff -r ebe5f9fae217 Cython/Compiler/Parsing.py --- a/Cython/Compiler/Parsing.py Fri Jul 11 16:52:31 2008 +0200 +++ b/Cython/Compiler/Parsing.py Fri Jul 11 12:17:34 2008 -0300 @@ -1615,7 +1615,7 @@ def p_c_simple_base_type(s, self_flag, n # Treat trailing [] on type as buffer access if s.sy == '[': if is_basic: - p.error("Basic C types do not support buffer access") + s.error("Basic C types do not support buffer access") return p_buffer_access(s, type_node) else: return type_node 3) But now, the code above seems to do not plat nice with the following: Error converting Pyrex file to C: ------------------------------------------------------------ ... cdef extern from *: int foo(int x[]) int bla(int[]) ^ ------------------------------------------------------------ /tmp/qq.pyx:3:15: Basic C types do not support buffer access -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From stefan_ml at behnel.de Fri Jul 11 18:23:27 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 11 Jul 2008 18:23:27 +0200 Subject: [Cython] problems with current cython-devel repo In-Reply-To: References: Message-ID: <487788FF.6040707@behnel.de> Hi, Lisandro Dalcin wrote: > Hi all, after some time of inactivity, I'm back. I've just pulled > cython-devel repo and I'm having some trouble. Yes, so do I. :-/ > 1) It seems that the transforms code does not play nicely with the > compiled Scanners.so. After installing Cython, I have to manually > remove Scanners.so to get Cython running. Not sure what the problem is, I'll have to dig into that. > 2) There is a typo, trivial to fix (just change a 'p' to a 's'), the Thanks. > 3) But now, the code above seems to do not plat nice with the following: > > Error converting Pyrex file to C: > ------------------------------------------------------------ > ... > cdef extern from *: > int foo(int x[]) > int bla(int[]) > ^ > ------------------------------------------------------------ > /tmp/qq.pyx:3:15: Basic C types do not support buffer access Hmm, ok, I commented that code out for now and added a test case. Should be back up working (at least for this part). Stefan From dalcinl at gmail.com Fri Jul 11 18:55:58 2008 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Fri, 11 Jul 2008 13:55:58 -0300 Subject: [Cython] problems with current cython-devel repo In-Reply-To: <487788FF.6040707@behnel.de> References: <487788FF.6040707@behnel.de> Message-ID: Now I've got another failure with my code. The following works (): def winsize(size): try: w, h = size except TypeError: w = size h = size return w, h But the following not: def winsize(size): try: w, h = size except TypeError: w = h = size return w, h So it seems there is a problem with a parallel assignement appearing in the body of an 'except' clause. On 7/11/08, Stefan Behnel wrote: > Hi, > > > Lisandro Dalcin wrote: > > Hi all, after some time of inactivity, I'm back. I've just pulled > > cython-devel repo and I'm having some trouble. > > > Yes, so do I. :-/ > > > > > 1) It seems that the transforms code does not play nicely with the > > compiled Scanners.so. After installing Cython, I have to manually > > remove Scanners.so to get Cython running. > > > Not sure what the problem is, I'll have to dig into that. > > > > > 2) There is a typo, trivial to fix (just change a 'p' to a 's'), the > > > Thanks. > > > > > 3) But now, the code above seems to do not plat nice with the following: > > > > Error converting Pyrex file to C: > > ------------------------------------------------------------ > > ... > > cdef extern from *: > > int foo(int x[]) > > int bla(int[]) > > ^ > > ------------------------------------------------------------ > > /tmp/qq.pyx:3:15: Basic C types do not support buffer access > > > Hmm, ok, I commented that code out for now and added a test case. > > Should be back up working (at least for this part). > > Stefan > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From stefan_ml at behnel.de Fri Jul 11 19:06:07 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 11 Jul 2008 19:06:07 +0200 Subject: [Cython] problems with current cython-devel repo In-Reply-To: References: <487788FF.6040707@behnel.de> Message-ID: <487792FF.5020407@behnel.de> Lisandro Dalcin wrote: > Now I've got another failure with my code. > > The following works (): > > def winsize(size): > try: > w, h = size > except TypeError: > w = size > h = size > return w, h > > But the following not: > > def winsize(size): > try: > w, h = size > except TypeError: > w = h = size > return w, h > > > So it seems there is a problem with a parallel assignement appearing > in the body of an 'except' clause. Not only there. Parallel assignments seem to be pretty much broken currently. I'm working on it. Try revision 771 if you need something working now. Stefan From stefan_ml at behnel.de Sat Jul 12 06:16:53 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 12 Jul 2008 06:16:53 +0200 Subject: [Cython] Interesting Py3k thread on generator expressions Message-ID: <48783035.5070802@behnel.de> Hi, there's an interesting thread on genexps on the Py3k list, in case someone is interested in implementing them. :) http://mail.python.org/pipermail/python-3000/2008-July/014311.html Stefan From robertwb at math.washington.edu Sat Jul 12 07:47:46 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 11 Jul 2008 22:47:46 -0700 Subject: [Cython] generators In-Reply-To: <48777670.7030907@behnel.de> References: <4875AEC8.9070006@behnel.de> <48777670.7030907@behnel.de> Message-ID: <9328F0F5-9347-4100-ABCC-95DA9F49B966@math.washington.edu> On Jul 11, 2008, at 8:04 AM, Stefan Behnel wrote: > Hi, > > Robert Bradshaw wrote: >> This is in fact very much in line with how closures would be >> implemented--there would be an extension class that would hold the >> state >> (and garbage collection would happen when this object went out of >> scope, >> so no issues for generators or closures). > > Just coming up with a couple of corner cases to make sure they will > work in > this solution. Could you comment on how they are handled? > > 1) > > def f(): > x = 1 > def a(): print x > def b(): print x > return (a,b) There will be a cdef class created for a and b, as well as one representing the scope of f, with a __call__ method containing the body of the function. The def a() and def b() lines would instantiate this class and assign them to local variable a and b (which have a reference to the containing scope). When both of these are collected, the scope will be as well. > 2) > > for i in range(10): > def a(): print i > a() > > (outputs 0-9 in Python) This isn't an inner function at all. To fix the scoping rules, "def" would create a function at the top level, and then do an assignment to the variable "a." > 3) > > for i in range(10): > if i == 0: > def a(): print i > a() > > (outputs 0-9 in Python) Exactly the same as (2). There is nothing special about the scoping here, it's "outer" scope is the module scope, just as in the standard def function. The reason Cython has this restriction is that functions can only be created once. - Robert From stefan_ml at behnel.de Sat Jul 12 08:06:12 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 12 Jul 2008 08:06:12 +0200 Subject: [Cython] generators In-Reply-To: <9328F0F5-9347-4100-ABCC-95DA9F49B966@math.washington.edu> References: <4875AEC8.9070006@behnel.de> <48777670.7030907@behnel.de> <9328F0F5-9347-4100-ABCC-95DA9F49B966@math.washington.edu> Message-ID: <487849D4.5080006@behnel.de> Hi, Robert Bradshaw wrote: > On Jul 11, 2008, at 8:04 AM, Stefan Behnel wrote: >> 1) >> >> def f(): >> x = 1 >> def a(): print x >> def b(): print x >> return (a,b) > > There will be a cdef class created for a and b, as well as one > representing the scope of f, with a __call__ method containing the > body of the function. The def a() and def b() lines would instantiate > this class and assign them to local variable a and b (which have a > reference to the containing scope). When both of these are collected, > the scope will be as well. Yes, that sounds correct to me. >> 2) >> >> for i in range(10): >> def a(): print i >> a() >> >> (outputs 0-9 in Python) > > This isn't an inner function at all. To fix the scoping rules, "def" > would create a function at the top level, and then do an assignment > to the variable "a." I actually meant it to be an inner function of a function, so this wasn't a clear example. But I can see that this would be handled the same way as above. So, yes, I'm happy with that implementation. I just came up with these examples when I was discussing the topic with some people at EuroPython, and I couldn't really tell them how we would handle that. Stefan From languitar at semipol.de Mon Jul 14 13:25:04 2008 From: languitar at semipol.de (Johannes Wienke) Date: Mon, 14 Jul 2008 13:25:04 +0200 Subject: [Cython] Raising exceptions under python 2.5 Message-ID: <487B3790.2090801@semipol.de> Hi again, I've developed my project under python 2.4 and now I got to get it running under 2.5. The main problem I notice at the moment with the cython party is raising exceptions. Running the project I get constant errors of the form: Traceback (most recent call last): File "/tmp/shIP/test/shiptests/datatest.py", line 49, in testStartData self.__dataStore.addDataObserver(DataStore.START_IDENT, plugin1) File "data.pyx", line 535, in ship.data.DataStore.addDataObserver (build/temp.linux-i686-2.5/pyrex/data.c:4314) File "data.pyx", line 472, in ship.data.DataStore.__addToObserverDict (build/temp.linux-i686-2.5/pyrex/data.c:3908) File "data.pyx", line 558, in ship.data.DataStore.__findObserverItem (build/temp.linux-i686-2.5/pyrex/data.c:4478) TypeError: raise: exception must be an old-style class or instance The line contains: raise KeyError("No _ObserverItem found belonging to '%s'." % observer) I thought this is the preferred way of raising exceptions. Why is cython / python complaining about this? Thanks for the help Johannes -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature Url : http://codespeak.net/pipermail/cython-dev/attachments/20080714/1e042005/attachment.pgp From jwienke at techfak.uni-bielefeld.de Mon Jul 14 15:27:06 2008 From: jwienke at techfak.uni-bielefeld.de (Johannes Wienke) Date: Mon, 14 Jul 2008 15:27:06 +0200 Subject: [Cython] Raising exceptions under python 2.5 In-Reply-To: <487B3790.2090801@semipol.de> References: <487B3790.2090801@semipol.de> Message-ID: <487B542A.8040407@techfak.uni-bielefeld.de> Am 07/14/2008 01:25 PM schrieb Johannes Wienke: > Hi again, > > I've developed my project under python 2.4 and now I got to get it > running under 2.5. The main problem I notice at the moment with the > cython party is raising exceptions. Running the project I get constant > errors of the form: > > Traceback (most recent call last): > File "/tmp/shIP/test/shiptests/datatest.py", line 49, in testStartData > self.__dataStore.addDataObserver(DataStore.START_IDENT, plugin1) > File "data.pyx", line 535, in ship.data.DataStore.addDataObserver > (build/temp.linux-i686-2.5/pyrex/data.c:4314) > File "data.pyx", line 472, in ship.data.DataStore.__addToObserverDict > (build/temp.linux-i686-2.5/pyrex/data.c:3908) > File "data.pyx", line 558, in ship.data.DataStore.__findObserverItem > (build/temp.linux-i686-2.5/pyrex/data.c:4478) > TypeError: raise: exception must be an old-style class or instance > > The line contains: > raise KeyError("No _ObserverItem found belonging to '%s'." % observer) > > I thought this is the preferred way of raising exceptions. Why is cython > / python complaining about this? I'm sorry, my fault, compiled against python 2.4. - Johannes -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature Url : http://codespeak.net/pipermail/cython-dev/attachments/20080714/49b10a1e/attachment.pgp From stefan_ml at behnel.de Mon Jul 14 21:20:02 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 14 Jul 2008 21:20:02 +0200 Subject: [Cython] Raising exceptions under python 2.5 In-Reply-To: <487B3790.2090801@semipol.de> References: <487B3790.2090801@semipol.de> Message-ID: <487BA6E2.90305@behnel.de> Hi, just a quick comment. Johannes Wienke wrote: > raise KeyError("No _ObserverItem found belonging to '%s'." % observer) it's actually faster to write raise KeyError, "No _ObserverItem found belonging to '%s'." % observer even if that's not the preferred way of doing it in Python. Stefan From greg.ewing at canterbury.ac.nz Tue Jul 15 01:52:59 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 15 Jul 2008 11:52:59 +1200 Subject: [Cython] generators In-Reply-To: <4875C415.6010706@student.matnat.uio.no> References: <4875AEC8.9070006@behnel.de> <4875C415.6010706@student.matnat.uio.no> Message-ID: <487BE6DB.6000501@canterbury.ac.nz> Dag Sverre Seljebotn wrote: > To be even more specific, it would help if self.result_code everywhere > was calculated during code generation rather than type analysis You might want to take a look at the latest Pyrex, as I've more or less done this already. The code generation phase now calls the result() method of a node to get its result code instead of accessing the result_code attribute directly (and not all nodes even have a result_code any more). -- Greg From greg.ewing at canterbury.ac.nz Tue Jul 15 02:26:40 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 15 Jul 2008 12:26:40 +1200 Subject: [Cython] Raising exceptions under python 2.5 In-Reply-To: <487BA6E2.90305@behnel.de> References: <487B3790.2090801@semipol.de> <487BA6E2.90305@behnel.de> Message-ID: <487BEEC0.4000802@canterbury.ac.nz> Stefan Behnel wrote: > it's actually faster to write > > raise KeyError, "No _ObserverItem found belonging to '%s'." % observer But only if the exception is both raised and caught in Pyrex or Cython code (because no exception instance is created). Otherwise it makes little difference. -- Greg From cswiercz at gmail.com Wed Jul 16 00:53:47 2008 From: cswiercz at gmail.com (Chris Swierczewski) Date: Tue, 15 Jul 2008 15:53:47 -0700 Subject: [Cython] Wrapping C++ Classes Message-ID: Hello, I'm having a compile issue with wrapping a certain C++ class. Allow me to describe the situation first before stating the issue. The Situation: the opentick API is a collection of cross-platform, cross-language libraries (and headers in the C++ flavor) for receiving live streams of financial data. (See www.opentick.com) Among the API/source languages available, there is a collection of cross-platform C++ headers along with a static library. What is meant by cross-platform? Well, depending on which platform you're using, different parts of the code are called and different macro definitions are made. For example, when compiling on a Linux system one must write #define __LINUX at the top of their code before calling the appropriate headers. Now, the main class is defined in the following way: class DLL_EXP OTClient: { public: ... } When running in the windows environment (that is, if you #define __WIN), DLL_EXP is set to "__declspec(dllexport)" which, to my understanding, has to do with using .dll files for callbacks akin to how twisted works. However, when compiling in the Linux environment, (#define __LINUX) the definition of the macro is "empty". That is, at some point before the definition of the class, we simply have #define DLL_EXP implying that no .dll calls need to be made. My Question (Finally): How do I go about wrapping this C++ class with this strange little define hanging around the class declaration? Methinks I'm having compile issues precisely because of this addition. I checked the Cython wiki and the Sage Wiki on wrapping C++ classes and couldn't find anything that looked like this. Thank you very much in advance for any help! (And thank you for such an awesome technology!) -- Chris Swierczewski cswiercz at gmail.com mobile: 253 2233721 From michael.abshoff at googlemail.com Wed Jul 16 00:59:25 2008 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Tue, 15 Jul 2008 15:59:25 -0700 Subject: [Cython] Wrapping C++ Classes In-Reply-To: References: Message-ID: <487D2BCD.5060103@gmail.com> Chris Swierczewski wrote: > Hello, > > I'm having a compile issue with wrapping a certain C++ class. Allow me > to describe the situation first before stating the issue. > > The Situation: the opentick API is a collection of cross-platform, > cross-language libraries (and headers in the C++ flavor) for receiving > live streams of financial data. (See www.opentick.com) Among the > API/source languages available, there is a collection of > cross-platform C++ headers along with a static library. What is meant > by cross-platform? Well, depending on which platform you're using, > different parts of the code are called and different macro definitions > are made. For example, when compiling on a Linux system one must write > > #define __LINUX > > at the top of their code before calling the appropriate headers. Now, > the main class is defined in the following way: > > class DLL_EXP OTClient: > { > public: > ... > } > > When running in the windows environment (that is, if you #define > __WIN), DLL_EXP is set to "__declspec(dllexport)" which, to my > understanding, has to do with using .dll files for callbacks akin to > how twisted works. However, when compiling in the Linux environment, > (#define __LINUX) the definition of the macro is "empty". That is, at > some point before the definition of the class, we simply have > > #define DLL_EXP > > implying that no .dll calls need to be made. > > My Question (Finally): How do I go about wrapping this C++ class with > this strange little define hanging around the class declaration? > Methinks I'm having compile issues precisely because of this addition. > I checked the Cython wiki and the Sage Wiki on wrapping C++ classes > and couldn't find anything that looked like this. Thank you very much > in advance for any help! (And thank you for such an awesome > technology!) > Hi, Have you tried passing -D__Linux at the build time of the extension? I saw your discussion with Glenn and the removal of the __declspec(dllexport) constructs as suggested by him will bite us in the ass down the road since we intent to support MSVC for which the above constructs are needed. Cheers, Michael From cswiercz at gmail.com Wed Jul 16 01:05:06 2008 From: cswiercz at gmail.com (Chris Swierczewski) Date: Tue, 15 Jul 2008 16:05:06 -0700 Subject: [Cython] Wrapping C++ Classes In-Reply-To: <487D2BCD.5060103@gmail.com> References: <487D2BCD.5060103@gmail.com> Message-ID: Michael, > Have you tried passing -D__Linux at the build time of the extension? I didn't know that you could pass those kinds of things at the Sage Extension declaration. What I did was make a copy of the header and hard-coded the definition. (OTClient.h ==> OTClient__LINUX.h containing #define __LINUX) I'll have to try that right now, though I'm not sure if it will make a difference. > I saw your discussion with Glenn and the removal of the > __declspec(dllexport) constructs as suggested by him will bite us in the > ass down the road since we intent to support MSVC for which the above > constructs are needed. Yeah, I was a bit weary of taking that route. The opentick libraries look way too intricate to begin mucking around with that. -- Chris From cswiercz at gmail.com Wed Jul 16 01:18:58 2008 From: cswiercz at gmail.com (Chris Swierczewski) Date: Tue, 15 Jul 2008 16:18:58 -0700 Subject: [Cython] Wrapping C++ Classes In-Reply-To: <487D2BCD.5060103@gmail.com> References: <487D2BCD.5060103@gmail.com> Message-ID: Michael, > Have you tried passing -D__Linux at the build time of the extension? Just to be sure, does that mean adding opentick = Extension('sage.finance.opentick', ... extra_compile_args = ["-D __Linux"] ) If so, then I still get the following error: Updating Cython code.... sage/finance/opentick.pyx --> /Users/cswiercz/sage/local//lib/python/site-packages//sage/finance/opentick.pyx Building sage/finance/opentick.cpp because it depends on sage/finance/opentick.pyx. python2.5 `which cython` --embed-positions --incref-local-binop -I/Users/cswiercz/sage/devel/sage-opentick -o sage/finance/opentick.cpp sage/finance/opentick.pyx Error converting Pyrex file to C: ------------------------------------------------------------ ... cdef class Opentick: cdef c_OTClient *otclient ^ ------------------------------------------------------------ /Users/cswiercz/sage/devel/sage-opentick/sage/finance/opentick.pxd:2:20: Syntax error in C variable declaration sage: Error running cython. sage: There was an error installing modified sage library code. when I have the following in opentick.pxd cdef extern from "OTClient.h": ctypedef struct c_OTClient "OTClient": pass c_OTClient *new_OTClient "new OTClient" () void del_OTClient "delete" (c_OTClient *otclient) -- Chris From greg.ewing at canterbury.ac.nz Wed Jul 16 02:57:59 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 16 Jul 2008 12:57:59 +1200 Subject: [Cython] Wrapping C++ Classes In-Reply-To: References: Message-ID: <487D4797.5050708@canterbury.ac.nz> Chris Swierczewski wrote: > For example, when compiling on a Linux system one must write > > #define __LINUX I would write a small .h file containing the appropriate #define and use a 'cdef extern from' declaration to include it. -- Greg From robertwb at math.washington.edu Wed Jul 16 08:56:58 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Tue, 15 Jul 2008 23:56:58 -0700 Subject: [Cython] Wrapping C++ Classes In-Reply-To: References: <487D2BCD.5060103@gmail.com> Message-ID: <5F6C6AF3-C60C-4C20-BDEB-6514CDF84E84@math.washington.edu> On Jul 15, 2008, at 4:18 PM, Chris Swierczewski wrote: > Michael, > >> Have you tried passing -D__Linux at the build time of the extension? > > Just to be sure, does that mean adding > > opentick = Extension('sage.finance.opentick', > ... > extra_compile_args = ["-D > __Linux"] ) > > If so, then I still get the following error: > > Updating Cython code.... > sage/finance/opentick.pyx --> > /Users/cswiercz/sage/local//lib/python/site-packages//sage/finance/ > opentick.pyx > > Building sage/finance/opentick.cpp because it depends on > sage/finance/opentick.pyx. > python2.5 `which cython` --embed-positions --incref-local-binop > -I/Users/cswiercz/sage/devel/sage-opentick -o > sage/finance/opentick.cpp sage/finance/opentick.pyx > > Error converting Pyrex file to C: > ------------------------------------------------------------ > ... > cdef class Opentick: > cdef c_OTClient *otclient > ^ > ------------------------------------------------------------ > > /Users/cswiercz/sage/devel/sage-opentick/sage/finance/opentick.pxd: > 2:20: > Syntax error in C variable declaration > sage: Error running cython. > sage: There was an error installing modified sage library code. > > when I have the following in opentick.pxd > > cdef extern from "OTClient.h": > ctypedef struct c_OTClient "OTClient": > pass > c_OTClient *new_OTClient "new OTClient" () > void del_OTClient "delete" (c_OTClient *otclient) I am unable to reproduce this. What I have is: ----------------- opentick.pxd --------------- cdef extern from "OTClient.h": ctypedef struct c_OTClient "OTClient": pass c_OTClient *new_OTClient "new OTClient" () void del_OTClient "delete" (c_OTClient *otclient) cdef class Opentick: pass ----------------- opentick.pyx --------------- cdef class Opentick: pass --------------------------------------------------- which compiles fine to c. Perhaps you're redefining something somewhere? - Robert From robertwb at math.washington.edu Wed Jul 16 09:15:55 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Wed, 16 Jul 2008 00:15:55 -0700 Subject: [Cython] Mapping a really strange typedef in Cython In-Reply-To: <48676F15.1090607@semipol.de> References: <48676F15.1090607@semipol.de> Message-ID: <22E10F70-3084-4B0C-B0EE-7B38B2BDE23B@math.washington.edu> On Jun 29, 2008, at 4:16 AM, Johannes Wienke wrote: > Hi again, > > I'm struggling to get a really strange typedef working in Cython. The > original C (preprocessor) code looks like this: > > #define IW_CTAB_SIZE 256 > typedef guchar (*iwColtab)[3]; > > extern iwColtab iw_def_col_tab; > > /* Color formats of the data for prev_render...() */ > #define IW_RGB ((iwColtab)0) > > // some more formats > > #define IW_BW ((iwColtab)7) > #define IW_GRAY IW_BW > #define IW_INDEX (iw_def_col_tab) > > #define IW_COLFORMAT_MAX IW_BW /* last 'special' definition */ > > The purpose of this strange construct was that one can use normal > color > spaces without struggling, but one is also able to use a custom > color table. > > This definition was originally used like this: > > void prev_colConvert (iwColtab ctab, gint r, gint g, gint b, > gint *rd, gint *gd, gint *bd) > { > if (ctab == IW_RGB) { > *rd = r; > *gd = g; > *bd = b; > } else if (ctab == IW_BGR) { > *rd = b; > *gd = g; > *bd = r; > } else if (ctab > IW_COLFORMAT_MAX && r>=0 && r < IW_CTAB_SIZE){ > *rd = ctab[r][0]; > *gd = ctab[r][1]; > *bd = ctab[r][2]; > } > > } > > I need to be able to do the same thing in cython with this definition. > My current approach was simply using a enum to emulate it: > > cdef extern from "gui/Gimage.h": > > ctypedef enum iwColtab: > IW_RGB > # ... > IW_GRAY > IW_CTAB_SIZE > IW_COLFORMAT_MAX > > The normal color modes work like this, but using the color table > doesn't > work: > > Error converting Pyrex file to C: > ------------------------------------------------------------ > ... > elif ctab == IW_YUV: > return Color(1.164 * (r - 16) + 1.596 * (b - 128), > 1.164 * (r - 16) - 0.813 * (b - 128) - 0.391 * > (g - > 128), > 1.164 * (r - 16) + 2.018 * (g - 128)) > elif (ctab > IW_COLFORMAT_MAX and r >= 0 and r < IW_CTAB_SIZE): > return Color(ctab[r][0], ctab[r][1], ctab[r][2]) > ^ > ------------------------------------------------------------ > > image_utils.pyx:144:25: Attempting to index non-array type 'iwColtab' > > I really have no idea how to solve this. Has anyone got a solution > for this? > > Thanks for the help > Johannes I'm not sure--what is the definition of iwColtab? An enum won't work here because one can't index into it (in Cython or in C) so I think the only thing to do here is to have two separate types (the enum, and iwColtab) and do some casting. - Robert -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 186 bytes Desc: This is a digitally signed message part Url : http://codespeak.net/pipermail/cython-dev/attachments/20080716/faba004e/attachment.pgp From robertwb at math.washington.edu Wed Jul 16 09:21:20 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Wed, 16 Jul 2008 00:21:20 -0700 Subject: [Cython] Make the Cython workflow easier In-Reply-To: References: <20080619144328.GC32587@phare.normalesup.org> <485A92F9.8040203@behnel.de> <20080619183809.GB10875@phare.normalesup.org> <96144A15-25C9-4091-AB9C-6A575746AF31@math.washington.edu> <20080619184426.GC10875@phare.normalesup.org> <485B3E5A.8030605@behnel.de> <20080620122525.GC3651@phare.normalesup.org> <485BF242.80408@student.matnat.uio.no> Message-ID: <8018EF36-0E09-4E0B-A7EF-1BE24511AFB0@math.washington.edu> On Jun 20, 2008, at 11:13 AM, Fernando Perez wrote: > On Fri, Jun 20, 2008 at 11:09 AM, Dag Sverre Seljebotn > wrote: >> Lisandro Dalcin wrote: >>> Gael, not sure if this will help you, but I wrote this script >>> based in >>> distutils for building *.pyx files on the fly (perhaps with others >>> companion C/C++ sources). You basically use it as this: >>> >> Also remember that Pex already has this feature (as a wrapper/build >> system for Cython): >> >> pexlang.sourceforge.net > > This is just a generic request: when the dust settles a bit and you > guys have a clearer picture of where the whole pex/cython integration > regarding numpy is going to go, could you provide a little summary? > I'm super interested in this (and I can guarantee I'm not the only > one) but right now things seem to be in a strong state of flux, so I'm > OK waiting. The big part is Dag's GSoC project on buffer support. There are a lot of other small improvements which we would like to incorporate which can be found at http://wiki.cython.org/enhancements/pex . > Furthermore, it would be great if you think some of this will be in > reasonable state (even alpha, but solid enough for knowing where it > will go) by August, to present it at scipy 2008: That is certainly the goal. > > http://conference.scipy.org/ > > Robert B. is already coming to present a cython tutorial, so perhaps a > short talk on this would be OK for him to give. It would be very well > received. > > Cheers, > > f > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev From robertwb at math.washington.edu Fri Jul 18 09:28:28 2008 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 18 Jul 2008 00:28:28 -0700 Subject: [Cython] Object not released when calling cpdef method from inherited class ? In-Reply-To: <1215488230.20373.14.camel@nikopol> References: <1215488230.20373.14.camel@nikopol> Message-ID: <62C56E05-9EC6-4095-AFCD-7DA5D5E7DB72@math.washington.edu> Fixed, see http://trac.cython.org/cython_trac/ticket/24 On Jul 7, 2008, at 8:37 PM, Guillaume Chereau wrote: > Hello all, > > I am currently using cython for a project related to openmoko cell > phone : http://charlie137-2.blogspot.com/2008/07/introducing- > tichy.html > > I noticed that the memory is constantly increasing when I use > cython. I > tracked down the problem to the case where I create a subclass of a > cython class and then redefine a cpdef method, asking it to call the > parent method, but ONLY if the cpdef method then call an other cpdef > method ! > > Here is the smallest example I could come with that fails : > > > == test.pyx == > > cdef class A: > cpdef func(self): > return > > cpdef test(self): > self.func() > > > == main.py == > > import test > import gc > > class B(test.A): > def test(self): > test.A.test(self) > > b = B() > > for i in range(10): > b.test() > gc.collect() > print len(gc.get_objects()) > > > The output will show that some objects are not released. > > Is it a known bug ? Is there a way to avoid it ? > > I tried with both Cython 0.9.6 and Cython 0.9.8 > > cheers, > -Charlie > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 186 bytes Desc: This is a digitally signed message part Url : http://codespeak.net/pipermail/cython-dev/attachments/20080718/619d97af/attachment.pgp From charlie at openmoko.org Fri Jul 18 10:27:41 2008 From: charlie at openmoko.org (Guillaume Chereau) Date: Fri, 18 Jul 2008 16:27:41 +0800 Subject: [Cython] Object not released when calling cpdef method from inherited class ? In-Reply-To: <62C56E05-9EC6-4095-AFCD-7DA5D5E7DB72@math.washington.edu> References: <1215488230.20373.14.camel@nikopol> <62C56E05-9EC6-4095-AFCD-7DA5D5E7DB72@math.washington.edu> Message-ID: <1216369661.7529.48.camel@nikopol> Ah excellent ! Thanks a lot ! -Charlie On Fri, 2008-07-18 at 00:28 -0700, Robert Bradshaw wrote: > Fixed, see http://trac.cython.org/cython_trac/ticket/24 > > On Jul 7, 2008, at 8:37 PM, Guillaume Chereau wrote: > > > Hello all, > > > > I am currently using cython for a project related to openmoko cell > > phone : http://charlie137-2.blogspot.com/2008/07/introducing- > > tichy.html > > > > I noticed that the memory is constantly increasing when I use > > cython. I > > tracked down the problem to the case where I create a subclass of a > > cython class and then redefine a cpdef method, asking it to call the > > parent method, but ONLY if the cpdef method then call an other cpdef > > method ! > > > > Here is the smallest example I could come with that fails : > > > > > > == test.pyx == > > > > cdef class A: > > cpdef func(self): > > return > > > > cpdef test(self): > > self.func() > > > > > > == main.py == > > > > import test > > import gc > > > > class B(test.A): > > def test(self): > > test.A.test(self) > > > > b = B() > > > > for i in range(10): > > b.test() > > gc.collect() > > print len(gc.get_objects()) > > > > > > The output will show that some objects are not released. > > > > Is it a known bug ? Is there a way to avoid it ? > > > > I tried with both Cython 0.9.6 and Cython 0.9.8 > > > > cheers, > > -Charlie > > _______________________________________________ > > Cython-dev mailing list > > Cython-dev at codespeak.net > > http://codespeak.net/mailman/listinfo/cython-dev > > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://codespeak.net/pipermail/cython-dev/attachments/20080718/2578dbdc/attachment.pgp From stefan_ml at behnel.de Sat Jul 19 10:58:35 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 19 Jul 2008 10:58:35 +0200 Subject: [Cython] problems with current cython-devel repo In-Reply-To: References: <487788FF.6040707@behnel.de> Message-ID: <4881ACBB.4030207@behnel.de> Hi, Lisandro Dalcin wrote: > Now I've got another failure with my code. > > The following works (): > > def winsize(size): > try: > w, h = size > except TypeError: > w = size > h = size > return w, h > > But the following not: > > def winsize(size): > try: > w, h = size > except TypeError: > w = h = size > return w, h This should be fixed now. Stefan From stefan_ml at behnel.de Sat Jul 19 11:37:00 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 19 Jul 2008 11:37:00 +0200 Subject: [Cython] rev 809 - cdef exception propagation Message-ID: <4881B5BC.3030402@behnel.de> Hi, revision 809 implements a nice feature which lets cdef functions propagate exceptions by default. However, the change also introduced a problem with nogil functions, when called without the GIL being held. I added a run/nogil.pyx test case for now which crashes for me. Looking through the change I get the impression that the signature modification is done too late in the processing. To me, it feels more like this should be something that should be handled at the tree level and reflected directly in the function signature. What do you think? Stefan From dagss at student.matnat.uio.no Sat Jul 19 11:53:48 2008 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Sat, 19 Jul 2008 11:53:48 +0200 Subject: [Cython] Standard include path Message-ID: <4881B9AC.3060909@student.matnat.uio.no> Two issues that may or may not overlap: My test case has a call to malloc (tips to get around this are welcome but I'd still like to have a look at include paths). So I'm cimporting stdlib, meaning I have to export INCLUDE=Includes prior to running tests (which I'd like to avoid as apparently no other tests relies on this?) There are two ways of going about this: - Have runtests.py always include the Includes dir - Have Cython itself always include the Includes dir (i.e. always ../../Includes relative to Cython/Compiler/Main.py). I prefer the latter one; thoughts? I suppose this might also mean installing Includes "somewhere" on setup.py (or is that already done?) Also there's another import issue: For more sophisticated transforms, I've (or Robert, can't remember) come up with a __cython__ namespace which basically contains "any symbol the compiler might like to use". For instance: TreeFragment(u"__cython__.PyObject_GetBuffer(<__cython__.PyObject*>TMP, &BUFINFO, 0)") Eventually all the __Pyx-utils might be declared in this one too (though I'm certainly not going to do it for the sake of it). Then the idea is that __cython__ is *always* cimported (and available to the user as well). Any strong feelings against simply making this a __cython__.pxd in the aforementioned standard include path? Note the consequences: The compiler will be hard-wired to work with a certain pxd file. The alternative is to create a cython-scope similar to builtin-scope programatically (or stick everything that is needed in symtab.py directly). -- Dag Sverre From dagss at student.matnat.uio.no Sat Jul 19 14:14:18 2008 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Sat, 19 Jul 2008 14:14:18 +0200 Subject: [Cython] More concerns over extern ctypedefs Message-ID: <4881DA9A.3070406@student.matnat.uio.no> Consider this code: cdef extern from "t.h": ctypedef long hullo def foo(hullo h): pass I noticed that it creates code like this: PyArg_ParseTupleAndKeywords(__pyx_args, __pyx_kwds, "l", __pyx_argnames, &__pyx_v_h)) Which could probably destroy the stack if we don't have the exact typedef, right? Which means: One should either write our own parsing code instead (generate for each parameter signature), or generate the typestring at runtime, or insert C checks for the sizeof(the typedefs). The latter will destroy the current approach for numpy.int64 and friends and remove what I consider a nice feature. Just as I took all that care to allow approximate typedefs for buffers :-) Just noting it down, I suppose for now we'll live with it. But need to be more careful about recommending approximate typedefs. -- Dag Sverre From dagss at student.matnat.uio.no Sat Jul 19 14:21:06 2008 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Sat, 19 Jul 2008 14:21:06 +0200 Subject: [Cython] More concerns over extern ctypedefs In-Reply-To: <4881DA9A.3070406@student.matnat.uio.no> References: <4881DA9A.3070406@student.matnat.uio.no> Message-ID: <4881DC32.3030103@student.matnat.uio.no> Dag Sverre Seljebotn wrote: > Consider this code: > > cdef extern from "t.h": > ctypedef long hullo > > def foo(hullo h): pass > > I noticed that it creates code like this: > PyArg_ParseTupleAndKeywords(__pyx_args, __pyx_kwds, "l", __pyx_argnames, > &__pyx_v_h)) > > Which could probably destroy the stack if we don't have the exact > typedef, right? Which means: One should either write our own parsing > code instead (generate for each parameter signature), or generate the > typestring at runtime, or insert C checks for the sizeof(the typedefs). Just realized that a dynamic typestring could be generated at module load time, which should be perfectly unnoticeable. So this might be the way to go. Each extern ctypedef would grow a function similar to what is mentioned in the "buffer must know exact C types thread", but switching on size instead: switch (sizeof(T)