From magnus at hetland.org Fri Jan 2 22:02:33 2009 From: magnus at hetland.org (Magnus Lie Hetland) Date: Fri, 2 Jan 2009 22:02:33 +0100 Subject: [Cython] Array parameter... Message-ID: Hi! I'm writing a method of an extension class that will take an array as an argument (or, rather, a pointer and a size, for example). Now, I don't, strictly speaking, need to access this method from Python (i.e., it can be internal to the Cython code), but I'm writing my unit tests in Python, so it'd be nice... On the other hand, I'm thinking I wouldn't want to use list or the like as the type (in a cpdef) just to accomplish this, as that would, most likely, slow things down in actual Cython-to-Cython (or C-to-C) calls. Is there a standard way of doing this sort of thing? My thinking is that I can write a simple wrapper for the Python use, which does the proper conversion, but I don't know if that's ideal... (It doesn't *have* to be an array. I just need to pass in a variable number of doubles in an efficient manner.) -- Magnus Lie Hetland http://hetland.org From robertwb at math.washington.edu Fri Jan 2 22:11:08 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 2 Jan 2009 13:11:08 -0800 Subject: [Cython] Array parameter... In-Reply-To: References: Message-ID: <057C12EE-F3E5-4A44-9711-7DB589D575A0@math.washington.edu> On Jan 2, 2009, at 1:02 PM, Magnus Lie Hetland wrote: > Hi! > > I'm writing a method of an extension class that will take an array as > an argument (or, rather, a pointer and a size, for example). Now, I > don't, strictly speaking, need to access this method from Python > (i.e., it can be internal to the Cython code), but I'm writing my unit > tests in Python, so it'd be nice... On the other hand, I'm thinking I > wouldn't want to use list or the like as the type (in a cpdef) just to > accomplish this, as that would, most likely, slow things down in > actual Cython-to-Cython (or C-to-C) calls. Using a list to wrap a list of doubles will *vastly* slow things down, probably by a factor of 100 or more. > Is there a standard way of doing this sort of thing? My thinking is > that I can write a simple wrapper for the Python use, which does the > proper conversion, but I don't know if that's ideal... (It doesn't > *have* to be an array. I just need to pass in a variable number of > doubles in an efficient manner.) The standard way to do things now is to write a cdef function that takes a double* and length argument, then write a python wrapper for testing. You could also look at using NumPy arrays with the buffer interface, depending on what you're doing. - Robert From magnus at hetland.org Fri Jan 2 22:31:32 2009 From: magnus at hetland.org (Magnus Lie Hetland) Date: Fri, 2 Jan 2009 22:31:32 +0100 Subject: [Cython] Array parameter... In-Reply-To: <057C12EE-F3E5-4A44-9711-7DB589D575A0@math.washington.edu> References: <057C12EE-F3E5-4A44-9711-7DB589D575A0@math.washington.edu> Message-ID: <1ADCF1DA-7C9B-450F-BD74-77368DBA9A2C@hetland.org> On Jan 2, 2009, at 22:11 , Robert Bradshaw wrote: > The standard way to do things now is to write a cdef function that > takes a double* and length argument, then write a python wrapper for > testing. Yeah, that's what I had planned. Sounds like a good solution, I guess. (Now I just need a naming convention to separate the two ;-) > You could also look at using NumPy arrays with the buffer > interface, depending on what you're doing. Indeed. I played with that a bit; it might be the best bet, perhaps (although sticking with only Cython has the advantage that I can ship the generated C code, and the user needn't install anything beyond Python). -- Magnus Lie Hetland http://hetland.org From xi at gamma.dn.ua Fri Jan 2 22:41:38 2009 From: xi at gamma.dn.ua (Kirill Simonov) Date: Fri, 02 Jan 2009 23:41:38 +0200 Subject: [Cython] long long and VC6 Message-ID: <495E8A12.5040902@gamma.dn.ua> Hi, I'm using Cython 0.10.3 to build an extension module and the generated C file does not compile under VC6, which I use to create a Windows binary package for Python 2.3. The culprit is the following expression: PyObject *j = (likely(i >= 0) || !is_unsigned) ? PyInt_FromLong(i) : PyLong_FromUnsignedLongLong((sizeof(unsigned long long) > sizeof(Py_ssize_t) ? (1ULL << (sizeof(Py_ssize_t)*8)) : 0) + i); in the function __Pyx_GetItemInt. It looks like VC6 does not know about "long long". I suppose Cython should respect HAVE_LONG_LONG and use PY_LONG_LONG macro instead of "long long". Thanks, Kirill From robertwb at math.washington.edu Fri Jan 2 23:20:43 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 2 Jan 2009 14:20:43 -0800 Subject: [Cython] long long and VC6 In-Reply-To: <495E8A12.5040902@gamma.dn.ua> References: <495E8A12.5040902@gamma.dn.ua> Message-ID: On Jan 2, 2009, at 1:41 PM, Kirill Simonov wrote: > Hi, > > I'm using Cython 0.10.3 to build an extension module and the > generated C > file does not compile under VC6, which I use to create a Windows > binary > package for Python 2.3. The culprit is the following expression: > > PyObject *j = (likely(i >= 0) || !is_unsigned) ? PyInt_FromLong(i) : > PyLong_FromUnsignedLongLong((sizeof(unsigned long long) > > sizeof(Py_ssize_t) ? (1ULL << (sizeof(Py_ssize_t)*8)) : 0) + i); > > in the function __Pyx_GetItemInt. It looks like VC6 does not know > about > "long long". I suppose Cython should respect HAVE_LONG_LONG and use > PY_LONG_LONG macro instead of "long long". Hmm... I don't have any experience with VC6, but it looks like a bug to me. Could you file a bug report on trac? Also, do you know if PY_LONG_LONG guaranteed to be 64+ bits? - Robert From xi at gamma.dn.ua Sat Jan 3 00:14:48 2009 From: xi at gamma.dn.ua (Kirill Simonov) Date: Sat, 03 Jan 2009 01:14:48 +0200 Subject: [Cython] long long and VC6 In-Reply-To: References: <495E8A12.5040902@gamma.dn.ua> Message-ID: <495E9FE8.7020806@gamma.dn.ua> Hi Robert, >> >> I'm using Cython 0.10.3 to build an extension module and the >> generated C >> file does not compile under VC6, which I use to create a Windows >> binary >> package for Python 2.3. The culprit is the following expression: >> >> PyObject *j = (likely(i >= 0) || !is_unsigned) ? PyInt_FromLong(i) : >> PyLong_FromUnsignedLongLong((sizeof(unsigned long long) > >> sizeof(Py_ssize_t) ? (1ULL << (sizeof(Py_ssize_t)*8)) : 0) + i); >> >> in the function __Pyx_GetItemInt. It looks like VC6 does not know >> about >> "long long". I suppose Cython should respect HAVE_LONG_LONG and use >> PY_LONG_LONG macro instead of "long long". > > Hmm... I don't have any experience with VC6, but it looks like a bug > to me. Could you file a bug report on trac? Also, do you know if > PY_LONG_LONG guaranteed to be 64+ bits? I can't file a bug report, got Forbidden here: http://trac.cython.org/cython_trac/newticket I don't know about PY_LONG_LONG on 64+, but long long is C99, so I suppose it's a reasonably safe assumption. Thanks, Kirill. From michael.abshoff at googlemail.com Sat Jan 3 00:22:07 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Fri, 02 Jan 2009 15:22:07 -0800 Subject: [Cython] long long and VC6 In-Reply-To: <495E9FE8.7020806@gamma.dn.ua> References: <495E8A12.5040902@gamma.dn.ua> <495E9FE8.7020806@gamma.dn.ua> Message-ID: <495EA19F.3080309@gmail.com> Kirill Simonov wrote: > Hi Robert, Hi, >>> I'm using Cython 0.10.3 to build an extension module and the >>> generated C >>> file does not compile under VC6, which I use to create a Windows >>> binary >>> package for Python 2.3. The culprit is the following expression: >>> >>> PyObject *j = (likely(i >= 0) || !is_unsigned) ? PyInt_FromLong(i) : >>> PyLong_FromUnsignedLongLong((sizeof(unsigned long long) > >>> sizeof(Py_ssize_t) ? (1ULL << (sizeof(Py_ssize_t)*8)) : 0) + i); >>> >>> in the function __Pyx_GetItemInt. It looks like VC6 does not know >>> about >>> "long long". I suppose Cython should respect HAVE_LONG_LONG and use >>> PY_LONG_LONG macro instead of "long long". >> Hmm... I don't have any experience with VC6, but it looks like a bug >> to me. Could you file a bug report on trac? Also, do you know if >> PY_LONG_LONG guaranteed to be 64+ bits? > > I can't file a bug report, got Forbidden here: > > http://trac.cython.org/cython_trac/newticket > > I don't know about PY_LONG_LONG on 64+, but long long is C99, so I > suppose it's a reasonably safe assumption. I am not sure what Python 2.3 does, but the standard workaround for VC 6 is to define long long as __int64. But if PY_LONG_LONG is supported I would obviously recommend to use that. > Thanks, > Kirill. Cheers, Michael > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From xi at gamma.dn.ua Sat Jan 3 13:06:40 2009 From: xi at gamma.dn.ua (Kirill Simonov) Date: Sat, 03 Jan 2009 14:06:40 +0200 Subject: [Cython] long long and VC6 In-Reply-To: <495EA19F.3080309@gmail.com> References: <495E8A12.5040902@gamma.dn.ua> <495E9FE8.7020806@gamma.dn.ua> <495EA19F.3080309@gmail.com> Message-ID: <495F54D0.3050200@gamma.dn.ua> >>>> I'm using Cython 0.10.3 to build an extension module and the >>>> generated C >>>> file does not compile under VC6, which I use to create a Windows >>>> binary >>>> package for Python 2.3. The culprit is the following expression: >>>> >>>> PyObject *j = (likely(i >= 0) || !is_unsigned) ? PyInt_FromLong(i) : >>>> PyLong_FromUnsignedLongLong((sizeof(unsigned long long) > >>>> sizeof(Py_ssize_t) ? (1ULL << (sizeof(Py_ssize_t)*8)) : 0) + i); >>>> >>>> in the function __Pyx_GetItemInt. It looks like VC6 does not know >>>> about >>>> "long long". I suppose Cython should respect HAVE_LONG_LONG and use >>>> PY_LONG_LONG macro instead of "long long". >>> Hmm... I don't have any experience with VC6, but it looks like a bug >>> to me. Could you file a bug report on trac? Also, do you know if >>> PY_LONG_LONG guaranteed to be 64+ bits? >> I can't file a bug report, got Forbidden here: >> >> http://trac.cython.org/cython_trac/newticket >> >> I don't know about PY_LONG_LONG on 64+, but long long is C99, so I >> suppose it's a reasonably safe assumption. > > I am not sure what Python 2.3 does, but the standard workaround for VC 6 > is to define long long as __int64. But if PY_LONG_LONG is supported I > would obviously recommend to use that. Indeed you are right. After replacing "unsigned long long" with "unsigned PY_LONG_LONG" and "1ULL" with "((unsigned PY_LONG_LONG) 1)", it compiles under VC6. Thanks, Kirill. From dagss at student.matnat.uio.no Sat Jan 3 16:57:00 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: 03 Jan 2009 16:57:00 +0100 Subject: [Cython] Array parameter... Message-ID: <3313846656.4392892@smtp.netcom.no> Another thing you could do is create your own mini-array Cython class. E.g.: A class "DoubleArray" has C fields "ptr" and "len", and a subclass "PyDoubleArray" allows initializing those using a Python list (and managing the memory needed to hold the converted result etc.). That's about 20 lines, nothing fancy... So from the unit tests you'd pass inn "PyDoubleArray" and from Cython code raw "DoubleArray". Your methods could then be "cpdef" and use the same interface. Doesn't quite work out if you want to create C-callable code though. Dag Sverre Seljebotn -----Original Message----- From: Magnus Lie Hetland Date: Friday, Jan 2, 2009 10:31 pm Subject: Re: [Cython] Array parameter... To: cython-dev at codespeak.netReply-To: cython-dev at codespeak.net On Jan 2, 2009, at 22:11 , Robert Bradshaw wrote: > >> The standard way to do things now is to write a cdef function that > takes a double* and length argument, then write a python wrapper for > testing. > >Yeah, that's what I had planned. Sounds like a good solution, I guess. >(Now I just need a naming convention to separate the two ;-) > >> You could also look at using NumPy arrays with the buffer > interface, depending on what you're doing. > >Indeed. I played with that a bit; it might be the best bet, perhaps >(although sticking with only Cython has the advantage that I can ship >the generated C code, and the user needn't install anything beyond >Python). > >-- >Magnus Lie Hetland >http://hetland.org > > >_______________________________________________ >Cython-dev mailing list >Cython-dev at codespeak.net >http://codespeak.net/mailman/listinfo/cython-dev > From magnus at hetland.org Sat Jan 3 17:40:07 2009 From: magnus at hetland.org (Magnus Lie Hetland) Date: Sat, 3 Jan 2009 17:40:07 +0100 Subject: [Cython] Array parameter... In-Reply-To: <3313846656.4392892@smtp.netcom.no> References: <3313846656.4392892@smtp.netcom.no> Message-ID: On Jan 3, 2009, at 16:57 , Dag Sverre Seljebotn wrote: > Another thing you could do is create your own mini-array Cython > class. E.g.: A class "DoubleArray" has C fields "ptr" and "len", and > a subclass "PyDoubleArray" allows initializing those using a Python > list (and managing the memory needed to hold the converted result > etc.). That's about 20 lines, nothing fancy... > > So from the unit tests you'd pass inn "PyDoubleArray" and from > Cython code raw "DoubleArray". Cool idea. I thought about using some form of struct, but I didn't think quite hard enough, I guess (i.e., the subclassing bit :) > Your methods could then be "cpdef" and use the same interface. Indeed. If it turns out that this double array is the main cause of "wrapping", that would save me quite a bit of cookie-cutter conversion code (as it might apply to several methods). > Doesn't quite work out if you want to create C-callable code though. Right. Not an issue for now; I guess I'll just deal with that if it crops up. Thanks, - M -- Magnus Lie Hetland http://hetland.org From robertwb at math.washington.edu Sun Jan 4 00:21:16 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sat, 3 Jan 2009 15:21:16 -0800 Subject: [Cython] long long and VC6 In-Reply-To: <495E9FE8.7020806@gamma.dn.ua> References: <495E8A12.5040902@gamma.dn.ua> <495E9FE8.7020806@gamma.dn.ua> Message-ID: <681F9CAA-49DE-458C-8A01-71CC725403EB@math.washington.edu> On Jan 2, 2009, at 3:14 PM, Kirill Simonov wrote: > Hi Robert, >>> >>> I'm using Cython 0.10.3 to build an extension module and the >>> generated C >>> file does not compile under VC6, which I use to create a Windows >>> binary >>> package for Python 2.3. The culprit is the following expression: >>> >>> PyObject *j = (likely(i >= 0) || !is_unsigned) ? PyInt_FromLong(i) : >>> PyLong_FromUnsignedLongLong((sizeof(unsigned long long) > >>> sizeof(Py_ssize_t) ? (1ULL << (sizeof(Py_ssize_t)*8)) : 0) + i); >>> >>> in the function __Pyx_GetItemInt. It looks like VC6 does not know >>> about >>> "long long". I suppose Cython should respect HAVE_LONG_LONG and use >>> PY_LONG_LONG macro instead of "long long". >> >> Hmm... I don't have any experience with VC6, but it looks like a bug >> to me. Could you file a bug report on trac? Also, do you know if >> PY_LONG_LONG guaranteed to be 64+ bits? > > I can't file a bug report, got Forbidden here: > > http://trac.cython.org/cython_trac/newticket You need to log in to make new tickets. This is solely because we started getting spammed pretty bad when it was open. If you don't have an account, please send me an htpasswd file offlist and I'll put it in for you. - Robert From dagss at student.matnat.uio.no Wed Jan 7 15:12:55 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Wed, 07 Jan 2009 15:12:55 +0100 Subject: [Cython] Refcount nanny status Message-ID: <4964B867.3060402@student.matnat.uio.no> I've had little time for Cython lately and so refcount nanny development has been slow. But when development is going at this pace I figured it was better to upload what I had so it doesn't start bit-rotting on my disk. It is NOT FINISHED, but the ideas and framework is mostly in place, it is just a matter of working out all the places it fails. I'll continue it myself when I get time, but feel free... By default it is disabled and Cython works as before. To enable it: cd Cython/Runtime ./build.sh # yes, ugh! Let's deal with that when the rest is done cd ../.. python runtests.py --refnanny ... One can then get reference count errors for two reasons: - Most likely: The required calls to __Pyx_GOTREF and __Pyx_GIVEREF is missing in a given situation - Otherwise: There's a genuine reference counting problem. Here's the details: - Cython generates code that can always be run with or without the refnanny -- to run it with, compile the C code with CYTHON_REFNANNY defined. - Some boilerplate code is added when entering and exiting all functions (which should compile into nothing when not using the nanny) - Many calls to Py_INCREF etc. are replaced with __Pyx_INCREF etc. (but not in utility code which is in general trusted and doesn't use the nanny) - New calls are added: __Pyx_GOTREF, __Pyx_GIVEREF, __Pyx_XGIVEREF which are called after getting a reference from another function and before giving away a reference to a function/array/struct field. I.e: __Pyx_GIVEREF(obj) PyList_SET_ITEM(..., obj, ...) and obj = Py_Get... if (!obj) { ... } __Pyx_GOTREF(obj) (Also, XGIVEREF is called on __pyx_r prior to returning.) - There's a hook module in Cython/Runtime/refnanny.pyx. For now this is manually compiled with a Linux-only shell script, this must of course be done something about. See [1] for more plans for this module. - runtests.py has the --refnanny flag to turn on the refnanny (when it is done, I expect this to be the default). What this does is loading the hook module and compiling modules with CYTHON_REFNANNY defined. End result: - Extra decrefs are intercepted and only logged. - Functions continue as normal when this happen, but when the function returns for any reason a normal exception is raised if there's a leak or there was an extra decref somewhere. [1] The module can be written to contain as advanced diagnostics as we like, and can be inspected by runtests.py to read logs etc.. It would be entirely possible to extend this to allow, say, putting assertions on the refcount operations Cython code performs in the doctests that runtests.py runs: def f(): """ Test that this functions does the reference counting we would like. >>> import refnanny >>> a = 2 >>> refnanny.monitor_object(a) >>> f(a) >>> print refnanny.operations_on(a) INCREFS=10, DECREFS=10 """ # ... That's an argument for keeping it as a seperate module, but that's all I'm going to do about it for now. Dag Sverre From dagss at student.matnat.uio.no Wed Jan 7 16:40:55 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Wed, 07 Jan 2009 16:40:55 +0100 Subject: [Cython] Question on Code.entry_as_pyobject Message-ID: <4964CD07.6000906@student.matnat.uio.no> I have a problem with Code.entry_as_pyobject -- it is used in Code.put_var_decref to cast the entry to PyObject in the DECREF, however if there's no base class for a class the cast doesn't happen (a base class is explicitly checked for and if it is there a cast doesn't happen). Does this make sense? Should I change put_var_decref to use another method to cast (so that non-PyObject types are always casted), or change entry_as_pyobject? Dag Sverre From magnus at hetland.org Thu Jan 8 17:05:57 2009 From: magnus at hetland.org (Magnus Lie Hetland) Date: Thu, 8 Jan 2009 17:05:57 +0100 Subject: [Cython] cimporting and packages (and renaming during compile) Message-ID: Hi! Maybe I just haven't read the docs well enough... But I've got a few modules with source names such as foo_core.pyx, foo_base.pyx and foo_util.pyx. When I compile them, they end up in the foo package, as foo.core, foo.base and foo.util. Works well for importing into Python, but the problem is that the util module is primarily for use in the other two Cython modules ... and I can't get the cimport to work. I've added a foo_util.pxd (and there certainly may be errors in that), but the main problem seems to be the naming -- there isn't a correspondence (that Cython knows about) between the source file names and the module names. Or perhaps that isn't a problem? The error I keep getting is "Name 'some_function' not declared in module 'foo.util'". Any pointers? -- Magnus Lie Hetland http://hetland.org From magnus at hetland.org Thu Jan 8 17:24:22 2009 From: magnus at hetland.org (Magnus Lie Hetland) Date: Thu, 8 Jan 2009 17:24:22 +0100 Subject: [Cython] cimporting and packages (and renaming during compile) In-Reply-To: References: Message-ID: <43B7F2DE-EEA5-49CC-A8A6-76DEF4B459B9@hetland.org> On Jan 8, 2009, at 17:05, Magnus Lie Hetland wrote: > The error I keep getting is "Name 'some_function' not declared in > module 'foo.util'". > > Any pointers? Found the (perhaps obvious :-) solution here: http://wiki.cython.org/PackageHierarchy Sorry to be too trigger happy with my question :-> - M -- Magnus Lie Hetland http://hetland.org From robertwb at math.washington.edu Fri Jan 9 00:27:37 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Thu, 8 Jan 2009 15:27:37 -0800 Subject: [Cython] Question on Code.entry_as_pyobject In-Reply-To: <4964CD07.6000906@student.matnat.uio.no> References: <4964CD07.6000906@student.matnat.uio.no> Message-ID: <79004DAC-CE67-46B1-BC18-5FA2B18F1DAA@math.washington.edu> On Jan 7, 2009, at 7:40 AM, Dag Sverre Seljebotn wrote: > I have a problem with Code.entry_as_pyobject -- it is used in > Code.put_var_decref to cast the entry to PyObject in the DECREF, > however > if there's no base class for a class the cast doesn't happen (a base > class is explicitly checked for and if it is there a cast doesn't > happen). Does this make sense? Should I change put_var_decref to use > another method to cast (so that non-PyObject types are always casted), > or change entry_as_pyobject? entry_as_pyobject should always be a PyObject*. In general, I like avoiding excessive casting, so if it is already a PyObject* we shouldn't re-cast it. - Robert From robertwb at math.washington.edu Fri Jan 9 00:31:26 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Thu, 8 Jan 2009 15:31:26 -0800 Subject: [Cython] Refcount nanny status In-Reply-To: <4964B867.3060402@student.matnat.uio.no> References: <4964B867.3060402@student.matnat.uio.no> Message-ID: <974D9C23-56ED-42B3-AAF5-C496AA5F00AD@math.washington.edu> Thanks! I think this will help immensely for watching for memory leaks in Sage. On Jan 7, 2009, at 6:12 AM, Dag Sverre Seljebotn wrote: > I've had little time for Cython lately and so refcount nanny > development > has been slow. But when development is going at this pace I figured it > was better to upload what I had so it doesn't start bit-rotting on > my disk. > > It is NOT FINISHED, but the ideas and framework is mostly in place, it > is just a matter of working out all the places it fails. I'll continue > it myself when I get time, but feel free... > > By default it is disabled and Cython works as before. To enable it: > cd Cython/Runtime > ./build.sh # yes, ugh! Let's deal with that when the rest is done > cd ../.. > python runtests.py --refnanny ... > > One can then get reference count errors for two reasons: > - Most likely: The required calls to __Pyx_GOTREF and __Pyx_GIVEREF is > missing in a given situation > - Otherwise: There's a genuine reference counting problem. > > Here's the details: > - Cython generates code that can always be run with or without the > refnanny -- to run it with, compile the C code with CYTHON_REFNANNY > defined. > - Some boilerplate code is added when entering and exiting all > functions > (which should compile into nothing when not using the nanny) > - Many calls to Py_INCREF etc. are replaced with __Pyx_INCREF etc. > (but > not in utility code which is in general trusted and doesn't use the > nanny) > - New calls are added: __Pyx_GOTREF, __Pyx_GIVEREF, __Pyx_XGIVEREF > which > are called after getting a reference from another function and before > giving away a reference to a function/array/struct field. I.e: > > __Pyx_GIVEREF(obj) > PyList_SET_ITEM(..., obj, ...) > > and > > obj = Py_Get... > if (!obj) { ... } > __Pyx_GOTREF(obj) > > (Also, XGIVEREF is called on __pyx_r prior to returning.) > > - There's a hook module in Cython/Runtime/refnanny.pyx. For now > this is > manually compiled with a Linux-only shell script, this must of > course be > done something about. See [1] for more plans for this module. > - runtests.py has the --refnanny flag to turn on the refnanny (when it > is done, I expect this to be the default). What this does is > loading the > hook module and compiling modules with CYTHON_REFNANNY defined. > > End result: > - Extra decrefs are intercepted and only logged. > - Functions continue as normal when this happen, but when the function > returns for any reason a normal exception is raised if there's a > leak or > there was an extra decref somewhere. > > [1] The module can be written to contain as advanced diagnostics as we > like, and can be inspected by runtests.py to read logs etc.. It > would be > entirely possible to extend this to allow, say, putting assertions on > the refcount operations Cython code performs in the doctests that > runtests.py runs: > > def f(): > """ > Test that this functions does the reference counting we would like. > >>>> import refnanny >>>> a = 2 >>>> refnanny.monitor_object(a) >>>> f(a) >>>> print refnanny.operations_on(a) > INCREFS=10, DECREFS=10 > """ > # ... > > That's an argument for keeping it as a seperate module, but that's all > I'm going to do about it for now. > > Dag Sverre > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev From michael.abshoff at googlemail.com Fri Jan 9 00:51:52 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Thu, 08 Jan 2009 15:51:52 -0800 Subject: [Cython] Refcount nanny status In-Reply-To: <974D9C23-56ED-42B3-AAF5-C496AA5F00AD@math.washington.edu> References: <4964B867.3060402@student.matnat.uio.no> <974D9C23-56ED-42B3-AAF5-C496AA5F00AD@math.washington.edu> Message-ID: <49669198.2090006@gmail.com> Robert Bradshaw wrote: > Thanks! I think this will help immensely for watching for memory > leaks in Sage. Yep, this is great. Given that Dag seems to have been slowed down by real world things I am curious what needs to be done to get this into a more usable state? Linux only is fine for me for now, but if this depeneds on 0.11-devel I would probably be not be able to use this. Cheers, Michael > On Jan 7, 2009, at 6:12 AM, Dag Sverre Seljebotn wrote: > >> I've had little time for Cython lately and so refcount nanny >> development >> has been slow. But when development is going at this pace I figured it >> was better to upload what I had so it doesn't start bit-rotting on >> my disk. >> >> It is NOT FINISHED, but the ideas and framework is mostly in place, it >> is just a matter of working out all the places it fails. I'll continue >> it myself when I get time, but feel free... >> >> By default it is disabled and Cython works as before. To enable it: >> cd Cython/Runtime >> ./build.sh # yes, ugh! Let's deal with that when the rest is done >> cd ../.. >> python runtests.py --refnanny ... >> >> One can then get reference count errors for two reasons: >> - Most likely: The required calls to __Pyx_GOTREF and __Pyx_GIVEREF is >> missing in a given situation >> - Otherwise: There's a genuine reference counting problem. >> >> Here's the details: >> - Cython generates code that can always be run with or without the >> refnanny -- to run it with, compile the C code with CYTHON_REFNANNY >> defined. >> - Some boilerplate code is added when entering and exiting all >> functions >> (which should compile into nothing when not using the nanny) >> - Many calls to Py_INCREF etc. are replaced with __Pyx_INCREF etc. >> (but >> not in utility code which is in general trusted and doesn't use the >> nanny) >> - New calls are added: __Pyx_GOTREF, __Pyx_GIVEREF, __Pyx_XGIVEREF >> which >> are called after getting a reference from another function and before >> giving away a reference to a function/array/struct field. I.e: >> >> __Pyx_GIVEREF(obj) >> PyList_SET_ITEM(..., obj, ...) >> >> and >> >> obj = Py_Get... >> if (!obj) { ... } >> __Pyx_GOTREF(obj) >> >> (Also, XGIVEREF is called on __pyx_r prior to returning.) >> >> - There's a hook module in Cython/Runtime/refnanny.pyx. For now >> this is >> manually compiled with a Linux-only shell script, this must of >> course be >> done something about. See [1] for more plans for this module. >> - runtests.py has the --refnanny flag to turn on the refnanny (when it >> is done, I expect this to be the default). What this does is >> loading the >> hook module and compiling modules with CYTHON_REFNANNY defined. >> >> End result: >> - Extra decrefs are intercepted and only logged. >> - Functions continue as normal when this happen, but when the function >> returns for any reason a normal exception is raised if there's a >> leak or >> there was an extra decref somewhere. >> >> [1] The module can be written to contain as advanced diagnostics as we >> like, and can be inspected by runtests.py to read logs etc.. It >> would be >> entirely possible to extend this to allow, say, putting assertions on >> the refcount operations Cython code performs in the doctests that >> runtests.py runs: >> >> def f(): >> """ >> Test that this functions does the reference counting we would like. >> >>>>> import refnanny >>>>> a = 2 >>>>> refnanny.monitor_object(a) >>>>> f(a) >>>>> print refnanny.operations_on(a) >> INCREFS=10, DECREFS=10 >> """ >> # ... >> >> That's an argument for keeping it as a seperate module, but that's all >> I'm going to do about it for now. >> >> Dag Sverre >> _______________________________________________ >> Cython-dev mailing list >> Cython-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/cython-dev > > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From robertwb at math.washington.edu Fri Jan 9 01:00:05 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Thu, 8 Jan 2009 16:00:05 -0800 Subject: [Cython] Refcount nanny status In-Reply-To: <49669198.2090006@gmail.com> References: <4964B867.3060402@student.matnat.uio.no> <974D9C23-56ED-42B3-AAF5-C496AA5F00AD@math.washington.edu> <49669198.2090006@gmail.com> Message-ID: <8537FD98-2AFE-4C5E-A44B-3870F5AA03DB@math.washington.edu> On Jan 8, 2009, at 3:51 PM, Michael Abshoff wrote: > Robert Bradshaw wrote: >> Thanks! I think this will help immensely for watching for memory >> leaks in Sage. > > Yep, this is great. Given that Dag seems to have been slowed down by > real world things I am curious what needs to be done to get this > into a > more usable state? Linux only is fine for me for now, but if this > depeneds on 0.11-devel I would probably be not be able to use this. I am pretty sure it depends on 0.11-devel. In fact, it was written in part *for* 0.11 as the temporary variable handle has been revised and we want to catch any refcounting errors this may have introduced. - Robert From michael.abshoff at googlemail.com Fri Jan 9 01:10:38 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Thu, 08 Jan 2009 16:10:38 -0800 Subject: [Cython] Refcount nanny status In-Reply-To: <8537FD98-2AFE-4C5E-A44B-3870F5AA03DB@math.washington.edu> References: <4964B867.3060402@student.matnat.uio.no> <974D9C23-56ED-42B3-AAF5-C496AA5F00AD@math.washington.edu> <49669198.2090006@gmail.com> <8537FD98-2AFE-4C5E-A44B-3870F5AA03DB@math.washington.edu> Message-ID: <496695FE.8030803@gmail.com> Robert Bradshaw wrote: > On Jan 8, 2009, at 3:51 PM, Michael Abshoff wrote: > >> Robert Bradshaw wrote: >>> Thanks! I think this will help immensely for watching for memory >>> leaks in Sage. >> Yep, this is great. Given that Dag seems to have been slowed down by >> real world things I am curious what needs to be done to get this >> into a >> more usable state? Linux only is fine for me for now, but if this >> depeneds on 0.11-devel I would probably be not be able to use this. > > I am pretty sure it depends on 0.11-devel. In fact, it was written in > part *for* 0.11 as the temporary variable handle has been revised and > we want to catch any refcounting errors this may have introduced. Sure, I am tempted to drop in 0.11-devel and see if it compiles the Sage library and if it then even starts Sage :). Hopefully 0.11 will come soon and we will find a way to integrate the refcount nanny as a special mode during doctesting of Sage, i.e. during compile time of Cython activate it and if we then somehow could have the doctesting framework have some mode where *every* reference was watched this would truly rock. If this is expensive performance wise it doesn't matter since it would be a special debug mode :) > - Robert Cheers, Michael > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From robertwb at math.washington.edu Fri Jan 9 01:16:19 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Thu, 8 Jan 2009 16:16:19 -0800 Subject: [Cython] Refcount nanny status In-Reply-To: <496695FE.8030803@gmail.com> References: <4964B867.3060402@student.matnat.uio.no> <974D9C23-56ED-42B3-AAF5-C496AA5F00AD@math.washington.edu> <49669198.2090006@gmail.com> <8537FD98-2AFE-4C5E-A44B-3870F5AA03DB@math.washington.edu> <496695FE.8030803@gmail.com> Message-ID: On Jan 8, 2009, at 4:10 PM, Michael Abshoff wrote: > Robert Bradshaw wrote: >> On Jan 8, 2009, at 3:51 PM, Michael Abshoff wrote: >> >>> Robert Bradshaw wrote: >>>> Thanks! I think this will help immensely for watching for memory >>>> leaks in Sage. >>> Yep, this is great. Given that Dag seems to have been slowed down by >>> real world things I am curious what needs to be done to get this >>> into a >>> more usable state? Linux only is fine for me for now, but if this >>> depeneds on 0.11-devel I would probably be not be able to use this. >> >> I am pretty sure it depends on 0.11-devel. In fact, it was written in >> part *for* 0.11 as the temporary variable handle has been revised and >> we want to catch any refcounting errors this may have introduced. > > Sure, I am tempted to drop in 0.11-devel and see if it compiles the > Sage > library and if it then even starts Sage :). I've been meaning to do the same too, but just haven't had time. I haven't tried compiling Sage with 0.11 for too long... > Hopefully 0.11 will come soon Yep. I think the primary bottleneck is the above (and fixing any bugs we find there). I would also like a "memleak" mode for doctests, like the timing code, where it would run each block, say, twice, then get the memory usage, then run the block again and see if it went up at all. This would be great to have in general, but would also make me much happier knowing I didn't introduce new memory leaks. > and we will find a way to integrate the refcount nanny as a special > mode during doctesting of Sage, i.e. during compile time of Cython > activate it and if we then somehow could have the doctesting framework > have some mode where *every* reference was watched this would truly > rock. If this is expensive performance wise it doesn't matter since it > would be a special debug mode :) Yeah. Even if it was slower, it wouldn't be as slow as Valgrind. - Robert From michael.abshoff at googlemail.com Fri Jan 9 02:01:11 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Thu, 08 Jan 2009 17:01:11 -0800 Subject: [Cython] Refcount nanny status In-Reply-To: References: <4964B867.3060402@student.matnat.uio.no> <974D9C23-56ED-42B3-AAF5-C496AA5F00AD@math.washington.edu> <49669198.2090006@gmail.com> <8537FD98-2AFE-4C5E-A44B-3870F5AA03DB@math.washington.edu> <496695FE.8030803@gmail.com> Message-ID: <4966A1D7.4000107@gmail.com> Robert Bradshaw wrote: > On Jan 8, 2009, at 4:10 PM, Michael Abshoff wrote: Hi Robert, >> I am tempted to drop in 0.11-devel and see if it compiles the >> Sage >> library and if it then even starts Sage :). > > I've been meaning to do the same too, but just haven't had time. I > haven't tried compiling Sage with 0.11 for too long... Well, I don't get very far: AttributeError: 'CoerceToTempNode' object has no attribute 'old_temp' Traceback (most recent call last): File "/scratch/mabshoff/junk/sage-3.2.4.alpha0/local/bin//cython", line 8, in main(command_line = 1) File "/scratch/mabshoff/junk/sage-3.2.4.alpha0/local/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 730, in main result = compile(sources, options) File "/scratch/mabshoff/junk/sage-3.2.4.alpha0/local/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 707, in compile return compile_multiple(source, options) File "/scratch/mabshoff/junk/sage-3.2.4.alpha0/local/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 677, in compile_multiple result = run_pipeline(source, options) File "/scratch/mabshoff/junk/sage-3.2.4.alpha0/local/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 539, in run_pipeline err, enddata = context.run_pipeline(pipeline, source) File "/scratch/mabshoff/junk/sage-3.2.4.alpha0/local/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 198, in run_pipeline data = phase(data) File "/scratch/mabshoff/junk/sage-3.2.4.alpha0/local/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 139, in generate_pyx_code module_node.process_implementation(options, result) File "/scratch/mabshoff/junk/sage-3.2.4.alpha0/local/lib/python2.5/site-packages/Cython/Compiler/ModuleNode.py", line 71, in process_implementation self.generate_c_code(env, options, result) File "/scratch/mabshoff/junk/sage-3.2.4.alpha0/local/lib/python2.5/site-packages/Cython/Compiler/ModuleNode.py", line 266, in generate_c_code self.body.generate_function_definitions(env, code) File "/scratch/mabshoff/junk/sage-3.2.4.alpha0/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 325, in generate_function_definitions stat.generate_function_definitions(env, code) File "/scratch/mabshoff/junk/sage-3.2.4.alpha0/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 325, in generate_function_definitions stat.generate_function_definitions(env, code) File "/scratch/mabshoff/junk/sage-3.2.4.alpha0/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 1049, in generate_function_definitions self.body.generate_execution_code(code) File "/scratch/mabshoff/junk/sage-3.2.4.alpha0/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 331, in generate_execution_code stat.generate_execution_code(code) File "/scratch/mabshoff/junk/sage-3.2.4.alpha0/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 3076, in generate_execution_code self.dup.free_temps(code) File "/scratch/mabshoff/junk/sage-3.2.4.alpha0/local/lib/python2.5/site-packages/Cython/Compiler/ExprNodes.py", line 712, in free_temps self.release_temp_result(code) File "/scratch/mabshoff/junk/sage-3.2.4.alpha0/local/lib/python2.5/site-packages/Cython/Compiler/ExprNodes.py", line 666, in release_temp_result if self.old_temp: AttributeError: 'CoerceToTempNode' object has no attribute 'old_temp' This is head of 0.11-devel and I made sure the old Cython was removed. >> Hopefully 0.11 will come soon > > Yep. I think the primary bottleneck is the above (and fixing any bugs > we find there). I would also like a "memleak" mode for doctests, like > the timing code, where it would run each block, say, twice, then get > the memory usage, then run the block again and see if it went up at > all. This would be great to have in general, but would also make me > much happier knowing I didn't introduce new memory leaks. I think running things in a loop until they no longer grow the heap is great, but running it twice will likely not even come close to cutting it. I have played with muppy which offers a mode to do deltas of two heap states and I have plans to add this to Sage once I find time, i.e. I would run a doctest in a loop a reasonable number of times and if it still leaks after say 100 reps we have a problem. But this is mostly in the "it would be really nice to have" stage at this point :) It might be interesting to check out how muppy would fit into this leak testing framework. It is similar to guppy, but seems faster and more elegant. guppy seems to be more powerful overall, but if I get what I want out of muppy I wouldn't care. >> and we will find a way to integrate the refcount nanny as a special >> mode during doctesting of Sage, i.e. during compile time of Cython >> activate it and if we then somehow could have the doctesting framework >> have some mode where *every* reference was watched this would truly >> rock. If this is expensive performance wise it doesn't matter since it >> would be a special debug mode :) > > Yeah. Even if it was slower, it wouldn't be as slow as Valgrind Sure, but the memleaks we target here are very different in nature and trying to find them with the "other" tool respectively is an exercise in futility :( And valgrind offers other fun tools like exp-omega, exp-ptrcheck and so on :) > - Robert Cheers, Michael > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From robertwb at math.washington.edu Fri Jan 9 02:05:52 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Thu, 8 Jan 2009 17:05:52 -0800 Subject: [Cython] Refcount nanny status In-Reply-To: <4966A1D7.4000107@gmail.com> References: <4964B867.3060402@student.matnat.uio.no> <974D9C23-56ED-42B3-AAF5-C496AA5F00AD@math.washington.edu> <49669198.2090006@gmail.com> <8537FD98-2AFE-4C5E-A44B-3870F5AA03DB@math.washington.edu> <496695FE.8030803@gmail.com> <4966A1D7.4000107@gmail.com> Message-ID: <4C4F08BD-1ED8-4EA6-B6E6-869D1FACFD93@math.washington.edu> On Jan 8, 2009, at 5:01 PM, Michael Abshoff wrote: > Robert Bradshaw wrote: >> On Jan 8, 2009, at 4:10 PM, Michael Abshoff wrote: > > Hi Robert, > > > >>> I am tempted to drop in 0.11-devel and see if it compiles the >>> Sage >>> library and if it then even starts Sage :). >> >> I've been meaning to do the same too, but just haven't had time. I >> haven't tried compiling Sage with 0.11 for too long... > > Well, I don't get very far: > > AttributeError: 'CoerceToTempNode' object has no attribute 'old_temp' > Traceback (most recent call last): [...] > line 666, in release_temp_result > if self.old_temp: > AttributeError: 'CoerceToTempNode' object has no attribute 'old_temp' > > This is head of 0.11-devel and I made sure the old Cython was removed. :( > >>> Hopefully 0.11 will come soon >> >> Yep. I think the primary bottleneck is the above (and fixing any bugs >> we find there). I would also like a "memleak" mode for doctests, like >> the timing code, where it would run each block, say, twice, then get >> the memory usage, then run the block again and see if it went up at >> all. This would be great to have in general, but would also make me >> much happier knowing I didn't introduce new memory leaks. > > I think running things in a loop until they no longer grow the heap is > great, but running it twice will likely not even come close to cutting > it. As a regression tool, I would be happy if the memory increase remained the same with the old and new Cython (or, e.g., between runs of Sage). At least it could probably pinpoint egregious issues. I bet twice is enough for 90+% of the code out there. > I have played with muppy which offers a mode to do deltas of two > heap states and I have plans to add this to Sage once I find time, > i.e. > I would run a doctest in a loop a reasonable number of times and if it > still leaks after say 100 reps we have a problem. But this is > mostly in > the "it would be really nice to have" stage at this point :) > > It might be interesting to check out how muppy would fit into this > leak > testing framework. It is similar to guppy, but seems faster and more > elegant. guppy seems to be more powerful overall, but if I get what I > want out of muppy I wouldn't care. > >>> and we will find a way to integrate the refcount nanny as a special >>> mode during doctesting of Sage, i.e. during compile time of Cython >>> activate it and if we then somehow could have the doctesting >>> framework >>> have some mode where *every* reference was watched this would truly >>> rock. If this is expensive performance wise it doesn't matter >>> since it >>> would be a special debug mode :) >> >> Yeah. Even if it was slower, it wouldn't be as slow as Valgrind > > Sure, but the memleaks we target here are very different in nature and > trying to find them with the "other" tool respectively is an > exercise in > futility :( > > And valgrind offers other fun tools like exp-omega, exp-ptrcheck > and so > on :) Yes, I was commenting on the fact that we don't worry about performance loss in these cases. - Robert From michael.abshoff at googlemail.com Fri Jan 9 02:11:40 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Thu, 08 Jan 2009 17:11:40 -0800 Subject: [Cython] Refcount nanny status In-Reply-To: <4C4F08BD-1ED8-4EA6-B6E6-869D1FACFD93@math.washington.edu> References: <4964B867.3060402@student.matnat.uio.no> <974D9C23-56ED-42B3-AAF5-C496AA5F00AD@math.washington.edu> <49669198.2090006@gmail.com> <8537FD98-2AFE-4C5E-A44B-3870F5AA03DB@math.washington.edu> <496695FE.8030803@gmail.com> <4966A1D7.4000107@gmail.com> <4C4F08BD-1ED8-4EA6-B6E6-869D1FACFD93@math.washington.edu> Message-ID: <4966A44C.90703@gmail.com> Robert Bradshaw wrote: > On Jan 8, 2009, at 5:01 PM, Michael Abshoff wrote: Hi Robert, >> Well, I don't get very far: >> >> AttributeError: 'CoerceToTempNode' object has no attribute 'old_temp' >> Traceback (most recent call last): > > [...] > >> line 666, in release_temp_result >> if self.old_temp: >> AttributeError: 'CoerceToTempNode' object has no attribute 'old_temp' >> >> This is head of 0.11-devel and I made sure the old Cython was removed. > > :( > If someone fixes this I will get right back to testing. I will see what is going on, maybe this is fixable for me in which case it should be trivial :) > > As a regression tool, I would be happy if the memory increase > remained the same with the old and new Cython (or, e.g., between runs > of Sage). At least it could probably pinpoint egregious issues. I bet > twice is enough for 90+% of the code out there. > Well, the main issue for me here is to hunt down the small root causes if leaks since I can find plenty of large scope examples that go nuts. But all the low hanging fruits in the C heap, i.e. sparse linear algebra have been fixed, so now the issues are also with Cython/Python and I don't know enough about this yet to make headway on my own. Obviously that ought to change over time, especially when I get some nice tools like the reference count nanny. >> >> And valgrind offers other fun tools like exp-omega, exp-ptrcheck >> and so >> on :) > > Yes, I was commenting on the fact that we don't worry about > performance loss in these cases. Yep, I agree 100%. > - Robert > Cheers, Michael > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From robertwb at math.washington.edu Fri Jan 9 02:15:22 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Thu, 8 Jan 2009 17:15:22 -0800 Subject: [Cython] Refcount nanny status In-Reply-To: <4966A44C.90703@gmail.com> References: <4964B867.3060402@student.matnat.uio.no> <974D9C23-56ED-42B3-AAF5-C496AA5F00AD@math.washington.edu> <49669198.2090006@gmail.com> <8537FD98-2AFE-4C5E-A44B-3870F5AA03DB@math.washington.edu> <496695FE.8030803@gmail.com> <4966A1D7.4000107@gmail.com> <4C4F08BD-1ED8-4EA6-B6E6-869D1FACFD93@math.washington.edu> <4966A44C.90703@gmail.com> Message-ID: On Jan 8, 2009, at 5:11 PM, Michael Abshoff wrote: > If someone fixes this I will get right back to testing. I will see > what > is going on, maybe this is fixable for me in which case it should be > trivial :) I think a lot of this kind of error is rather trivial, it's just that no one handled a certain case for new-style temps. I've just had too many other higher-priority things to do to attack this. >> >> As a regression tool, I would be happy if the memory increase >> remained the same with the old and new Cython (or, e.g., between runs >> of Sage). At least it could probably pinpoint egregious issues. I bet >> twice is enough for 90+% of the code out there. >> > > Well, the main issue for me here is to hunt down the small root causes > if leaks since I can find plenty of large scope examples that go nuts. > But all the low hanging fruits in the C heap, i.e. sparse linear > algebra > have been fixed, so now the issues are also with Cython/Python and I > don't know enough about this yet to make headway on my own. Obviously > that ought to change over time, especially when I get some nice tools > like the reference count nanny. I think Cython 0.10 was pretty tight (perhaps not completely bug free, but mostly). I think 0.11 hasn't even been tested enough yet. - Robert From michael.abshoff at googlemail.com Fri Jan 9 03:16:58 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Thu, 08 Jan 2009 18:16:58 -0800 Subject: [Cython] Refcount nanny status In-Reply-To: References: <4964B867.3060402@student.matnat.uio.no> <974D9C23-56ED-42B3-AAF5-C496AA5F00AD@math.washington.edu> <49669198.2090006@gmail.com> <8537FD98-2AFE-4C5E-A44B-3870F5AA03DB@math.washington.edu> <496695FE.8030803@gmail.com> <4966A1D7.4000107@gmail.com> <4C4F08BD-1ED8-4EA6-B6E6-869D1FACFD93@math.washington.edu> <4966A44C.90703@gmail.com> Message-ID: <4966B39A.1010200@gmail.com> Robert Bradshaw wrote: > On Jan 8, 2009, at 5:11 PM, Michael Abshoff wrote: Hi, >> If someone fixes this I will get right back to testing. I will see >> what >> is going on, maybe this is fixable for me in which case it should be >> trivial :) > > I think a lot of this kind of error is rather trivial, it's just that > no one handled a certain case for new-style temps. I've just had too > many other higher-priority things to do to attack this. old_temp was introduced in rev 1447: http://hg.cython.org/cython-devel/rev/36de07933246 But there is no other file in the tree where it is used and/or mentioned. I guess this might be a merge gone wrong and I have no clue what the correct fix would be after staring at it for 30 minutes. Either way, I have run out of time and I am hopeful Stefan or Dag will fix it in the morning :) Cheers, Michael > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From dagss at student.matnat.uio.no Fri Jan 9 08:09:00 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: 09 Jan 2009 08:09:00 +0100 Subject: [Cython] Refcount nanny status Message-ID: <3314333379.699613@smtp.netcom.no> 80% of the work so far was done on the 0.10 branch, one could go back to developing the rest there... Backporting the rest should be easy. Have a look on the branch history. Non-Linux is almost only a matter of writing a setup.py file (once it actually works on Linux). Dag Sverre Seljebotn -----Original Message----- From: Robert Bradshaw Date: Friday, Jan 9, 2009 1:00 am Subject: Re: [Cython] Refcount nanny status To: cython-dev at codespeak.netReply-To: cython-dev at codespeak.net On Jan 8, 2009, at 3:51 PM, Michael Abshoff wrote: > >> Robert Bradshaw wrote: >> Thanks! I think this will help immensely for watching for memory >> leaks in Sage. > >> Yep, this is great. Given that Dag seems to have been slowed down by > real world things I am curious what needs to be done to get this > into a > more usable state? Linux only is fine for me for now, but if this > depeneds on 0.11-devel I would probably be not be able to use this. > >I am pretty sure it depends on 0.11-devel. In fact, it was written in >part *for* 0.11 as the temporary variable handle has been revised and >we want to catch any refcounting errors this may have introduced. > >- Robert > >_______________________________________________ >Cython-dev mailing list >Cython-dev at codespeak.net >http://codespeak.net/mailman/listinfo/cython-dev > From dagss at student.matnat.uio.no Fri Jan 9 08:19:00 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: 09 Jan 2009 08:19:00 +0100 Subject: [Cython] Refcount nanny status Message-ID: <3314333946.756336@smtp.netcom.no> I don't have time for fixing it, but old_temp in NewTempsExprNode was introduced solely to do "print self.old_temp" during debugging. Sorry, there should have beena comment. Dag Sverre Seljebotn -----Original Message----- From: Michael Abshoff Date: Friday, Jan 9, 2009 3:17 am Subject: Re: [Cython] Refcount nanny status To: cython-dev at codespeak.netReply-To: cython-dev at codespeak.net Robert Bradshaw wrote: > On Jan 8, 2009, at 5:11 PM, Michael Abshoff wrote: > >Hi, > >>> If someone fixes this I will get right back to testing. I will see >> what >> is going on, maybe this is fixable for me in which case it should be >> trivial :) > > I think a lot of this kind of error is rather trivial, it's just that > no one handled a certain case for new-style temps. I've just had too > many other higher-priority things to do to attack this. > > >old_temp was introduced in rev 1447: > > http://hg.cython.org/cython-devel/rev/36de07933246 > >But there is no other file in the tree where it is used and/or >mentioned. I guess this might be a merge gone wrong and I have no clue >what the correct fix would be after staring at it for 30 minutes. Either >way, I have run out of time and I am hopeful Stefan or Dag will fix it >in the morning :) > >Cheers, > >Michael > > >> _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > > >_______________________________________________ >Cython-dev mailing list >Cython-dev at codespeak.net >http://codespeak.net/mailman/listinfo/cython-dev > From michael.abshoff at googlemail.com Fri Jan 9 08:32:13 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Thu, 08 Jan 2009 23:32:13 -0800 Subject: [Cython] Refcount nanny status In-Reply-To: <3314333946.756336@smtp.netcom.no> References: <3314333946.756336@smtp.netcom.no> Message-ID: <4966FD7D.8030003@gmail.com> Dag Sverre Seljebotn wrote: Hi Dag, > I don't have time for fixing it, but old_temp in NewTempsExprNode was introduced > solely to do "print self.old_temp" during debugging. Sorry, there should have > beena comment. Thanks for the heads up, I need to get my hand dirtier with the Cython codebase any instead of just complaining to Robert that $FOO is broken :) > Dag Sverre Seljebotn Cheers, Michael From stefan_ml at behnel.de Fri Jan 9 08:28:24 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 09 Jan 2009 08:28:24 +0100 Subject: [Cython] Refcount nanny status In-Reply-To: <4966A1D7.4000107@gmail.com> References: <4964B867.3060402@student.matnat.uio.no> <974D9C23-56ED-42B3-AAF5-C496AA5F00AD@math.washington.edu> <49669198.2090006@gmail.com> <8537FD98-2AFE-4C5E-A44B-3870F5AA03DB@math.washington.edu> <496695FE.8030803@gmail.com> <4966A1D7.4000107@gmail.com> Message-ID: <4966FC98.6040602@behnel.de> Hi, Michael Abshoff wrote: > I think running things in a loop until they no longer grow the heap is > great, but running it twice will likely not even come close to cutting > it. It should work pretty well (and pretty slowly, obviously) if you run a full GC after each repetition and before you check the refcounts. Stefan From stefan_ml at behnel.de Fri Jan 9 08:43:00 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 09 Jan 2009 08:43:00 +0100 Subject: [Cython] Refcount nanny status In-Reply-To: <4966B39A.1010200@gmail.com> References: <4964B867.3060402@student.matnat.uio.no> <974D9C23-56ED-42B3-AAF5-C496AA5F00AD@math.washington.edu> <49669198.2090006@gmail.com> <8537FD98-2AFE-4C5E-A44B-3870F5AA03DB@math.washington.edu> <496695FE.8030803@gmail.com> <4966A1D7.4000107@gmail.com> <4C4F08BD-1ED8-4EA6-B6E6-869D1FACFD93@math.washington.edu> <4966A44C.90703@gmail.com> <4966B39A.1010200@gmail.com> Message-ID: <49670004.1050201@behnel.de> Hi, Michael Abshoff wrote: > old_temp was introduced in rev 1447: > > http://hg.cython.org/cython-devel/rev/36de07933246 > > But there is no other file in the tree where it is used and/or > mentioned. I guess this might be a merge gone wrong No, it's something Dag added to have the compiler notice when temps are freed multiple times, or freed without being allocated. The crash you notice shows that there is a problem with the temp (de-)allocation. However, the reason why you get an AttributeError instead of a meaningful error message is also a bug. > I have no clue > what the correct fix would be after staring at it for 30 minutes. Either > way, I have run out of time and I am hopeful Stefan or Dag will fix it > in the morning :) :) A stripped-down test case would definitely help here. Stefan From stefan_ml at behnel.de Fri Jan 9 09:52:27 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 09 Jan 2009 09:52:27 +0100 Subject: [Cython] Refcount nanny status In-Reply-To: <3314333946.756336@smtp.netcom.no> References: <3314333946.756336@smtp.netcom.no> Message-ID: <4967104B.4000908@behnel.de> Dag Sverre Seljebotn wrote: > I don't have time for fixing it, but old_temp in NewTempsExprNode was > introduced solely to do "print self.old_temp" during debugging. Sorry, > there should have beena comment. I fixed the crash (but not the underlying temp problem). You should at least get a meaningful error message out of it now. Stefan From michael.abshoff at googlemail.com Sat Jan 10 02:47:30 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Fri, 09 Jan 2009 17:47:30 -0800 Subject: [Cython] Refcount nanny status In-Reply-To: <4967104B.4000908@behnel.de> References: <3314333946.756336@smtp.netcom.no> <4967104B.4000908@behnel.de> Message-ID: <4967FE32.6000109@gmail.com> Stefan Behnel wrote: > Dag Sverre Seljebotn wrote: >> I don't have time for fixing it, but old_temp in NewTempsExprNode was >> introduced solely to do "print self.old_temp" during debugging. Sorry, >> there should have beena comment. > > I fixed the crash (but not the underlying temp problem). Thanks > You should at > least get a meaningful error message out of it now. Now the first file in the Sage library builds, but the second one from 3.2.3 fails: python2.5 `which cython` --embed-positions --incref-local-binop -I/scratch/mabshoff/junk/sage-3.2.3/devel/sage-main -o sage/categories/action.c sage/categories/action.pyx Traceback (most recent call last): File "/scratch/mabshoff/junk/sage-3.2.3/local/bin//cython", line 8, in main(command_line = 1) File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 730, in main result = compile(sources, options) File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 707, in compile return compile_multiple(source, options) File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 677, in compile_multiple result = run_pipeline(source, options) File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 539, in run_pipeline err, enddata = context.run_pipeline(pipeline, source) File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 198, in run_pipeline data = phase(data) File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 139, in generate_pyx_code module_node.process_implementation(options, result) File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/ModuleNode.py", line 71, in process_implementation self.generate_c_code(env, options, result) File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/ModuleNode.py", line 266, in generate_c_code self.body.generate_function_definitions(env, code) File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 325, in generate_function_definitions stat.generate_function_definitions(env, code) File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 325, in generate_function_definitions stat.generate_function_definitions(env, code) File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 2620, in generate_function_definitions self.entry.type.scope, code) File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 325, in generate_function_definitions stat.generate_function_definitions(env, code) File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 1049, in generate_function_definitions self.body.generate_execution_code(code) File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 331, in generate_execution_code stat.generate_execution_code(code) File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 3558, in generate_execution_code if_clause.generate_execution_code(code, end_label) File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 3600, in generate_execution_code self.body.generate_execution_code(code) File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 331, in generate_execution_code stat.generate_execution_code(code) File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/Nodes.py", line 3076, in generate_execution_code self.dup.free_temps(code) File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/ExprNodes.py", line 713, in free_temps self.release_temp_result(code) File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/ExprNodes.py", line 672, in release_temp_result self.__class__.__name__)) RuntimeError: no temp, but release requested in CoerceToTempNode I am not quite sure what to do next, but I am hoping RobertWB or somwbody else has an idea what the problem might be :) > Stefan Cheers, Michael > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From michael.abshoff at googlemail.com Sat Jan 10 04:43:30 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Fri, 09 Jan 2009 19:43:30 -0800 Subject: [Cython] http://docs.cython.org/index.html still shows the 0.9.8.1.1 docs Message-ID: <49681962.6080302@gmail.com> Hi, Somebody ought to update the documentation since it still claims to show the docs from Cython v0.9.8.1.1. Cheers, Michael From ggellner at uoguelph.ca Sat Jan 10 05:23:19 2009 From: ggellner at uoguelph.ca (Gabriel Gellner) Date: Fri, 9 Jan 2009 23:23:19 -0500 Subject: [Cython] http://docs.cython.org/index.html still shows the 0.9.8.1.1 docs In-Reply-To: <49681962.6080302@gmail.com> References: <49681962.6080302@gmail.com> Message-ID: <20090110042319.GA10749@encolpuis> On Fri, Jan 09, 2009 at 07:43:30PM -0800, Michael Abshoff wrote: > Hi, > > Somebody ought to update the documentation since it still claims to show > the docs from Cython v0.9.8.1.1. > Fixed. Also the docs will compile with latex again. Gabriel From hoytak at cs.ubc.ca Sat Jan 10 07:21:18 2009 From: hoytak at cs.ubc.ca (Hoyt Koepke) Date: Fri, 9 Jan 2009 22:21:18 -0800 Subject: [Cython] typo in ExprNode.py Message-ID: <4db580fd0901092221u48993e1fv5be8f1894b37ab39@mail.gmail.com> Hello, beginswith() in ExprNode.py should be startswith() Here's the changeset: # HG changeset patch # User hoytak at hk-laptop.site # Date 1231568090 28800 # Node ID f77199c86f6f461971d4ce0817ed68aa80e25cc5 # Parent ee4817ddb9900cdec49ba086eb31c9f2c64ee338 Fixed typo; beginswith -> startswith diff -r ee4817ddb990 -r f77199c86f6f Cython/Compiler/ExprNodes.py --- a/Cython/Compiler/ExprNodes.py Fri Jan 09 08:44:58 2009 +0100 +++ b/Cython/Compiler/ExprNodes.py Fri Jan 09 22:14:50 2009 -0800 @@ -2608,13 +2608,13 @@ def calculate_constant_result(self): attr = self.attribute - if attr.beginswith("__") and attr.endswith("__"): + if attr.startswith("__") and attr.endswith("__"): return self.constant_result = getattr(self.obj.constant_result, attr) def compile_time_value(self, denv): attr = self.attribute - if attr.beginswith("__") and attr.endswith("__"): + if attr.startswith("__") and attr.endswith("__"): self.error("Invalid attribute name '%s' in compile-time expression" % attr) return None Also attached... --Hoyt P.S. Is this the best way to send little updates like these? Should I send them to the whole list? ++++++++++++++++++++++++++++++++++++++++++++++++ + Hoyt Koepke + University of Washington Department of Statistics + http://www.stat.washington.edu/~hoytak/ + hoytak at gmail.com ++++++++++++++++++++++++++++++++++++++++++ -------------- next part -------------- A non-text attachment was scrubbed... Name: changeset_diff Type: application/octet-stream Size: 1038 bytes Desc: not available Url : http://codespeak.net/pipermail/cython-dev/attachments/20090109/dab78c70/attachment.obj From hoytak at cs.ubc.ca Sat Jan 10 07:40:36 2009 From: hoytak at cs.ubc.ca (Hoyt Koepke) Date: Fri, 9 Jan 2009 22:40:36 -0800 Subject: [Cython] another error in ExprNodes.py Message-ID: <4db580fd0901092240p4e8399f6y2895a439aed63ccf@mail.gmail.com> Hello, In trying to compile a new cython file with a fair bit of code in it, I get the following traceback: 22:23 hk-laptop:../clsynth/clsynth > cython transforms.pyx Traceback (most recent call last): File "/home/hoytak/sysroot/bin/cython", line 8, in main(command_line = 1) File "/home/hoytak/sysroot/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 730, in main result = compile(sources, options) File "/home/hoytak/sysroot/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 707, in compile return compile_multiple(source, options) File "/home/hoytak/sysroot/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 677, in compile_multiple result = run_pipeline(source, options) File "/home/hoytak/sysroot/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 539, in run_pipeline err, enddata = context.run_pipeline(pipeline, source) File "/home/hoytak/sysroot/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 198, in run_pipeline data = phase(data) File "/home/hoytak/sysroot/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 497, in parse tree = context.parse(source_desc, scope, pxd = 0, full_module_name = full_module_name) File "/home/hoytak/sysroot/lib/python2.5/site-packages/Cython/Compiler/Main.py", line 429, in parse tree = Parsing.p_module(s, pxd, full_module_name) File "Parsing.py", line 2478, in Cython.Compiler.Parsing.p_module (/home/hoytak/sysroot/src/cython/Cython/Compiler/Parsing.c:52412) File "Parsing.py", line 2490, in Cython.Compiler.Parsing.p_module (/home/hoytak/sysroot/src/cython/Cython/Compiler/Parsing.c:52183) File "Parsing.py", line 1551, in Cython.Compiler.Parsing.p_statement_list (/home/hoytak/sysroot/src/cython/Cython/Compiler/Parsing.c:33272) File "Parsing.py", line 1556, in Cython.Compiler.Parsing.p_statement_list (/home/hoytak/sysroot/src/cython/Cython/Compiler/Parsing.c:33090) File "Parsing.py", line 1480, in Cython.Compiler.Parsing.p_statement (/home/hoytak/sysroot/src/cython/Cython/Compiler/Parsing.c:32939) File "Parsing.py", line 1489, in Cython.Compiler.Parsing.p_statement (/home/hoytak/sysroot/src/cython/Cython/Compiler/Parsing.c:31516) File "Parsing.py", line 1447, in Cython.Compiler.Parsing.p_DEF_statement (/home/hoytak/sysroot/src/cython/Cython/Compiler/Parsing.c:30670) File "/home/hoytak/sysroot/lib/python2.5/site-packages/Cython/Compiler/ExprNodes.py", line 2621, in compile_time_value obj = self.arg.compile_time_value(denv) AttributeError: 'AttributeNode' object has no attribute 'arg' I haven't been able to track down what exactly in my code is causing this, but if someone could tell what type of code would invoke the above nodes, I'll try to isolate it in my code and write a test case. I suspect this bit of code in cython hasn't really been tested much, per my last email... Thanks, --Hoyt ++++++++++++++++++++++++++++++++++++++++++++++++ + Hoyt Koepke + University of Washington Department of Statistics + http://www.stat.washington.edu/~hoytak/ + hoytak at gmail.com ++++++++++++++++++++++++++++++++++++++++++ From michael.abshoff at googlemail.com Sat Jan 10 08:54:28 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Fri, 09 Jan 2009 23:54:28 -0800 Subject: [Cython] http://docs.cython.org/index.html still shows the 0.9.8.1.1 docs In-Reply-To: <20090110042319.GA10749@encolpuis> References: <49681962.6080302@gmail.com> <20090110042319.GA10749@encolpuis> Message-ID: <49685434.7010809@gmail.com> Gabriel Gellner wrote: > On Fri, Jan 09, 2009 at 07:43:30PM -0800, Michael Abshoff wrote: >> Hi, >> >> Somebody ought to update the documentation since it still claims to show >> the docs from Cython v0.9.8.1.1. >> > Fixed. Also the docs will compile with latex again. Mhh, the website still says: "Cython v0.9.8.1.1 documentation ? ? Copyright 2008, Stefan Behnel, Robert Bradshaw, Grew Ewing, William Stein, Gary Furnish, Dag Seljebotn, Gabriel Gellner. Last updated on Dec 19, 2008." I did check with a browser that had never visited the site, so it is not a caching issue. So either someone has to press the button for the doc site or something else is going wrong. Gabriel: Did you already update the website at doc.cython.org or has somebody to do something? Or is this periodically updated via some script from some third party source? > Gabriel Cheers, Michael > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From stefan_ml at behnel.de Sat Jan 10 09:08:54 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 10 Jan 2009 09:08:54 +0100 Subject: [Cython] typo in ExprNode.py In-Reply-To: <4db580fd0901092221u48993e1fv5be8f1894b37ab39@mail.gmail.com> References: <4db580fd0901092221u48993e1fv5be8f1894b37ab39@mail.gmail.com> Message-ID: <49685796.1090006@behnel.de> Hi, Hoyt Koepke wrote: > beginswith() in ExprNode.py should be startswith() Thanks for catching that, I applied the patch. I would expect the same bug in Pyrex, BTW. It looks like we lack a test case for this specific part of the code. Could you come up with something here? http://wiki.cython.org/HackerGuide#head-6afd0aa52fa6a1de32f6d554b5d30556699e4f18 Might fit in here: tests/run/consts.pyx tests/run/ct_DEF.pyx > P.S. Is this the best way to send little updates like these? Should I > send them to the whole list? I think it's perfect for small to medium sized patches. More complex stuff should go to the bug tracker, but for patches like this, a quick review while reading your mail and a quick import is the most efficient way to apply them. Stefan From robertwb at math.washington.edu Sat Jan 10 09:48:25 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sat, 10 Jan 2009 00:48:25 -0800 Subject: [Cython] http://docs.cython.org/index.html still shows the 0.9.8.1.1 docs In-Reply-To: <49681962.6080302@gmail.com> References: <49681962.6080302@gmail.com> Message-ID: Yes. The Cython docs currently explain 0.11 features, which isn't officially released yet, which is why I haven't done that. On Jan 9, 2009, at 7:43 PM, Michael Abshoff wrote: > Hi, > > Somebody ought to update the documentation since it still claims to > show > the docs from Cython v0.9.8.1.1. > > Cheers, > > Michael > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev From robertwb at math.washington.edu Sat Jan 10 09:49:50 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sat, 10 Jan 2009 00:49:50 -0800 Subject: [Cython] http://docs.cython.org/index.html still shows the 0.9.8.1.1 docs In-Reply-To: <49685434.7010809@gmail.com> References: <49681962.6080302@gmail.com> <20090110042319.GA10749@encolpuis> <49685434.7010809@gmail.com> Message-ID: <6A63FC69-F113-4C17-BF26-2B1FF28B2CDE@math.washington.edu> On Jan 9, 2009, at 11:54 PM, Michael Abshoff wrote: > Gabriel Gellner wrote: >> On Fri, Jan 09, 2009 at 07:43:30PM -0800, Michael Abshoff wrote: >>> Hi, >>> >>> Somebody ought to update the documentation since it still claims >>> to show >>> the docs from Cython v0.9.8.1.1. >>> >> Fixed. Also the docs will compile with latex again. > > Mhh, the website still says: "Cython v0.9.8.1.1 documentation ? > ? Copyright 2008, Stefan Behnel, Robert Bradshaw, Grew Ewing, William > Stein, Gary Furnish, Dag Seljebotn, Gabriel Gellner. Last updated > on Dec > 19, 2008." > > I did check with a browser that had never visited the site, so it > is not > a caching issue. So either someone has to press the button for > the doc > site or something else is going wrong. > > Gabriel: Did you already update the website at doc.cython.org or has > somebody to do something? Or is this periodically updated via some > script from some third party source? Sorry to double post, but I update them when the new release comes out. The docs don't currently distinguish between new features and old, which is why I haven't done that yet. - Robert From robertwb at math.washington.edu Sat Jan 10 09:51:33 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sat, 10 Jan 2009 00:51:33 -0800 Subject: [Cython] typo in ExprNode.py In-Reply-To: <49685796.1090006@behnel.de> References: <4db580fd0901092221u48993e1fv5be8f1894b37ab39@mail.gmail.com> <49685796.1090006@behnel.de> Message-ID: <71FC34A2-6364-4BC0-AAC9-A5C777D3490B@math.washington.edu> On Jan 10, 2009, at 12:08 AM, Stefan Behnel wrote: > Hi, > > Hoyt Koepke wrote: >> beginswith() in ExprNode.py should be startswith() > > Thanks for catching that, I applied the patch. I would expect the > same bug > in Pyrex, BTW. > > It looks like we lack a test case for this specific part of the > code. Could > you come up with something here? > > http://wiki.cython.org/ > HackerGuide#head-6afd0aa52fa6a1de32f6d554b5d30556699e4f18 > > Might fit in here: > > tests/run/consts.pyx > tests/run/ct_DEF.pyx > > >> P.S. Is this the best way to send little updates like these? >> Should I >> send them to the whole list? > > I think it's perfect for small to medium sized patches. More > complex stuff > should go to the bug tracker, but for patches like this, a quick > review > while reading your mail and a quick import is the most efficient > way to > apply them. I agree. I try and make track tickets for history and completeness (e.g. it makes release notes much easier) but for something like this it is (currently) the best way to go about it. If no one responds, make sure a ticket gets filed so it doesn't get lost in the archives. - Robert From robertwb at math.washington.edu Sat Jan 10 09:55:01 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sat, 10 Jan 2009 00:55:01 -0800 Subject: [Cython] another error in ExprNodes.py In-Reply-To: <4db580fd0901092240p4e8399f6y2895a439aed63ccf@mail.gmail.com> References: <4db580fd0901092240p4e8399f6y2895a439aed63ccf@mail.gmail.com> Message-ID: <12FFEE29-BC42-4F8E-8E14-DEAE796DEAA6@math.washington.edu> On Jan 9, 2009, at 10:40 PM, Hoyt Koepke wrote: > Hello, > > In trying to compile a new cython file with a fair bit of code in it, > I get the following traceback: [...] We're getting similar errors trying to compile Sage. A lot of stuff was refactored, and I think this is fallout from that. The hope is that such bugs will be easy to fix. We need higher coverage in our test suite. > I haven't been able to track down what exactly in my code is causing > this, but if someone could tell what type of code would invoke the > above nodes, I'll try to isolate it in my code and write a test case. > > I suspect this bit of code in cython hasn't really been tested much, > per my last email... It would be really useful to have a mode/mechanism to get the position of the code when an error like this occurs. I can think of several ways to do it, but all would significantly impact speed. - Robert From stefan_ml at behnel.de Sat Jan 10 12:31:57 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 10 Jan 2009 12:31:57 +0100 Subject: [Cython] another error in ExprNodes.py In-Reply-To: <4db580fd0901092240p4e8399f6y2895a439aed63ccf@mail.gmail.com> References: <4db580fd0901092240p4e8399f6y2895a439aed63ccf@mail.gmail.com> Message-ID: <4968872D.1030206@behnel.de> Hi, Hoyt Koepke wrote: > In trying to compile a new cython file with a fair bit of code in it, > I get the following traceback: > > 22:23 hk-laptop:../clsynth/clsynth > cython transforms.pyx > Traceback (most recent call last): > [...] > File "Parsing.py", line 1447, in > Cython.Compiler.Parsing.p_DEF_statement > (/home/hoytak/sysroot/src/cython/Cython/Compiler/Parsing.c:30670) > File "/home/hoytak/sysroot/lib/python2.5/site-packages/Cython/Compiler/ExprNodes.py", > line 2621, in compile_time_value > obj = self.arg.compile_time_value(denv) > AttributeError: 'AttributeNode' object has no attribute 'arg' > > I haven't been able to track down what exactly in my code is causing > this, That was easy to fix, just a wrong field name in the code. > but if someone could tell what type of code would invoke the > above nodes, I'll try to isolate it in my code and write a test case. It looks like you access an attribute of a compile-time value in a DEF statement. > I suspect this bit of code in cython hasn't really been tested much, > per my last email... Yes, both bugs you spotted were in code that comes unchanged from Pyrex, and it looks like no-one ever used it for the last (two?) years that it's already in there. Stefan From stefan_ml at behnel.de Sat Jan 10 12:46:50 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 10 Jan 2009 12:46:50 +0100 Subject: [Cython] another error in ExprNodes.py In-Reply-To: <12FFEE29-BC42-4F8E-8E14-DEAE796DEAA6@math.washington.edu> References: <4db580fd0901092240p4e8399f6y2895a439aed63ccf@mail.gmail.com> <12FFEE29-BC42-4F8E-8E14-DEAE796DEAA6@math.washington.edu> Message-ID: <49688AAA.40100@behnel.de> Hi, Robert Bradshaw wrote: > It would be really useful to have a mode/mechanism to get the > position of the code when an error like this occurs. I can think of > several ways to do it, but all would significantly impact speed. Yes, I thought about that, too, a couple of times already. The stack traces are not very helpful, what you'd need is a dump of the syntax tree nodes that are along the trace path. We could do that by catching all exceptions in Main.py, collecting all objects that belong to method calls in the stack trace, and dumping all that inherit from Node. This won't work for compiled code, but exceptions inside the parser are rare anyway. Most of the compiler crashes I had to deal with so far were in Nodes or ExprNodes. Something even simpler could work for transforms, where we control the traversal in framework code. Here, we could just catch all exceptions in visit() and dump the current tree path. Stefan From stefan_ml at behnel.de Sat Jan 10 12:51:09 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 10 Jan 2009 12:51:09 +0100 Subject: [Cython] another error in ExprNodes.py In-Reply-To: <49688AAA.40100@behnel.de> References: <4db580fd0901092240p4e8399f6y2895a439aed63ccf@mail.gmail.com> <12FFEE29-BC42-4F8E-8E14-DEAE796DEAA6@math.washington.edu> <49688AAA.40100@behnel.de> Message-ID: <49688BAD.6060904@behnel.de> Stefan Behnel wrote: > Robert Bradshaw wrote: >> It would be really useful to have a mode/mechanism to get the >> position of the code when an error like this occurs. I can think of >> several ways to do it, but all would significantly impact speed. > > Yes, I thought about that, too, a couple of times already. http://trac.cython.org/cython_trac/ticket/179 Stefan From stefan_ml at behnel.de Sat Jan 10 17:00:38 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 10 Jan 2009 17:00:38 +0100 Subject: [Cython] another error in ExprNodes.py In-Reply-To: <49688AAA.40100@behnel.de> References: <4db580fd0901092240p4e8399f6y2895a439aed63ccf@mail.gmail.com> <12FFEE29-BC42-4F8E-8E14-DEAE796DEAA6@math.washington.edu> <49688AAA.40100@behnel.de> Message-ID: <4968C626.6070002@behnel.de> Hi again, Stefan Behnel wrote: > Robert Bradshaw wrote: >> It would be really useful to have a mode/mechanism to get the >> position of the code when an error like this occurs. I can think of >> several ways to do it, but all would significantly impact speed. > > Yes, I thought about that, too, a couple of times already. The stack traces > are not very helpful, what you'd need is a dump of the syntax tree nodes > that are along the trace path. > > We could do that by catching all exceptions in Main.py, collecting all > objects that belong to method calls in the stack trace, and dumping all > that inherit from Node. I implemented this. Here's an example of a compiler crash while compiling Visitor.py, that I triggered by explicitly raising an exception somewhere in the compiler code. That should make future compiler crashes a lot easier to track down. Stefan Compiling module Cython.Compiler.Visitor ... Error converting Pyrex file to C: ------------------------------------------------------------ ... cdef class BasicVisitor: cdef dict dispatch_table cpdef visit(self, obj) cdef class TreeVisitor(BasicVisitor): cdef public list access_path ^ ------------------------------------------------------------ /home/sbehnel/source/Python/cython/cython-work/Cython/Compiler/Visitor.pxd:6:21: Compiler crash in AnalyseExpressionsTransform ModuleNode.body = StatListNode(Visitor.pxd:1:0, is_literal = 0, is_name = 0) StatListNode.stats[1] = CClassDefNode(Visitor.pxd:5:5, api = 0, as_name = u'TreeVisitor', base_class_module = '', base_class_name = u'BasicVisitor', class_name = u'TreeVisitor', in_pxd = True, is_literal = 0, is_name = 0, module_name = '', typedef_flag = 0, visibility = 'private') CClassDefNode.body = StatListNode(Visitor.pxd:6:4, is_literal = 0, is_name = 0) StatListNode.stats[0] = StatListNode(Visitor.pxd:6:9, is_literal = 0, is_name = 0) StatListNode.stats[0] = PropertyNode(Visitor.pxd:6:21, is_literal = 0, is_name = 0, name = u'access_path') PropertyNode.body = StatListNode(Visitor.pxd:6:21, is_literal = 0, is_name = 0) StatListNode.stats[0] = DefNode(Visitor.pxd:6:21, is_literal = 0, is_name = 0, is_wrapper = 0, name = u'__get__', needs_closure = False, num_kwonly_args = 0, num_required_args = 1, num_required_kw_args = 0, pxd_locals = [...], reqd_kw_flags_cname = '0') File 'Nodes.py', line 3314, in analyse_expressions: ReturnStatNode(Visitor.pxd:6:21, is_literal = 0, is_name = 0, temps_in_use = [...]) File 'ExprNodes.py', line 2633, in analyse_types: AttributeNode(Visitor.pxd:6:21, attribute = u'access_path', is_attribute = 1, is_called = 0, is_literal = 0, is_name = 0, is_py_attr = 0, is_sequence_constructor = 0, is_target = 0, is_temp = 0, member = u'access_path', needs_none_check = True, op = '->') File 'ExprNodes.py', line 2715, in analyse_as_ordinary_attribute: AttributeNode(Visitor.pxd:6:21, attribute = u'access_path', is_attribute = 1, is_called = 0, is_literal = 0, is_name = 0, is_py_attr = 0, is_sequence_constructor = 0, is_target = 0, is_temp = 0, member = u'access_path', needs_none_check = True, op = '->') File 'ExprNodes.py', line 2729, in analyse_attribute: AttributeNode(Visitor.pxd:6:21, attribute = u'access_path', is_attribute = 1, is_called = 0, is_literal = 0, is_name = 0, is_py_attr = 0, is_sequence_constructor = 0, is_target = 0, is_temp = 0, member = u'access_path', needs_none_check = True, op = '->') Compiler crash traceback from this point on: File "/home/sbehnel/source/Python/cython/cython-work/Cython/Compiler/ExprNodes.py", line 2743, in analyse_attribute raise InternalError(None) InternalError: Internal compiler error: None From hoytak at cs.ubc.ca Sat Jan 10 20:26:10 2009 From: hoytak at cs.ubc.ca (Hoyt Koepke) Date: Sat, 10 Jan 2009 11:26:10 -0800 Subject: [Cython] typo in ExprNode.py In-Reply-To: <71FC34A2-6364-4BC0-AAC9-A5C777D3490B@math.washington.edu> References: <4db580fd0901092221u48993e1fv5be8f1894b37ab39@mail.gmail.com> <49685796.1090006@behnel.de> <71FC34A2-6364-4BC0-AAC9-A5C777D3490B@math.washington.edu> Message-ID: <4db580fd0901101126oc289c51u42b0a39afd71f651@mail.gmail.com> Okay, I isolated the problem and created a small test case for this section of code. --Hoyt # HG changeset patch # User hoytak at gmail.com # Date 1231615306 28800 # Node ID 7992887cb01ec1c9aa029bc0e15f4806e99b1c3e # Parent f77199c86f6f461971d4ce0817ed68aa80e25cc5 New test case for non-constant DEF statements. diff -r f77199c86f6f -r 7992887cb01e tests/compile/nonconst_def.pyx --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tests/compile/nonconst_def.pyx Sat Jan 10 11:21:46 2009 -0800 @@ -0,0 +1,3 @@ +import os +DEF ospath = os.path + (also attached) On Sat, Jan 10, 2009 at 12:51 AM, Robert Bradshaw wrote: > On Jan 10, 2009, at 12:08 AM, Stefan Behnel wrote: > >> Hi, >> >> Hoyt Koepke wrote: >>> beginswith() in ExprNode.py should be startswith() >> >> Thanks for catching that, I applied the patch. I would expect the >> same bug >> in Pyrex, BTW. >> >> It looks like we lack a test case for this specific part of the >> code. Could >> you come up with something here? >> >> http://wiki.cython.org/ >> HackerGuide#head-6afd0aa52fa6a1de32f6d554b5d30556699e4f18 >> >> Might fit in here: >> >> tests/run/consts.pyx >> tests/run/ct_DEF.pyx >> >> >>> P.S. Is this the best way to send little updates like these? >>> Should I >>> send them to the whole list? >> >> I think it's perfect for small to medium sized patches. More >> complex stuff >> should go to the bug tracker, but for patches like this, a quick >> review >> while reading your mail and a quick import is the most efficient >> way to >> apply them. > > I agree. I try and make track tickets for history and completeness > (e.g. it makes release notes much easier) but for something like this > it is (currently) the best way to go about it. > > If no one responds, make sure a ticket gets filed so it doesn't get > lost in the archives. > > - Robert > > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > -- ++++++++++++++++++++++++++++++++++++++++++++++++ + Hoyt Koepke + University of Washington Department of Statistics + http://www.stat.washington.edu/~hoytak/ + hoytak at gmail.com ++++++++++++++++++++++++++++++++++++++++++ -------------- next part -------------- A non-text attachment was scrubbed... Name: new_test_changeset Type: application/octet-stream Size: 450 bytes Desc: not available Url : http://codespeak.net/pipermail/cython-dev/attachments/20090110/797f437c/attachment-0001.obj From hoytak at cs.ubc.ca Sat Jan 10 20:32:07 2009 From: hoytak at cs.ubc.ca (Hoyt Koepke) Date: Sat, 10 Jan 2009 11:32:07 -0800 Subject: [Cython] typo in ExprNode.py In-Reply-To: <4db580fd0901101126oc289c51u42b0a39afd71f651@mail.gmail.com> References: <4db580fd0901092221u48993e1fv5be8f1894b37ab39@mail.gmail.com> <49685796.1090006@behnel.de> <71FC34A2-6364-4BC0-AAC9-A5C777D3490B@math.washington.edu> <4db580fd0901101126oc289c51u42b0a39afd71f651@mail.gmail.com> Message-ID: <4db580fd0901101132u4c93dd0ag77773c3b8c10bc6a@mail.gmail.com> Wait, I think there may be something I don't realize about that bit of code. After merging in the new changes, I realize that I wasn't doing it correctly; i.e. I didn't realize that DEF was only for compile time expressions. Are the tests in compile/ meant to compile without any errors? If so, then forget the previous test case. --Hoyt On Sat, Jan 10, 2009 at 11:26 AM, Hoyt Koepke wrote: > Okay, I isolated the problem and created a small test case for this > section of code. > > --Hoyt > > # HG changeset patch > # User hoytak at gmail.com > # Date 1231615306 28800 > # Node ID 7992887cb01ec1c9aa029bc0e15f4806e99b1c3e > # Parent f77199c86f6f461971d4ce0817ed68aa80e25cc5 > New test case for non-constant DEF statements. > > diff -r f77199c86f6f -r 7992887cb01e tests/compile/nonconst_def.pyx > --- /dev/null Thu Jan 01 00:00:00 1970 +0000 > +++ b/tests/compile/nonconst_def.pyx Sat Jan 10 11:21:46 2009 -0800 > @@ -0,0 +1,3 @@ > +import os > +DEF ospath = os.path > + > > (also attached) > > > On Sat, Jan 10, 2009 at 12:51 AM, Robert Bradshaw > wrote: >> On Jan 10, 2009, at 12:08 AM, Stefan Behnel wrote: >> >>> Hi, >>> >>> Hoyt Koepke wrote: >>>> beginswith() in ExprNode.py should be startswith() >>> >>> Thanks for catching that, I applied the patch. I would expect the >>> same bug >>> in Pyrex, BTW. >>> >>> It looks like we lack a test case for this specific part of the >>> code. Could >>> you come up with something here? >>> >>> http://wiki.cython.org/ >>> HackerGuide#head-6afd0aa52fa6a1de32f6d554b5d30556699e4f18 >>> >>> Might fit in here: >>> >>> tests/run/consts.pyx >>> tests/run/ct_DEF.pyx >>> >>> >>>> P.S. Is this the best way to send little updates like these? >>>> Should I >>>> send them to the whole list? >>> >>> I think it's perfect for small to medium sized patches. More >>> complex stuff >>> should go to the bug tracker, but for patches like this, a quick >>> review >>> while reading your mail and a quick import is the most efficient >>> way to >>> apply them. >> >> I agree. I try and make track tickets for history and completeness >> (e.g. it makes release notes much easier) but for something like this >> it is (currently) the best way to go about it. >> >> If no one responds, make sure a ticket gets filed so it doesn't get >> lost in the archives. >> >> - Robert >> >> _______________________________________________ >> Cython-dev mailing list >> Cython-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/cython-dev >> > > > > -- > > ++++++++++++++++++++++++++++++++++++++++++++++++ > + Hoyt Koepke > + University of Washington Department of Statistics > + http://www.stat.washington.edu/~hoytak/ > + hoytak at gmail.com > ++++++++++++++++++++++++++++++++++++++++++ > -- ++++++++++++++++++++++++++++++++++++++++++++++++ + Hoyt Koepke + University of Washington Department of Statistics + http://www.stat.washington.edu/~hoytak/ + hoytak at gmail.com ++++++++++++++++++++++++++++++++++++++++++ From stefan_ml at behnel.de Sun Jan 11 08:44:02 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 11 Jan 2009 08:44:02 +0100 Subject: [Cython] typo in ExprNode.py In-Reply-To: <4db580fd0901101132u4c93dd0ag77773c3b8c10bc6a@mail.gmail.com> References: <4db580fd0901092221u48993e1fv5be8f1894b37ab39@mail.gmail.com> <49685796.1090006@behnel.de> <71FC34A2-6364-4BC0-AAC9-A5C777D3490B@math.washington.edu> <4db580fd0901101126oc289c51u42b0a39afd71f651@mail.gmail.com> <4db580fd0901101132u4c93dd0ag77773c3b8c10bc6a@mail.gmail.com> Message-ID: <4969A342.7060200@behnel.de> Hi, Hoyt Koepke wrote: > Wait, I think there may be something I don't realize about that bit of > code. After merging in the new changes, I realize that I wasn't doing > it correctly; i.e. I didn't realize that DEF was only for compile time > expressions. > > Are the tests in compile/ meant to compile without any errors? If so, > then forget the previous test case. Thanks for providing it anyway. I moved it over to tests/errors/ and added the expected error messages. http://hg.cython.org/cython-devel/rev/6d42bbf30e7f Stefan From igorsyl at gmail.com Tue Jan 13 06:48:56 2009 From: igorsyl at gmail.com (Igor Sylvester) Date: Mon, 12 Jan 2009 23:48:56 -0600 Subject: [Cython] PyExc_IOError Message-ID: Has anyone had problems with dynamically loading PyExc_IOError when embedding a cython module? While calling init below: *in main.c: *void init { Py_Initialize(); initmymodule(); do_somethind(); Py_Finalize(); } *in module.c:* cdef public void do_something(): import exceptions print exceptions.IOError # this prints , so I don't know what to make of the error below import numpy # exception is thrown here, and numpy library is in pythonpath Exception: Exception exceptions.ImportError: '/usr/lib/python2.5/lib-dynload/time.so: undefined symbol: PyExc_IOError' in 'mymodule.myfunction' ignored I'm not yet sure which library in numpy causes the exception but this example is easy to reproduce. Thanks. Igor -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20090112/31d38518/attachment.htm From michael.abshoff at googlemail.com Tue Jan 13 07:00:29 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Mon, 12 Jan 2009 22:00:29 -0800 Subject: [Cython] PyExc_IOError In-Reply-To: References: Message-ID: <496C2DFD.7090905@gmail.com> Igor Sylvester wrote: Hi Igor, > Has anyone had problems with dynamically loading PyExc_IOError when > embedding a cython module? > While calling init below: > > *in main.c: > *void init { > Py_Initialize(); > initmymodule(); > do_somethind(); > Py_Finalize(); > } > > *in module.c:* > cdef public void do_something(): > import > exceptions > > print exceptions.IOError # this prints , > so I don't know what to make of the error below > import numpy # exception is thrown here, and numpy library is in > pythonpath > > Exception: > Exception exceptions.ImportError: > '/usr/lib/python2.5/lib-dynload/time.so: undefined symbol: > PyExc_IOError' in 'mymodule.myfunction' ignored > > I'm not yet sure which library in numpy causes the exception but this > example is easy to reproduce. How did you build the extension, i.e. did you link against python2[3|4|5] ? > Thanks. > Igor Cheers, Michael > ------------------------------------------------------------------------ > > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev From igorsyl at gmail.com Tue Jan 13 07:12:41 2009 From: igorsyl at gmail.com (Igor Sylvester) Date: Tue, 13 Jan 2009 00:12:41 -0600 Subject: [Cython] PyExc_IOError In-Reply-To: <496C2DFD.7090905@gmail.com> References: <496C2DFD.7090905@gmail.com> Message-ID: I built it against *Python 2.5.1. compile:* gcc -c -I/usr/lib/python2.5/site-packages/numpy-1.0.4.0002-py2.5-linux-x86_64.egg/numpy/core/include -I/usr/bin/python/2.5.1/include/python2.5 -fPIC -fno-omit-frame-pointer -pthread -fexceptions -ansi -D_GNU_SOURCE -DMX_COMPAT_32 -g main.c compile line for the other c-file is identical to main.c's *link:* gcc -g -pthread -shared main.o module.o -L/usr/bin/python/2.5.1/lib * -lpython2.5* -ldl -lm -lpthread -lutil -lmx -lmex -lmat -lm -lm -lstdc++ On Tue, Jan 13, 2009 at 12:00 AM, Michael Abshoff < michael.abshoff at googlemail.com> wrote: > Igor Sylvester wrote: > > Hi Igor, > > > Has anyone had problems with dynamically loading PyExc_IOError when > > embedding a cython module? > > While calling init below: > > > > *in main.c: > > *void init { > > Py_Initialize(); > > initmymodule(); > > do_somethind(); > > Py_Finalize(); > > } > > > > *in module.c:* > > cdef public void do_something(): > > import > > exceptions > > > > print exceptions.IOError # this prints , > > so I don't know what to make of the error below > > import numpy # exception is thrown here, and numpy library is in > > pythonpath > > > > Exception: > > Exception exceptions.ImportError: > > '/usr/lib/python2.5/lib-dynload/time.so: undefined symbol: > > PyExc_IOError' in 'mymodule.myfunction' ignored > > > > I'm not yet sure which library in numpy causes the exception but this > > example is easy to reproduce. > > How did you build the extension, i.e. did you link against python2[3|4|5] ? > > > > Thanks. > > Igor > > Cheers, > > Michael > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > Cython-dev mailing list > > Cython-dev at codespeak.net > > http://codespeak.net/mailman/listinfo/cython-dev > > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20090113/d6f5be03/attachment.htm From michael.abshoff at googlemail.com Tue Jan 13 07:20:35 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Mon, 12 Jan 2009 22:20:35 -0800 Subject: [Cython] PyExc_IOError In-Reply-To: References: <496C2DFD.7090905@gmail.com> Message-ID: <496C32B3.907@gmail.com> Igor Sylvester wrote: > I built it against *Python 2.5.1. > > compile:* > gcc -c > -I/usr/lib/python2.5/site-packages/numpy-1.0.4.0002-py2.5-linux-x86_64.egg/numpy/core/include > -I/usr/bin/python/2.5.1/include/python2.5 -fPIC -fno-omit-frame-pointer > -pthread -fexceptions -ansi -D_GNU_SOURCE -DMX_COMPAT_32 -g main.c > > compile line for the other c-file is identical to main.c's > > *link:* > gcc -g -pthread -shared main.o module.o -L/usr/bin/python/2.5.1/lib > *-lpython2.5* -ldl -lm -lpthread -lutil -lmx -lmex -lmat -lm -lm -lstdc++ > Ok, so far so good. Is /usr/bin/python/2.5.1/lib in LD_LIBRARY_PATH ? Out of curiosity: Are you building something against matlab, i.e. are you linking against MATLAB's mex? Cheers, Michael From igorsyl at gmail.com Tue Jan 13 07:31:43 2009 From: igorsyl at gmail.com (Igor Sylvester) Date: Tue, 13 Jan 2009 00:31:43 -0600 Subject: [Cython] PyExc_IOError In-Reply-To: <496C32B3.907@gmail.com> References: <496C2DFD.7090905@gmail.com> <496C32B3.907@gmail.com> Message-ID: Yes, the python .so is in my LD path. I can import the module from python without changing any environment variables. As the compiler flag reveals, I am building a Matlab extension. I need some numpy functionality which I don't want to re-code. Perhaps Matlab uses a different method to dynamically load libraries? On Tue, Jan 13, 2009 at 12:20 AM, Michael Abshoff < michael.abshoff at googlemail.com> wrote: > Igor Sylvester wrote: > > I built it against *Python 2.5.1. > > > > compile:* > > gcc -c > > > -I/usr/lib/python2.5/site-packages/numpy-1.0.4.0002-py2.5-linux-x86_64.egg/numpy/core/include > > -I/usr/bin/python/2.5.1/include/python2.5 -fPIC -fno-omit-frame-pointer > > -pthread -fexceptions -ansi -D_GNU_SOURCE -DMX_COMPAT_32 -g main.c > > > > compile line for the other c-file is identical to main.c's > > > > *link:* > > gcc -g -pthread -shared main.o module.o -L/usr/bin/python/2.5.1/lib > > *-lpython2.5* -ldl -lm -lpthread -lutil -lmx -lmex -lmat -lm -lm -lstdc++ > > > > Ok, so far so good. Is /usr/bin/python/2.5.1/lib in LD_LIBRARY_PATH ? > > Out of curiosity: Are you building something against matlab, i.e. are > you linking against MATLAB's mex? > > Cheers, > > Michael > > > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20090113/af38bb34/attachment-0001.htm From michael.abshoff at googlemail.com Tue Jan 13 07:40:20 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Mon, 12 Jan 2009 22:40:20 -0800 Subject: [Cython] PyExc_IOError In-Reply-To: References: <496C2DFD.7090905@gmail.com> <496C32B3.907@gmail.com> Message-ID: <496C3754.3040606@gmail.com> Igor Sylvester wrote: Hi, > Yes, the python .so is in my LD path. Ok. > I can import the module from python without changing any environment > variables. Good. > As the compiler flag reveals, I am building a Matlab extension. I need > some numpy functionality which I don't want to re-code. You could try to link libpyhthon.a statically. > Perhaps Matlab uses a different method to dynamically load libraries? Which release are you using? IIRC at least recent ones can show the content of env from within Matlab, so that might give you a clue. Cheers, Michael From igorsyl at gmail.com Tue Jan 13 14:34:20 2009 From: igorsyl at gmail.com (Igor Sylvester) Date: Tue, 13 Jan 2009 07:34:20 -0600 Subject: [Cython] PyExc_IOError In-Reply-To: <496C3754.3040606@gmail.com> References: <496C2DFD.7090905@gmail.com> <496C32B3.907@gmail.com> <496C3754.3040606@gmail.com> Message-ID: Statically linking python is of no avail. Adding python's time.so to the linker list doesn't help either. I believe it still tries to dlopen a library it isn't finding. I'll start looking at the source to see what's going on. It looks like other people have had similar problems but I haven't been able to find a concrete solution. http://lists.cistron.nl/pipermail/freeradius-users/2007-October/msg00032.html I'm using R2008a. Do you know the relevant commands? On Tue, Jan 13, 2009 at 12:40 AM, Michael Abshoff < michael.abshoff at googlemail.com> wrote: > Igor Sylvester wrote: > > Hi, > > > Yes, the python .so is in my LD path. > > Ok. > > > I can import the module from python without changing any environment > > variables. > > Good. > > > As the compiler flag reveals, I am building a Matlab extension. I need > > some numpy functionality which I don't want to re-code. > > You could try to link libpyhthon.a statically. > > > Perhaps Matlab uses a different method to dynamically load libraries? > > Which release are you using? IIRC at least recent ones can show the > content of env from within Matlab, so that might give you a clue. > > Cheers, > > Michael > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20090113/8e5e46b6/attachment.htm From michael.abshoff at googlemail.com Tue Jan 13 15:52:56 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Tue, 13 Jan 2009 06:52:56 -0800 Subject: [Cython] PyExc_IOError In-Reply-To: References: <496C2DFD.7090905@gmail.com> <496C32B3.907@gmail.com> <496C3754.3040606@gmail.com> Message-ID: <496CAAC8.1080902@gmail.com> Igor Sylvester wrote: Hi Igor, > Statically linking python is of no avail. Adding python's time.so to > the linker list doesn't help either. I believe it still tries to dlopen > a library it isn't finding. I'll start looking at the source to see > what's going on. Stracing Matlab might help here. When dlopen() complains about not finding something it is often that an external symbol in a dynamic lib cannot be found. dlopen's error message is a little retarded in that aspect and I found out about it after patching python's dlopen code and after a fifteen hour session of debugging Sage on Cygwin I learned that some dynamic libs can be loaded and others not. That lead me to the rather obvious idea to use a dependency tracker script to find all involved dynamic libs. One thing that might be different between Matlab and python are the RTLD_* flags used by Matlab itself. I am clueless if this is documented or if you have to poke around in Matlab to find out. To check the various flags check out /usr/include/dlfcn.h (or in case it isn't there on your Unix-y box grep around for RTLD_ in /usr/include. Another thing: For me on OSX time.so is in local/lib/python2.5/lib-dynload and I kind of doubt that this dynamic lib is picked up by your LD_LIBRARY_PATH, so you might want to copy missing dynamic libs in a location where you are 100% sure that LD_LIBRARY_PATH finds them. > It looks like other people have had similar problems but I haven't been > able to find a concrete solution. > http://lists.cistron.nl/pipermail/freeradius-users/2007-October/msg00032.html > > I'm using R2008a. Do you know the relevant commands? Nope, I used to write C++ extensions for 2006a or so and I would rather not touch Matlab ever again. In fact, I deleted it from my old laptop and never installed it on my new one. One last comment: If you post to another list like the numpy list about the same problem you might want to mention that you asked about the problem on another list and give a link to the archives. Cheers, Michael From robertwb at math.washington.edu Thu Jan 15 11:16:08 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Thu, 15 Jan 2009 02:16:08 -0800 Subject: [Cython] Refcount nanny status In-Reply-To: <4967FE32.6000109@gmail.com> References: <3314333946.756336@smtp.netcom.no> <4967104B.4000908@behnel.de> <4967FE32.6000109@gmail.com> Message-ID: <91C3B5A5-87FF-4C46-8F78-1BF70EA52CBB@math.washington.edu> On Jan 9, 2009, at 5:47 PM, Michael Abshoff wrote: > Stefan Behnel wrote: >> Dag Sverre Seljebotn wrote: >>> I don't have time for fixing it, but old_temp in NewTempsExprNode >>> was >>> introduced solely to do "print self.old_temp" during debugging. >>> Sorry, >>> there should have beena comment. >> >> I fixed the crash (but not the underlying temp problem). > > Thanks > >> You should at >> least get a meaningful error message out of it now. > > Now the first file in the Sage library builds, but the second one from > 3.2.3 fails: [...] > RuntimeError: no temp, but release requested in CoerceToTempNode > > I am not quite sure what to do next, but I am hoping RobertWB or > somwbody else has an idea what the problem might be :) OK, I looked into this and got Cython to work through the entire Sage library. It has issues compiling http://hg.sagemath.org/sage-main/file/b0aa7ef45b3c/sage/combinat/ matrices/dancing_links.pyx http://hg.sagemath.org/sage-main/file/b0aa7ef45b3c/sage/combinat/ matrices/dancing_links_c.h sage/combinat/matrices/dancing_links_c.h: In member function ?column* dancing_links::smallest_column()?: sage/combinat/matrices/dancing_links_c.h:73: error: ?set? was not declared in this scope sage/combinat/matrices/dancing_links_c.h:73: error: expected primary- expression before ?*? token sage/combinat/matrices/dancing_links_c.h:73: error: expected primary- expression before ?>? token sage/combinat/matrices/dancing_links_c.h:73: error: ?seenColumns? was not declared in this scope and I don't know enough C++ to diagnose the error message. Any ideas? - Robert From michael.abshoff at googlemail.com Thu Jan 15 11:30:19 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Thu, 15 Jan 2009 02:30:19 -0800 Subject: [Cython] Refcount nanny status In-Reply-To: <91C3B5A5-87FF-4C46-8F78-1BF70EA52CBB@math.washington.edu> References: <3314333946.756336@smtp.netcom.no> <4967104B.4000908@behnel.de> <4967FE32.6000109@gmail.com> <91C3B5A5-87FF-4C46-8F78-1BF70EA52CBB@math.washington.edu> Message-ID: <496F103B.4020302@gmail.com> Robert Bradshaw wrote: > On Jan 9, 2009, at 5:47 PM, Michael Abshoff wrote: > >> Stefan Behnel wrote: >>> Dag Sverre Seljebotn wrote: >>>> I don't have time for fixing it, but old_temp in NewTempsExprNode >>>> was >>>> introduced solely to do "print self.old_temp" during debugging. >>>> Sorry, >>>> there should have beena comment. >>> I fixed the crash (but not the underlying temp problem). >> Thanks >> >>> You should at >>> least get a meaningful error message out of it now. >> Now the first file in the Sage library builds, but the second one from >> 3.2.3 fails: > > [...] Hi Robert, >> RuntimeError: no temp, but release requested in CoerceToTempNode >> >> I am not quite sure what to do next, but I am hoping RobertWB or >> somwbody else has an idea what the problem might be :) > > OK, I looked into this and got Cython to work through the entire Sage > library. It has issues compiling > > http://hg.sagemath.org/sage-main/file/b0aa7ef45b3c/sage/combinat/ > matrices/dancing_links.pyx > http://hg.sagemath.org/sage-main/file/b0aa7ef45b3c/sage/combinat/ > matrices/dancing_links_c.h > > sage/combinat/matrices/dancing_links_c.h: In member function ?column* > dancing_links::smallest_column()?: > sage/combinat/matrices/dancing_links_c.h:73: error: ?set? was not > declared in this scope > sage/combinat/matrices/dancing_links_c.h:73: error: expected primary- > expression before ?*? token > sage/combinat/matrices/dancing_links_c.h:73: error: expected primary- > expression before ?>? token > sage/combinat/matrices/dancing_links_c.h:73: error: ?seenColumns? was > not declared in this scope > > and I don't know enough C++ to diagnose the error message. Any ideas? Which gcc are you using? Do you have your changes anywhere so I can play with them? Is set in any way now a reserved Cython word? Maybe it is as simple as prefixing set with std::, but I would prefer to play with it. Pinging Carlo Hamalainen (listed at the top of the file) might also be a good idea :) > - Robert Cheers, Michael > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From stefan_ml at behnel.de Thu Jan 15 12:39:42 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 15 Jan 2009 12:39:42 +0100 (CET) Subject: [Cython] Refcount nanny status In-Reply-To: <496F103B.4020302@gmail.com> References: <3314333946.756336@smtp.netcom.no> <4967104B.4000908@behnel.de> <4967FE32.6000109@gmail.com> <91C3B5A5-87FF-4C46-8F78-1BF70EA52CBB@math.washington.edu> <496F103B.4020302@gmail.com> Message-ID: <65054.213.61.181.86.1232019582.squirrel@groupware.dvs.informatik.tu-darmstadt.de> Hi, Michael Abshoff wrote: > Is set in any way now a reserved Cython word? No, but it's a known builtin type. > Maybe it is as simple as prefixing set with std:: If the C++ STL type is meant, that sounds promising. At least, it sounds to me like Cython is confusing some names here. Stefan From jholg at gmx.de Thu Jan 15 13:21:54 2009 From: jholg at gmx.de (jholg at gmx.de) Date: Thu, 15 Jan 2009 13:21:54 +0100 Subject: [Cython] [cython-dev] bugtracker login? Message-ID: <20090115122154.172130@gmx.net> Hi, maybe I'm missing something: How to file a ticket in the cython issue tracker? I can't seem to find the possibility to register. Cheers, Holger -- Sensationsangebot verl?ngert: GMX FreeDSL - Telefonanschluss + DSL f?r nur 16,37 Euro/mtl.!* http://dsl.gmx.de/?ac=OM.AD.PD003K1308T4569a From michael.abshoff at googlemail.com Thu Jan 15 13:30:03 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Thu, 15 Jan 2009 04:30:03 -0800 Subject: [Cython] [cython-dev] bugtracker login? In-Reply-To: <20090115122154.172130@gmx.net> References: <20090115122154.172130@gmx.net> Message-ID: <496F2C4B.4030504@gmail.com> jholg at gmx.de wrote: > Hi, Hi Holger > maybe I'm missing something: > > How to file a ticket in the cython issue tracker? I can't seem to find the possibility to register. Registration is disabled due to massive spamming when it is left open. Send an htdigest password to Robert Bradshaw and once he is up he will add it to the trac install. > Cheers, > Holger Cheers, Michael From robertwb at math.washington.edu Thu Jan 15 20:33:00 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Thu, 15 Jan 2009 11:33:00 -0800 Subject: [Cython] Refcount nanny status In-Reply-To: <496F103B.4020302@gmail.com> References: <3314333946.756336@smtp.netcom.no> <4967104B.4000908@behnel.de> <4967FE32.6000109@gmail.com> <91C3B5A5-87FF-4C46-8F78-1BF70EA52CBB@math.washington.edu> <496F103B.4020302@gmail.com> Message-ID: On Jan 15, 2009, at 2:30 AM, Michael Abshoff wrote: > Robert Bradshaw wrote: >> On Jan 9, 2009, at 5:47 PM, Michael Abshoff wrote: >> >>> Stefan Behnel wrote: >>>> Dag Sverre Seljebotn wrote: >>>>> I don't have time for fixing it, but old_temp in NewTempsExprNode >>>>> was >>>>> introduced solely to do "print self.old_temp" during debugging. >>>>> Sorry, >>>>> there should have beena comment. >>>> I fixed the crash (but not the underlying temp problem). >>> Thanks >>> >>>> You should at >>>> least get a meaningful error message out of it now. >>> Now the first file in the Sage library builds, but the second one >>> from >>> 3.2.3 fails: >> >> [...] > > Hi Robert, > >>> RuntimeError: no temp, but release requested in CoerceToTempNode >>> >>> I am not quite sure what to do next, but I am hoping RobertWB or >>> somwbody else has an idea what the problem might be :) >> >> OK, I looked into this and got Cython to work through the entire Sage >> library. It has issues compiling >> >> http://hg.sagemath.org/sage-main/file/b0aa7ef45b3c/sage/combinat/ >> matrices/dancing_links.pyx >> http://hg.sagemath.org/sage-main/file/b0aa7ef45b3c/sage/combinat/ >> matrices/dancing_links_c.h >> >> sage/combinat/matrices/dancing_links_c.h: In member function ?column* >> dancing_links::smallest_column()?: >> sage/combinat/matrices/dancing_links_c.h:73: error: ?set? was not >> declared in this scope >> sage/combinat/matrices/dancing_links_c.h:73: error: expected primary- >> expression before ?*? token >> sage/combinat/matrices/dancing_links_c.h:73: error: expected primary- >> expression before ?>? token >> sage/combinat/matrices/dancing_links_c.h:73: error: ?seenColumns? was >> not declared in this scope >> >> and I don't know enough C++ to diagnose the error message. Any ideas? > > Which gcc are you using? The one that ships with OS X 10.4 > Do you have your changes anywhere so I can play with them? I pushed them to sage-devel, and an spkg is up in http:// sage.math.washington.edu/home/robertwb/cython/ too. > Is set in any way now a reserved Cython word? Maybe it is as simple as > prefixing set with std::, but I would prefer to play with it. "set" doesn't appear anywhere in the .pyx file, and Cython compiles it just fine, so it'd be nice if it was something like this but I doubt it. It fails when it gets to the gcc stage. > Pinging > Carlo Hamalainen (listed at the top of the file) might also be a good > idea :) - Robert From robertwb at math.washington.edu Thu Jan 15 20:38:43 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Thu, 15 Jan 2009 11:38:43 -0800 Subject: [Cython] [cython-dev] bugtracker login? In-Reply-To: <496F2C4B.4030504@gmail.com> References: <20090115122154.172130@gmx.net> <496F2C4B.4030504@gmail.com> Message-ID: On Jan 15, 2009, at 4:30 AM, Michael Abshoff wrote: > jholg at gmx.de wrote: >> Hi, > > Hi Holger > >> maybe I'm missing something: >> >> How to file a ticket in the cython issue tracker? I can't seem to >> find the possibility to register. > > Registration is disabled due to massive spamming when it is left open. > Send an htdigest password to Robert Bradshaw and once he is up he will > add it to the trac install. Yes. I am sorry for this annoyance, but please do--we value your input. - Robert From michael.abshoff at googlemail.com Thu Jan 15 23:23:55 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Thu, 15 Jan 2009 14:23:55 -0800 Subject: [Cython] Refcount nanny status In-Reply-To: References: <3314333946.756336@smtp.netcom.no> <4967104B.4000908@behnel.de> <4967FE32.6000109@gmail.com> <91C3B5A5-87FF-4C46-8F78-1BF70EA52CBB@math.washington.edu> <496F103B.4020302@gmail.com> Message-ID: <496FB77B.7080301@gmail.com> Robert Bradshaw wrote: > On Jan 15, 2009, at 2:30 AM, Michael Abshoff wrote: >>> and I don't know enough C++ to diagnose the error message. Any ideas? >> Which gcc are you using? > > The one that ships with OS X 10.4 Apple doesn't ship XCode with OSX, but has some version of XCode on the companion DVD and AFAIK that version changes over time. Since you compile Sage on that box I assume it works, so the version isn't that relevant then I guess :) >> Do you have your changes anywhere so I can play with them? > > I pushed them to sage-devel, and an spkg is up in http:// > sage.math.washington.edu/home/robertwb/cython/ too. Ok, I will take a look hopefully today. >> Is set in any way now a reserved Cython word? Maybe it is as simple as >> prefixing set with std::, but I would prefer to play with it. > > "set" doesn't appear anywhere in the .pyx file, and Cython compiles > it just fine, so it'd be nice if it was something like this but I > doubt it. It fails when it gets to the gcc stage. Ok. >> Pinging >> Carlo Hamalainen (listed at the top of the file) might also be a good >> idea :) > > - Robert Cheers, Michael > > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From robertwb at math.washington.edu Thu Jan 15 23:53:51 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Thu, 15 Jan 2009 14:53:51 -0800 Subject: [Cython] Refcount nanny status In-Reply-To: <496FB77B.7080301@gmail.com> References: <3314333946.756336@smtp.netcom.no> <4967104B.4000908@behnel.de> <4967FE32.6000109@gmail.com> <91C3B5A5-87FF-4C46-8F78-1BF70EA52CBB@math.washington.edu> <496F103B.4020302@gmail.com> <496FB77B.7080301@gmail.com> Message-ID: <4C68636F-640D-4F07-9027-B63B99EDE268@math.washington.edu> On Jan 15, 2009, at 2:23 PM, Michael Abshoff wrote: > Robert Bradshaw wrote: >> On Jan 15, 2009, at 2:30 AM, Michael Abshoff wrote: > > > >>>> and I don't know enough C++ to diagnose the error message. Any >>>> ideas? >>> Which gcc are you using? >> >> The one that ships with OS X 10.4 > > Apple doesn't ship XCode with OSX, but has some version of XCode on > the > companion DVD and AFAIK that version changes over time. Since you > compile Sage on that box I assume it works, so the version isn't that > relevant then I guess :) Yep, I regulary compile Sage with this one :). For the record d-128-208-204-152:~ robert$ gcc -v Using built-in specs. Target: i686-apple-darwin8 Configured with: /private/var/tmp/gcc/gcc-5363.obj~28/src/configure -- disable-checking -enable-werror --prefix=/usr --mandir=/share/man -- enable-languages=c,objc,c++,obj-c++ --program-transform-name=/^[cg] [^.-]*$/s/$/-4.0/ --with-gxx-include-dir=/include/c++/4.0.0 --with- slibdir=/usr/lib --build=powerpc-apple-darwin8 --with-arch=nocona -- with-tune=generic --program-prefix= --host=i686-apple-darwin8 -- target=i686-apple-darwin8 Thread model: posix gcc version 4.0.1 (Apple Computer, Inc. build 5363) > >>> Do you have your changes anywhere so I can play with them? >> >> I pushed them to sage-devel, and an spkg is up in http:// >> sage.math.washington.edu/home/robertwb/cython/ too. > > Ok, I will take a look hopefully today. Excellent. - Robert > From michael.abshoff at googlemail.com Fri Jan 16 00:35:57 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Thu, 15 Jan 2009 15:35:57 -0800 Subject: [Cython] Sage 3.2.3 vs. cython 0.11-devel In-Reply-To: <4C68636F-640D-4F07-9027-B63B99EDE268@math.washington.edu> References: <3314333946.756336@smtp.netcom.no> <4967104B.4000908@behnel.de> <4967FE32.6000109@gmail.com> <91C3B5A5-87FF-4C46-8F78-1BF70EA52CBB@math.washington.edu> <496F103B.4020302@gmail.com> <496FB77B.7080301@gmail.com> <4C68636F-640D-4F07-9027-B63B99EDE268@math.washington.edu> Message-ID: <496FC85D.4010004@gmail.com> Robert Bradshaw wrote: > On Jan 15, 2009, at 2:23 PM, Michael Abshoff wrote: >>>> Do you have your changes anywhere so I can play with them? >>> I pushed them to sage-devel, and an spkg is up in http:// >>> sage.math.washington.edu/home/robertwb/cython/ too. >> Ok, I will take a look hopefully today. > > Excellent. Ok, adding std:: makes the code compile, but this is not the real fix. The next issue is this: gcc -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/scratch/mabshoff/junk/sage-3.2.3/local/include/fplll -I/scratch/mabshoff/junk/sage-3.2.3/local//include -I/scratch/mabshoff/junk/sage-3.2.3/local//include/csage -I/scratch/mabshoff/junk/sage-3.2.3/devel//sage/sage/ext -I/scratch/mabshoff/junk/sage-3.2.3/local/include/python2.5 -c sage/libs/fplll/fplll.cpp -o build/temp.linux-x86_64-2.5/sage/libs/fplll/fplll.o -w cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++ In file included from /scratch/mabshoff/junk/sage-3.2.3/local//include/fplll/wrapper.h:59, from /scratch/mabshoff/junk/sage-3.2.3/local//include/fplll/fplll.h:45, from sage/libs/fplll/fplll.cpp:136: /scratch/mabshoff/junk/sage-3.2.3/local//include/fplll/wrapper.cpp: In member function ?int wrapper::heuristicLoop(int)?: /scratch/mabshoff/junk/sage-3.2.3/local//include/fplll/wrapper.cpp:142: error: call of overloaded ?min(int, int&)? is ambiguous /scratch/mabshoff/junk/sage-3.2.3/local//include/fplll/wrapper.cpp:69: note: candidates are: int min(int, int) /usr/include/c++/4.2/bits/stl_algobase.h:182: note: const _Tp& std::min(const _Tp&, const _Tp&) [with _Tp = int] /scratch/mabshoff/junk/sage-3.2.3/local//include/NTL/tools.h:158: note: long int NTL::min(long int, int) /scratch/mabshoff/junk/sage-3.2.3/local//include/NTL/tools.h:155: note: long int NTL::min(int, long int) /scratch/mabshoff/junk/sage-3.2.3/local//include/NTL/tools.h:152: note: long int NTL::min(long int, long int) /scratch/mabshoff/junk/sage-3.2.3/local//include/NTL/tools.h:149: note: int NTL::min(int, int) /scratch/mabshoff/junk/sage-3.2.3/local//include/fplll/wrapper.cpp: In member function ?int wrapper::LLL()?: /scratch/mabshoff/junk/sage-3.2.3/local//include/fplll/wrapper.cpp:208: error: call of overloaded ?min(int, int&)? is ambiguous /scratch/mabshoff/junk/sage-3.2.3/local//include/fplll/wrapper.cpp:69: note: candidates are: int min(int, int) /usr/include/c++/4.2/bits/stl_algobase.h:182: note: const _Tp& std::min(const _Tp&, const _Tp&) [with _Tp = int] /scratch/mabshoff/junk/sage-3.2.3/local//include/NTL/tools.h:158: note: long int NTL::min(long int, int) /scratch/mabshoff/junk/sage-3.2.3/local//include/NTL/tools.h:155: note: long int NTL::min(int, long int) /scratch/mabshoff/junk/sage-3.2.3/local//include/NTL/tools.h:152: note: long int NTL::min(long int, long int) /scratch/mabshoff/junk/sage-3.2.3/local//include/NTL/tools.h:149: note: int NTL::min(int, int) /scratch/mabshoff/junk/sage-3.2.3/local//include/fplll/wrapper.cpp:218: error: call of overloaded ?min(int, int&)? is ambiguous /scratch/mabshoff/junk/sage-3.2.3/local//include/fplll/wrapper.cpp:69: note: candidates are: int min(int, int) /usr/include/c++/4.2/bits/stl_algobase.h:182: note: const _Tp& std::min(const _Tp&, const _Tp&) [with _Tp = int] /scratch/mabshoff/junk/sage-3.2.3/local//include/NTL/tools.h:158: note: long int NTL::min(long int, int) /scratch/mabshoff/junk/sage-3.2.3/local//include/NTL/tools.h:155: note: long int NTL::min(int, long int) /scratch/mabshoff/junk/sage-3.2.3/local//include/NTL/tools.h:152: note: long int NTL::min(long int, long int) /scratch/mabshoff/junk/sage-3.2.3/local//include/NTL/tools.h:149: note: int NTL::min(int, int) error: command 'gcc' failed with exit status 1 sage: There was an error installing modified sage library code. Ok, I cursed the person who defined a function called "min" in the global namespace and prefixing the calls with "::" makes the extension compile. The this error popped up: gcc -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DGSL_DISABLE_DEPRECATED=1 -I/scratch/mabshoff/junk/sage-3.2.3/local//include -I/scratch/mabshoff/junk/sage-3.2.3/local//include/csage -I/scratch/mabshoff/junk/sage-3.2.3/devel//sage/sage/ext -I/scratch/mabshoff/junk/sage-3.2.3/local/include/python2.5 -c sage/rings/real_double.c -o build/temp.linux-x86_64-2.5/sage/rings/real_double.o -w In file included from sage/rings/real_double.c:148: /scratch/mabshoff/junk/sage-3.2.3/local//include/gsl/gsl_sf_bessel.h:287: error: expected ?;?, ?,? or ?)? before ?gmax? /scratch/mabshoff/junk/sage-3.2.3/local//include/gsl/gsl_sf_bessel.h:295: error: expected ?;?, ?,? or ?)? before ?gmax? /scratch/mabshoff/junk/sage-3.2.3/local//include/gsl/gsl_sf_bessel.h:334: error: expected ?;?, ?,? or ?)? before ?gmax? /scratch/mabshoff/junk/sage-3.2.3/local//include/gsl/gsl_sf_bessel.h:387: error: expected ?;?, ?,? or ?)? before ?gmax? /scratch/mabshoff/junk/sage-3.2.3/local//include/gsl/gsl_sf_bessel.h:437: error: expected ?;?, ?,? or ?)? before ?gmax? In file included from sage/rings/real_double.c:167: /scratch/mabshoff/junk/sage-3.2.3/local//include/gsl/gsl_sf_legendre.h:53: error: expected ?;?, ?,? or ?)? before ?gmax? /scratch/mabshoff/junk/sage-3.2.3/local//include/gsl/gsl_sf_legendre.h:63: error: expected ?;?, ?,? or ?)? before ?gmax? /scratch/mabshoff/junk/sage-3.2.3/local//include/gsl/gsl_sf_legendre.h:132: error: expected ?;?, ?,? or ?)? before ?gmax? /scratch/mabshoff/junk/sage-3.2.3/local//include/gsl/gsl_sf_legendre.h:143: error: expected ?;?, ?,? or ?)? before ?gmax? /scratch/mabshoff/junk/sage-3.2.3/local//include/gsl/gsl_sf_legendre.h:172: error: expected ?;?, ?,? or ?)? before ?gmax? /scratch/mabshoff/junk/sage-3.2.3/local//include/gsl/gsl_sf_legendre.h:184: error: expected ?;?, ?,? or ?)? before ?gmax? /scratch/mabshoff/junk/sage-3.2.3/local//include/gsl/gsl_sf_legendre.h:194: error: expected ?;?, ?,? or ?)? before ?gmax? /scratch/mabshoff/junk/sage-3.2.3/local//include/gsl/gsl_sf_legendre.h:311: error: expected ?;?, ?,? or ?)? before ?gmax? error: command 'gcc' failed with exit status 1 sage: There was an error installing modified sage library code. This is getting strange, so for the hell of it I disabled the pari import since gmax is defined there: --- a/sage/rings/real_double.pyx Mon Jan 05 23:03:45 2009 -0800 +++ b/sage/rings/real_double.pyx Thu Jan 15 15:27:34 2009 -0800 @@ -1,4 +1,4 @@ -r""" +""" Double Precision Real Numbers EXAMPLES: @@ -30,7 +30,7 @@ import math, operator -cimport sage.libs.pari.gen +#cimport sage.libs.pari.gen import sage.libs.pari.gen @@ -1025,8 +1025,8 @@ sage: RDF(1.5)._pari_() 1.50000000000000 """ - cdef sage.libs.pari.gen.PariInstance P = sage.libs.pari.gen.pari - return P.double_to_gen_c(self._value) + #cdef sage.libs.pari.gen.PariInstance P = sage.libs.pari.gen.pari + return 1 #P.double_to_gen_c(self._value) ########################################### This made Cython throw this: Installing c_lib scons: `install' is up to date. Updating Cython code.... Building modified file sage/rings/real_double.pyx. python2.5 `which cython` --embed-positions --incref-local-binop -I/scratch/mabshoff/junk/sage-3.2.3/devel/sage-main -o sage/rings/real_double.c sage/rings/real_double.pyx Traceback (most recent call last): File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/Optimize.py", line 557, in _calculate_const node.calculate_constant_result() File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/ExprNodes.py", line 4028, in calculate_constant_result self.operand2.constant_result) ZeroDivisionError: float division Traceback (most recent call last): File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/Optimize.py", line 557, in _calculate_const node.calculate_constant_result() File "/scratch/mabshoff/junk/sage-3.2.3/local/lib/python2.5/site-packages/Cython/Compiler/ExprNodes.py", line 4028, in calculate_constant_result self.operand2.constant_result) ZeroDivisionError: float division sage/rings/real_double.pyx --> /scratch/mabshoff/junk/sage-3.2.3/local//lib/python/site-packages//sage/rings/real_double.pyx Time to execute 1 commands: 7.37845206261 seconds Finished compiling Cython code (time = 7.78979706764 seconds) running install Anyway, moving on since the pattern of the problem is shown with the next failure: gcc -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/scratch/mabshoff/junk/sage-3.2.3/local//include -I/scratch/mabshoff/junk/sage-3.2.3/local//include/csage -I/scratch/mabshoff/junk/sage-3.2.3/devel//sage/sage/ext -I/scratch/mabshoff/junk/sage-3.2.3/local/include/python2.5 -c sage/rings/real_rqdf.cpp -o build/temp.linux-x86_64-2.5/sage/rings/real_rqdf.o -w cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++ In file included from /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_real.h:30, from sage/rings/real_rqdf.cpp:140: /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/dd_real.h:282:30: error: macro "min" requires 2 arguments, but only 1 given In file included from sage/rings/real_rqdf.cpp:140: /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_real.h:274:81: error: macro "max" passed 3 arguments, but takes just 2 /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_real.h:276:81: error: macro "min" passed 3 arguments, but takes just 2 /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_real.h:306:30: error: macro "min" requires 2 arguments, but only 1 given In file included from /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_real.h:324, from sage/rings/real_rqdf.cpp:140: /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_inline.h:1012:36: error: macro "max" passed 3 arguments, but takes just 2 /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_inline.h:1021:36: error: macro "min" passed 3 arguments, but takes just 2 In file included from /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_real.h:30, from sage/rings/real_rqdf.cpp:140: /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/dd_real.h:282: error: function definition does not declare parameters In file included from sage/rings/real_rqdf.cpp:140: /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_real.h:273: error: expected unqualified-id before ?const? /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_real.h:273: error: expected `)' before ?const? /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_real.h:273: error: expected `)' before ?const? /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_real.h:274: error: field ?max? has incomplete type /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_real.h:275: error: expected unqualified-id before ?const? /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_real.h:275: error: expected `)' before ?const? /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_real.h:275: error: expected `)' before ?const? /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_real.h:276: error: field ?min? has incomplete type /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_real.h:306: error: function definition does not declare parameters In file included from /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_real.h:324, from sage/rings/real_rqdf.cpp:140: /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_inline.h:1007: error: expected unqualified-id before ?const? /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_inline.h:1007: error: expected `)' before ?const? /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_inline.h:1007: error: expected `)' before ?const? /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_inline.h:1011: error: function definition does not declare parameters /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_inline.h:1016: error: expected unqualified-id before ?const? /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_inline.h:1016: error: expected `)' before ?const? /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_inline.h:1016: error: expected `)' before ?const? /scratch/mabshoff/junk/sage-3.2.3/local//include/qd/qd_inline.h:1020: error: function definition does not declare parameters and again we are having a problem with min and so on. Cython seems to be at fault for all the above issues. I.e. for some reason we end up using symbols from Sage's library like set, min, gmax and so on that in 0.10 we did not use. For C++ we can monkey patch around them which is a bad idea and for C code we seem to be screwed for now. Thoughts? I know way to little about Cython internals to have an idea what is going on here :) > - Robert Cheers, Michael > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From robertwb at math.washington.edu Fri Jan 16 01:52:07 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Thu, 15 Jan 2009 16:52:07 -0800 Subject: [Cython] Sage 3.2.3 vs. cython 0.11-devel In-Reply-To: <496FC85D.4010004@gmail.com> References: <3314333946.756336@smtp.netcom.no> <4967104B.4000908@behnel.de> <4967FE32.6000109@gmail.com> <91C3B5A5-87FF-4C46-8F78-1BF70EA52CBB@math.washington.edu> <496F103B.4020302@gmail.com> <496FB77B.7080301@gmail.com> <4C68636F-640D-4F07-9027-B63B99EDE268@math.washington.edu> <496FC85D.4010004@gmail.com> Message-ID: <6FDEC0FB-2B7C-455C-A570-7CE0A77A3691@math.washington.edu> On Jan 15, 2009, at 3:35 PM, Michael Abshoff wrote: > Robert Bradshaw wrote: >> On Jan 15, 2009, at 2:23 PM, Michael Abshoff wrote: > > > >>>>> Do you have your changes anywhere so I can play with them? >>>> I pushed them to sage-devel, and an spkg is up in http:// >>>> sage.math.washington.edu/home/robertwb/cython/ too. >>> Ok, I will take a look hopefully today. >> >> Excellent. > > Ok, adding std:: makes the code compile, but this is not the real fix. [...] > and again we are having a problem with min and so on. Cython seems > to be > at fault for all the above issues. I.e. for some reason we end up > using symbols from Sage's library like set, min, gmax and so on > that in > 0.10 we did not use. For C++ we can monkey patch around them which > is a > bad idea and for C code we seem to be screwed for now. > > Thoughts? I know way to little about Cython internals to have an idea > what is going on here :) Thanks for looking into this--from what I understand of what you sent it's symbols like "min" and "set" that are getting exported to the namespace that weren't before? I'm not sure what happened between 0.10.3 and now that would call this--did anyone include any new headers? - Robert From michael.abshoff at googlemail.com Fri Jan 16 02:04:05 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Thu, 15 Jan 2009 17:04:05 -0800 Subject: [Cython] Sage 3.2.3 vs. cython 0.11-devel In-Reply-To: <6FDEC0FB-2B7C-455C-A570-7CE0A77A3691@math.washington.edu> References: <3314333946.756336@smtp.netcom.no> <4967104B.4000908@behnel.de> <4967FE32.6000109@gmail.com> <91C3B5A5-87FF-4C46-8F78-1BF70EA52CBB@math.washington.edu> <496F103B.4020302@gmail.com> <496FB77B.7080301@gmail.com> <4C68636F-640D-4F07-9027-B63B99EDE268@math.washington.edu> <496FC85D.4010004@gmail.com> <6FDEC0FB-2B7C-455C-A570-7CE0A77A3691@math.washington.edu> Message-ID: <496FDD05.6010605@gmail.com> Robert Bradshaw wrote: > On Jan 15, 2009, at 3:35 PM, Michael Abshoff wrote: Hi Robert, >> Ok, adding std:: makes the code compile, but this is not the real fix. > > [...] > >> and again we are having a problem with min and so on. Cython seems >> to be >> at fault for all the above issues. I.e. for some reason we end up >> using symbols from Sage's library like set, min, gmax and so on >> that in >> 0.10 we did not use. For C++ we can monkey patch around them which >> is a >> bad idea and for C code we seem to be screwed for now. >> >> Thoughts? I know way to little about Cython internals to have an idea >> what is going on here :) > > Thanks for looking into this--from what I understand of what you sent > it's symbols like "min" and "set" that are getting exported to the > namespace that weren't before? Yes, I avoided the term namespace since there is no meaning like that in C as I assume you all know well. Obviously calling "internal" functions stuff like min or max is a disaster waiting to happen. On the other hand it is ironic that std's set was picked up, but that can be easily fixed for C++ code, but not C code. Given that gmax from gsl and pari collided lead me to the conclusion you also reached above. > I'm not sure what happened between > 0.10.3 and now that would call this--did anyone include any new headers? I don't know, but I assume you mean this in the context of Cython. > - Robert Cheers, Michael > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From robertwb at math.washington.edu Fri Jan 16 02:08:07 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Thu, 15 Jan 2009 17:08:07 -0800 Subject: [Cython] Sage 3.2.3 vs. cython 0.11-devel In-Reply-To: <496FDD05.6010605@gmail.com> References: <3314333946.756336@smtp.netcom.no> <4967104B.4000908@behnel.de> <4967FE32.6000109@gmail.com> <91C3B5A5-87FF-4C46-8F78-1BF70EA52CBB@math.washington.edu> <496F103B.4020302@gmail.com> <496FB77B.7080301@gmail.com> <4C68636F-640D-4F07-9027-B63B99EDE268@math.washington.edu> <496FC85D.4010004@gmail.com> <6FDEC0FB-2B7C-455C-A570-7CE0A77A3691@math.washington.edu> <496FDD05.6010605@gmail.com> Message-ID: <260C9AEF-A860-4FFD-8E8F-E78846A752ED@math.washington.edu> On Jan 15, 2009, at 5:04 PM, Michael Abshoff wrote: > Robert Bradshaw wrote: >> On Jan 15, 2009, at 3:35 PM, Michael Abshoff wrote: > > > > Hi Robert, > >>> Ok, adding std:: makes the code compile, but this is not the real >>> fix. >> >> [...] >> >>> and again we are having a problem with min and so on. Cython seems >>> to be >>> at fault for all the above issues. I.e. for some reason we end up >>> using symbols from Sage's library like set, min, gmax and so on >>> that in >>> 0.10 we did not use. For C++ we can monkey patch around them which >>> is a >>> bad idea and for C code we seem to be screwed for now. >>> >>> Thoughts? I know way to little about Cython internals to have an >>> idea >>> what is going on here :) >> >> Thanks for looking into this--from what I understand of what you sent >> it's symbols like "min" and "set" that are getting exported to the >> namespace that weren't before? > > > Yes, I avoided the term namespace since there is no meaning like > that in > C as I assume you all know well. Yeah. Scope would be a better word, and there's only one global one. > Obviously calling "internal" functions > stuff like min or max is a disaster waiting to happen. On the other > hand > it is ironic that std's set was picked up, but that can be easily > fixed > for C++ code, but not C code. Given that gmax from gsl and pari > collided > lead me to the conclusion you also reached above. > >> I'm not sure what happened between >> 0.10.3 and now that would call this--did anyone include any new >> headers? > > I don't know, but I assume you mean this in the context of Cython. Yes, I'm meaning in the context of Cython. Either that or someone changed how (C++?) files get processed. I certainly haven't done anything in that area, but it's possible that someone else did. A random thought, this couldn't be an artifact of all the char* -> const char* changes, could it be? - Robert From michael.abshoff at googlemail.com Fri Jan 16 02:14:00 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Thu, 15 Jan 2009 17:14:00 -0800 Subject: [Cython] Sage 3.2.3 vs. cython 0.11-devel In-Reply-To: <260C9AEF-A860-4FFD-8E8F-E78846A752ED@math.washington.edu> References: <3314333946.756336@smtp.netcom.no> <4967104B.4000908@behnel.de> <4967FE32.6000109@gmail.com> <91C3B5A5-87FF-4C46-8F78-1BF70EA52CBB@math.washington.edu> <496F103B.4020302@gmail.com> <496FB77B.7080301@gmail.com> <4C68636F-640D-4F07-9027-B63B99EDE268@math.washington.edu> <496FC85D.4010004@gmail.com> <6FDEC0FB-2B7C-455C-A570-7CE0A77A3691@math.washington.edu> <496FDD05.6010605@gmail.com> <260C9AEF-A860-4FFD-8E8F-E78846A752ED@math.washington.edu> Message-ID: <496FDF58.1090206@gmail.com> Robert Bradshaw wrote: > On Jan 15, 2009, at 5:04 PM, Michael Abshoff wrote: Hi Robert, >>> Thanks for looking into this--from what I understand of what you sent >>> it's symbols like "min" and "set" that are getting exported to the >>> namespace that weren't before? >> >> Yes, I avoided the term namespace since there is no meaning like >> that in >> C as I assume you all know well. > > Yeah. Scope would be a better word, and there's only one global one. :) - having a discussion about Cython, C and C++ at the same time makes using technical terms more than a little ambiguous. >>> I'm not sure what happened between >>> 0.10.3 and now that would call this--did anyone include any new >>> headers? >> I don't know, but I assume you mean this in the context of Cython. > > Yes, I'm meaning in the context of Cython. Either that or someone > changed how (C++?) files get processed. I certainly haven't done > anything in that area, but it's possible that someone else did. This isn't just C++, i.e. the pari failures, the quaddouble and others are all pure C issues unless NTL is pulled for example in those cases. > A random thought, this couldn't be an artifact of all the char* -> > const char* changes, could it be? Given that this is also a C issue I cannot see how this could cause the problem, but given my experience over the last year on porting Sage I can tell you that the impossible seems to happen weekly and I have to eat my hat because of that quite often :) > - Robert Cheers, Michael > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From michael.abshoff at googlemail.com Fri Jan 16 04:29:34 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Thu, 15 Jan 2009 19:29:34 -0800 Subject: [Cython] setup.py does not refer to the Apache license Message-ID: <496FFF1E.7040807@gmail.com> Hi, you guys switched to the Apache [2?] license a while ago, but setup.py seems to have not been updated: classifiers = [ "Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "License :: OSI Approved :: Python Software Foundation License", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: C", "Topic :: Software Development :: Code Generators", "Topic :: Software Development :: Compilers", "Topic :: Software Development :: Libraries :: Python Modules" ], Cheers, Michael From robertwb at math.washington.edu Fri Jan 16 05:23:42 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Thu, 15 Jan 2009 20:23:42 -0800 Subject: [Cython] Sage 3.2.3 vs. cython 0.11-devel In-Reply-To: <496FDF58.1090206@gmail.com> References: <3314333946.756336@smtp.netcom.no> <4967104B.4000908@behnel.de> <4967FE32.6000109@gmail.com> <91C3B5A5-87FF-4C46-8F78-1BF70EA52CBB@math.washington.edu> <496F103B.4020302@gmail.com> <496FB77B.7080301@gmail.com> <4C68636F-640D-4F07-9027-B63B99EDE268@math.washington.edu> <496FC85D.4010004@gmail.com> <6FDEC0FB-2B7C-455C-A570-7CE0A77A3691@math.washington.edu> <496FDD05.6010605@gmail.com> <260C9AEF-A860-4FFD-8E8F-E78846A752ED@math.washington.edu> <496FDF58.1090206@gmail.com> Message-ID: <9D435F92-48E5-4D48-BC77-96E928C59341@math.washington.edu> On Jan 15, 2009, at 5:14 PM, Michael Abshoff wrote: > Robert Bradshaw wrote: >> On Jan 15, 2009, at 5:04 PM, Michael Abshoff wrote: > > > > Hi Robert, > >>>> Thanks for looking into this--from what I understand of what you >>>> sent >>>> it's symbols like "min" and "set" that are getting exported to the >>>> namespace that weren't before? >>> >>> Yes, I avoided the term namespace since there is no meaning like >>> that in >>> C as I assume you all know well. >> >> Yeah. Scope would be a better word, and there's only one global one. > > :) - having a discussion about Cython, C and C++ at the same time > makes > using technical terms more than a little ambiguous. > >>>> I'm not sure what happened between >>>> 0.10.3 and now that would call this--did anyone include any new >>>> headers? >>> I don't know, but I assume you mean this in the context of Cython. >> >> Yes, I'm meaning in the context of Cython. Either that or someone >> changed how (C++?) files get processed. I certainly haven't done >> anything in that area, but it's possible that someone else did. > > This isn't just C++, i.e. the pari failures, the quaddouble and others > are all pure C issues unless NTL is pulled for example in those cases. OK, I think it's because the ordering of #includes has been changed. I'm responsible for this--fixed some things, but apparently broke others. Well, at least I have somewhere to look now. - Robert From robertwb at math.washington.edu Fri Jan 16 09:59:13 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 16 Jan 2009 00:59:13 -0800 Subject: [Cython] Sage 3.2.3 vs. cython 0.11-devel In-Reply-To: <9D435F92-48E5-4D48-BC77-96E928C59341@math.washington.edu> References: <3314333946.756336@smtp.netcom.no> <4967104B.4000908@behnel.de> <4967FE32.6000109@gmail.com> <91C3B5A5-87FF-4C46-8F78-1BF70EA52CBB@math.washington.edu> <496F103B.4020302@gmail.com> <496FB77B.7080301@gmail.com> <4C68636F-640D-4F07-9027-B63B99EDE268@math.washington.edu> <496FC85D.4010004@gmail.com> <6FDEC0FB-2B7C-455C-A570-7CE0A77A3691@math.washington.edu> <496FDD05.6010605@gmail.com> <260C9AEF-A860-4FFD-8E8F-E78846A752ED@math.washington.edu> <496FDF58.1090206@gmail.com> <9D435F92-48E5-4D48-BC77-96E928C59341@math.washington.edu> Message-ID: On Jan 15, 2009, at 8:23 PM, Robert Bradshaw wrote: > On Jan 15, 2009, at 5:14 PM, Michael Abshoff wrote: > > OK, I think it's because the ordering of #includes has been changed. > I'm responsible for this--fixed some things, but apparently broke > others. Well, at least I have somewhere to look now. Sage now compiles, a ticket is up at http://trac.sagemath.org/ sage_trac/ticket/4987 Not all doctests pass, but it looks like it's in pretty good shape. - Robert From dagss at student.matnat.uio.no Fri Jan 16 10:36:52 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Fri, 16 Jan 2009 10:36:52 +0100 Subject: [Cython] Fortran types in C Message-ID: <49705534.2030003@student.matnat.uio.no> As I have to interface a lot with some Fortran code these days, and a full-fledged Cython<->Fortran-binding *may* come out of it (if I continue to manage using Cython for this task it would be a likely result in about a year, at least I structure my hacky scripts in that direction), I feel free to ask a somewhat tangential question: What is the best way of declaring Fortran types in C? I understand there is likely no perfect cross-platform solution here, but is the naive approach taken by f2py (C int == integer(4) == default Fortran integer) sufficient? Including libgfortran.h or similar is too compiler dependant and complicated (but I'd be willing to use a single header file with ifdefs for platforms and compilers, if anyone have such a thing...) Looking to a Cython binding, I think there would be a seperate type hierarchy for Fortran types ("cython.fortran.real8" etc.), using Fortran specific typedefs in C. Furthermore it would be very sweet to automatically be able to pass Python buffer objects as F90 assumed-shape arrays :-) (I think a Fortran wrapper module would then have to be generated, as there's no consistent ABI for those. The same goes for strings.). Dag Sverre From stefan_ml at behnel.de Fri Jan 16 13:19:56 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 16 Jan 2009 13:19:56 +0100 Subject: [Cython] setup.py does not refer to the Apache license In-Reply-To: <496FFF1E.7040807@gmail.com> References: <496FFF1E.7040807@gmail.com> Message-ID: <49707B6C.10309@behnel.de> Hi, Michael Abshoff wrote: > you guys switched to the Apache [2?] license a while ago, but setup.py > seems to have not been updated: > > classifiers = [ > "Development Status :: 5 - Production/Stable", Given the current trunk status, I find this a lot more surprising. :) > "License :: OSI Approved :: Python Software Foundation License", Thanks, fixed. Stefan From michael.abshoff at googlemail.com Fri Jan 16 14:24:38 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Fri, 16 Jan 2009 05:24:38 -0800 Subject: [Cython] setup.py does not refer to the Apache license In-Reply-To: <49707B6C.10309@behnel.de> References: <496FFF1E.7040807@gmail.com> <49707B6C.10309@behnel.de> Message-ID: <49708A96.1080302@gmail.com> Stefan Behnel wrote: > Hi, > > Michael Abshoff wrote: >> you guys switched to the Apache [2?] license a while ago, but setup.py >> seems to have not been updated: >> >> classifiers = [ >> "Development Status :: 5 - Production/Stable", > > Given the current trunk status, I find this a lot more surprising. :) > Well, Robert fixed the compilation issues with the Sage library and stated that "most doctests pass". I was a little curious and ended up discovering that roughly 130 or so failed, many with segfaults. While technically true, I would not call 130 failed doctests out of 950 or so "most". But the issue is more than likely caused by a few bugs in the new Cython, so poking around should reveal those. >> "License :: OSI Approved :: Python Software Foundation License", > > Thanks, fixed. I guess I should have send a patch, but I was too lazy. Next time I will send a patch I guess :) > Stefan Cheers, Michael > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From robertwb at math.washington.edu Fri Jan 16 20:43:03 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 16 Jan 2009 11:43:03 -0800 Subject: [Cython] Fortran types in C In-Reply-To: <49705534.2030003@student.matnat.uio.no> References: <49705534.2030003@student.matnat.uio.no> Message-ID: On Jan 16, 2009, at 1:36 AM, Dag Sverre Seljebotn wrote: > As I have to interface a lot with some Fortran code these days, and a > full-fledged Cython<->Fortran-binding *may* come out of it (if I > continue to manage using Cython for this task it would be a likely > result in about a year, at least I structure my hacky scripts in that > direction), I know virtually nothing about Fortran, but this sure sounds cool and I'm sure the number-crunching communities would really like it. > I feel free to ask a somewhat tangential question: > > What is the best way of declaring Fortran types in C? I understand > there > is likely no perfect cross-platform solution here, but is the naive > approach taken by f2py (C int == integer(4) == default Fortran > integer) > sufficient? Including libgfortran.h or similar is too compiler > dependant > and complicated (but I'd be willing to use a single header file with > ifdefs for platforms and compilers, if anyone have such a thing...) > > Looking to a Cython binding, I think there would be a seperate type > hierarchy for Fortran types ("cython.fortran.real8" etc.), using > Fortran > specific typedefs in C. Furthermore it would be very sweet to > automatically be able to pass Python buffer objects as F90 assumed- > shape > arrays :-) (I think a Fortran wrapper module would then have to be > generated, as there's no consistent ABI for those. The same goes for > strings.). I would follow whatever conventions are followed for C-Fortran interfaces, if that makes any sense. - Robert From robertwb at math.washington.edu Fri Jan 16 20:45:22 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 16 Jan 2009 11:45:22 -0800 Subject: [Cython] setup.py does not refer to the Apache license In-Reply-To: <49708A96.1080302@gmail.com> References: <496FFF1E.7040807@gmail.com> <49707B6C.10309@behnel.de> <49708A96.1080302@gmail.com> Message-ID: <6A556DB6-5A0C-4587-AAA0-1F3BB4AE6311@math.washington.edu> On Jan 16, 2009, at 5:24 AM, Michael Abshoff wrote: > Stefan Behnel wrote: >> Hi, >> >> Michael Abshoff wrote: >>> you guys switched to the Apache [2?] license a while ago, but >>> setup.py >>> seems to have not been updated: >>> >>> classifiers = [ >>> "Development Status :: 5 - Production/Stable", >> >> Given the current trunk status, I find this a lot more surprising. :) :). Well, hopefully it'll be true again by the time we release. > Well, Robert fixed the compilation issues with the Sage library and > stated that "most doctests pass". I was a little curious and ended up > discovering that roughly 130 or so failed, many with segfaults. While > technically true, I would not call 130 failed doctests out of 950 > or so > "most". But the issue is more than likely caused by a few bugs in the > new Cython, so poking around should reveal those. I call it most in the sense that it shows the vast majority of generated code must be correct. Of course, there still is a lot to be done. - Robert From michael.abshoff at googlemail.com Sat Jan 17 00:20:18 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Fri, 16 Jan 2009 15:20:18 -0800 Subject: [Cython] Sage 3.2.3 vs. cython 0.11-devel In-Reply-To: References: <3314333946.756336@smtp.netcom.no> <4967104B.4000908@behnel.de> <4967FE32.6000109@gmail.com> <91C3B5A5-87FF-4C46-8F78-1BF70EA52CBB@math.washington.edu> <496F103B.4020302@gmail.com> <496FB77B.7080301@gmail.com> <4C68636F-640D-4F07-9027-B63B99EDE268@math.washington.edu> <496FC85D.4010004@gmail.com> <6FDEC0FB-2B7C-455C-A570-7CE0A77A3691@math.washington.edu> <496FDD05.6010605@gmail.com> <260C9AEF-A860-4FFD-8E8F-E78846A752ED@math.washington.edu> <496FDF58.1090206@gmail.com> <9D435F92-48E5-4D48-BC77-96E928C59341@math.washington.edu> Message-ID: <49711632.2000808@gmail.com> Robert Bradshaw wrote: > On Jan 15, 2009, at 8:23 PM, Robert Bradshaw wrote: Hi Robert, >> On Jan 15, 2009, at 5:14 PM, Michael Abshoff wrote: >> >> OK, I think it's because the ordering of #includes has been changed. >> I'm responsible for this--fixed some things, but apparently broke >> others. Well, at least I have somewhere to look now. > > Sage now compiles, a ticket is up at http://trac.sagemath.org/ > sage_trac/ticket/4987 > > Not all doctests pass, but it looks like it's in pretty good shape. Any chance the same issue exists in Cython 0.10.3? With Sage on OSX 10.5 in 64 bit mode I am seeing memory corruption issues when minpoly() calls libsingular's minpoly from rather odd places. The main difference between the 32 and 64 bit builds are that linSingular is compiled with a different allocator, i.e. mmap vs. malloc. > - Robert Cheers, Michael > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From michael.abshoff at googlemail.com Sat Jan 17 00:34:50 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Fri, 16 Jan 2009 15:34:50 -0800 Subject: [Cython] Sage 3.2.3 vs. cython 0.11-devel In-Reply-To: References: <3314333946.756336@smtp.netcom.no> <4967104B.4000908@behnel.de> <4967FE32.6000109@gmail.com> <91C3B5A5-87FF-4C46-8F78-1BF70EA52CBB@math.washington.edu> <496F103B.4020302@gmail.com> <496FB77B.7080301@gmail.com> <4C68636F-640D-4F07-9027-B63B99EDE268@math.washington.edu> <496FC85D.4010004@gmail.com> <6FDEC0FB-2B7C-455C-A570-7CE0A77A3691@math.washington.edu> <496FDD05.6010605@gmail.com> <260C9AEF-A860-4FFD-8E8F-E78846A752ED@math.washington.edu> <496FDF58.1090206@gmail.com> <9D435F92-48E5-4D48-BC77-96E928C59341@math.washington.edu> Message-ID: <4971199A.7040207@gmail.com> Robert Bradshaw wrote: > On Jan 15, 2009, at 8:23 PM, Robert Bradshaw wrote: > >> On Jan 15, 2009, at 5:14 PM, Michael Abshoff wrote: >> >> OK, I think it's because the ordering of #includes has been changed. >> I'm responsible for this--fixed some things, but apparently broke >> others. Well, at least I have somewhere to look now. > > Sage now compiles, a ticket is up at http://trac.sagemath.org/ > sage_trac/ticket/4987 > > Not all doctests pass, but it looks like it's in pretty good shape. And having thought about the minpoly() issue a little more it now very much reminds me of the issue we saw at SD8.5 when the new-coercion-merge build on sage.math exhibited a mysterious segfault that only I was able to reproduce. Another thing was a mysterious doctest segfault that at some point magically went away and also involved a call to minpoly() in libSingular in a codepath that could have never been taken. > - Robert Cheers, Michael > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From robertwb at math.washington.edu Sat Jan 17 10:59:08 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sat, 17 Jan 2009 01:59:08 -0800 Subject: [Cython] Temp allocation bug Message-ID: It seems there's a bug in temp allocation, basically a temp is allocated but never released (even though it's freed). See http://hg.cython.org/cython-devel/file/4f0327bdebc9/tests/run/ call_crash.pyx - Robert From ilmarw at simula.no Sat Jan 17 13:41:17 2009 From: ilmarw at simula.no (Ilmar Wilbers) Date: Sat, 17 Jan 2009 13:41:17 +0100 Subject: [Cython] Problems accessing ndarray attributes Message-ID: <4971D1ED.4060504@simula.no> Hi list, While trying to wrap an external C library, I have stumbled upon some question. A lot of them were answered by going through the mailing list archive, however a few remain. I have the following function with arguments: def tridag(np.ndarray[DTYPE_t, ndim=1, negative_indices=False] a) I want to do some assertions on the array a: assert a.dtype == np.float32 assert a.flags['C_CONTIGUOUS'] but keep getting an error "Attempting to index non-array type 'int'". Checking the type of a.flags indicates that it is in int. What am I missing? ilmar From stefan_ml at behnel.de Sat Jan 17 13:41:28 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 17 Jan 2009 13:41:28 +0100 Subject: [Cython] Temp allocation bug In-Reply-To: References: Message-ID: <4971D1F8.9020603@behnel.de> Robert Bradshaw wrote: > It seems there's a bug in temp allocation, basically a temp is > allocated but never released (even though it's freed). > > See http://hg.cython.org/cython-devel/file/4f0327bdebc9/tests/run/ > call_crash.pyx The problem is that AttributeNode is not a NewTempExprNode and thus fails to pass on the request for freeing the temp. Changing its baseclass fixes this problem, but it also breaks tons of other test cases. It might just be one problem that kills all of them, but in any case, this needs some investigation. I'll try to look into it this weekend if I find the time. Stefan From dagss at student.matnat.uio.no Sat Jan 17 16:01:05 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Sat, 17 Jan 2009 16:01:05 +0100 (CET) Subject: [Cython] Problems accessing ndarray attributes In-Reply-To: <4971D1ED.4060504@simula.no> References: <4971D1ED.4060504@simula.no> Message-ID: <5ec8c9e0f4016f936a9bdfb8d7b8b56c.squirrel@webmail.uio.no> Hi Ilmar, Ilmar Wilbers wrote: > Hi list, > > While trying to wrap an external C library, I have stumbled upon some > question. A lot of them were answered by going through the mailing list > archive, however a few remain. > > I have the following function with arguments: > def tridag(np.ndarray[DTYPE_t, ndim=1, negative_indices=False] a) > > I want to do some assertions on the array a: > assert a.dtype == np.float32 > assert a.flags['C_CONTIGUOUS'] > but keep getting an error "Attempting to index non-array type 'int'". > Checking the type of a.flags indicates that it is in int. What am I > missing? You have encountered one of the less polished aspects of Cython I'm afraid. The flags field is typed as int to give efficient access for or-style flags checking, which precludes the object behaviour you use. To make a long story short, here's your current options: - assert (a).flags[....] - The assertion for datatype is already made if DTYPE_t is float32_t. Use mode="c" as a flag to ndarray to do the other assertion automatically (on entering the function). This is definitely something that should be more done something about (i.e. this should Just Work), I'll try to file an enhancement ticket for it. Dag Sverre From ilmarw at simula.no Sat Jan 17 16:56:09 2009 From: ilmarw at simula.no (Ilmar Wilbers) Date: Sat, 17 Jan 2009 16:56:09 +0100 Subject: [Cython] Problems accessing ndarray attributes In-Reply-To: <5ec8c9e0f4016f936a9bdfb8d7b8b56c.squirrel@webmail.uio.no> References: <4971D1ED.4060504@simula.no> <5ec8c9e0f4016f936a9bdfb8d7b8b56c.squirrel@webmail.uio.no> Message-ID: <4971FF99.2060702@simula.no> Dag Sverre Seljebotn wrote: > Hi Ilmar, > > Ilmar Wilbers wrote: > >> Hi list, >> >> While trying to wrap an external C library, I have stumbled upon some >> question. A lot of them were answered by going through the mailing list >> archive, however a few remain. >> >> I have the following function with arguments: >> def tridag(np.ndarray[DTYPE_t, ndim=1, negative_indices=False] a) >> >> I want to do some assertions on the array a: >> assert a.dtype == np.float32 >> assert a.flags['C_CONTIGUOUS'] >> but keep getting an error "Attempting to index non-array type 'int'". >> Checking the type of a.flags indicates that it is in int. What am I >> missing? >> > > You have encountered one of the less polished aspects of Cython I'm > afraid. The flags field is typed as int to give efficient access for > or-style flags checking, which precludes the object behaviour you use. > > To make a long story short, here's your current options: > - assert (a).flags[....] > - The assertion for datatype is already made if DTYPE_t is float32_t. Use > mode="c" as a flag to ndarray to do the other assertion automatically (on > entering the function). > > This is definitely something that should be more done something about > (i.e. this should Just Work), I'll try to file an enhancement ticket for > it. > > OK, thank you, Dag Sverre. I realize that the mode is not really important for 1D arrays, but was merely working my way up to matrices. Another question I was having, which I realize probably belongs on the numpy list, is the following: The C function I am calling takes in float arrays, hence I do this: cdef float * p_a = a.data in order to get the correct precision. This requires the assertion of DTYPE_t being float32_t, as simply writing a.dtype=np.float32 doesn't seem to have any effect on the actual data when creating the pointer (but no errors, though). This requires the explicit use of a = zeros(n, numpy.float32) in the calling Python code, which I was hoping to avoid. Alternatively, I can use a loop like this: # Compare arrays, copy values if not same type: old = [a, b, c, r] new = [a_, b_, c_, r_] for i in range(len(new)): if not old[i].dtype == np.float32: new[i][:] = old[i][:] else: new[i] = old[i] which is ugly and requires more memory. My question is, can I use an alternative to cdef float * p_a = a.data in case the arrays have dtype float64, for instance, for simply writing cdef float * p_a = a.data doesn't help. Aka: is it possible to not make assumptions on the array dtype? Using np.float without 32 or 64 works fine, but the data sent to the C function is wrong. I guess float in C has the same precision as float32 in numpy and double in C has the same precision as float64 in numpy? Thanks for the awesome software, by the way. ilmar From dagss at student.matnat.uio.no Sat Jan 17 19:15:54 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Sat, 17 Jan 2009 19:15:54 +0100 Subject: [Cython] Passing numpy arrays to C using Cython In-Reply-To: <4971FF99.2060702@simula.no> References: <4971D1ED.4060504@simula.no> <5ec8c9e0f4016f936a9bdfb8d7b8b56c.squirrel@webmail.uio.no> <4971FF99.2060702@simula.no> Message-ID: <4972205A.6000002@student.matnat.uio.no> First: This question (how to safely pass numpy arrays to C or Fortran code using Cython) actually comes up a lot. I'm wondering about perhaps making it a feature in Cython itself, borrowing a few syntax things from Fortran. (That would clean up a lot of code I'm using myself with Cython these days, so it could get realized in some months). Basically, perhaps one should allow declaring functions like this: cdef extern from "myheader.h": int myfunc(double[0:nrows, 0:ncols] data, int nrows, int ncols) (here data is really "double* data" in the C header) and then call it like this: cdef np.ndarray[double, ndim=2] arr = x myfunc(arr) nrows and ncols would be inferred from the shape of "arr", and Cython would transparently make sure contiguous memory is passed, making a copy if necesarry. (One could leave out the auto-nrows-ncols-bit for now though.) This would be backed by the buffer PEP and not be NumPy-specific. The thing lacking from the syntax above is how to specify whether data only needs to be passed in, or whether it is written to. "const double[...]" perhaps... Thoughts? Then to your question: Ilmar Wilbers wrote: > Dag Sverre Seljebotn wrote: >> Hi Ilmar, >> >> Ilmar Wilbers wrote: >> >>> Hi list, >>> >>> While trying to wrap an external C library, I have stumbled upon some >>> question. A lot of them were answered by going through the mailing list >>> archive, however a few remain. >>> >>> I have the following function with arguments: >>> def tridag(np.ndarray[DTYPE_t, ndim=1, negative_indices=False] a) >>> >>> I want to do some assertions on the array a: >>> assert a.dtype == np.float32 >>> assert a.flags['C_CONTIGUOUS'] >>> but keep getting an error "Attempting to index non-array type 'int'". >>> Checking the type of a.flags indicates that it is in int. What am I >>> missing? >>> >> You have encountered one of the less polished aspects of Cython I'm >> afraid. The flags field is typed as int to give efficient access for >> or-style flags checking, which precludes the object behaviour you use. >> >> To make a long story short, here's your current options: >> - assert (a).flags[....] >> - The assertion for datatype is already made if DTYPE_t is float32_t. Use >> mode="c" as a flag to ndarray to do the other assertion automatically (on >> entering the function). >> >> This is definitely something that should be more done something about >> (i.e. this should Just Work), I'll try to file an enhancement ticket for >> it. >> >> > OK, thank you, Dag Sverre. > > I realize that the mode is not really important for 1D arrays, but was > merely working my way up to matrices. Mode is important for 1D arrays in this setting -- if you have strided memory, you cannot pass it on to C. I.e. x = arange(10) x = x[::2] Here no memory is changed on the second line, only a stride is set skipping every other element. > This requires the explicit use of a = zeros(n, numpy.float32) in the > calling Python code, which I was hoping to avoid. You can also use arr.astype(), which will make a copy without filling with zeros. If that isn't flexible enough (cannot set C or Fortran mode?), one can also use np.empty, which only allocates memory and doesn't fill with zeros either. > My question is, can I use an alternative to > cdef float * p_a = a.data > > in case the arrays have dtype float64, for instance, for simply writing > cdef float * p_a = a.data > > doesn't help. Aka: is it possible to not make assumptions on the array > dtype? Using np.float without 32 or 64 works fine, but the data sent to > the C function is wrong. I guess float in C has the same precision as > float32 in numpy and double in C has the same precision as float64 in numpy? If you simply write "np.ndarray[double] arr", you WILL get a runtime exception if arr is assigned something that does not contain a C double. So this is already handled automatically and you do not need to code the check yourself -- simply do cdef np.ndarray[double] a = x cdef double * p_a = a.data Yes, usually double == float64. -- Dag Sverre From dagss at student.matnat.uio.no Sat Jan 17 19:34:11 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Sat, 17 Jan 2009 19:34:11 +0100 Subject: [Cython] Passing numpy arrays to C using Cython In-Reply-To: <4972205A.6000002@student.matnat.uio.no> References: <4971D1ED.4060504@simula.no> <5ec8c9e0f4016f936a9bdfb8d7b8b56c.squirrel@webmail.uio.no> <4971FF99.2060702@simula.no> <4972205A.6000002@student.matnat.uio.no> Message-ID: <497224A3.80803@student.matnat.uio.no> Dag Sverre Seljebotn wrote: > First: This question (how to safely pass numpy arrays to C or Fortran > code using Cython) actually comes up a lot. I'm wondering about perhaps > making it a feature in Cython itself, borrowing a few syntax things from > Fortran. (That would clean up a lot of code I'm using myself with Cython > these days, so it could get realized in some months). (BTW, next time I sit down with Cython it will definitely be a contribution towards making the trunk stable again. Just so that's clear :-) ) -- Dag Sverre From ondrej at certik.cz Sat Jan 17 20:10:57 2009 From: ondrej at certik.cz (Ondrej Certik) Date: Sat, 17 Jan 2009 11:10:57 -0800 Subject: [Cython] Passing numpy arrays to C using Cython In-Reply-To: <4972205A.6000002@student.matnat.uio.no> References: <4971D1ED.4060504@simula.no> <5ec8c9e0f4016f936a9bdfb8d7b8b56c.squirrel@webmail.uio.no> <4971FF99.2060702@simula.no> <4972205A.6000002@student.matnat.uio.no> Message-ID: <85b5c3130901171110j2390b7abu728ca5151b3eba46@mail.gmail.com> On Sat, Jan 17, 2009 at 10:15 AM, Dag Sverre Seljebotn wrote: > First: This question (how to safely pass numpy arrays to C or Fortran > code using Cython) actually comes up a lot. I'm wondering about perhaps > making it a feature in Cython itself, borrowing a few syntax things from > Fortran. (That would clean up a lot of code I'm using myself with Cython > these days, so it could get realized in some months). > > Basically, perhaps one should allow declaring functions like this: > > cdef extern from "myheader.h": > int myfunc(double[0:nrows, 0:ncols] data, int nrows, int ncols) > > (here data is really "double* data" in the C header) and then call it > like this: > > cdef np.ndarray[double, ndim=2] arr = x > myfunc(arr) > > nrows and ncols would be inferred from the shape of "arr", and Cython > would transparently make sure contiguous memory is passed, making a copy > if necesarry. (One could leave out the auto-nrows-ncols-bit for now though.) > > This would be backed by the buffer PEP and not be NumPy-specific. > > The thing lacking from the syntax above is how to specify whether data > only needs to be passed in, or whether it is written to. "const > double[...]" perhaps... > > Thoughts? I thought that's what the buffer interface (you implemented over the summer) makes easy? Or you want to make it even easier. :) Ondrej From hoytak at cs.ubc.ca Sat Jan 17 20:54:28 2009 From: hoytak at cs.ubc.ca (Hoyt Koepke) Date: Sat, 17 Jan 2009 11:54:28 -0800 Subject: [Cython] Cython-devel from 1620 on doesn't build Message-ID: <4db580fd0901171154g38e3cfb4h120862a742e0fca5@mail.gmail.com> Hello, The latest version of cython-devel, 1627:d9e7d28e9045, doesn't compile for me, failing with a syntax error in one of the generated files. I get the same error from 1620 on. Here's the end of the traceback: creating build/temp.linux-i686-2.5 creating build/temp.linux-i686-2.5/home creating build/temp.linux-i686-2.5/home/hoytak creating build/temp.linux-i686-2.5/home/hoytak/sysroot creating build/temp.linux-i686-2.5/home/hoytak/sysroot/src creating build/temp.linux-i686-2.5/home/hoytak/sysroot/src/cython creating build/temp.linux-i686-2.5/home/hoytak/sysroot/src/cython/Cython creating build/temp.linux-i686-2.5/home/hoytak/sysroot/src/cython/Cython/Plex gcc -pthread -pg -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/hoytak/sysroot/include/python2.5 -c /home/hoytak/sysroot/src/cython/Cython/Plex/Scanners.c -o build/temp.linux-i686-2.5/home/hoytak/sysroot/src/cython/Cython/Plex/Scanners.o /home/hoytak/sysroot/src/cython/Cython/Plex/Scanners.c:7747: error: expected identifier or '(' before numeric constant error: command 'gcc' failed with exit status 1 The culprit is some sort of comment snippet (?) ( "2) */", 4th line in): static void __pyx_init_filenames(void) { __pyx_f = __pyx_filenames; } 2) */ static void __Pyx_RaiseDoubleKeywordsError( const char* func_name, PyObject* kw_name) { PyErr_Format(PyExc_TypeError, #if PY_MAJOR_VERSION >= 3 "%s() got multiple values for keyword argument '%U'", func_name, kw_name); #else "%s() got multiple values for keyword argument '%s'", func_name, PyString_AS_STRING(kw_name)); #endif } I'm sure one of you can fix it in an few seconds... --Hoyt ++++++++++++++++++++++++++++++++++++++++++++++++ + Hoyt Koepke + University of Washington Department of Statistics + http://www.stat.washington.edu/~hoytak/ + hoytak at gmail.com ++++++++++++++++++++++++++++++++++++++++++ From dagss at student.matnat.uio.no Sat Jan 17 21:38:46 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Sat, 17 Jan 2009 21:38:46 +0100 (CET) Subject: [Cython] Passing numpy arrays to C using Cython In-Reply-To: <85b5c3130901171110j2390b7abu728ca5151b3eba46@mail.gmail.com> References: <4971D1ED.4060504@simula.no> <5ec8c9e0f4016f936a9bdfb8d7b8b56c.squirrel@webmail.uio.no> <4971FF99.2060702@simula.no> <4972205A.6000002@student.matnat.uio.no> <85b5c3130901171110j2390b7abu728ca5151b3eba46@mail.gmail.com> Message-ID: Ondrej Certik write: > On Sat, Jan 17, 2009 at 10:15 AM, Dag Sverre Seljebotn > wrote: >> Basically, perhaps one should allow declaring functions like this: >> >> cdef extern from "myheader.h": >> int myfunc(double[0:nrows, 0:ncols] data, int nrows, int ncols) >> >> (here data is really "double* data" in the C header) and then call it >> like this: >> >> cdef np.ndarray[double, ndim=2] arr = x >> myfunc(arr) > > I thought that's what the buffer interface (you implemented over the > summer) makes easy? Or you want to make it even easier. :) The buffer interface in summer was about making writing numerical programs in Cython itself easy; it made sure that arr[i,j,k] compiles to something efficient in Cython. This is about making calling into external C libraries, which expects a pure double* or similar, easier; by allowing automatic coercion from Python buffer objects to pointers. At least at first I think it would only be allowed for declarations of external functions. It would of course build quite heavily on what I did in summer. Dag Sverre From ilmarw at simula.no Sat Jan 17 21:41:17 2009 From: ilmarw at simula.no (Ilmar Wilbers) Date: Sat, 17 Jan 2009 21:41:17 +0100 Subject: [Cython] Passing numpy arrays to C using Cython In-Reply-To: <4972205A.6000002@student.matnat.uio.no> References: <4971D1ED.4060504@simula.no> <5ec8c9e0f4016f936a9bdfb8d7b8b56c.squirrel@webmail.uio.no> <4971FF99.2060702@simula.no> <4972205A.6000002@student.matnat.uio.no> Message-ID: <4972426D.5050504@simula.no> Dag Sverre Seljebotn wrote: > First: This question (how to safely pass numpy arrays to C or Fortran > code using Cython) actually comes up a lot. I'm wondering about perhaps > making it a feature in Cython itself, borrowing a few syntax things from > Fortran. (That would clean up a lot of code I'm using myself with Cython > these days, so it could get realized in some months). > > Basically, perhaps one should allow declaring functions like this: > > cdef extern from "myheader.h": > int myfunc(double[0:nrows, 0:ncols] data, int nrows, int ncols) > > (here data is really "double* data" in the C header) and then call it > like this: > > cdef np.ndarray[double, ndim=2] arr = x > myfunc(arr) > > nrows and ncols would be inferred from the shape of "arr", and Cython > would transparently make sure contiguous memory is passed, making a copy > if necesarry. (One could leave out the auto-nrows-ncols-bit for now though.) > > This would be backed by the buffer PEP and not be NumPy-specific. > > The thing lacking from the syntax above is how to specify whether data > only needs to be passed in, or whether it is written to. "const > double[...]" perhaps... > > Thoughts? > My thoughts are that I would expect such a functionality to be present, yes. > Then to your question: > > >> >> I realize that the mode is not really important for 1D arrays, but was >> merely working my way up to matrices. >> > > Mode is important for 1D arrays in this setting -- if you have strided > memory, you cannot pass it on to C. I.e. > > x = arange(10) > x = x[::2] > > Here no memory is changed on the second line, only a stride is set > skipping every other element. > You are right, I hadn't considered that. >> This requires the explicit use of a = zeros(n, numpy.float32) in the >> calling Python code, which I was hoping to avoid. >> > > You can also use arr.astype(), which will make a copy without filling > with zeros. If that isn't flexible enough (cannot set C or Fortran > mode?), one can also use np.empty, which only allocates memory and > doesn't fill with zeros either. > > The zeros part was just for demonstration, my point was that I had choose the type of array that could be sent to the cython function, it can not be determined at runtime except if I start juggling with several temporary arrays. >> My question is, can I use an alternative to >> cdef float * p_a = a.data >> >> in case the arrays have dtype float64, for instance, for simply writing >> cdef float * p_a = a.data >> >> doesn't help. Aka: is it possible to not make assumptions on the array >> dtype? Using np.float without 32 or 64 works fine, but the data sent to >> the C function is wrong. I guess float in C has the same precision as >> float32 in numpy and double in C has the same precision as float64 in numpy? >> > > If you simply write "np.ndarray[double] arr", you WILL get a runtime > exception if arr is assigned something that does not contain a C double. > So this is already handled automatically and you do not need to code > the check yourself -- simply do > > cdef np.ndarray[double] a = x > cdef double * p_a = a.data > > Yes, usually double == float64. > > You mean that x as argument to the Cython function is given without buffer arguments? I want the users to be able to call the Cython function with any type of numpy array, and to use the data from that array as float* for a C function. This is what I do now: def tridag(a, b): cdef np.ndarray[double, ndim=2] a2 = a cdef np.ndarray[double, ndim=2] b2 = b but while this will work for float, float64 and double, this will not work for float32. Is there any way at all to allow the user calling tridag to sent all types of floting point number array (float32 as well as float64)? It might very well be that I am missing a point here, if so, I'm sorry. Also, my questions might not be frased in the most obvious way, as I am still trying to grasps new concepts. Static typing never was my favorite :-) ilmar From dagss at student.matnat.uio.no Sat Jan 17 22:01:19 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Sat, 17 Jan 2009 22:01:19 +0100 (CET) Subject: [Cython] Passing numpy arrays to C using Cython In-Reply-To: <4972426D.5050504@simula.no> References: <4971D1ED.4060504@simula.no> <5ec8c9e0f4016f936a9bdfb8d7b8b56c.squirrel@webmail.uio.no> <4971FF99.2060702@simula.no> <4972205A.6000002@student.matnat.uio.no> <4972426D.5050504@simula.no> Message-ID: Ilmar Wilbers wrote: > I want the users to be able to call the Cython function with any type of > numpy array, and to use the data from that array as float* for a C > function. This is what I do now: > def tridag(a, b): > cdef np.ndarray[double, ndim=2] a2 = a > cdef np.ndarray[double, ndim=2] b2 = b > > but while this will work for float, float64 and double, this will not > work for float32. Is there any way at all to allow the user calling > tridag to sent all types of floting point number array (float32 as well > as float64)? If the C function expects float*, change the type of those buffers to float. And yes, then you need to manually convert from arrays of other types (i.e. do arr.astype(np.float32)). If the user has supplied data in a different format, then that data needs to be copied somehow anyway, for the compiled C code to be able to understand it. So, roughly: cdef np.ndarray[float, ndim=2, mode="c"] a2 try: a2 = a except: a2 = a.astype(np.float32) call_c_func(a2.data) if a2 is not a: a[...] = a2 # can be skipped if data not modified by function Yes, it's some manual work, but as I noted, Cython really hasn't got features for automatic wrapping of C functions in this respect -- my work in summer was primarily geared towards use of Cython itself for programming. Dag Sverre From robertwb at math.washington.edu Sat Jan 17 22:20:56 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sat, 17 Jan 2009 13:20:56 -0800 Subject: [Cython] Temp allocation bug In-Reply-To: <4971D1F8.9020603@behnel.de> References: <4971D1F8.9020603@behnel.de> Message-ID: <5DF5288F-6BEF-41EB-8307-E588037E2BEA@math.washington.edu> On Jan 17, 2009, at 4:41 AM, Stefan Behnel wrote: > Robert Bradshaw wrote: >> It seems there's a bug in temp allocation, basically a temp is >> allocated but never released (even though it's freed). >> >> See http://hg.cython.org/cython-devel/file/4f0327bdebc9/tests/run/ >> call_crash.pyx > > The problem is that AttributeNode is not a NewTempExprNode and thus > fails > to pass on the request for freeing the temp. Changing its baseclass > fixes > this problem, but it also breaks tons of other test cases. It might > just be > one problem that kills all of them, but in any case, this needs some > investigation. I'll try to look into it this weekend if I find the > time. Is this going to be a general problem with any inside a NewTempExprNode (plain) ExprNode? That could be bad. On that note, is there any reason that temp freeing is done as a separate step now instead of as part of the disposal code? (Someone more familiar with the new code probably has a trivial answer to this.) - Robert From robertwb at math.washington.edu Sat Jan 17 22:33:00 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sat, 17 Jan 2009 13:33:00 -0800 Subject: [Cython] Cython-devel from 1620 on doesn't build In-Reply-To: <4db580fd0901171154g38e3cfb4h120862a742e0fca5@mail.gmail.com> References: <4db580fd0901171154g38e3cfb4h120862a742e0fca5@mail.gmail.com> Message-ID: <2CD01AA0-400E-41A1-BF3C-D4B67FA214DA@math.washington.edu> Sorry, this is some debugging output that apparently gets mangled in this case. I've disabled it for now, as it's easy to turn on and off, try pulling again. - Robert On Jan 17, 2009, at 11:54 AM, Hoyt Koepke wrote: > Hello, > > The latest version of cython-devel, 1627:d9e7d28e9045, doesn't compile > for me, failing with a syntax error in one of the generated files. I > get the same error from 1620 on. Here's the end of the traceback: > > creating build/temp.linux-i686-2.5 > creating build/temp.linux-i686-2.5/home > creating build/temp.linux-i686-2.5/home/hoytak > creating build/temp.linux-i686-2.5/home/hoytak/sysroot > creating build/temp.linux-i686-2.5/home/hoytak/sysroot/src > creating build/temp.linux-i686-2.5/home/hoytak/sysroot/src/cython > creating build/temp.linux-i686-2.5/home/hoytak/sysroot/src/cython/ > Cython > creating build/temp.linux-i686-2.5/home/hoytak/sysroot/src/cython/ > Cython/Plex > gcc -pthread -pg -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall > -Wstrict-prototypes -fPIC -I/home/hoytak/sysroot/include/python2.5 -c > /home/hoytak/sysroot/src/cython/Cython/Plex/Scanners.c -o > build/temp.linux-i686-2.5/home/hoytak/sysroot/src/cython/Cython/ > Plex/Scanners.o > /home/hoytak/sysroot/src/cython/Cython/Plex/Scanners.c:7747: error: > expected identifier or '(' before numeric constant > error: command 'gcc' failed with exit status 1 > > The culprit is some sort of comment snippet (?) ( "2) */", 4th line > in): > > static void __pyx_init_filenames(void) { > __pyx_f = __pyx_filenames; > } > 2) */ > > static void __Pyx_RaiseDoubleKeywordsError( > const char* func_name, > PyObject* kw_name) > { > PyErr_Format(PyExc_TypeError, > #if PY_MAJOR_VERSION >= 3 > "%s() got multiple values for keyword argument '%U'", > func_name, kw_name); > #else > "%s() got multiple values for keyword argument '%s'", > func_name, > PyString_AS_STRING(kw_name)); > #endif > } > > I'm sure one of you can fix it in an few seconds... > > --Hoyt > > > ++++++++++++++++++++++++++++++++++++++++++++++++ > + Hoyt Koepke > + University of Washington Department of Statistics > + http://www.stat.washington.edu/~hoytak/ > + hoytak at gmail.com > ++++++++++++++++++++++++++++++++++++++++++ > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev From dagss at student.matnat.uio.no Sat Jan 17 23:20:26 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Sat, 17 Jan 2009 23:20:26 +0100 (CET) Subject: [Cython] Temp allocation bug In-Reply-To: <5DF5288F-6BEF-41EB-8307-E588037E2BEA@math.washington.edu> References: <4971D1F8.9020603@behnel.de> <5DF5288F-6BEF-41EB-8307-E588037E2BEA@math.washington.edu> Message-ID: <481ae881afa431805c9c3d3959a2ed82.squirrel@webmail.uio.no> Robert wrote: > Is this going to be a general problem with any inside a > NewTempExprNode (plain) ExprNode? That could be bad. On that note, is > there any reason that temp freeing is done as a separate step now > instead of as part of the disposal code? (Someone more familiar with > the new code probably has a trivial answer to this.) Some code is generated like this: /*allocate node*/ if (something) { block A /* deallocate node */ } else { block B /* deallocate node */ } If temp releasing is tied up with disposal, the temp can be reallocated and reused in block B, leading to disaster. Also it would be released twice. Etc. If the C control flow matched the node tree, this wouldn't be a problem, but it doesn't. Pyrex didn't have this problem, but that meant that deallocation (i.e. decrefs) is done later than necesarry. Dag Sverre From robertwb at math.washington.edu Sat Jan 17 23:37:01 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sat, 17 Jan 2009 14:37:01 -0800 Subject: [Cython] Temp allocation bug In-Reply-To: <481ae881afa431805c9c3d3959a2ed82.squirrel@webmail.uio.no> References: <4971D1F8.9020603@behnel.de> <5DF5288F-6BEF-41EB-8307-E588037E2BEA@math.washington.edu> <481ae881afa431805c9c3d3959a2ed82.squirrel@webmail.uio.no> Message-ID: On Jan 17, 2009, at 2:20 PM, Dag Sverre Seljebotn wrote: > Robert wrote: >> Is this going to be a general problem with any inside a >> NewTempExprNode (plain) ExprNode? That could be bad. On that note, is >> there any reason that temp freeing is done as a separate step now >> instead of as part of the disposal code? (Someone more familiar with >> the new code probably has a trivial answer to this.) > > Some code is generated like this: > > /*allocate node*/ > if (something) { > block A > /* deallocate node */ > } else { > block B > /* deallocate node */ > } > > If temp releasing is tied up with disposal, the temp can be > reallocated > and reused in block B, leading to disaster. Also it would be released > twice. Etc. > > If the C control flow matched the node tree, this wouldn't be a > problem, > but it doesn't. > > Pyrex didn't have this problem, but that meant that deallocation (i.e. > decrefs) is done later than necesarry. Ah, I see. Well, I'd rather deallocate too late than deallocate incorrectly. Does this mean that it's all or nothing with the NewTempExprNodes, or is a mix and match supposed to work? - Robert From hoytak at cs.ubc.ca Sat Jan 17 23:54:34 2009 From: hoytak at cs.ubc.ca (Hoyt Koepke) Date: Sat, 17 Jan 2009 14:54:34 -0800 Subject: [Cython] Cython-devel from 1620 on doesn't build In-Reply-To: <2CD01AA0-400E-41A1-BF3C-D4B67FA214DA@math.washington.edu> References: <4db580fd0901171154g38e3cfb4h120862a742e0fca5@mail.gmail.com> <2CD01AA0-400E-41A1-BF3C-D4B67FA214DA@math.washington.edu> Message-ID: <4db580fd0901171454r71b3ed67kfe7d8e590a328917@mail.gmail.com> Yup, it works fine. Thanks!!!! --Hoyt On Sat, Jan 17, 2009 at 1:33 PM, Robert Bradshaw wrote: > Sorry, this is some debugging output that apparently gets mangled in > this case. I've disabled it for now, as it's easy to turn on and off, > try pulling again. > > - Robert > > > On Jan 17, 2009, at 11:54 AM, Hoyt Koepke wrote: > >> Hello, >> >> The latest version of cython-devel, 1627:d9e7d28e9045, doesn't compile >> for me, failing with a syntax error in one of the generated files. I >> get the same error from 1620 on. Here's the end of the traceback: >> >> creating build/temp.linux-i686-2.5 >> creating build/temp.linux-i686-2.5/home >> creating build/temp.linux-i686-2.5/home/hoytak >> creating build/temp.linux-i686-2.5/home/hoytak/sysroot >> creating build/temp.linux-i686-2.5/home/hoytak/sysroot/src >> creating build/temp.linux-i686-2.5/home/hoytak/sysroot/src/cython >> creating build/temp.linux-i686-2.5/home/hoytak/sysroot/src/cython/ >> Cython >> creating build/temp.linux-i686-2.5/home/hoytak/sysroot/src/cython/ >> Cython/Plex >> gcc -pthread -pg -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall >> -Wstrict-prototypes -fPIC -I/home/hoytak/sysroot/include/python2.5 -c >> /home/hoytak/sysroot/src/cython/Cython/Plex/Scanners.c -o >> build/temp.linux-i686-2.5/home/hoytak/sysroot/src/cython/Cython/ >> Plex/Scanners.o >> /home/hoytak/sysroot/src/cython/Cython/Plex/Scanners.c:7747: error: >> expected identifier or '(' before numeric constant >> error: command 'gcc' failed with exit status 1 >> >> The culprit is some sort of comment snippet (?) ( "2) */", 4th line >> in): >> >> static void __pyx_init_filenames(void) { >> __pyx_f = __pyx_filenames; >> } >> 2) */ >> >> static void __Pyx_RaiseDoubleKeywordsError( >> const char* func_name, >> PyObject* kw_name) >> { >> PyErr_Format(PyExc_TypeError, >> #if PY_MAJOR_VERSION >= 3 >> "%s() got multiple values for keyword argument '%U'", >> func_name, kw_name); >> #else >> "%s() got multiple values for keyword argument '%s'", >> func_name, >> PyString_AS_STRING(kw_name)); >> #endif >> } >> >> I'm sure one of you can fix it in an few seconds... >> >> --Hoyt >> >> >> ++++++++++++++++++++++++++++++++++++++++++++++++ >> + Hoyt Koepke >> + University of Washington Department of Statistics >> + http://www.stat.washington.edu/~hoytak/ >> + hoytak at gmail.com >> ++++++++++++++++++++++++++++++++++++++++++ >> _______________________________________________ >> Cython-dev mailing list >> Cython-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/cython-dev > > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > -- ++++++++++++++++++++++++++++++++++++++++++++++++ + Hoyt Koepke + University of Washington Department of Statistics + http://www.stat.washington.edu/~hoytak/ + hoytak at gmail.com ++++++++++++++++++++++++++++++++++++++++++ From dagss at student.matnat.uio.no Sun Jan 18 00:02:00 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: 18 Jan 2009 00:02:00 +0100 Subject: [Cython] Temp allocation bug Message-ID: <3315081734.645558@smtp.netcom.no> It is supposed to work, that's why generate_disposal_code and free_temps are seperate calls. The former can be called multiple times in generate_..._code, the latter must be called exactly once. The temporary variable will be held on to until the end of the C branch, which is optimized away by gcc. You'll find examples somewhere of this, but I can't remember where at the moment. Dag Sverre Seljebotn -----Original Message----- From: Robert Bradshaw Date: Saturday, Jan 17, 2009 11:37 pm Subject: Re: [Cython] Temp allocation bug To: cython-dev at codespeak.netReply-To: cython-dev at codespeak.net On Jan 17, 2009, at 2:20 PM, Dag Sverre Seljebotn wrote: > >> Robert wrote: >> Is this going to be a general problem with any inside a >> NewTempExprNode (plain) ExprNode? That could be bad. On that note, is >> there any reason that temp freeing is done as a separate step now >> instead of as part of the disposal code? (Someone more familiar with >> the new code probably has a trivial answer to this.) > >> Some code is generated like this: > >> /*allocate node*/ > if (something) { > block A > /* deallocate node */ > } else { > block B > /* deallocate node */ > } > >> If temp releasing is tied up with disposal, the temp can be > reallocated > and reused in block B, leading to disaster. Also it would be released > twice. Etc. > >> If the C control flow matched the node tree, this wouldn't be a > problem, > but it doesn't. > >> Pyrex didn't have this problem, but that meant that deallocation (i.e. > decrefs) is done later than necesarry. > >Ah, I see. > >Well, I'd rather deallocate too late than deallocate incorrectly. >Does this mean that it's all or nothing with the NewTempExprNodes, or >is a mix and match supposed to work? > >- Robert > >_______________________________________________ >Cython-dev mailing list >Cython-dev at codespeak.net >http://codespeak.net/mailman/listinfo/cython-dev > From robertwb at math.washington.edu Sun Jan 18 00:11:47 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sat, 17 Jan 2009 15:11:47 -0800 Subject: [Cython] Temp allocation bug In-Reply-To: <3315081734.645558@smtp.netcom.no> References: <3315081734.645558@smtp.netcom.no> Message-ID: <4572FA69-4B26-4481-B3DE-3C078DC6698A@math.washington.edu> On Jan 17, 2009, at 3:02 PM, Dag Sverre Seljebotn wrote: > It is supposed to work, that's why generate_disposal_code and > free_temps are seperate calls. The former can be called multiple > times in generate_..._code, the latter must be called exactly once. > The temporary variable will be held on to until the end of the C > branch, which is optimized away by gcc. > > You'll find examples somewhere of this, but I can't remember where > at the moment. Sure. I'm was just musing about it not meshing well with old vs. new temps, which is the real issue that needs attention. - Robert > Dag Sverre Seljebotn > -----Original Message----- > From: Robert Bradshaw > Date: Saturday, Jan 17, 2009 11:37 pm > Subject: Re: [Cython] Temp allocation bug > To: cython-dev at codespeak.netReply-To: cython-dev at codespeak.net > > On Jan 17, 2009, at 2:20 PM, Dag Sverre Seljebotn wrote: >> >>> Robert wrote: >>> Is this going to be a general problem with any inside a >>> NewTempExprNode (plain) ExprNode? That could be bad. On that >>> note, is >>> there any reason that temp freeing is done as a separate step now >>> instead of as part of the disposal code? (Someone more familiar with >>> the new code probably has a trivial answer to this.) >> >>> Some code is generated like this: >> >>> /*allocate node*/ >> if (something) { >> block A >> /* deallocate node */ >> } else { >> block B >> /* deallocate node */ >> } >> >>> If temp releasing is tied up with disposal, the temp can be >> reallocated >> and reused in block B, leading to disaster. Also it would be released >> twice. Etc. >> >>> If the C control flow matched the node tree, this wouldn't be a >> problem, >> but it doesn't. >> >>> Pyrex didn't have this problem, but that meant that deallocation >>> (i.e. >> decrefs) is done later than necesarry. >> >> Ah, I see. >> >> Well, I'd rather deallocate too late than deallocate incorrectly. >> Does this mean that it's all or nothing with the NewTempExprNodes, or >> is a mix and match supposed to work? >> >> - Robert >> >> _______________________________________________ >> Cython-dev mailing list >> Cython-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/cython-dev >> > > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev From hoytak at cs.ubc.ca Sun Jan 18 05:57:05 2009 From: hoytak at cs.ubc.ca (Hoyt Koepke) Date: Sat, 17 Jan 2009 20:57:05 -0800 Subject: [Cython] self not typed in __richcmp__ Message-ID: <4db580fd0901172057k78bde99aw4bd402aacabdcb9b@mail.gmail.com> Hello, Something seems really wrong here, maybe with my code. It seems that the self argument of __richcmp__ is not typed correctly. Before I file a bug report, is __richcmp__ treated as a static method with the arguments still being each of the tested objects? Here's what's happening. With this code: {{{ cdef class A: cdef int _a def __init__(self, int a): self._a = a def __richcmp__(self, b, int t): if not isinstance(b, A): return False if t == 2: return self._a == (b)._a def test(): a1 = A(1) a2 = A(1) assert a1 == a2 }}} I get the following results: {{{ In [1]: import richcmp_self_typing In [2]: richcmp_self_typing.test() ERROR: An unexpected error occurred while tokenizing input The following traceback may be corrupted or invalid The error message is: ('EOF in multi-line statement', (230, 0)) --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) /home/hoytak/workspace/cython-tests/modloadvars/ in () /home/hoytak/workspace/cython-tests/modloadvars/richcmp_self_typing.so in richcmp_self_typing.test (richcmp_self_typing.c:515)() 16 def test(): 17 a1 = A(1) 18 a2 = A(1) 19 ---> 20 assert a1 == a2 /home/hoytak/workspace/cython-tests/modloadvars/richcmp_self_typing.so in richcmp_self_typing.A.__richcmp__ (richcmp_self_typing.c:404)() 12 13 if t == 2: ---> 14 return self._a == (b)._a 15 16 def test(): AttributeError: 'richcmp_self_typing.A' object has no attribute '_a' > /home/hoytak/workspace/cython-tests/modloadvars/richcmp_self_typing.pyx(14)richcmp_self_typing.A.__richcmp__ (richcmp_self_typing.c:404)() 13 if t == 2: ---> 14 return self._a == (b)._a 15 }}} However, when I explicitly type the self (note last line), it works. {{{ cdef class A: cdef int _a def __init__(self, int a): self._a = a def __richcmp__(self, b, int t): if not isinstance(b, A): return False if t == 2: return (self)._a == (b)._a }}} Thanks! --Hoyt -- ++++++++++++++++++++++++++++++++++++++++++++++++ + Hoyt Koepke + University of Washington Department of Statistics + http://www.stat.washington.edu/~hoytak/ + hoytak at gmail.com ++++++++++++++++++++++++++++++++++++++++++ From stefan_ml at behnel.de Sun Jan 18 08:42:59 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 18 Jan 2009 08:42:59 +0100 Subject: [Cython] Temp allocation bug In-Reply-To: <5DF5288F-6BEF-41EB-8307-E588037E2BEA@math.washington.edu> References: <4971D1F8.9020603@behnel.de> <5DF5288F-6BEF-41EB-8307-E588037E2BEA@math.washington.edu> Message-ID: <4972DD83.5090407@behnel.de> Hi, Robert Bradshaw wrote: > On Jan 17, 2009, at 4:41 AM, Stefan Behnel wrote: > >> Robert Bradshaw wrote: >>> It seems there's a bug in temp allocation, basically a temp is >>> allocated but never released (even though it's freed). >>> >>> See http://hg.cython.org/cython-devel/file/4f0327bdebc9/tests/run/ >>> call_crash.pyx >> The problem is that AttributeNode is not a NewTempExprNode and thus >> fails >> to pass on the request for freeing the temp. Changing its baseclass >> fixes >> this problem, but it also breaks tons of other test cases. It might >> just be >> one problem that kills all of them, but in any case, this needs some >> investigation. I'll try to look into it this weekend if I find the >> time. > > Is this going to be a general problem with any inside a > NewTempExprNode (plain) ExprNode? I can't answer that in general, but the good news is that there are very few ExprNodes left, especially those that can contain subexpressions. Fixing the AttributeNode will get us pretty far. Sadly, I can't do that this weekend. So, if someone else could jump in and (at least) analyse the resulting problems after the switch, that would be great. Stefan From stefan_ml at behnel.de Sun Jan 18 08:50:37 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 18 Jan 2009 08:50:37 +0100 Subject: [Cython] self not typed in __richcmp__ In-Reply-To: <4db580fd0901172057k78bde99aw4bd402aacabdcb9b@mail.gmail.com> References: <4db580fd0901172057k78bde99aw4bd402aacabdcb9b@mail.gmail.com> Message-ID: <4972DF4D.7010803@behnel.de> Hi, Hoyt Koepke wrote: > Something seems really wrong here, maybe with my code. It seems that > the self argument of __richcmp__ is not typed correctly. Before I > file a bug report, is __richcmp__ treated as a static method with the > arguments still being each of the tested objects? Sort of. It's mapped to PyObject_RichCompare(), which takes two arbitrary objects and compares them. http://docs.python.org/c-api/object.html The C-API does this as the first object may not support the required operation, in which case the operands will end up being reversed and passed to the second object. I just noticed that this isn't in the FAQ in our Wiki. It's definitely worth going there. Stefan From dagss at student.matnat.uio.no Sun Jan 18 12:48:10 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Sun, 18 Jan 2009 12:48:10 +0100 Subject: [Cython] Temp allocation bug In-Reply-To: <4972DD83.5090407@behnel.de> References: <4971D1F8.9020603@behnel.de> <5DF5288F-6BEF-41EB-8307-E588037E2BEA@math.washington.edu> <4972DD83.5090407@behnel.de> Message-ID: <497316FA.9080101@student.matnat.uio.no> Stefan Behnel wrote: > Hi, > > Robert Bradshaw wrote: >> On Jan 17, 2009, at 4:41 AM, Stefan Behnel wrote: >> >>> Robert Bradshaw wrote: >>>> It seems there's a bug in temp allocation, basically a temp is >>>> allocated but never released (even though it's freed). >>>> >>>> See http://hg.cython.org/cython-devel/file/4f0327bdebc9/tests/run/ >>>> call_crash.pyx >>> The problem is that AttributeNode is not a NewTempExprNode and thus >>> fails >>> to pass on the request for freeing the temp. Changing its baseclass >>> fixes >>> this problem, but it also breaks tons of other test cases. It might >>> just be >>> one problem that kills all of them, but in any case, this needs some >>> investigation. I'll try to look into it this weekend if I find the >>> time. >> Is this going to be a general problem with any inside a >> NewTempExprNode (plain) ExprNode? > > I can't answer that in general, but the good news is that there are very > few ExprNodes left, especially those that can contain subexpressions. > Fixing the AttributeNode will get us pretty far. Sadly, I can't do that > this weekend. So, if someone else could jump in and (at least) analyse the > resulting problems after the switch, that would be great. My intention with NewTempExpr node was simultaneous use, and I don't see any obstacles in principle. In this case it was a rather embarrassing and simple mistake, fix is up. However, making AttributeNode a NewTempExprNode didn't cause any more failures for me -- were they unreliable segfaults, or were you trying it on lxml? -- Dag Sverre From stefan_ml at behnel.de Sun Jan 18 13:44:48 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 18 Jan 2009 13:44:48 +0100 Subject: [Cython] Temp allocation bug In-Reply-To: <497316FA.9080101@student.matnat.uio.no> References: <4971D1F8.9020603@behnel.de> <5DF5288F-6BEF-41EB-8307-E588037E2BEA@math.washington.edu> <4972DD83.5090407@behnel.de> <497316FA.9080101@student.matnat.uio.no> Message-ID: <49732440.6030208@behnel.de> Hi, Dag Sverre Seljebotn wrote: > In this case it was a rather embarrassing and simple mistake, fix is up. Thanks! > However, making AttributeNode a NewTempExprNode didn't cause any more > failures for me -- were they unreliable segfaults, or were you trying it > on lxml? I ran the test suite and got a lot of gcc parser errors about unexpected '->' tokens and such. Looks ok now with your fix applied, apart from a Cython compiler crash in e_bufaccess. Stefan From erdewit at zonnet.nl Sun Jan 18 15:00:16 2009 From: erdewit at zonnet.nl (Ewald R. de Wit) Date: Sun, 18 Jan 2009 15:00:16 +0100 Subject: [Cython] Distutils and pxd files Message-ID: <200901181500.16640.erdewit@zonnet.nl> I have a problem using the example from http://docs.cython.org/docs/sharing_declarations.html#sharing-extension-types I'm compiling it with the setup.py file from below, with the command python setup.py build_ext --inplace When I run it I get the following error: $ python -c 'import Landscaping' Traceback (most recent call last): File "", line 1, in File "Landscaping.pyx", line 6, in Landscaping (Landscaping.c:347) print "Shrubbery size is %d x %d" % (sh.width, sh.height) AttributeError: 'Shrubbing.Shrubbery' object has no attribute 'height' It looks like the Shrubbing.pxd file, where the hight and width attributes are defined, is ignored. How can I fix this? -- -- Ewald ------------------------------------------------------------------ # setup.py from distutils.core import setup from distutils.extension import Extension from Cython.Distutils import build_ext setup( cmdclass = {'build_ext': build_ext}, ext_modules = [ Extension("Shrubbing", ["Shrubbing.pyx"]), Extension("Landscaping", ["Landscaping.pyx"])] ) ------------------------------------------------------------------ From stefan_ml at behnel.de Sun Jan 18 19:45:16 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 18 Jan 2009 19:45:16 +0100 Subject: [Cython] Temp allocation bug In-Reply-To: <49732440.6030208@behnel.de> References: <4971D1F8.9020603@behnel.de> <5DF5288F-6BEF-41EB-8307-E588037E2BEA@math.washington.edu> <4972DD83.5090407@behnel.de> <497316FA.9080101@student.matnat.uio.no> <49732440.6030208@behnel.de> Message-ID: <497378BC.1010609@behnel.de> Stefan Behnel wrote: > Looks ok now with your fix applied, apart from a > Cython compiler crash in e_bufaccess. ... which was actually due to a bug in the compiler crash reporting code. Now all tests pass again. Stefan From casevh at gmail.com Mon Jan 19 08:22:31 2009 From: casevh at gmail.com (Case Vanhorsen) Date: Sun, 18 Jan 2009 23:22:31 -0800 Subject: [Cython] error with cimport python_version Message-ID: <99e0b9530901182322n701072aagbb888359cd46e77f@mail.gmail.com> I'm probably doing something wrong, but I'm getting an error when I try to compile the following: cimport python_version print hex(python_version.PY_VERSION_HEX) The error message is: using Cython running build_ext cythoning test.pyx to test.c Error converting Pyrex file to C: ------------------------------------------------------------ ... # 0xA (alpha) # 0xB (beta) # 0xC (release candidate) # 0xF (final) char[] PY_VERSION ^ ------------------------------------------------------------ /usr/local/lib/python2.6/site-packages/Cython/Includes/python_version.pxd:31:8: Buffer types only allowed as function local variables Error converting Pyrex file to C: ------------------------------------------------------------ ... # 0xB (beta) # 0xC (release candidate) # 0xF (final) char[] PY_VERSION char[] PY_PATCHLEVEL_REVISION ^ ------------------------------------------------------------ /usr/local/lib/python2.6/site-packages/Cython/Includes/python_version.pxd:32:8: Buffer types only allowed as function local variables If I comment out the last two lines of python_version, it does compile properly and I get the correct hex value. I tried both 0.10.3 and cython-devel. Suggestions? Case From dagss at student.matnat.uio.no Mon Jan 19 14:44:00 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: 19 Jan 2009 14:44:00 +0100 Subject: [Cython] error with cimport python_version Message-ID: <3315221075.1464273@smtp.netcom.no> This is a bug in Cython which we must fix. As a temporary solution you can replace "char[]" with "char*" in that pxd file. Dag Sverre Seljebotn -----Original Message----- From: "Case Vanhorsen" Date: Monday, Jan 19, 2009 8:23 am Subject: [Cython] error with cimport python_version To: cython-dev at codespeak.netReply-To: cython-dev at codespeak.net I'm probably doing something wrong, but I'm getting an error when I >try to compile the following: > >cimport python_version > > >print hex(python_version.PY_VERSION_HEX) > > >The error message is: > >using Cython >running build_ext >cythoning test.pyx to test.c > >Error converting Pyrex file to C: >------------------------------------------------------------ >... > # 0xA (alpha) > # 0xB (beta) > # 0xC (release candidate) > # 0xF (final) > > char[] PY_VERSION > ^ >------------------------------------------------------------ > >/usr/local/lib/python2.6/site-packages/Cython/Includes/python_version.pxd:31:8: Buffer types only allowed as function local variables > >Error converting Pyrex file to C: >------------------------------------------------------------ >... > # 0xB (beta) > # 0xC (release candidate) > # 0xF (final) > > char[] PY_VERSION > char[] PY_PATCHLEVEL_REVISION > ^ >------------------------------------------------------------ > >/usr/local/lib/python2.6/site-packages/Cython/Includes/python_version.pxd:32:8: Buffer types only allowed as function local variables > >If I comment out the last two lines of python_version, it does compile properly and I get the correct hex value. I tried both 0.10.3 and >cython-devel. > >Suggestions? > >Case >_______________________________________________ >Cython-dev mailing list >Cython-dev at codespeak.net >http://codespeak.net/mailman/listinfo/cython-dev > From robertwb at math.washington.edu Mon Jan 19 19:51:42 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Mon, 19 Jan 2009 10:51:42 -0800 Subject: [Cython] Distutils and pxd files In-Reply-To: <200901181500.16640.erdewit@zonnet.nl> References: <200901181500.16640.erdewit@zonnet.nl> Message-ID: <80931750-351B-4599-AF17-95948E31325D@math.washington.edu> On Jan 18, 2009, at 6:00 AM, Ewald R. de Wit wrote: > I have a problem using the example from > http://docs.cython.org/docs/sharing_declarations.html#sharing- > extension-types > > I'm compiling it with the setup.py file from below, with the command > python setup.py build_ext --inplace > > When I run it I get the following error: > > $ python -c 'import Landscaping' > Traceback (most recent call last): > File "", line 1, in > File "Landscaping.pyx", line 6, in Landscaping (Landscaping.c:347) > print "Shrubbery size is %d x %d" % (sh.width, sh.height) > AttributeError: 'Shrubbing.Shrubbery' object has no attribute 'height' > > It looks like the Shrubbing.pxd file, where the hight and width > attributes are > defined, is ignored. How can I fix this? Did you declare sh to be of type Shrubbery? - Robert > From ilmarw at simula.no Mon Jan 19 20:25:42 2009 From: ilmarw at simula.no (Ilmar Wilbers) Date: Mon, 19 Jan 2009 20:25:42 +0100 Subject: [Cython] Distutils and pxd files In-Reply-To: <80931750-351B-4599-AF17-95948E31325D@math.washington.edu> References: <200901181500.16640.erdewit@zonnet.nl> <80931750-351B-4599-AF17-95948E31325D@math.washington.edu> Message-ID: <4974D3B6.40408@simula.no> There is a type on that page, it should be length rather than height, as only width and length are defined in the pdx file. ilmar Robert Bradshaw wrote: > On Jan 18, 2009, at 6:00 AM, Ewald R. de Wit wrote: > > >> I have a problem using the example from >> http://docs.cython.org/docs/sharing_declarations.html#sharing- >> extension-types >> >> I'm compiling it with the setup.py file from below, with the command >> python setup.py build_ext --inplace >> >> When I run it I get the following error: >> >> $ python -c 'import Landscaping' >> Traceback (most recent call last): >> File "", line 1, in >> File "Landscaping.pyx", line 6, in Landscaping (Landscaping.c:347) >> print "Shrubbery size is %d x %d" % (sh.width, sh.height) >> AttributeError: 'Shrubbing.Shrubbery' object has no attribute 'height' >> >> It looks like the Shrubbing.pxd file, where the hight and width >> attributes are >> defined, is ignored. How can I fix this? >> > > Did you declare sh to be of type Shrubbery? > > - Robert > > > > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From ilmarw at simula.no Mon Jan 19 20:26:55 2009 From: ilmarw at simula.no (Ilmar Wilbers) Date: Mon, 19 Jan 2009 20:26:55 +0100 Subject: [Cython] Distutils and pxd files In-Reply-To: <4974D3B6.40408@simula.no> References: <200901181500.16640.erdewit@zonnet.nl> <80931750-351B-4599-AF17-95948E31325D@math.washington.edu> <4974D3B6.40408@simula.no> Message-ID: <4974D3FF.7080805@simula.no> That, of course, should read 'typo', not 'type', ironically. ilmar Ilmar Wilbers wrote: > There is a type on that page, it should be length rather than height, as > only width and length are defined in the pdx file. > > ilmar > > Robert Bradshaw wrote: > >> On Jan 18, 2009, at 6:00 AM, Ewald R. de Wit wrote: >> >> >> >>> I have a problem using the example from >>> http://docs.cython.org/docs/sharing_declarations.html#sharing- >>> extension-types >>> >>> I'm compiling it with the setup.py file from below, with the command >>> python setup.py build_ext --inplace >>> >>> When I run it I get the following error: >>> >>> $ python -c 'import Landscaping' >>> Traceback (most recent call last): >>> File "", line 1, in >>> File "Landscaping.pyx", line 6, in Landscaping (Landscaping.c:347) >>> print "Shrubbery size is %d x %d" % (sh.width, sh.height) >>> AttributeError: 'Shrubbing.Shrubbery' object has no attribute 'height' >>> >>> It looks like the Shrubbing.pxd file, where the hight and width >>> attributes are >>> defined, is ignored. How can I fix this? >>> >>> >> Did you declare sh to be of type Shrubbery? >> >> - Robert >> >> >> >> _______________________________________________ >> Cython-dev mailing list >> Cython-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/cython-dev >> >> > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From erdewit at zonnet.nl Mon Jan 19 21:04:03 2009 From: erdewit at zonnet.nl (Ewald R. de Wit) Date: Mon, 19 Jan 2009 21:04:03 +0100 Subject: [Cython] Distutils and pxd files In-Reply-To: <4974D3B6.40408@simula.no> References: <200901181500.16640.erdewit@zonnet.nl> <80931750-351B-4599-AF17-95948E31325D@math.washington.edu> <4974D3B6.40408@simula.no> Message-ID: <200901192104.03580.erdewit@zonnet.nl> On Monday 19 January 2009 Ilmar Wilbers wrote: > There is a type on that page, it should be length rather than height, as > only width and length are defined in the pdx file. Thanks for spotting this Ilmar, I thought something was wrong with the setup.py file. Cython is new for me and after getting a bit of practice it is just so exciting. joyfully, -- -- Ewald From ilmarw at simula.no Mon Jan 19 21:14:49 2009 From: ilmarw at simula.no (Ilmar Wilbers) Date: Mon, 19 Jan 2009 21:14:49 +0100 Subject: [Cython] Distutils and pxd files In-Reply-To: <200901192104.03580.erdewit@zonnet.nl> References: <200901181500.16640.erdewit@zonnet.nl> <80931750-351B-4599-AF17-95948E31325D@math.washington.edu> <4974D3B6.40408@simula.no> <200901192104.03580.erdewit@zonnet.nl> Message-ID: <4974DF39.1000000@simula.no> That's sort of where I am... Geen probleem! ilmar Ewald R. de Wit wrote: > On Monday 19 January 2009 Ilmar Wilbers wrote: > > >> There is a type on that page, it should be length rather than height, as >> only width and length are defined in the pdx file. >> > > Thanks for spotting this Ilmar, I thought something was wrong with the > setup.py file. Cython is new for me and after getting a bit of practice > it is just so exciting. > > joyfully, > > -- > -- Ewald > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From robertwb at math.washington.edu Mon Jan 19 21:28:16 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Mon, 19 Jan 2009 12:28:16 -0800 Subject: [Cython] Temp allocation bug In-Reply-To: <497316FA.9080101@student.matnat.uio.no> References: <4971D1F8.9020603@behnel.de> <5DF5288F-6BEF-41EB-8307-E588037E2BEA@math.washington.edu> <4972DD83.5090407@behnel.de> <497316FA.9080101@student.matnat.uio.no> Message-ID: <51EDF4C5-DBA0-41BA-B194-6EFE3913747C@math.washington.edu> On Jan 18, 2009, at 3:48 AM, Dag Sverre Seljebotn wrote: > Stefan Behnel wrote: >> Hi, >> >> Robert Bradshaw wrote: >>> On Jan 17, 2009, at 4:41 AM, Stefan Behnel wrote: >>> >>>> Robert Bradshaw wrote: >>>>> It seems there's a bug in temp allocation, basically a temp is >>>>> allocated but never released (even though it's freed). >>>>> >>>>> See http://hg.cython.org/cython-devel/file/4f0327bdebc9/tests/run/ >>>>> call_crash.pyx >>>> The problem is that AttributeNode is not a NewTempExprNode and thus >>>> fails >>>> to pass on the request for freeing the temp. Changing its baseclass >>>> fixes >>>> this problem, but it also breaks tons of other test cases. It might >>>> just be >>>> one problem that kills all of them, but in any case, this needs >>>> some >>>> investigation. I'll try to look into it this weekend if I find the >>>> time. >>> Is this going to be a general problem with any inside a >>> NewTempExprNode (plain) ExprNode? >> >> I can't answer that in general, but the good news is that there >> are very >> few ExprNodes left, especially those that can contain subexpressions. >> Fixing the AttributeNode will get us pretty far. Sadly, I can't do >> that >> this weekend. So, if someone else could jump in and (at least) >> analyse the >> resulting problems after the switch, that would be great. > > My intention with NewTempExpr node was simultaneous use, and I > don't see > any obstacles in principle. > > In this case it was a rather embarrassing and simple mistake, fix > is up. Excellent, thanks! I'm compiling now and will run tests, will get back to you when it's done (usually take a couple of hours). - Robert From ilmarw at simula.no Mon Jan 19 21:44:33 2009 From: ilmarw at simula.no (Ilmar Wilbers) Date: Mon, 19 Jan 2009 21:44:33 +0100 Subject: [Cython] C API declarations Message-ID: <4974E631.10309@simula.no> Hi, I am having some serious problems with the C API declarations, chapter 7.2.2 i n the manual. Using the exact two example files does not work for me, I keep getting: marty.c:3: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?car? marty.c: In function ?main?: marty.c:7: error: ?car? undeclared (first use in this function) marty.c:7: error: (Each undeclared identifier is reported only once marty.c:7: error: for each function it appears in.) Using the other way, with public and including delorean.h instead of delorean_api.h: In file included from marty.c:1: delorean.h:24: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?initdelorean? marty.c:3: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?car? marty.c: In function ?main?: marty.c:7: error: ?car? undeclared (first use in this function) marty.c:7: error: (Each undeclared identifier is reported only once marty.c:7: error: for each function it appears in.) Is there a problem with the header files that I don't understand? I run Cython on delorean.pyx before running gcc -c -I/usr/include/python2.5 -lpython2.5 marty.c Has anyone tried these examples lately? Sincerly, Ilmar From robertwb at math.washington.edu Mon Jan 19 23:55:51 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Mon, 19 Jan 2009 14:55:51 -0800 Subject: [Cython] Temp allocation bug In-Reply-To: <51EDF4C5-DBA0-41BA-B194-6EFE3913747C@math.washington.edu> References: <4971D1F8.9020603@behnel.de> <5DF5288F-6BEF-41EB-8307-E588037E2BEA@math.washington.edu> <4972DD83.5090407@behnel.de> <497316FA.9080101@student.matnat.uio.no> <51EDF4C5-DBA0-41BA-B194-6EFE3913747C@math.washington.edu> Message-ID: <33C9B5C6-CB03-4F68-891A-10AD759C9986@math.washington.edu> On Jan 19, 2009, at 12:28 PM, Robert Bradshaw wrote: > On Jan 18, 2009, at 3:48 AM, Dag Sverre Seljebotn wrote: > >> Stefan Behnel wrote: >>> Hi, >>> >>> Robert Bradshaw wrote: >>>> On Jan 17, 2009, at 4:41 AM, Stefan Behnel wrote: >>>> >>>>> Robert Bradshaw wrote: >>>>>> It seems there's a bug in temp allocation, basically a temp is >>>>>> allocated but never released (even though it's freed). >>>>>> >>>>>> See http://hg.cython.org/cython-devel/file/4f0327bdebc9/tests/ >>>>>> run/ >>>>>> call_crash.pyx >>>>> The problem is that AttributeNode is not a NewTempExprNode and >>>>> thus >>>>> fails >>>>> to pass on the request for freeing the temp. Changing its >>>>> baseclass >>>>> fixes >>>>> this problem, but it also breaks tons of other test cases. It >>>>> might >>>>> just be >>>>> one problem that kills all of them, but in any case, this needs >>>>> some >>>>> investigation. I'll try to look into it this weekend if I find the >>>>> time. >>>> Is this going to be a general problem with any inside a >>>> NewTempExprNode (plain) ExprNode? >>> >>> I can't answer that in general, but the good news is that there >>> are very >>> few ExprNodes left, especially those that can contain >>> subexpressions. >>> Fixing the AttributeNode will get us pretty far. Sadly, I can't do >>> that >>> this weekend. So, if someone else could jump in and (at least) >>> analyse the >>> resulting problems after the switch, that would be great. >> >> My intention with NewTempExpr node was simultaneous use, and I >> don't see >> any obstacles in principle. >> >> In this case it was a rather embarrassing and simple mistake, fix >> is up. > > Excellent, thanks! I'm compiling now and will run tests, will get > back to you when it's done (usually take a couple of hours). That took care of most of them, we're down to 6 failures in the entire suite. - Robert From djagoe at xype.com Tue Jan 20 11:43:20 2009 From: djagoe at xype.com (David Jagoe) Date: Tue, 20 Jan 2009 10:43:20 -0000 Subject: [Cython] Cross-platform extensions Message-ID: <99C7B0B1050E4149A7ECDC9693A19D5A1FDB93@XYPE-SV-01-DEV.xype.local> Hi, I am investigating Cython as a solution for interfacing Python with some existing C code. My application is cross-platform (running on Linux, Windows, SunOs and possibly HPUX). It seems that there are three possible ways of deploying a Cython application cross-platform: 1. Run Cython on all the platforms (and use Cython.DistUtils to manage deployment) 2. Run Cython on Linux only - do the pyx -> c compilation step on Linux - Use standard distutils + gcc on each platform to build the application 3. Cross-compile on Linux Option 3 seems like a nightmare. Option 1 seems ideal assuming that Cython is easy to deploy on all of the above platforms. Option 2 seems to be perfectly possible, but I'm not sure: does the Cython compiler generate C that can be compiled on any platform with python dev headers and the appropriate compiler available? I would really appreciate any feedback on the pros and cons of each of the options from anyone with experience or knowledge in this area. I also apologise for the scant details on the platforms - at this stage I'm not sure exactly which SunOs/HPUX versions I will be using. So I'm really looking for general guidance and best-practice rather than a specific solution. Thanks in advance! Cheers, David -- David Jagoe Xype Ltd Phone: +44 (0) 117 9062146 Email: djagoe at xype.com From stefan_ml at behnel.de Tue Jan 20 13:13:16 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 20 Jan 2009 13:13:16 +0100 (CET) Subject: [Cython] Cross-platform extensions In-Reply-To: <99C7B0B1050E4149A7ECDC9693A19D5A1FDB93@XYPE-SV-01-DEV.xype.local> References: <99C7B0B1050E4149A7ECDC9693A19D5A1FDB93@XYPE-SV-01-DEV.xype.local> Message-ID: <42347.213.61.181.86.1232453596.squirrel@groupware.dvs.informatik.tu-darmstadt.de> David Jagoe wrote: > It seems that there are three > possible ways of deploying a Cython application cross-platform: > > 1. Run Cython on all the platforms (and use Cython.DistUtils to manage > deployment) > > 2. Run Cython on Linux only > - do the pyx -> c compilation step on Linux > - Use standard distutils + gcc on each platform to build the > application > > [...] > I'm not sure: does the Cython compiler > generate C that can be compiled on any platform with python dev headers > and the appropriate compiler available? 2) is actually the preferred way of doing it. Cython does not generate platform-specific code, regardless of the system you run it on. The usual way is to build a source distribution once (including the generated .c files in MANIFEST.in) and then just build that on other systems using distutils but without Cython. Stefan From davidjagoe at gmail.com Tue Jan 20 14:32:54 2009 From: davidjagoe at gmail.com (David Jagoe) Date: Tue, 20 Jan 2009 13:32:54 +0000 Subject: [Cython] Cross-platform extensions In-Reply-To: <42347.213.61.181.86.1232453596.squirrel@groupware.dvs.informatik.tu-darmstadt.de> References: <99C7B0B1050E4149A7ECDC9693A19D5A1FDB93@XYPE-SV-01-DEV.xype.local> <42347.213.61.181.86.1232453596.squirrel@groupware.dvs.informatik.tu-darmstadt.de> Message-ID: 2009/1/20 Stefan Behnel : > David Jagoe wrote: >> It seems that there are three >> possible ways of deploying a Cython application cross-platform: >> >> 1. Run Cython on all the platforms (and use Cython.DistUtils to manage >> deployment) >> >> 2. Run Cython on Linux only >> - do the pyx -> c compilation step on Linux >> - Use standard distutils + gcc on each platform to build the >> application >> >> [...] >> I'm not sure: does the Cython compiler >> generate C that can be compiled on any platform with python dev headers >> and the appropriate compiler available? > > 2) is actually the preferred way of doing it. Cython does not generate > platform-specific code, regardless of the system you run it on. > > The usual way is to build a source distribution once (including the > generated .c files in MANIFEST.in) and then just build that on other > systems using distutils but without Cython. > Excellent, thank you very much for your quick response! Cheers, David From ilmarw at simula.no Tue Jan 20 21:35:35 2009 From: ilmarw at simula.no (Ilmar Wilbers) Date: Tue, 20 Jan 2009 21:35:35 +0100 Subject: [Cython] Sending pointers as arguments between functions Message-ID: <49763597.7010209@simula.no> Hi, Would it be possible to send a pointer from one function to another? For instance if I have the following function: def array2vector(a): cdef float *p = a.data cdef float *v = vector(1, a.shape[0]) for i in range(a.shape[0]): v[i+1] = a[i] return p is it then possible to grab this pointer in another function: cdef float *p = array2vector(a) at all? If so, how, because trying the above gives me the error Casting temporary Python object to non-numeric non-Python type Sincerly, Ilmar From robertwb at math.washington.edu Tue Jan 20 21:39:08 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Tue, 20 Jan 2009 12:39:08 -0800 Subject: [Cython] Sending pointers as arguments between functions In-Reply-To: <49763597.7010209@simula.no> References: <49763597.7010209@simula.no> Message-ID: <78E76DDA-845A-4578-BAE2-CF3144981925@math.washington.edu> On Jan 20, 2009, at 12:35 PM, Ilmar Wilbers wrote: > Hi, > > Would it be possible to send a pointer from one function to > another? For > instance if I have the following function: > def array2vector(a): > cdef float *p = a.data > cdef float *v = vector(1, a.shape[0]) > for i in range(a.shape[0]): > v[i+1] = a[i] > return p > > is it then possible to grab this pointer in another function: > cdef float *p = array2vector(a) > > at all? If so, how, because trying the above gives me the error > Casting temporary Python object to non-numeric non-Python type Yes, you either have to make it a cdef function (where the arguments and return type can be declared to be float*) or wrap it in another object (such as a numpy array). - Robert From hoytak at cs.ubc.ca Wed Jan 21 21:56:42 2009 From: hoytak at cs.ubc.ca (Hoyt Koepke) Date: Wed, 21 Jan 2009 12:56:42 -0800 Subject: [Cython] self not typed in __richcmp__ In-Reply-To: <4972DF4D.7010803@behnel.de> References: <4db580fd0901172057k78bde99aw4bd402aacabdcb9b@mail.gmail.com> <4972DF4D.7010803@behnel.de> Message-ID: <4db580fd0901211256q4a5e0f1eu6e81162e2185e056@mail.gmail.com> Ah, that makes sense. Sorry I took so long to reply. I just got done with one of those hws that pushes everything else aside. I can't edit the wiki currently, but I would be happy to put this in there if someone would give me access. --Hoyt On Sat, Jan 17, 2009 at 11:50 PM, Stefan Behnel wrote: > Hi, > > Hoyt Koepke wrote: >> Something seems really wrong here, maybe with my code. It seems that >> the self argument of __richcmp__ is not typed correctly. Before I >> file a bug report, is __richcmp__ treated as a static method with the >> arguments still being each of the tested objects? > > Sort of. It's mapped to PyObject_RichCompare(), which takes two arbitrary > objects and compares them. > > http://docs.python.org/c-api/object.html > > The C-API does this as the first object may not support the required > operation, in which case the operands will end up being reversed and passed > to the second object. > > I just noticed that this isn't in the FAQ in our Wiki. It's definitely > worth going there. > > Stefan > > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > -- ++++++++++++++++++++++++++++++++++++++++++++++++ + Hoyt Koepke + University of Washington Department of Statistics + http://www.stat.washington.edu/~hoytak/ + hoytak at gmail.com ++++++++++++++++++++++++++++++++++++++++++ From robertwb at math.washington.edu Wed Jan 21 21:59:38 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Wed, 21 Jan 2009 12:59:38 -0800 Subject: [Cython] self not typed in __richcmp__ In-Reply-To: <4db580fd0901211256q4a5e0f1eu6e81162e2185e056@mail.gmail.com> References: <4db580fd0901172057k78bde99aw4bd402aacabdcb9b@mail.gmail.com> <4972DF4D.7010803@behnel.de> <4db580fd0901211256q4a5e0f1eu6e81162e2185e056@mail.gmail.com> Message-ID: On Jan 21, 2009, at 12:56 PM, Hoyt Koepke wrote: > Ah, that makes sense. > > Sorry I took so long to reply. I just got done with one of those hws > that pushes everything else aside. > > I can't edit the wiki currently, but I would be happy to put this in > there if someone would give me access. You should be able to create an account by clicking on "login." > > --Hoyt > > On Sat, Jan 17, 2009 at 11:50 PM, Stefan Behnel > wrote: >> Hi, >> >> Hoyt Koepke wrote: >>> Something seems really wrong here, maybe with my code. It seems >>> that >>> the self argument of __richcmp__ is not typed correctly. Before I >>> file a bug report, is __richcmp__ treated as a static method with >>> the >>> arguments still being each of the tested objects? >> >> Sort of. It's mapped to PyObject_RichCompare(), which takes two >> arbitrary >> objects and compares them. >> >> http://docs.python.org/c-api/object.html >> >> The C-API does this as the first object may not support the required >> operation, in which case the operands will end up being reversed >> and passed >> to the second object. >> >> I just noticed that this isn't in the FAQ in our Wiki. It's >> definitely >> worth going there. >> >> Stefan >> >> _______________________________________________ >> Cython-dev mailing list >> Cython-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/cython-dev >> > > > > -- > > ++++++++++++++++++++++++++++++++++++++++++++++++ > + Hoyt Koepke > + University of Washington Department of Statistics > + http://www.stat.washington.edu/~hoytak/ > + hoytak at gmail.com > ++++++++++++++++++++++++++++++++++++++++++ > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev From hoytak at cs.ubc.ca Wed Jan 21 22:16:59 2009 From: hoytak at cs.ubc.ca (Hoyt Koepke) Date: Wed, 21 Jan 2009 13:16:59 -0800 Subject: [Cython] self not typed in __richcmp__ In-Reply-To: References: <4db580fd0901172057k78bde99aw4bd402aacabdcb9b@mail.gmail.com> <4972DF4D.7010803@behnel.de> <4db580fd0901211256q4a5e0f1eu6e81162e2185e056@mail.gmail.com> Message-ID: <4db580fd0901211316r3523996eod2087baf0c3e038d@mail.gmail.com> > You should be able to create an account by clicking on "login." Ah, of course. It's been one of those days... On a separate note, when I try to do a search for __richcmp__ in the fulltext search page (once logged on), it gives me an error: [Errno 20] Not a directory: 'data/pages/current/revisions/99999999' The error page gives a link to report the bug, which points to a MoinMoinBug page that says it's no longer active. I attached the error output here; Should I file a bug report with the main bug tracker? --Hoyt ++++++++++++++++++++++++++++++++++++++++++++++++ + Hoyt Koepke + University of Washington Department of Statistics + http://www.stat.washington.edu/~hoytak/ + hoytak at gmail.com ++++++++++++++++++++++++++++++++++++++++++ -------------- next part -------------- A non-text attachment was scrubbed... Name: FindPage.html.gz Type: application/x-gzip Size: 3449 bytes Desc: not available Url : http://codespeak.net/pipermail/cython-dev/attachments/20090121/1024b9ad/attachment.bin From robertwb at math.washington.edu Thu Jan 22 06:35:30 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Wed, 21 Jan 2009 21:35:30 -0800 Subject: [Cython] self not typed in __richcmp__ In-Reply-To: <4db580fd0901211316r3523996eod2087baf0c3e038d@mail.gmail.com> References: <4db580fd0901172057k78bde99aw4bd402aacabdcb9b@mail.gmail.com> <4972DF4D.7010803@behnel.de> <4db580fd0901211256q4a5e0f1eu6e81162e2185e056@mail.gmail.com> <4db580fd0901211316r3523996eod2087baf0c3e038d@mail.gmail.com> Message-ID: <9B6921A8-8447-4784-8C24-B945C0EA015A@math.washington.edu> Yeah, looks like a MoinMoin bug to me. On Jan 21, 2009, at 1:16 PM, Hoyt Koepke wrote: >> You should be able to create an account by clicking on "login." > > Ah, of course. It's been one of those days... > > On a separate note, when I try to do a search for __richcmp__ in the > fulltext search page (once logged on), it gives me an error: > > [Errno 20] Not a directory: 'data/pages/current/revisions/99999999' > > The error page gives a link to report the bug, which points to a > MoinMoinBug page that says it's no longer active. > > I attached the error output here; Should I file a bug report with the > main bug tracker? > > --Hoyt > > > ++++++++++++++++++++++++++++++++++++++++++++++++ > + Hoyt Koepke > + University of Washington Department of Statistics > + http://www.stat.washington.edu/~hoytak/ > + hoytak at gmail.com > +++++++++++++++++++++++++++++++++++++++++ > +_______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev From robertwb at math.washington.edu Sun Jan 25 00:13:52 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sat, 24 Jan 2009 15:13:52 -0800 Subject: [Cython] Cython 0.11 beta Message-ID: <36ECFF66-1EC5-4FAB-AEFB-3C42AC697162@math.washington.edu> After much development, it looks like Cython 0.11 is getting stable enough to release. Please try out the beta, which is the current tip of sage-devel. http://cython.org/Cython-0.11.beta.tar.gz - Robert From stefan_ml at behnel.de Sun Jan 25 09:50:56 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 25 Jan 2009 09:50:56 +0100 Subject: [Cython] Cython 0.11 beta In-Reply-To: <36ECFF66-1EC5-4FAB-AEFB-3C42AC697162@math.washington.edu> References: <36ECFF66-1EC5-4FAB-AEFB-3C42AC697162@math.washington.edu> Message-ID: <497C27F0.1000904@behnel.de> Robert Bradshaw wrote: > After much development, it looks like Cython 0.11 is getting stable > enough to release. Please try out the beta, which is the current tip > of sage-devel. Thanks, Robert. I bet you meant "cython-devel", though. ;) I recently went through the remaining bugs for 0.11 and filtered out some that can wait for future releases. We should decide which of the remaining bugs we consider blockers or critical for this release, and then concentrate on fixing those for the release candidate. http://trac.cython.org/cython_trac/report/3 For my part, I think the bugs that generate wrong or dangerous code for built-in types are worth fixing (#166, #158). Also, most of what results in a compiler crash or a runtime crash is probably critical. I cannot really comment on the buffer bugs. Some of them look more like missing features (i.e. minor issues) rather than major bugs. There are also a few bugs that (I guess) you added just as a reminder for yourself. It would still be good if you could add some comments to make them clearer to others (if only to prevent people from submitting duplicates or from not submitting non-duplicates), especially if they depend on finishing up other things before being worth another take in their own right. I assume that's the case for closures? A little status report in that ticket would be very helpful. It's the only major (non 0.11) bug that keeps us from compiling the majority of the Python regression test suite. Finally, it would be good if we had one test case for each open bug. I added a directory 'tests/bugs' to keep them. The naming convention is: descriptivename_Txyz.pyx, 'xyz' being the ticket number in trac. They will not be part of a normal test run, but you can enable them by selecting a bug test explicitly like this (e.g. for ticket 5): runtests.py -T5 Stefan From robertwb at math.washington.edu Sun Jan 25 11:38:49 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sun, 25 Jan 2009 02:38:49 -0800 Subject: [Cython] Cython 0.11 beta In-Reply-To: <497C27F0.1000904@behnel.de> References: <36ECFF66-1EC5-4FAB-AEFB-3C42AC697162@math.washington.edu> <497C27F0.1000904@behnel.de> Message-ID: <42381E3C-02DA-4FE6-A755-657C195C00CA@math.washington.edu> On Jan 25, 2009, at 12:50 AM, Stefan Behnel wrote: > > Robert Bradshaw wrote: >> After much development, it looks like Cython 0.11 is getting stable >> enough to release. Please try out the beta, which is the current tip >> of sage-devel. > > Thanks, Robert. I bet you meant "cython-devel", though. ;) Yep, oops. > I recently went through the remaining bugs for 0.11 and filtered > out some > that can wait for future releases. We should decide which of the > remaining > bugs we consider blockers or critical for this release, and then > concentrate on fixing those for the release candidate. > > http://trac.cython.org/cython_trac/report/3 Thanks. I'll try to go through that on Monday. FYI, I was typically using the "wishlist" milestone as things that would be nice to have, but aren't bugs or deficiencies per se, nor have any urgency. In the meantime, I hope other people try it out so we (and they) don't have any surprises when we release. Sage compiles and passes all 75000+ doctests, I assume the same for lxml? > For my part, I think the bugs that generate wrong or dangerous code > for > built-in types are worth fixing (#166, #158). Also, most of what > results in > a compiler crash or a runtime crash is probably critical. > > I cannot really comment on the buffer bugs. Some of them look more > like > missing features (i.e. minor issues) rather than major bugs. Hopefully Dag can comment on these, but I think your assessment is correct. > There are also a few bugs that (I guess) you added just as a > reminder for > yourself. It would still be good if you could add some comments to > make > them clearer to others (if only to prevent people from submitting > duplicates or from not submitting non-duplicates), especially if they > depend on finishing up other things before being worth another take in > their own right. Yep, I'll do this. > I assume that's the case for closures? A little status > report in that ticket would be very helpful. It's the only major > (non 0.11) > bug that keeps us from compiling the majority of the Python > regression test > suite. No progress has been made on that recently, I have ideas but haven't had a solid block of time to sit down and work on it. > Finally, it would be good if we had one test case for each open bug. I > added a directory 'tests/bugs' to keep them. The naming convention is: > descriptivename_Txyz.pyx, 'xyz' being the ticket number in trac. > They will > not be part of a normal test run, but you can enable them by > selecting a > bug test explicitly like this (e.g. for ticket 5): > > runtests.py -T5 Good idea. - Robert From tolot at solar-empire.de Sun Jan 25 15:45:31 2009 From: tolot at solar-empire.de (Marc Christiansen) Date: Sun, 25 Jan 2009 15:45:31 +0100 Subject: [Cython] Cython 0.11 beta References: <36ECFF66-1EC5-4FAB-AEFB-3C42AC697162@math.washington.edu> Message-ID: Robert Bradshaw wrote: > After much development, it looks like Cython 0.11 is getting stable > enough to release. Please try out the beta, which is the current tip > of sage-devel. > > http://cython.org/Cython-0.11.beta.tar.gz Ok, problems I found so far: 1) With Python 2.5.2, tests/run/charencoding.pyx gives a SyntaxError: File "/tmp/Cython-0.11.beta/BUILD/run/c/charencoding.so", line ?, in charencoding Failed example: expected = b''.join([chr(i) for i in range(0x10,0xFF,0x11)] + [chr(0xFF)]) Exception raised: Traceback (most recent call last): File "/usr/lib64/python2.5/doctest.py", line 1228, in __run compileflags, 1) in test.globs File "", line 1 expected = b''.join([chr(i) for i in range(0x10,0xFF,0x11)] + [chr(0xFF)]) ^ SyntaxError: invalid syntax 2) After a pyximport.install(), "import some_module" does not work if some_module.pyx is in the current dir, because os.path.isdir("") == False (line 179 pyximport/pyximport.py). I suggest something like the following patch: --- a/pyximport/pyximport.py Sun Jan 25 02:54:09 2009 +0100 +++ b/pyximport/pyximport.py Sun Jan 25 03:06:19 2009 +0100 @@ -176,7 +176,13 @@ paths = sys.path join_path = os.path.join is_file = os.path.isfile - for path in filter(os.path.isdir, paths): + for path in paths: + # os.path.isdir('') == False + # os.listdir('') => OSError: [Errno 2] No such file or directory: '' + if path == '': + path = '.' + if not os.path.isdir(path): + continue for filename in os.listdir(path): if filename == pyx_module_name: return PyxLoader(fullname, join_path(path, filename), 3) A minor one. python2.6 -t -c "import pyximport" complains. pyximport/pyxbuild.py: inconsistent use of tabs and spaces in indentation Cython/Compiler/Nodes.py: inconsistent use of tabs and spaces in indentation I haven't tested it with own code yet. Ciao Marc From stefan_ml at behnel.de Sun Jan 25 19:21:48 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 25 Jan 2009 19:21:48 +0100 Subject: [Cython] Cython 0.11 beta In-Reply-To: References: <36ECFF66-1EC5-4FAB-AEFB-3C42AC697162@math.washington.edu> Message-ID: <497CADBC.5060607@behnel.de> Hi, Marc Christiansen wrote: > With Python 2.5.2, tests/run/charencoding.pyx gives a SyntaxError: Thanks, I fixed that (and a couple of other bugs in the test runner). > After a pyximport.install(), "import some_module" does not work if > some_module.pyx is in the current dir, because os.path.isdir("") == > False (line 179 pyximport/pyximport.py). Fixed. The current pyximport implementation is actually pretty new, so it merits some testing. > A minor one. python2.6 -t -c "import pyximport" complains. > > pyximport/pyxbuild.py: inconsistent use of tabs and spaces in indentation > Cython/Compiler/Nodes.py: inconsistent use of tabs and spaces in indentation I noticed that pyximport used a wild mix of tabs and spaces before, guess I forgot to fix pyxbuild as well. Greg Ewing traditionally used tabs in his source code, that's where the left-over in Nodes.py came from. Should be fixed now as well. Stefan From magnus at hetland.org Mon Jan 26 11:19:38 2009 From: magnus at hetland.org (Magnus Lie Hetland) Date: Mon, 26 Jan 2009 11:19:38 +0100 Subject: [Cython] Pi in the sky: Alternate back-end? Message-ID: Hi! I'm not really suggesting this as project in the near future -- just thought I'd hear if it would even be viable. That is, supplementing the current C generating code in Cython with (e.g.) Java generating code (with the Java code to be used in concert with Jython). This way it would be possible to write high-performance code in the Cython language/dialect and use it both with CPython and Jython (and, to some extent, in C and Java). As I haven't looked at the innards of Cython (nor those of Jython, really) I don't know how large a task this would be. Just musing. - M -- Magnus Lie Hetland http://hetland.org From ilmarw at simula.no Mon Jan 26 11:27:31 2009 From: ilmarw at simula.no (Ilmar Wilbers) Date: Mon, 26 Jan 2009 11:27:31 +0100 Subject: [Cython] Pi in the sky: Alternate back-end? In-Reply-To: References: Message-ID: <497D9013.6030803@simula.no> This is really not remotely answering your request, but is anyone using Jython for high-performance code (or Java, for that matter)? I'm sure there is, but could you point me in a direction, i.e. are there any numerical modules for Jython as of today? When looking I couldn't really find anything. ilmar Magnus Lie Hetland wrote: > Hi! > > I'm not really suggesting this as project in the near future -- just > thought I'd hear if it would even be viable. That is, supplementing > the current C generating code in Cython with (e.g.) Java generating > code (with the Java code to be used in concert with Jython). This way > it would be possible to write high-performance code in the Cython > language/dialect and use it both with CPython and Jython (and, to some > extent, in C and Java). > > As I haven't looked at the innards of Cython (nor those of Jython, > really) I don't know how large a task this would be. Just musing. > > - M > From stefan_ml at behnel.de Mon Jan 26 12:38:44 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 26 Jan 2009 12:38:44 +0100 (CET) Subject: [Cython] Pi in the sky: Alternate back-end? In-Reply-To: References: Message-ID: <34784.213.61.181.86.1232969924.squirrel@groupware.dvs.informatik.tu-darmstadt.de> Magnus Lie Hetland wrote: > I'm not really suggesting this as project in the near future -- just > thought I'd hear if it would even be viable. That is, supplementing > the current C generating code in Cython with (e.g.) Java generating > code (with the Java code to be used in concert with Jython). This way > it would be possible to write high-performance code in the Cython > language/dialect and use it both with CPython and Jython (and, to some > extent, in C and Java). Knowing the compiler internals, I don't think this would make much sense. Both the Cython compiler and the Cython language are closely tied to CPython and the C language. It should not be too hard to split the C generation phase from the rest, but most of what happens before that is already targeted towards C code generation and the CPython environment. Plus, you'd loose basically everything from the language that starts with a "cdef". If the PyPy project ever gets to release anything truely usable, that should be much better suited to targeting other environments. That said, there is a project that tries to port CPython extension modules to IronPython, already with some success from what I heard. An adapted Cython backend might be very well suited to simplify the interfacing and porting overhead here. Haven't heard of anything in the Java area, though. Stefan From michael.abshoff at googlemail.com Mon Jan 26 12:48:26 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Mon, 26 Jan 2009 03:48:26 -0800 Subject: [Cython] Pi in the sky: Alternate back-end? In-Reply-To: <34784.213.61.181.86.1232969924.squirrel@groupware.dvs.informatik.tu-darmstadt.de> References: <34784.213.61.181.86.1232969924.squirrel@groupware.dvs.informatik.tu-darmstadt.de> Message-ID: <497DA30A.1010104@gmail.com> Stefan Behnel wrote: > Magnus Lie Hetland wrote: >> I'm not really suggesting this as project in the near future -- just >> thought I'd hear if it would even be viable. That is, supplementing >> the current C generating code in Cython with (e.g.) Java generating >> code (with the Java code to be used in concert with Jython). This way >> it would be possible to write high-performance code in the Cython >> language/dialect and use it both with CPython and Jython (and, to some >> extent, in C and Java). > > Knowing the compiler internals, I don't think this would make much sense. > Both the Cython compiler and the Cython language are closely tied to > CPython and the C language. It should not be too hard to split the C > generation phase from the rest, but most of what happens before that is > already targeted towards C code generation and the CPython environment. > Plus, you'd loose basically everything from the language that starts with > a "cdef". Well, using JNI I could imagine this could be made to work, but one has to assume this will be rather painful since the C API in Python is a lot closer to the metal than JNI in Java AFAIK. The again: Jython seems to be the most useful inside browsers and using non-standard extensions modules kind of defeats that use case. I do not know if Jython is faster than CPython, but why would you want to use it other than the embedding advantage? I am curious here since I never had more than superficial contact with Jython. > If the PyPy project ever gets to release anything truely usable, that > should be much better suited to targeting other environments. *ever* :) > That said, there is a project that tries to port CPython extension modules > to IronPython, already with some success from what I heard. An adapted > Cython backend might be very well suited to simplify the interfacing and > porting overhead here. Haven't heard of anything in the Java area, though. Do you have any pointers here? > Stefan Cheers, Michael > _____________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From stefan_ml at behnel.de Mon Jan 26 13:45:39 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 26 Jan 2009 13:45:39 +0100 (CET) Subject: [Cython] Pi in the sky: Alternate back-end? In-Reply-To: <497DA30A.1010104@gmail.com> References: <34784.213.61.181.86.1232969924.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <497DA30A.1010104@gmail.com> Message-ID: <52784.213.61.181.86.1232973939.squirrel@groupware.dvs.informatik.tu-darmstadt.de> Michael Abshoff wrote: > Stefan Behnel wrote: >> That said, there is a project that tries to port CPython extension >> modules >> to IronPython, already with some success from what I heard. An adapted >> Cython backend might be very well suited to simplify the interfacing and >> porting overhead here. Haven't heard of anything in the Java area, >> though. > > Do you have any pointers here? Had to look it up myself, but here it is: http://code.google.com/p/ironclad/ They seem to have bz2 working (their first simple target) and major parts of NumPy. I never looked at it in detail, but I assume that most of their work deals with enumlating the ref-counting stuff and finding more or less simple replacements for the C-API functions. Stefan From magnus at hetland.org Mon Jan 26 14:04:56 2009 From: magnus at hetland.org (Magnus Lie Hetland) Date: Mon, 26 Jan 2009 14:04:56 +0100 Subject: [Cython] Pi in the sky: Alternate back-end? In-Reply-To: <497D9013.6030803@simula.no> References: <497D9013.6030803@simula.no> Message-ID: On Jan 26, 2009, at 11:27, Ilmar Wilbers wrote: > This is really not remotely answering your request, but is anyone > using > Jython for high-performance code (or Java, for that matter)? Jython -- I don't know. I think Jython is quite a bit slower than CPython (although that may have changed). As for plain Java, sure. I don't know about it being used for numerical stuff (though I'm sure it is), but I know of several people (in research and industry) using it for large-scale search engine work, for example. As far as I can tell, it's quite on par with C for that sort of stuff -- it's really not true that Java is slow anymore. > I'm sure there is, but could you point me in a direction, i.e. are > there any > numerical modules for Jython as of today? When looking I couldn't > really > find anything. Hm. I remember a Jython-compatible numarray implementation out there at one point, but couldn't find it right now. For me, it's more a matter of implementing various structures and algorithms myself (which is what I'm using Cython for right now). If I then want to reuse those structures in a project based on Java, for example, a solution like then one I suggested would be useful. -- Magnus Lie Hetland http://hetland.org From magnus at hetland.org Mon Jan 26 14:09:09 2009 From: magnus at hetland.org (Magnus Lie Hetland) Date: Mon, 26 Jan 2009 14:09:09 +0100 Subject: [Cython] Pi in the sky: Alternate back-end? In-Reply-To: <497DA30A.1010104@gmail.com> References: <34784.213.61.181.86.1232969924.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <497DA30A.1010104@gmail.com> Message-ID: <01875A8D-1FDF-434D-B3EC-98D2D414F413@hetland.org> On Jan 26, 2009, at 12:48, Michael Abshoff wrote: > I do not know if Jython is faster than CPython, but why would you > want to use it other than the embedding advantage? I am curious here > since I never had more than superficial contact with Jython. Basically, it would be to use code that's already written in Java. The Jython part wouldn't be expected to be fast :-) As for what Stefan said aboud losing cdef-stuff -- I'll take his word for the coupling between Cython and C, but the syntax would be just as useful for type declarations for Java (but with a completely different translator then, I guess :-) -- Magnus Lie Hetland http://hetland.org From dagss at student.matnat.uio.no Mon Jan 26 14:41:00 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: 26 Jan 2009 14:41:00 +0100 Subject: [Cython] Pi in the sky: Alternate back-end? Message-ID: <3315825703.354493@smtp.netcom.no> My take: I would think this plan would be both more benefitial and might be easier to implement technically: - Identify what features in Cython are needed in Javaland. This is likely mostly typing/early binding of interfaces. - Agree with Cython on a syntax for this. There are a couple of PEPs opening the road towards this. - Implement that spec in both Cython (easy) and Jython (don't know). The thing is, Java is much more dynamic than C (introspection, bytecode compiled scripting languages abound and a compiler is often integrated in the runtime etc.) and if Jython compiles to Java bytecode anyway (which I'm not sure of, but at least it could) then really the part missing is the early binding for full Java speed. A Cython for Jython would likely duplicate much of Jython, it might be better to start from Jython itself. Alternatively, if Jython already did this, look at porting the syntax they used to Cython. Dag Sverre Seljebotn -----Original Message----- From: Magnus Lie Hetland Date: Monday, Jan 26, 2009 2:09 pm Subject: Re: [Cython] Pi in the sky: Alternate back-end? To: cython-dev at codespeak.netReply-To: cython-dev at codespeak.net On Jan 26, 2009, at 12:48, Michael Abshoff wrote: > >> I do not know if Jython is faster than CPython, but why would you > want to use it other than the embedding advantage? I am curious here > since I never had more than superficial contact with Jython. > >Basically, it would be to use code that's already written in Java. The >Jython part wouldn't be expected to be fast :-) > >As for what Stefan said aboud losing cdef-stuff -- I'll take his word >for the coupling between Cython and C, but the syntax would be just as >useful for type declarations for Java (but with a completely different >translator then, I guess :-) > >-- >Magnus Lie Hetland >http://hetland.org > > >_______________________________________________ >Cython-dev mailing list >Cython-dev at codespeak.net >http://codespeak.net/mailman/listinfo/cython-dev > From dagss at student.matnat.uio.no Mon Jan 26 18:31:36 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Mon, 26 Jan 2009 18:31:36 +0100 Subject: [Cython] Cython 0.11 beta In-Reply-To: <497C27F0.1000904@behnel.de> References: <36ECFF66-1EC5-4FAB-AEFB-3C42AC697162@math.washington.edu> <497C27F0.1000904@behnel.de> Message-ID: <497DF378.6070806@student.matnat.uio.no> Stefan Behnel wrote: > Robert Bradshaw wrote: > >> After much development, it looks like Cython 0.11 is getting stable >> enough to release. Please try out the beta, which is the current tip >> of sage-devel. >> > > Thanks, Robert. I bet you meant "cython-devel", though. ;) > > I recently went through the remaining bugs for 0.11 and filtered out some > that can wait for future releases. We should decide which of the remaining > bugs we consider blockers or critical for this release, and then > concentrate on fixing those for the release candidate. > > http://trac.cython.org/cython_trac/report/3 > > For my part, I think the bugs that generate wrong or dangerous code for > built-in types are worth fixing (#166, #158). Also, most of what results in > a compiler crash or a runtime crash is probably critical. > Thanks. I've fixed one low hanging one and downgraded another buffer bug to minor. I couldn't reproduce #43 myself. I think #143 could be partially fixed and made a feature request instead by at least disallowing the current way of typing local vars in pxd files, without necesarrily introducing a new way of doing it. I'd prefer working on #151 next, if/when I can find the time for it (and will assign it to me if I start on it). Sage passes doctests: Does that include detecting any leaks? The refnanny is not yet working, so I'd be wary of releasing until that is done unless we know that Sage doesn't leak more when Cython is upgraded. Dag Sverre From dagss at student.matnat.uio.no Mon Jan 26 18:33:03 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Mon, 26 Jan 2009 18:33:03 +0100 Subject: [Cython] Cython 0.11 beta In-Reply-To: <497DF378.6070806@student.matnat.uio.no> References: <36ECFF66-1EC5-4FAB-AEFB-3C42AC697162@math.washington.edu> <497C27F0.1000904@behnel.de> <497DF378.6070806@student.matnat.uio.no> Message-ID: <497DF3CF.7020701@student.matnat.uio.no> Dag Sverre Seljebotn wrote: > Stefan Behnel wrote: > >> Robert Bradshaw wrote: >> >> >>> After much development, it looks like Cython 0.11 is getting stable >>> enough to release. Please try out the beta, which is the current tip >>> of sage-devel. >>> >>> >> Thanks, Robert. I bet you meant "cython-devel", though. ;) >> >> I recently went through the remaining bugs for 0.11 and filtered out some >> that can wait for future releases. We should decide which of the remaining >> bugs we consider blockers or critical for this release, and then >> concentrate on fixing those for the release candidate. >> >> http://trac.cython.org/cython_trac/report/3 >> >> For my part, I think the bugs that generate wrong or dangerous code for >> built-in types are worth fixing (#166, #158). Also, most of what results in >> a compiler crash or a runtime crash is probably critical. >> >> > Thanks. I've fixed one low hanging one and downgraded another buffer bug > to minor. > I'd assign it to milestone 0.11.1 or 0.12 if they were available...? In general I think one more milestone than the next release should be available in trac. Dag Sverre From robertwb at math.washington.edu Mon Jan 26 21:20:44 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Mon, 26 Jan 2009 12:20:44 -0800 Subject: [Cython] Cython 0.11 beta In-Reply-To: <497DF3CF.7020701@student.matnat.uio.no> References: <36ECFF66-1EC5-4FAB-AEFB-3C42AC697162@math.washington.edu> <497C27F0.1000904@behnel.de> <497DF378.6070806@student.matnat.uio.no> <497DF3CF.7020701@student.matnat.uio.no> Message-ID: <85CC3E68-08F2-4A27-ABE6-ED662D1DC9F6@math.washington.edu> On Jan 26, 2009, at 9:33 AM, Dag Sverre Seljebotn wrote: > Dag Sverre Seljebotn wrote: >> Stefan Behnel wrote: >> >>> Robert Bradshaw wrote: >>> >>> >>>> After much development, it looks like Cython 0.11 is getting stable >>>> enough to release. Please try out the beta, which is the current >>>> tip >>>> of sage-devel. >>>> >>>> >>> Thanks, Robert. I bet you meant "cython-devel", though. ;) >>> >>> I recently went through the remaining bugs for 0.11 and filtered >>> out some >>> that can wait for future releases. We should decide which of the >>> remaining >>> bugs we consider blockers or critical for this release, and then >>> concentrate on fixing those for the release candidate. >>> >>> http://trac.cython.org/cython_trac/report/3 >>> >>> For my part, I think the bugs that generate wrong or dangerous >>> code for >>> built-in types are worth fixing (#166, #158). Also, most of what >>> results in >>> a compiler crash or a runtime crash is probably critical. >>> >>> >> Thanks. I've fixed one low hanging one and downgraded another >> buffer bug >> to minor. >> > I'd assign it to milestone 0.11.1 or 0.12 if they were > available...? In > general I think one more milestone than the next release should be > available in trac. Yep. Done. - Robert From robertwb at math.washington.edu Mon Jan 26 21:33:46 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Mon, 26 Jan 2009 12:33:46 -0800 Subject: [Cython] Pi in the sky: Alternate back-end? In-Reply-To: <01875A8D-1FDF-434D-B3EC-98D2D414F413@hetland.org> References: <34784.213.61.181.86.1232969924.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <497DA30A.1010104@gmail.com> <01875A8D-1FDF-434D-B3EC-98D2D414F413@hetland.org> Message-ID: <9197EBCF-1E8A-45D7-8DBE-FFAEE52C3BA3@math.washington.edu> On Jan 26, 2009, at 5:09 AM, Magnus Lie Hetland wrote: > On Jan 26, 2009, at 12:48, Michael Abshoff wrote: > >> I do not know if Jython is faster than CPython, but why would you >> want to use it other than the embedding advantage? I am curious here >> since I never had more than superficial contact with Jython. > > Basically, it would be to use code that's already written in Java. The > Jython part wouldn't be expected to be fast :-) I think you can already do this with Jython, though it's not compiled ahead of time. > As for what Stefan said aboud losing cdef-stuff -- I'll take his word > for the coupling between Cython and C, but the syntax would be just as > useful for type declarations for Java (but with a completely different > translator then, I guess :-) You wouldn't lose all the cdef stuff, it could still be useful to type things like floats and ints, and perhaps specialized java types. However, you wouldn't have pointers, and a huge benefit from Cython is being able to link to existing C/C++ libraries. There is a lot of C/CPython specific stuff in Cython as well, stuff like refcounting and virtual function pointer tables are not needed in a JVM context. In short, I think it could be done, but may be a lot easier to adapt Jython to do what you want (if it doesn't already) than Cython. - Robert From magnus at hetland.org Mon Jan 26 21:43:43 2009 From: magnus at hetland.org (Magnus Lie Hetland) Date: Mon, 26 Jan 2009 21:43:43 +0100 Subject: [Cython] Pi in the sky: Alternate back-end? In-Reply-To: <9197EBCF-1E8A-45D7-8DBE-FFAEE52C3BA3@math.washington.edu> References: <34784.213.61.181.86.1232969924.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <497DA30A.1010104@gmail.com> <01875A8D-1FDF-434D-B3EC-98D2D414F413@hetland.org> <9197EBCF-1E8A-45D7-8DBE-FFAEE52C3BA3@math.washington.edu> Message-ID: <5C1DEF2B-3188-43B7-8879-1C1DC58E1777@hetland.org> On Jan 26, 2009, at 21:33 , Robert Bradshaw wrote: >> Basically, it would be to use code that's already written in Java. >> The >> Jython part wouldn't be expected to be fast :-) > > I think you can already do this with Jython, though it's not compiled > ahead of time. Not sure what you mean here. Jython isn't compiled ahead of time, true. If you want to use Java code in Jython, the Java code has to be pre-compiled. But that works well. What I was thinking was basically to avoid writing the "high-speed"- stuff in Java (and thereby avoiding lock-in to the JVM), and that perhas it could be generated from Cython files (so that the same source code could be used with C-based systems as well). [snip] > You wouldn't lose all the cdef stuff, it could still be useful to > type things like floats and ints, and perhaps specialized java types. > However, you wouldn't have pointers, Indeed. > and a huge benefit from Cython is being able to link to existing C/C+ > + libraries. Absolutely. [snip] > In short, I think it could be done, but may be a lot easier to adapt > Jython to do what you want (if it doesn't already) than Cython. OK :) -- Magnus Lie Hetland http://hetland.org From robertwb at math.washington.edu Mon Jan 26 22:01:12 2009 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Mon, 26 Jan 2009 13:01:12 -0800 Subject: [Cython] Pi in the sky: Alternate back-end? In-Reply-To: <5C1DEF2B-3188-43B7-8879-1C1DC58E1777@hetland.org> References: <34784.213.61.181.86.1232969924.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <497DA30A.1010104@gmail.com> <01875A8D-1FDF-434D-B3EC-98D2D414F413@hetland.org> <9197EBCF-1E8A-45D7-8DBE-FFAEE52C3BA3@math.washington.edu> <5C1DEF2B-3188-43B7-8879-1C1DC58E1777@hetland.org> Message-ID: <3DAFF182-C80D-4593-A18B-62264DFC79B3@math.washington.edu> On Jan 26, 2009, at 12:43 PM, Magnus Lie Hetland wrote: > On Jan 26, 2009, at 21:33 , Robert Bradshaw wrote: > >>> Basically, it would be to use code that's already written in Java. >>> The >>> Jython part wouldn't be expected to be fast :-) >> >> I think you can already do this with Jython, though it's not compiled >> ahead of time. > > Not sure what you mean here. Jython isn't compiled ahead of time, > true. If you want to use Java code in Jython, the Java code has to be > pre-compiled. But that works well. > > What I was thinking was basically to avoid writing the "high-speed"- > stuff in Java (and thereby avoiding lock-in to the JVM), and that > perhas it could be generated from Cython files (so that the same > source code could be used with C-based systems as well). That's a worthwhile goal, and could possibly be done by adapting the Jython interpreter to understand the Cython type annotations. From what I understand, Jython compiles Python code on the fly to bytecodes, rather than interpreting it like CPython. (Well, that's being a bit vague, as the JVM then interprets the bytecodes, and CPython has a notion of bytecodes too, but the JVM bytecodes are "closer to the metal" and its virtual machine has a lot more optimizations like JIT compilation.) - Robert From michael.abshoff at googlemail.com Tue Jan 27 16:00:00 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Tue, 27 Jan 2009 07:00:00 -0800 Subject: [Cython] Pi in the sky: Alternate back-end? In-Reply-To: <52784.213.61.181.86.1232973939.squirrel@groupware.dvs.informatik.tu-darmstadt.de> References: <34784.213.61.181.86.1232969924.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <497DA30A.1010104@gmail.com> <52784.213.61.181.86.1232973939.squirrel@groupware.dvs.informatik.tu-darmstadt.de> Message-ID: <497F2170.6020207@gmail.com> Stefan Behnel wrote: > Michael Abshoff wrote: >> Stefan Behnel wrote: >>> That said, there is a project that tries to port CPython extension >>> modules >>> to IronPython, already with some success from what I heard. An adapted >>> Cython backend might be very well suited to simplify the interfacing and >>> porting overhead here. Haven't heard of anything in the Java area, >>> though. >> Do you have any pointers here? > > Had to look it up myself, but here it is: > > http://code.google.com/p/ironclad/ Thanks. > They seem to have bz2 working (their first simple target) and major parts > of NumPy. I never looked at it in detail, but I assume that most of their > work deals with enumlating the ref-counting stuff and finding more or less > simple replacements for the C-API functions. I had heard about the ongoing numpy port, but I did not make the connection to this. In the end it cannot hurt to support IronPython even though it looks like they have some way to go. > Stefan Cheers, Michael > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From michael.abshoff at googlemail.com Tue Jan 27 16:01:38 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Tue, 27 Jan 2009 07:01:38 -0800 Subject: [Cython] Cython 0.11 beta In-Reply-To: <497DF378.6070806@student.matnat.uio.no> References: <36ECFF66-1EC5-4FAB-AEFB-3C42AC697162@math.washington.edu> <497C27F0.1000904@behnel.de> <497DF378.6070806@student.matnat.uio.no> Message-ID: <497F21D2.9080002@gmail.com> Dag Sverre Seljebotn wrote: > Stefan Behnel wrote: >> Robert Bradshaw wrote: > Sage passes doctests: Does that include detecting any leaks? The > refnanny is not yet working, so I'd be wary of releasing until that is > done unless we know that Sage doesn't leak more when Cython is upgraded. I don't think Robert did any testing, but I can do some valgrinding with the upcoming 3.3.alpha3 with the old and the new 0.11 snapshot to see if anything pops up. It will be a day or two until I have results. > Dag Sverre Cheers, Michael > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From muogoro at gmail.com Wed Jan 28 12:36:54 2009 From: muogoro at gmail.com (Daniele Pianu) Date: Wed, 28 Jan 2009 12:36:54 +0100 Subject: [Cython] problem with public C API for Python-implemented library Message-ID: <66a8c45d0901280336y2d9d7d1ep86dd599084abb140@mail.gmail.com> Hi to all!!!! In my recent project I'm working to allow a C user to work with a Python Module which presents a C API but it's mainly implemented writing Python code. The data structure passed between C API calls might be a C structure containing Python objects as fields. Here an example: ctypedef public struct SpamManager: PySpam wrapper cdef public api SpamManager* SpamManager_new(): # Alloc and return a SpamManager structure .... cdef public api int SpamManager_init( SpamManager* self, Spam* instance ): ..... # Init the wrapper with the given instance self.wrapper = PySpam( instance ) .... cdef public api int SpamManager_doSomething( SpamManager* self ): # Do something with my nice wrapper :) ...... cdef public api void SpamManager_clear( SpamManager* self ): # Clear structure .... cdef public api void SpamManager_delete( SpamManager* self ): # Delete structure .... Spam is a C structure and PySpam is the correspondent Cython extension type (or the Python wrapper, as you prefer :) ). The wrapper is initialized passing a C instance to the constructor (casting the instance to an unsigned long to obtain the pointer address). The SpamManager module works with the PySpam wrapper but allows a C user to interact with the Python wrapper using the SpamManager structure and function calls. The SpamManager structure must contain a PySpam field, so we can avoid the overhead of the wrapper initialization creating it only in the initialization phase (that is, in the SpamManager_init function call). The problem is that, in Cython, ctypedef-ed structures cannot have an object field (so my SpamManager definition raises a compiler error). I've partially rounded on the error declaring the wrapper attribute in the SpamManager structure as a PyObject*, but this trick force me to continually cast the wrapper attribute to every time I need to call a PySpam method using the wrapper instance. There's also the problem that I've to allocate the wrapper like this: .... tmp = PySpam( instance ) self.wrapper = tmp .... tmp is a temporary Python object, so it goes out of scope when the function terminates and, consequently, I've to manage wrapper's reference count. Is there a way to declare a C structure which has Python types as fields? Hints about different approaches to library implementation are also welcome. :) Thanks for you help (and excuse me for my poor english). Daniele From magnus at hetland.org Wed Jan 28 14:53:31 2009 From: magnus at hetland.org (Magnus Lie Hetland) Date: Wed, 28 Jan 2009 14:53:31 +0100 Subject: [Cython] For loop bug? Message-ID: <05DFE43E-A13E-4771-852F-F0A5A3728833@hetland.org> This is definitely a corner case -- and it may be an intentional incompatibility with Python (and, for all I know, I may have missed some documentation on it somewhere), but ... here goes: After a for-loop in Python, the index variable will retain its last value. E.g.: for i in range(10): pass print i # Prints out 9 In Cython, however, standard C-semantics are used for the counting for- loops, so the index variable is incremented one step beyond the value it had in the last iteration. I just wrote some code where this was an issue. It's easy to fix, of course (just use a statement like "i -= 1" or something) -- but it does mean an incompatibility between Python and Cython... I realise that adding an "i -= 1" statement in general, as a blanket solution, would be rather wasteful (as most code doesn't rely on this property). Perhaps it would be possible to do some analysis on whether the index variable is used without modification later? (I realise that this is intractable in the general case, but perhaps there could be some well-defined rules?) - M -- Magnus Lie Hetland http://hetland.org From stefan_ml at behnel.de Wed Jan 28 16:19:56 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 28 Jan 2009 16:19:56 +0100 (CET) Subject: [Cython] For loop bug? In-Reply-To: <05DFE43E-A13E-4771-852F-F0A5A3728833@hetland.org> References: <05DFE43E-A13E-4771-852F-F0A5A3728833@hetland.org> Message-ID: <37772.213.61.181.86.1233155996.squirrel@groupware.dvs.informatik.tu-darmstadt.de> Magnus Lie Hetland wrote: > This is definitely a corner case -- and it may be an intentional > incompatibility with Python (and, for all I know, I may have missed > some documentation on it somewhere), but ... here goes: > > After a for-loop in Python, the index variable will retain its last > value. E.g.: > > for i in range(10): pass > print i > # Prints out 9 > > In Cython, however, standard C-semantics are used for the counting for- > loops, so the index variable is incremented one step beyond the value > it had in the last iteration. > > I just wrote some code where this was an issue. It's easy to fix, of > course (just use a statement like "i -= 1" or something) -- but it > does mean an incompatibility between Python and Cython... Definitely a bug, please file a bug report. (if you don't have a trac account, please send a htpasswd line to Robert). > I realise that adding an "i -= 1" statement in general, as a blanket > solution, would be rather wasteful. Not at all. We only do the range() optimisation for C integers, and the C compiler will simply strip it out if it's not used. Stefan From magnus at hetland.org Wed Jan 28 16:28:15 2009 From: magnus at hetland.org (Magnus Lie Hetland) Date: Wed, 28 Jan 2009 16:28:15 +0100 Subject: [Cython] For loop bug? In-Reply-To: <37772.213.61.181.86.1233155996.squirrel@groupware.dvs.informatik.tu-darmstadt.de> References: <05DFE43E-A13E-4771-852F-F0A5A3728833@hetland.org> <37772.213.61.181.86.1233155996.squirrel@groupware.dvs.informatik.tu-darmstadt.de> Message-ID: <1BC0B0ED-A3F9-43CE-A3D0-3A4C5A900858@hetland.org> On Jan 28, 2009, at 16:19, Stefan Behnel wrote: > Definitely a bug, please file a bug report. Will do. > (if you don't have a trac account, please send a htpasswd line to > Robert). Done. >> I realise that adding an "i -= 1" statement in general, as a blanket >> solution, would be rather wasteful. > > Not at all. We only do the range() optimisation for C integers, and > the C > compiler will simply strip it out if it's not used. Ah, of course -- good point :) - M -- Magnus Lie Hetland http://hetland.org From dagss at student.matnat.uio.no Wed Jan 28 23:10:01 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Wed, 28 Jan 2009 23:10:01 +0100 Subject: [Cython] Range argument unsigned behaviour Message-ID: <4980D7B9.90800@student.matnat.uio.no> I recently got bitten by this: cdef int i cdef unsigned n = 3 for i in range(-n, n): ... Here, the loop will never be run. I first submitted http://trac.cython.org/cython_trac/ticket/184 which was invalid, but I have another proposal: Can we let the type of the parameters to the magic, optimize range function be determined by the type of the iterator? So, in the example above, range would be prototyped roughly cdef int range(int, int) while if "i" had been typed as unsigned, range would behave as cdef int range(unsigned, unsigned) -- Dag Sverre From dagss at student.matnat.uio.no Thu Jan 29 20:16:45 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Thu, 29 Jan 2009 20:16:45 +0100 Subject: [Cython] Cython 0.11 beta In-Reply-To: <497F21D2.9080002@gmail.com> References: <36ECFF66-1EC5-4FAB-AEFB-3C42AC697162@math.washington.edu> <497C27F0.1000904@behnel.de> <497DF378.6070806@student.matnat.uio.no> <497F21D2.9080002@gmail.com> Message-ID: <4982009D.60609@student.matnat.uio.no> Michael Abshoff wrote: > Dag Sverre Seljebotn wrote: >> Stefan Behnel wrote: >>> Robert Bradshaw wrote: > > > >> Sage passes doctests: Does that include detecting any leaks? The >> refnanny is not yet working, so I'd be wary of releasing until that is >> done unless we know that Sage doesn't leak more when Cython is upgraded. > > I don't think Robert did any testing, but I can do some valgrinding with > the upcoming 3.3.alpha3 with the old and the new 0.11 snapshot to see if > anything pops up. It will be a day or two until I have results. > My refcount nanny now seems to be in a state where it doesn't crash things (though there are spurious reports still from not following the refnanny convention), and I've discovered at least one genuine leak so far in the cython test suite. So in the event that you get lots of leaks, waiting for that to mature may be a better angle. -- Dag Sverre From michael.abshoff at googlemail.com Thu Jan 29 20:17:43 2009 From: michael.abshoff at googlemail.com (Michael Abshoff) Date: Thu, 29 Jan 2009 11:17:43 -0800 Subject: [Cython] Cython 0.11 beta In-Reply-To: <4982009D.60609@student.matnat.uio.no> References: <36ECFF66-1EC5-4FAB-AEFB-3C42AC697162@math.washington.edu> <497C27F0.1000904@behnel.de> <497DF378.6070806@student.matnat.uio.no> <497F21D2.9080002@gmail.com> <4982009D.60609@student.matnat.uio.no> Message-ID: <498200D7.3010900@gmail.com> Dag Sverre Seljebotn wrote: > Michael Abshoff wrote: >> Dag Sverre Seljebotn wrote: >>> Stefan Behnel wrote: >>>> Robert Bradshaw wrote: >> Hi Dag, >>> Sage passes doctests: Does that include detecting any leaks? The >>> refnanny is not yet working, so I'd be wary of releasing until that is >>> done unless we know that Sage doesn't leak more when Cython is upgraded. >> I don't think Robert did any testing, but I can do some valgrinding with >> the upcoming 3.3.alpha3 with the old and the new 0.11 snapshot to see if >> anything pops up. It will be a day or two until I have results. >> > > My refcount nanny now seems to be in a state where it doesn't crash > things (though there are spurious reports still from not following the > refnanny convention), and I've discovered at least one genuine leak so > far in the cython test suite. Excuse my ignorance, but how do I turn on the refnanny in all the code? I do not recall that being documented anywhere? I guess the best place might be the wiki for that. Did anybody fix that leak you mentioned above? > So in the event that you get lots of leaks, waiting for that to mature > may be a better angle. I have been distracted by other things, but I am about to start the valgrind run with the 0.10.3 to establish a baseline. This will take about 6 hours clock time if I use 12 to 18 CPUs at the same time. Once that is finished I will switch to the new Cython. I could also use another machine to do the second run in parallel, but some other users of the new Sage hardware might be less than happy about that :) Cheers, Michael From dagss at student.matnat.uio.no Thu Jan 29 20:48:54 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Thu, 29 Jan 2009 20:48:54 +0100 Subject: [Cython] Cython 0.11 beta In-Reply-To: <498200D7.3010900@gmail.com> References: <36ECFF66-1EC5-4FAB-AEFB-3C42AC697162@math.washington.edu> <497C27F0.1000904@behnel.de> <497DF378.6070806@student.matnat.uio.no> <497F21D2.9080002@gmail.com> <4982009D.60609@student.matnat.uio.no> <498200D7.3010900@gmail.com> Message-ID: <49820826.6030209@student.matnat.uio.no> >>>> Sage passes doctests: Does that include detecting any leaks? The >>>> refnanny is not yet working, so I'd be wary of releasing until that is >>>> done unless we know that Sage doesn't leak more when Cython is upgraded. >>> I don't think Robert did any testing, but I can do some valgrinding with >>> the upcoming 3.3.alpha3 with the old and the new 0.11 snapshot to see if >>> anything pops up. It will be a day or two until I have results. >>> >> My refcount nanny now seems to be in a state where it doesn't crash >> things (though there are spurious reports still from not following the >> refnanny convention), and I've discovered at least one genuine leak so >> far in the cython test suite. > > Excuse my ignorance, but how do I turn on the refnanny in all the code? > I do not recall that being documented anywhere? I guess the best place > might be the wiki for that. Did anybody fix that leak you mentioned above? No, I was just giving you a heads up for something I'm doing right now (and which is in experiment mode) -- running valgrind on Cython 0.11 may not be such a big point if I'm right about to get the refnanny in order, which will make those statistics outdated if the "refnanny effect" kicks in. I found the leak while playing with the refnanny: http://trac.cython.org/cython_trac/ticket/199 Establishing a baseline is always good though :-) Docs should be in wiki when it is more mature. There's some here: http://article.gmane.org/gmane.comp.python.cython.devel/4045/ Basically, once the Cython test suite runs with the nanny, this would be needed for SAGE: 1) Somebody should get distutils to compile Cython/Runtime/refnanny.pyx 2) SAGE needs to load refnanny.so or similar using RTLD_GLOBAL 3) Then, import it as a Python module 4) Then, run the C compiler with CYTHON_REFNANNY defined on Cython-generated sources Don't bother with it until it really works though :-) -- Dag Sverre From dagss at student.matnat.uio.no Thu Jan 29 22:41:03 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Thu, 29 Jan 2009 22:41:03 +0100 (CET) Subject: [Cython] FlattenInListTransform again Message-ID: <2d77a248bb76ca20433a60d0f0a1399a.squirrel@webmail.uio.no> Just a note that I've disabled FlattenInListTransform as that removed some leaks, making the assumption that speed regression is better than leaks. Just enable it in Main.py if you disagree/want to pursue it. http://trac.cython.org/cython_trac/ticket/199 Dag Sverre From stefan_ml at behnel.de Thu Jan 29 22:43:10 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 29 Jan 2009 22:43:10 +0100 Subject: [Cython] FlattenInListTransform again In-Reply-To: <2d77a248bb76ca20433a60d0f0a1399a.squirrel@webmail.uio.no> References: <2d77a248bb76ca20433a60d0f0a1399a.squirrel@webmail.uio.no> Message-ID: <498222EE.1040705@behnel.de> Dag Sverre Seljebotn wrote: > Just a note that I've disabled FlattenInListTransform as that removed some > leaks, making the assumption that speed regression is better than leaks. > Just enable it in Main.py if you disagree/want to pursue it. > > http://trac.cython.org/cython_trac/ticket/199 I've already fixed it, will commit it when I see the test suite pass nicely. Stefan From carl.witty at gmail.com Thu Jan 29 22:49:39 2009 From: carl.witty at gmail.com (Carl Witty) Date: Thu, 29 Jan 2009 13:49:39 -0800 Subject: [Cython] Range argument unsigned behaviour In-Reply-To: <4980D7B9.90800@student.matnat.uio.no> References: <4980D7B9.90800@student.matnat.uio.no> Message-ID: On Wed, Jan 28, 2009 at 2:10 PM, Dag Sverre Seljebotn wrote: > I recently got bitten by this: > > cdef int i > cdef unsigned n = 3 > for i in range(-n, n): > ... > > Here, the loop will never be run. > > I first submitted http://trac.cython.org/cython_trac/ticket/184 which > was invalid, but I have another proposal: > > Can we let the type of the parameters to the magic, optimize range > function be determined by the type of the iterator? > > So, in the example above, range would be prototyped roughly > > cdef int range(int, int) > > while if "i" had been typed as unsigned, range would behave as > > cdef int range(unsigned, unsigned) So if I understand correctly, the optimized range would act differently than an unoptimized range would? That seems like a bad idea, especially since the range optimization is on by default. Carl From dagss at student.matnat.uio.no Fri Jan 30 09:13:08 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Fri, 30 Jan 2009 09:13:08 +0100 Subject: [Cython] Range argument unsigned behaviour In-Reply-To: References: <4980D7B9.90800@student.matnat.uio.no> Message-ID: <4982B694.2090303@student.matnat.uio.no> Carl Witty wrote: > On Wed, Jan 28, 2009 at 2:10 PM, Dag Sverre Seljebotn > wrote: > >> I recently got bitten by this: >> >> cdef int i >> cdef unsigned n = 3 >> for i in range(-n, n): >> ... >> >> Here, the loop will never be run. >> >> I first submitted http://trac.cython.org/cython_trac/ticket/184 which >> was invalid, but I have another proposal: >> >> Can we let the type of the parameters to the magic, optimize range >> function be determined by the type of the iterator? >> >> So, in the example above, range would be prototyped roughly >> >> cdef int range(int, int) >> >> while if "i" had been typed as unsigned, range would behave as >> >> cdef int range(unsigned, unsigned) >> > > So if I understand correctly, the optimized range would act > differently than an unoptimized range would? That seems like a bad > idea, especially since the range optimization is on by default. > Hmm.. yes, range(-i, i) would be an empty list when i is unsigned... OK I suppose I give up :-) Dag Sverre From dagss at student.matnat.uio.no Fri Jan 30 10:56:19 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Fri, 30 Jan 2009 10:56:19 +0100 Subject: [Cython] Refnanny done Message-ID: <4982CEC3.5070803@student.matnat.uio.no> The test suite now runs successfully with the refnanny; the only difference in turning it on is that bug #200 is detected :-) There was practically no time penalty for the test suite. I propose that once the issue below is fixed, this is made the default behaviour for runtests.py, and that --no-refnanny will be the flag instead. Also, I think this could be a good time for lxml and Sage to try to run with the refnanny. See the bottom of http://wiki.cython.org/enhancements/refnanny for instructions. The only thing missing is that runtests.py should compile refnanny.pyx automatically rather than leave it for manual compilation. As about all I know about distutils is the name I was hoping someone else could pick that up: http://trac.cython.org/cython_trac/ticket/201. But lxml and Sage cannot benefit from this anyway, it must be integrated into their own build system. Dag Sverre From magnus at hetland.org Fri Jan 30 12:13:43 2009 From: magnus at hetland.org (Magnus Lie Hetland) Date: Fri, 30 Jan 2009 12:13:43 +0100 Subject: [Cython] Fix for #196 (for loop bug) Message-ID: Hi! I've put together a tiny patch for the for loop bug, and just thought I'd run it by you guys. As far as I understand, the "for ... from" node is also used for optimized range()-loops (i.e., over C integer indices) -- so that's the only place I fixed anything. (It seems to work for range too, now.) I've basically just mirrored the incop, creating an inverse named decop, to "undo" the last (superfluous) step, and then put a statement using this (such as "i--" if the incop is "i++", or "i-=2" if the incop is "i+=2") right after the closing parens of the for loop. As far as I can see, this should be safe, because if the code breaks out of the loop (using a goto in C), the last step will never occur, so the "undo" isn't needed. --- a/Cython/Compiler/Nodes.py Wed Dec 17 01:43:58 2008 -0800 +++ b/Cython/Compiler/Nodes.py Fri Jan 30 12:02:15 2009 +0100 @@ -3764,9 +3764,12 @@ self.bound1.generate_evaluation_code(code) self.bound2.generate_evaluation_code(code) offset, incop = self.relation_table[self.relation1] + decop = "++" if incop == "--" else "--" if self.step is not None: + step = self.step.result() self.step.generate_evaluation_code(code) - incop = "%s=%s" % (incop[0], self.step.result()) + incop = "%s=%s" % (incop[0], step) + decop = "%s=%s" % (decop[0], step) code.putln( "for (%s = %s%s; %s %s %s; %s%s) {" % ( self.loopvar_name, @@ -3778,7 +3781,7 @@ self.target.generate_assignment_code(self.py_loopvar_node, code) self.body.generate_execution_code(code) code.put_label(code.continue_label) - code.putln("}") + code.putln("} %s%s;" % (self.loopvar_name, decop)) break_label = code.break_label code.set_loop_labels(old_loop_labels) if self.else_clause: I've basically just tested it with loops from 0 to 10 and from 10 to 0, with steps 1 and 2, with both loop syntaxes (for ... from and for ... in range(...)). Of course it's very possible that I've missed something obvious somewhere... :-> If it seems OK, I could always write up some more proper tests and upload a patch including those to Trac for review. -- Magnus Lie Hetland http://hetland.org From dagss at student.matnat.uio.no Fri Jan 30 12:45:48 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Fri, 30 Jan 2009 12:45:48 +0100 Subject: [Cython] Fix for #196 (for loop bug) In-Reply-To: References: Message-ID: <4982E86C.5@student.matnat.uio.no> Comments interleaved. Magnus Lie Hetland wrote: > Hi! > > I've put together a tiny patch for the for loop bug, and just thought > I'd run it by you guys. As far as I understand, the "for ... from" > node is also used for optimized range()-loops (i.e., over C integer > indices) -- so that's the only place I fixed anything. (It seems to > work for range too, now.) > This is correct. for in range(...) is transformed before the code generation step into a forfrom. > I've basically just mirrored the incop, creating an inverse named > decop, to "undo" the last (superfluous) step, and then put a statement > using this (such as "i--" if the incop is "i++", or "i-=2" if the > incop is "i+=2") right after the closing parens of the for loop. As > far as I can see, this should be safe, because if the code breaks out > of the loop (using a goto in C), the last step will never occur, so > the "undo" isn't needed. > > --- a/Cython/Compiler/Nodes.py Wed Dec 17 01:43:58 2008 -0800 > +++ b/Cython/Compiler/Nodes.py Fri Jan 30 12:02:15 2009 +0100 > @@ -3764,9 +3764,12 @@ > self.bound1.generate_evaluation_code(code) > self.bound2.generate_evaluation_code(code) > offset, incop = self.relation_table[self.relation1] > + decop = "++" if incop == "--" else "--" > Cython supports Python 2.3, so you must change the syntax. > if self.step is not None: > + step = self.step.result() > self.step.generate_evaluation_code(code) > - incop = "%s=%s" % (incop[0], self.step.result()) > + incop = "%s=%s" % (incop[0], step) > + decop = "%s=%s" % (decop[0], step) > code.putln( > "for (%s = %s%s; %s %s %s; %s%s) {" % ( > self.loopvar_name, > @@ -3778,7 +3781,7 @@ > > self.target.generate_assignment_code(self.py_loopvar_node, code) > self.body.generate_execution_code(code) > code.put_label(code.continue_label) > - code.putln("}") > + code.putln("} %s%s;" % (self.loopvar_name, decop)) > break_label = code.break_label > code.set_loop_labels(old_loop_labels) > if self.else_clause: > > I've basically just tested it with loops from 0 to 10 and from 10 to > 0, with steps 1 and 2, with both loop syntaxes (for ... from and > for ... in range(...)). > > Of course it's very possible that I've missed something obvious > somewhere... :-> > One minor issue: What if the iterator variable expression is a function? I.e.: cdef int getstep(): return 10 for 0 <= i < n by getstep(): ... Then self.step.result() will return a function call, and your decop will invoke the function twice :-( What you do is call "code.funcstate.allocate_temp(integer_type_of_step)" to get a temporary variable which you can assign the step variable to, and then you use this temporary instead. Finally, after the decrement has happened, you call code.funcstate.release_temp(...) so that code.funcstate can hand out the same temporary for the next for loop (or whatever). See Code.py. Dag Sverre From dagss at student.matnat.uio.no Fri Jan 30 12:50:16 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Fri, 30 Jan 2009 12:50:16 +0100 Subject: [Cython] Fix for #196 (for loop bug) In-Reply-To: <4982E86C.5@student.matnat.uio.no> References: <4982E86C.5@student.matnat.uio.no> Message-ID: <4982E978.90902@student.matnat.uio.no> Dag Sverre Seljebotn wrote: > What you do is call "code.funcstate.allocate_temp(integer_type_of_step)" > To be slightly more helpful here, I believe the exact call would be code.funcstate.allocate_temp(step.type) I think step will be a "coercion node" if step is a Python object so you don't need to worry about refcounts etc. (but do include functions returning both Python and C types as "step" in your tests). Thanks for doing this! Dag Sverre From dagss at student.matnat.uio.no Fri Jan 30 12:51:08 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Fri, 30 Jan 2009 12:51:08 +0100 Subject: [Cython] Fix for #196 (for loop bug) In-Reply-To: <4982E978.90902@student.matnat.uio.no> References: <4982E86C.5@student.matnat.uio.no> <4982E978.90902@student.matnat.uio.no> Message-ID: <4982E9AC.2080802@student.matnat.uio.no> Dag Sverre Seljebotn wrote: > Dag Sverre Seljebotn wrote: > >> What you do is call "code.funcstate.allocate_temp(integer_type_of_step)" >> >> > > To be slightly more helpful here, I believe the exact call would be > > code.funcstate.allocate_temp(step.type) > Make that "self.step.type". Dag Sverre From dagss at student.matnat.uio.no Fri Jan 30 12:56:00 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Fri, 30 Jan 2009 12:56:00 +0100 Subject: [Cython] Fix for #196 (for loop bug) In-Reply-To: References: Message-ID: <4982EAD0.2060101@student.matnat.uio.no> Magnus Lie Hetland wrote: > Hi! > > I've put together a tiny patch for the for loop bug, and just thought > I'd run it by you guys. As far as I understand, the "for ... from" > node is also used for optimized range()-loops (i.e., over C integer > indices) -- so that's the only place I fixed anything. (It seems to > work for range too, now.) > I continously come to think about more stuff here :-) I think we should leave the semantics of the for-from statement alone at the moment, since it was introduced to mimick the C construct (and for all I know code may depend on the C behaviour here). The solution would be to have a flag in the ForFromStatNode, which you could turn on when the node is the result of a transform from a for-in-range. Which is done in Optimize.py line 140. Dag Sverre From magnus at hetland.org Fri Jan 30 14:03:05 2009 From: magnus at hetland.org (Magnus Lie Hetland) Date: Fri, 30 Jan 2009 14:03:05 +0100 Subject: [Cython] Fix for #196 (for loop bug) In-Reply-To: <4982EAD0.2060101@student.matnat.uio.no> References: <4982EAD0.2060101@student.matnat.uio.no> Message-ID: <3888E3C8-3CC3-42EA-81F2-41CC382F4A0D@hetland.org> On Jan 30, 2009, at 12:56, Dag Sverre Seljebotn wrote: > I continously come to think about more stuff here :-) Cool :) > I think we should leave the semantics of the for-from statement > alone at > the moment, since it was introduced to mimick the C construct (and for > all I know code may depend on the C behaviour here). Sounds reasonable. As for the step problem ... you wrote: > Then self.step.result() will return a function call, and your decop > will invoke the function twice :-( What you point out here is an interesting issue -- and I don't think it relates only to my decop. In fact, as far as I can see, the existing Cython behavior is quite odd here. For the for-from loop, this may be intended -- as you say, it mimics the C behavior (although it seems to fly in the face of "Python intuition"): Even with the existing Cython semantics, getstep() will be called after *every iteration*. It seems to me that in for 0 <= i < n by getstep(): pass one would expect (in Python) that the step expression be evaluated only once. If that is not the case, fine. Then the for-from loop can safely be left alone as it is. (If not, a local variable should probably be used here too, even with the decref out of the picture, no?) But there seems to be a problem with range() as well: cdef int i, j cdef int getstep(): return 2 for i in range(0,10,2): pass for j in range(0,10,getstep()): pass print i, j # (10, 8) This doesn't seem right? (I tried looking at the code generated for the loop with getstep, but it was a bit too hairy for me to spot the bug immediately :->) On the other hand, here it seems that the right thing is done wrt. the use of the step expression. As opposed to in the for-from loop, getstep() is called only once... Anyway... I'll have a look at adding the flag to the for-from-loop that will use decrementation if it's constructed from a range-loop. It seems that the local variable isn't an issue here (i.e. in for-from loops constructed from range) after all (or am I mistaken)? I'll hold off on adding a local variable to the plain for-from loop for now (and as yet I have no idea what's up with the 10 vs 8 above), as I don't know if the repeated evaluation of the step expression is desired. -- Magnus Lie Hetland http://hetland.org From magnus at hetland.org Fri Jan 30 14:11:59 2009 From: magnus at hetland.org (Magnus Lie Hetland) Date: Fri, 30 Jan 2009 14:11:59 +0100 Subject: [Cython] Fix for #196 (for loop bug) In-Reply-To: <3888E3C8-3CC3-42EA-81F2-41CC382F4A0D@hetland.org> References: <4982EAD0.2060101@student.matnat.uio.no> <3888E3C8-3CC3-42EA-81F2-41CC382F4A0D@hetland.org> Message-ID: On Jan 30, 2009, at 14:03, Magnus Lie Hetland wrote: > for i in range(0,10,2): pass > for j in range(0,10,getstep()): pass > > print i, j > # (10, 8) Ahm. I guess the explanation here was, in fact, quite obvious from the fact that the code was too hairy for me to see the bug: Unless I'm mistaken, the version with the function call isn't optimized and hence, its behavior is in line with Python semantics. Then again, if getstep is cdef-ed to return an int, why *isn't* this optimized to a for-from-loop? (And if it should be, the local variable issue that Dag Sverre mentioned is again relevant.) -- Magnus Lie Hetland http://hetland.org From dagss at student.matnat.uio.no Fri Jan 30 14:23:02 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Fri, 30 Jan 2009 14:23:02 +0100 Subject: [Cython] Fix for #196 (for loop bug) In-Reply-To: References: <4982EAD0.2060101@student.matnat.uio.no> <3888E3C8-3CC3-42EA-81F2-41CC382F4A0D@hetland.org> Message-ID: <4982FF36.7070504@student.matnat.uio.no> Magnus Lie Hetland wrote: > On Jan 30, 2009, at 14:03, Magnus Lie Hetland wrote: > > >> for i in range(0,10,2): pass >> for j in range(0,10,getstep()): pass >> >> print i, j >> # (10, 8) >> > > > Ahm. I guess the explanation here was, in fact, quite obvious from the > fact that the code was too hairy for me to see the bug: Unless I'm > mistaken, the version with the function call isn't optimized and > hence, its behavior is in line with Python semantics. > > Then again, if getstep is cdef-ed to return an int, why *isn't* this > optimized to a for-from-loop? (And if it should be, the local variable > issue that Dag Sverre mentioned is again relevant.) > Ah. Line 115 in Optimize.py: Apparently the step direction cannot be determined, which is why it isn't optimized unless it is a compile-time constant. So the expressions will only be constant and your fix is OK (except that the flag is needed to preserve for-from behaviour). (One could get around this by a more elaborate temporary variable shuffling prior to the loop with a runtime if-test (i.e. if step is negative, exchange bounds), but that's digressing way beyond a simple bugfix. Shows that it is really important with a regression test here though in case that is done in the future -- i.e. print something from the getstep() function so that the side-effects can be traced in the doctest) Dag Sverre From dagss at student.matnat.uio.no Fri Jan 30 14:30:57 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Fri, 30 Jan 2009 14:30:57 +0100 Subject: [Cython] Fix for #196 (for loop bug) In-Reply-To: <3888E3C8-3CC3-42EA-81F2-41CC382F4A0D@hetland.org> References: <4982EAD0.2060101@student.matnat.uio.no> <3888E3C8-3CC3-42EA-81F2-41CC382F4A0D@hetland.org> Message-ID: <49830111.7030201@student.matnat.uio.no> Magnus Lie Hetland wrote: > For the for-from loop, this may be intended -- as you say, it mimics > the C behavior (although it seems to fly in the face of "Python > intuition"): Even with the existing Cython semantics, getstep() will > be called after *every iteration*. > > It seems to me that in > > for 0 <= i < n by getstep(): pass > > one would expect (in Python) that the step expression be evaluated > only once. If that is not the case, fine. Then the for-from loop can > safely be left alone as it is. (If not, a local variable should > I think so -- the for-from loop is a C idiom from which one can expect C behaviour from my perspective, and if this has been the behaviour so far then backward compatability alone speaks against altering it. The range optimization has been around for much shorter and clearly should behave as Python range, so that story is different. (Well, my real perspective is that for-from should be deprecated and removed prior to a 1.0 release, but let's not have that war now :-) ) Dag Sverre From stefan_ml at behnel.de Fri Jan 30 14:46:28 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 30 Jan 2009 14:46:28 +0100 (CET) Subject: [Cython] Refnanny done In-Reply-To: <4982CEC3.5070803@student.matnat.uio.no> References: <4982CEC3.5070803@student.matnat.uio.no> Message-ID: <53109.213.61.181.86.1233323188.squirrel@groupware.dvs.informatik.tu-darmstadt.de> Dag Sverre Seljebotn wrote: > The test suite now runs successfully with the refnanny; the only > difference in turning it on is that bug #200 is detected :-) > > There was practically no time penalty for the test suite. I propose that > once the issue below is fixed, this is made the default behaviour for > runtests.py, and that --no-refnanny will be the flag instead. +1 > Also, I think this could be a good time for lxml and Sage to try to run > with the refnanny. See the bottom of > http://wiki.cython.org/enhancements/refnanny for instructions. I didn't try running the refnanny so far, but I'm very happy to hear that you consider it done. Thanks for the work you put into it. I'll certainly give it a try with lxml, but that won't be before the end of next week. > The only thing missing is that runtests.py should compile refnanny.pyx > automatically rather than leave it for manual compilation. As about all > I know about distutils is the name I was hoping someone else could pick > that up: http://trac.cython.org/cython_trac/ticket/201. But lxml and > Sage cannot benefit from this anyway, it must be integrated into their > own build system. You can leave that to pyximport/pyxbuild.py. Stefan From magnus at hetland.org Fri Jan 30 15:08:46 2009 From: magnus at hetland.org (Magnus Lie Hetland) Date: Fri, 30 Jan 2009 15:08:46 +0100 Subject: [Cython] Patch for #196 uploaded Message-ID: <5F0642D6-D666-4BD9-A8CA-056CB48C34EE@hetland.org> Suggested fix with a small test uploaded: http://trac.cython.org/cython_trac/attachment/ticket/196/for-dec-patch.txt - M -- Magnus Lie Hetland http://hetland.org From dagss at student.matnat.uio.no Fri Jan 30 19:50:42 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Fri, 30 Jan 2009 19:50:42 +0100 Subject: [Cython] Warning: python object pointer used Message-ID: <49834C02.6020202@student.matnat.uio.no> I need to convert a Python object to void* and back again, and also from object to PyObject*. Is there a way of doing this without getting warnings? If not, how can I introduce one? (Warning compiler directives? In that case we should start numbering the warnings...) warning: /home/dagss/cython/devel5/Cython/Runtime/refnanny.pyx:64:9: No conversion from void * to Python object, python object pointer used. warning: /home/dagss/cython/devel5/Cython/Runtime/refnanny.pyx:65:26: No conversion from python_exc.PyObject * to Python object, python object pointer used. ... -- Dag Sverre From dagss at student.matnat.uio.no Fri Jan 30 19:55:36 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Fri, 30 Jan 2009 19:55:36 +0100 Subject: [Cython] Warning: python object pointer used In-Reply-To: <49834C02.6020202@student.matnat.uio.no> References: <49834C02.6020202@student.matnat.uio.no> Message-ID: <49834D28.5090602@student.matnat.uio.no> Dag Sverre Seljebotn wrote: > I need to convert a Python object to void* and back again, and also from > object to PyObject*. Is there a way of doing this without getting > warnings? If not, how can I introduce one? (Warning compiler directives? > In that case we should start numbering the warnings...) As usual, sending an email spawns a solution in my head... cdef extern from *: cdef void* tovoid "(void*)"(object) Anyone better/anything cleaner though? (If not I'll file a ticket to make such explicit conversions magic functions in the cython module.) -- Dag Sverre From dalcinl at gmail.com Fri Jan 30 20:26:05 2009 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Fri, 30 Jan 2009 16:26:05 -0300 Subject: [Cython] Warning: python object pointer used In-Reply-To: <49834D28.5090602@student.matnat.uio.no> References: <49834C02.6020202@student.matnat.uio.no> <49834D28.5090602@student.matnat.uio.no> Message-ID: Any time I needed this in the past, I've used explicit Cython casts, like , , or . On Fri, Jan 30, 2009 at 3:55 PM, Dag Sverre Seljebotn wrote: > Dag Sverre Seljebotn wrote: >> I need to convert a Python object to void* and back again, and also from >> object to PyObject*. Is there a way of doing this without getting >> warnings? If not, how can I introduce one? (Warning compiler directives? >> In that case we should start numbering the warnings...) > > As usual, sending an email spawns a solution in my head... > > cdef extern from *: > cdef void* tovoid "(void*)"(object) > > Anyone better/anything cleaner though? (If not I'll file a ticket to > make such explicit conversions magic functions in the cython module.) > > -- > Dag Sverre > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From dalcinl at gmail.com Fri Jan 30 20:31:07 2009 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Fri, 30 Jan 2009 16:31:07 -0300 Subject: [Cython] Fix for #196 (for loop bug) In-Reply-To: <49830111.7030201@student.matnat.uio.no> References: <4982EAD0.2060101@student.matnat.uio.no> <3888E3C8-3CC3-42EA-81F2-41CC382F4A0D@hetland.org> <49830111.7030201@student.matnat.uio.no> Message-ID: On Fri, Jan 30, 2009 at 10:30 AM, Dag Sverre Seljebotn wrote: > I think so -- the for-from loop is a C idiom from which one can expect C > behaviour from my perspective, and if this has been the behaviour so far > then backward compatability alone speaks against altering it. The range > optimization has been around for much shorter and clearly should behave > as Python range, so that story is different. > Indeed. I've wrote code where the C semantics is assumed. > (Well, my real perspective is that for-from should be deprecated and > removed prior to a 1.0 release, but let's not have that war now :-) ) > No please! Again, the C semantics is useful in some cases, and removing it will make a lot of code out there stop compiling. > Dag Sverre > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From stefan_ml at behnel.de Fri Jan 30 20:44:57 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 30 Jan 2009 20:44:57 +0100 Subject: [Cython] Fix for #196 (for loop bug) In-Reply-To: <49830111.7030201@student.matnat.uio.no> References: <4982EAD0.2060101@student.matnat.uio.no> <3888E3C8-3CC3-42EA-81F2-41CC382F4A0D@hetland.org> <49830111.7030201@student.matnat.uio.no> Message-ID: <498358B9.20005@behnel.de> Hi, Dag Sverre Seljebotn wrote: > Magnus Lie Hetland wrote: >> For the for-from loop, this may be intended -- as you say, it mimics >> the C behavior (although it seems to fly in the face of "Python >> intuition"): Even with the existing Cython semantics, getstep() will >> be called after *every iteration*. >> >> It seems to me that in >> >> for 0 <= i < n by getstep(): pass >> >> one would expect (in Python) that the step expression be evaluated >> only once. If that is not the case, fine. Then the for-from loop can >> safely be left alone as it is. >> > I think so -- the for-from loop is a C idiom from which one can expect C > behaviour from my perspective, and if this has been the behaviour so far > then backward compatability alone speaks against altering it. The range > optimization has been around for much shorter and clearly should behave > as Python range, so that story is different. I vote for a) diverging behaviour between for-range and for-from for the loop-variable after loop termination, and b) the obvious evaluate-once semantics for the range() optimisation and the obvious (?) evaluate-on-each-step semantics for the for-from loop. And while I'm fine with removing the emphases from the for-from loop in the docs, I really don't think we should remove it from the language. However, it needs to be stated in the docs that it follows the semantics of the C for loop as far as possible, and that the usual Python for-range pattern is preferred for readability and semantic clarity. Stefan From dagss at student.matnat.uio.no Fri Jan 30 21:06:32 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Fri, 30 Jan 2009 21:06:32 +0100 Subject: [Cython] Fix for #196 (for loop bug) In-Reply-To: <498358B9.20005@behnel.de> References: <4982EAD0.2060101@student.matnat.uio.no> <3888E3C8-3CC3-42EA-81F2-41CC382F4A0D@hetland.org> <49830111.7030201@student.matnat.uio.no> <498358B9.20005@behnel.de> Message-ID: <49835DC8.6000502@student.matnat.uio.no> Stefan Behnel wrote: > Hi, > > Dag Sverre Seljebotn wrote: >> Magnus Lie Hetland wrote: >>> For the for-from loop, this may be intended -- as you say, it mimics >>> the C behavior (although it seems to fly in the face of "Python >>> intuition"): Even with the existing Cython semantics, getstep() will >>> be called after *every iteration*. >>> >>> It seems to me that in >>> >>> for 0 <= i < n by getstep(): pass >>> >>> one would expect (in Python) that the step expression be evaluated >>> only once. If that is not the case, fine. Then the for-from loop can >>> safely be left alone as it is. >>> >> I think so -- the for-from loop is a C idiom from which one can expect C >> behaviour from my perspective, and if this has been the behaviour so far >> then backward compatability alone speaks against altering it. The range >> optimization has been around for much shorter and clearly should behave >> as Python range, so that story is different. > > I vote for a) diverging behaviour between for-range and for-from for the > loop-variable after loop termination, and b) the obvious evaluate-once > semantics for the range() optimisation and the obvious (?) > evaluate-on-each-step semantics for the for-from loop. Lucky for you, that's what sits in the tree right now, thanks to Magnus :-) Your thoughts on for-from seems like a good compromise, too. +1 Dag Sverre From dagss at student.matnat.uio.no Fri Jan 30 21:07:42 2009 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Fri, 30 Jan 2009 21:07:42 +0100 Subject: [Cython] Warning: python object pointer used In-Reply-To: References: <49834C02.6020202@student.matnat.uio.no> <49834D28.5090602@student.matnat.uio.no> Message-ID: <49835E0E.4080106@student.matnat.uio.no> Lisandro Dalcin wrote: > Any time I needed this in the past, I've used explicit Cython casts, > like , , or . Me too, but if I go for that then apparently we'll all get a pageload of Cython warnings each time we run the test suite :-) Dag Sverre > > On Fri, Jan 30, 2009 at 3:55 PM, Dag Sverre Seljebotn > wrote: >> Dag Sverre Seljebotn wrote: >>> I need to convert a Python object to void* and back again, and also from >>> object to PyObject*. Is there a way of doing this without getting >>> warnings? If not, how can I introduce one? (Warning compiler directives? >>> In that case we should start numbering the warnings...) >> As usual, sending an email spawns a solution in my head... >> >> cdef extern from *: >> cdef void* tovoid "(void*)"(object) >> >> Anyone better/anything cleaner though? (If not I'll file a ticket to >> make such explicit conversions magic functions in the cython module.) >> >> -- >> Dag Sverre >> _______________________________________________ >> Cython-dev mailing list >> Cython-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/cython-dev >> > > > From peyroux at me.com Sat Jan 31 22:21:27 2009 From: peyroux at me.com (Jean-Alexandre Peyroux) Date: Sat, 31 Jan 2009 22:21:27 +0100 Subject: [Cython] char* and string object Message-ID: Hello, I think i did not understand the handling character string. This is an example : user.pyx : cdef class User: def __init__(self, char* name): self.cname = name cpdef printName(self): print "My name is %s" % self.cname user.pxd : cdef class User: cdef readonly char* cname cpdef printName(self) "compilation" is going well but i have a problem with the use : In [1]: from user import User In [2]: u = User("Peyroux") In [3]: u. u.__class__ u.__delattr__ u.__doc__ u.__getattribute__ u.__hash__ u.__init__ u.__new__ u.__pyx_vtable__ u.__reduce__ u.__reduce_ex__ u.__repr__ u.__setattr__ u.__str__ u.cname u.printName In [3]: u.printName Out[3]: In [4]: u.printName() My name is _3 In [5]: u.cname Out[5]: '_3' I do not understand. My string is completely random, as if it was pointing in the wrong location of memory. I missed step ? regards From kirr at landau.phys.spbu.ru Sun Jan 25 09:46:31 2009 From: kirr at landau.phys.spbu.ru (Kirill Smelkov) Date: Sun, 25 Jan 2009 08:46:31 -0000 Subject: [Cython] Cython 0.11 beta In-Reply-To: <36ECFF66-1EC5-4FAB-AEFB-3C42AC697162@math.washington.edu> References: <36ECFF66-1EC5-4FAB-AEFB-3C42AC697162@math.washington.edu> Message-ID: <20090125084610.GA4898@roro3.zxlink> On Sat, Jan 24, 2009 at 03:13:52PM -0800, Robert Bradshaw wrote: > After much development, it looks like Cython 0.11 is getting stable > enough to release. Please try out the beta, which is the current tip > of sage-devel. > > http://cython.org/Cython-0.11.beta.tar.gz I'm on Debian Lenny and for b99a8f46f298 I get test failures: --- 8< --- rm -fr BUILD python runtests.py -vv Running tests against Cython 0.11.beta Python 2.5.2 (r252:60911, Jan 4 2009, 17:40:26) [GCC 4.3.2] test_parserbehaviour_is_what_we_coded_for (Cython.Compiler.Tests.TestParseTreeTransforms.TestNormalizeTree) ... ok test_pass_eliminated (Cython.Compiler.Tests.TestParseTreeTransforms.TestNormalizeTree) ... ok test_statinexpr (Cython.Compiler.Tests.TestParseTreeTransforms.TestNormalizeTree) ... ok test_wrap_multistat (Cython.Compiler.Tests.TestParseTreeTransforms.TestNormalizeTree) ... ok test_wrap_offagain (Cython.Compiler.Tests.TestParseTreeTransforms.TestNormalizeTree) ... ok test_wrap_singlestat (Cython.Compiler.Tests.TestParseTreeTransforms.TestNormalizeTree) ... ok test_basic (Cython.Compiler.Tests.TestParseTreeTransforms.TestWithTransform) ... ok test_simplified (Cython.Compiler.Tests.TestParseTreeTransforms.TestWithTransform) ... ok test_basic (Cython.Compiler.Tests.TestTreeFragment.TestTreeFragments) ... ok test_copy_is_taken (Cython.Compiler.Tests.TestTreeFragment.TestTreeFragments) ... ok test_exprstat (Cython.Compiler.Tests.TestTreeFragment.TestTreeFragments) ... ok test_pos_is_transferred (Cython.Compiler.Tests.TestTreeFragment.TestTreeFragments) ... ok test_substitution (Cython.Compiler.Tests.TestTreeFragment.TestTreeFragments) ... ok test_substitutions_are_copied (Cython.Compiler.Tests.TestTreeFragment.TestTreeFragments) ... ok test_temps (Cython.Compiler.Tests.TestTreeFragment.TestTreeFragments) ... ok test_basic (Cython.Compiler.Tests.TestBuffer.TestBufferParsing) ... ok test_notype_as_expr1 (Cython.Compiler.Tests.TestBuffer.TestBufferParsing) ... ok test_notype_as_expr2 (Cython.Compiler.Tests.TestBuffer.TestBufferParsing) ... ok test_pos_after_key (Cython.Compiler.Tests.TestBuffer.TestBufferParsing) ... ok test_type_keyword (Cython.Compiler.Tests.TestBuffer.TestBufferParsing) ... ok test_type_pos (Cython.Compiler.Tests.TestBuffer.TestBufferParsing) ... ok test_decorator (Cython.Compiler.Tests.TestDecorators.TestDecorator) ... ok test_attribute (Cython.Tests.TestCodeWriter.TestCodeWriter) ... ok test_cdef_var (Cython.Tests.TestCodeWriter.TestCodeWriter) ... ok test_def (Cython.Tests.TestCodeWriter.TestCodeWriter) ... ok test_for_loop (Cython.Tests.TestCodeWriter.TestCodeWriter) ... ok test_if (Cython.Tests.TestCodeWriter.TestCodeWriter) ... ok test_ifelifelse (Cython.Tests.TestCodeWriter.TestCodeWriter) ... ok test_inplace_assignment (Cython.Tests.TestCodeWriter.TestCodeWriter) ... ok test_longness_and_signedness (Cython.Tests.TestCodeWriter.TestCodeWriter) ... ok test_print (Cython.Tests.TestCodeWriter.TestCodeWriter) ... ok test_signed_short (Cython.Tests.TestCodeWriter.TestCodeWriter) ... ok test_typed_args (Cython.Tests.TestCodeWriter.TestCodeWriter) ... ok Doctest: Cython.StringIOTree ... ok Doctest: Cython.Compiler.Options.parse_option_list ... ok Doctest: Cython.Compiler.Options.parse_option_value ... ok Doctest: Cython.Compiler.Visitor.TreeVisitor ... ok compiling (c) a_capi ... ok compiling (cpp) a_capi ... ok compiling (c) argdefault ... ok compiling (cpp) argdefault ... ok compiling (c) arrayargs ... ok compiling (cpp) arrayargs ... ok compiling (c) arrayptrcompat ... ok compiling (cpp) arrayptrcompat ... ok compiling (c) arraytoptrarg ... ok compiling (cpp) arraytoptrarg ... ok compiling (c) ass2longlong ... ok compiling (cpp) ass2longlong ... ok compiling (c) behnel4 ... ok compiling (cpp) behnel4 ... ok compiling (c) belchenko1 ... ok compiling (cpp) belchenko1 ... ok compiling (c) builtin ... ok compiling (cpp) builtin ... ok compiling (c) builtinfuncs ... ok compiling (cpp) builtinfuncs ... ok compiling (c) c_directives ... ok compiling (cpp) c_directives ... ok compiling (c) callingconvention ... ok compiling (cpp) callingconvention ... ok compiling (c) cargdef ... ok compiling (cpp) cargdef ... ok compiling (c) carrdecl ... ok compiling (cpp) carrdecl ... ok compiling (c) cascmp ... ok compiling (cpp) cascmp ... ok compiling (c) cassign ... ok compiling (cpp) cassign ... ok compiling (c) casttoexttype ... ok compiling (cpp) casttoexttype ... ok compiling (c) cdefexternfromstar ... ok compiling (cpp) cdefexternfromstar ... ok compiling (c) cenum ... ok compiling (cpp) cenum ... ok compiling (c) cforfromloop ... ok compiling (cpp) cforfromloop ... ok compiling (c) classmethargdefault ... ok compiling (cpp) classmethargdefault ... ok compiling (c) cnamespec ... cnamespec.c:270: warning: ?c_spam? defined but not used ok compiling (cpp) cnamespec ... ok compiling (c) cnumop ... ok compiling (cpp) cnumop ... ok compiling (c) coercearraytoptr ... ok compiling (cpp) coercearraytoptr ... ok compiling (c) coercetovoidptr ... ok compiling (cpp) coercetovoidptr ... ok compiling (c) complexbasetype ... ok compiling (cpp) complexbasetype ... ok compiling (c) constexpr ... ok compiling (cpp) constexpr ... ok compiling (c) coventry1 ... ok compiling (cpp) coventry1 ... ok compiling (c) cpdef ... ok compiling (cpp) cpdef ... ok compiling (c) cstructreturn ... ok compiling (cpp) cstructreturn ... ok compiling (c) ctypedef ... ok compiling (cpp) ctypedef ... ok compiling (c) ctypedefclass ... ok compiling (cpp) ctypedefclass ... ok compiling (c) ctypedefenum ... ok compiling (cpp) ctypedefenum ... ok compiling (c) ctypedefpubapi ... ok compiling (cpp) ctypedefpubapi ... ok compiling (c) ctypedefstruct ... ok compiling (cpp) ctypedefstruct ... ok compiling (c) ctypedefunion ... ok compiling (cpp) ctypedefunion ... ok compiling (c) cunsignedlong ... ok compiling (cpp) cunsignedlong ... ok compiling (c) cverylongtypes ... ok compiling (cpp) cverylongtypes ... ok compiling (c) declandimpl ... ok compiling (cpp) declandimpl ... ok compiling (c) declarations ... ok compiling (cpp) declarations ... ok compiling (c) del ... ok compiling (cpp) del ... ok compiling (c) delslice ... ok compiling (cpp) delslice ... ok compiling (c) docstrings ... ok compiling (cpp) docstrings ... ok compiling (c) doda1 ... doda1.c: In function ?__pyx_f_5doda1_foo?: doda1.c:285: warning: value computed is not used ok compiling (cpp) doda1 ... ok compiling (c) dotted_cimport ... ok compiling (cpp) dotted_cimport ... ok compiling (c) drake1 ... ok compiling (cpp) drake1 ... ok compiling (c) emptytry ... ok compiling (cpp) emptytry ... ok compiling (c) enumintcompat ... ok compiling (cpp) enumintcompat ... ok compiling (c) eqcmp ... ok compiling (cpp) eqcmp ... ok compiling (c) ewing1 ... ok compiling (cpp) ewing1 ... ok compiling (c) ewing3 ... ok compiling (cpp) ewing3 ... ok compiling (c) ewing4 ... ok compiling (cpp) ewing4 ... ok compiling (c) ewing5 ... ok compiling (cpp) ewing5 ... ok compiling (c) ewing6 ... ok compiling (cpp) ewing6 ... ok compiling (c) ewing7 ... ok compiling (cpp) ewing7 ... ok compiling (c) ewing8 ... ok compiling (cpp) ewing8 ... ok compiling (c) ewing9 ... ok compiling (cpp) ewing9 ... ok compiling (c) excvalcheck ... ok compiling (cpp) excvalcheck ... ok compiling (c) excvaldecl ... ok compiling (cpp) excvaldecl ... ok compiling (c) excvalreturn ... ok compiling (cpp) excvalreturn ... ok compiling (c) extargdefault ... ok compiling (cpp) extargdefault ... ok compiling (c) extcmethcall ... ok compiling (cpp) extcmethcall ... ok compiling (c) extcoerce ... ok compiling (cpp) extcoerce ... ok compiling (c) extdelattr ... ok compiling (cpp) extdelattr ... ok compiling (c) extdelitem ... ok compiling (cpp) extdelitem ... ok compiling (c) extdelslice ... ok compiling (cpp) extdelslice ... ok compiling (c) extdescrdel ... ok compiling (cpp) extdescrdel ... ok compiling (c) extdescrget ... ok compiling (cpp) extdescrget ... ok compiling (c) extdescrset ... ok compiling (cpp) extdescrset ... ok compiling (c) extern ... ok compiling (cpp) extern ... ok compiling (c) extexttype ... ok compiling (cpp) extexttype ... ok compiling (c) extforward ... ok compiling (cpp) extforward ... ok compiling (c) extgetattr ... ok compiling (cpp) extgetattr ... ok compiling (c) extgetitem ... ok compiling (cpp) extgetitem ... ok compiling (c) exthash ... ok compiling (cpp) exthash ... ok compiling (c) extimported ... ok compiling (cpp) extimported ... ok compiling (c) extimportedsubtype ... ok compiling (cpp) extimportedsubtype ... ok compiling (c) extindex ... ok compiling (cpp) extindex ... ok compiling (c) extinheritdel ... ok compiling (cpp) extinheritdel ... ok compiling (c) extinheritset ... ok compiling (cpp) extinheritset ... ok compiling (c) extpropertyall ... ok compiling (cpp) extpropertyall ... ok compiling (c) extpropertydel ... ok compiling (cpp) extpropertydel ... ok compiling (c) extpropertydoc ... ok compiling (cpp) extpropertydoc ... ok compiling (c) extpropertyget ... ok compiling (cpp) extpropertyget ... ok compiling (c) extpropertyset ... ok compiling (cpp) extpropertyset ... ok compiling (c) extpymemberdef ... ok compiling (cpp) extpymemberdef ... ok compiling (c) extsetattr ... ok compiling (cpp) extsetattr ... ok compiling (c) extsetitem ... ok compiling (cpp) extsetitem ... ok compiling (c) extsetslice ... ok compiling (cpp) extsetslice ... ok compiling (c) for ... ok compiling (cpp) for ... ok compiling (c) forfromelse ... ok compiling (cpp) forfromelse ... ok compiling (c) formfeed ... ok compiling (cpp) formfeed ... ok compiling (c) fromimport ... ok compiling (cpp) fromimport ... ok compiling (c) funcptr ... funcptr.c:241: warning: ?__pyx_f_7funcptr_grail? defined but not used ok compiling (cpp) funcptr ... ok compiling (c) gencall ... ok compiling (cpp) gencall ... ok compiling (c) globalonly ... ok compiling (cpp) globalonly ... ok compiling (c) globalstmt ... ok compiling (cpp) globalstmt ... ok compiling (c) globvardef ... ok compiling (cpp) globvardef ... ok compiling (c) gustafsson2 ... ok compiling (cpp) gustafsson2 ... ok compiling (c) hinsen2 ... ok compiling (cpp) hinsen2 ... ok compiling (c) huss2 ... ok compiling (cpp) huss2 ... ok compiling (c) ia_cdefblock ... ok compiling (cpp) ia_cdefblock ... ok compiling (c) import ... ok compiling (cpp) import ... ok compiling (c) index ... ok compiling (cpp) index ... ok compiling (c) indices ... ok compiling (cpp) indices ... ok compiling (c) ishimoto1 ... ok compiling (cpp) ishimoto1 ... ok compiling (c) ishimoto4 ... ok compiling (cpp) ishimoto4 ... ok compiling (c) jiba3 ... ok compiling (cpp) jiba3 ... ok compiling (c) jiba4 ... ok compiling (cpp) jiba4 ... ok compiling (c) jiba5 ... ok compiling (cpp) jiba5 ... ok compiling (c) jiba6 ... ok compiling (cpp) jiba6 ... ok compiling (c) johnson1 ... ok compiling (cpp) johnson1 ... ok compiling (c) johnson2 ... ok compiling (cpp) johnson2 ... ok compiling (c) khavkine1 ... ok compiling (cpp) khavkine1 ... ok compiling (c) kleckner1 ... ok compiling (cpp) kleckner1 ... ok compiling (c) lepage_2 ... ok compiling (cpp) lepage_2 ... ok compiling (c) longunsigned ... ok compiling (cpp) longunsigned ... ok compiling (c) magcmp ... ok compiling (cpp) magcmp ... ok compiling (c) nogil ... ok compiling (cpp) nogil ... ok compiling (c) nonctypedefclass ... ok compiling (cpp) nonctypedefclass ... ok compiling (c) none ... ok compiling (cpp) none ... ok compiling (c) notnonearg ... ok compiling (cpp) notnonearg ... ok compiling (c) nullptr ... ok compiling (cpp) nullptr ... ok compiling (c) omittedargnames ... ok compiling (cpp) omittedargnames ... ok compiling (c) pinard4 ... ok compiling (cpp) pinard4 ... ok compiling (c) pyclass ... ok compiling (cpp) pyclass ... ok compiling (c) pylong ... ok compiling (cpp) pylong ... ok compiling (c) r_pernici1 ... ok compiling (cpp) r_pernici1 ... ok compiling (c) signedtypes ... ok compiling (cpp) signedtypes ... ok compiling (c) slicex ... ok compiling (cpp) slicex ... ok compiling (c) specmethargdefault ... ok compiling (cpp) specmethargdefault ... ok compiling (c) specmethdocstring ... specmethdocstring.c:265: warning: ?__pyx_doc_17specmethdocstring_1C___init__? defined but not used specmethdocstring.c:291: warning: ?__pyx_doc_17specmethdocstring_1C_3foo___get__? defined but not used specmethdocstring.c:314: warning: ?__pyx_doc_17specmethdocstring_1C_3foo___set__? defined but not used ok compiling (cpp) specmethdocstring ... specmethdocstring.cpp:265: warning: ?__pyx_doc_17specmethdocstring_1C___init__? defined but not used specmethdocstring.cpp:291: warning: ?__pyx_doc_17specmethdocstring_1C_3foo___get__? defined but not used specmethdocstring.cpp:314: warning: ?__pyx_doc_17specmethdocstring_1C_3foo___set__? defined but not used ok compiling (c) specmethextarg ... ok compiling (cpp) specmethextarg ... ok compiling (c) traceback ... ok compiling (cpp) traceback ... ok compiling (c) tryexcept ... ok compiling (cpp) tryexcept ... ok compiling (c) tryfinally ... ok compiling (cpp) tryfinally ... ok compiling (c) typecast ... ok compiling (cpp) typecast ... ok compiling (c) types_and_names ... ok compiling (cpp) types_and_names ... ok compiling (c) watts2 ... ok compiling (cpp) watts2 ... ok compiling (c) while ... ok compiling (cpp) while ... ok compiling (c) withgil ... ok compiling (cpp) withgil ... ok compiling (c) callargs ... ok compiling (c) cdefkwargs ... ok compiling (c) cdefoptargs ... ok compiling (c) cdefspecial ... ok compiling (c) cmethbasematch ... ok compiling (c) e_addop ... ok compiling (c) e_argdefault ... ok compiling (c) e_ass ... ok compiling (c) e_assnone ... ok compiling (c) e_badexcvaltype ... ok compiling (c) e_badfuncargtype ... ok compiling (c) e_badpyparam ... ok compiling (c) e_badtypeuse ... ok compiling (c) e_bitop ... ok compiling (c) e_boolcoerce ... ok compiling (c) e_bufaccess ... ok compiling (c) e_bufaccess2 ... ok compiling (c) e_callnonfunction ... ok compiling (c) e_cdef_missing_declarator ... ok compiling (c) e_cdefassign ... ok compiling (c) e_cdefemptysue ... ok compiling (c) e_cenum ... ok compiling (c) e_cmethbasematch ... ok compiling (c) e_cmp ... ok compiling (c) e_cstruct ... ok compiling (c) e_ctypedefforward ... ok compiling (c) e_ctypedefornot ... ok compiling (c) e_declarations ... ok compiling (c) e_decorators ... ok compiling (c) e_del ... ok compiling (c) e_directives ... ok compiling (c) e_exceptclause ... ok compiling (c) e_excvalfunctype ... ok compiling (c) e_extweakref ... ok compiling (c) e_func_in_pxd ... ok compiling (c) e_index ... ok compiling (c) e_multass ... ok compiling (c) e_nargs ... ok compiling (c) e_nogilcmeth ... ok compiling (c) e_nogilfunctype ... ok compiling (c) e_notnone ... ok compiling (c) e_notnone2 ... ok compiling (c) e_numop ... ok compiling (c) e_powop ... ok compiling (c) e_pxdimpl ... ok compiling (c) e_pyobinstruct ... ok compiling (c) e_redeclmeth ... ok compiling (c) e_return ... ok compiling (c) e_sizeofincomplete ... ok compiling (c) e_slice ... ok compiling (c) e_strcoerce ... ok compiling (c) e_subop ... ok compiling (c) e_switch ... ok compiling (c) e_tempcast ... ok compiling (c) e_undefexttype ... ok compiling (c) e_unop ... ok compiling (c) e_while ... ok compiling (c) empty ... ok compiling (c) encoding ... ok compiling (c) extclassattrsetting ... ok compiling (c) futurebraces ... ok compiling (c) invalid_cast ... ok compiling (c) literal_lists ... ok compiling (c) nogil ... ok compiling (c) nogilcmeth ... ok compiling (c) nogilfunctype ... ok compiling (c) nonconst_def ... ok compiling (c) se_badindent ... ok compiling (c) se_badindent2 ... ok compiling (c) se_conddef ... ok compiling (c) se_mixtabspace ... ok compiling (c) se_multass ... ok compiling (c) se_nestdef ... ok compiling (c) undefinedname ... ok compiling (c) void_as_arg ... ok compiling (c) and running __getattribute__ ... running doctests in __getattribute__ ... Doctest: __getattribute__ ... ok compiling (cpp) and running __getattribute__ ... running doctests in __getattribute__ ... Doctest: __getattribute__ ... ok compiling (c) and running __getattribute_subclasses__ ... running doctests in __getattribute_subclasses__ ... Doctest: __getattribute_subclasses__ ... ok compiling (cpp) and running __getattribute_subclasses__ ... running doctests in __getattribute_subclasses__ ... Doctest: __getattribute_subclasses__ ... ok compiling (c) and running addloop ... running doctests in addloop ... Doctest: addloop ... ok compiling (cpp) and running addloop ... running doctests in addloop ... Doctest: addloop ... ok compiling (c) and running addop ... running doctests in addop ... Doctest: addop ... ok compiling (cpp) and running addop ... running doctests in addop ... Doctest: addop ... ok compiling (c) and running addressof ... running doctests in addressof ... Doctest: addressof ... ok compiling (cpp) and running addressof ... running doctests in addressof ... Doctest: addressof ... ok compiling (c) and running altet2 ... running doctests in altet2 ... Doctest: altet2 ... ok compiling (cpp) and running altet2 ... running doctests in altet2 ... Doctest: altet2 ... ok compiling (c) and running and ... running doctests in and ... Doctest: and ... ok compiling (cpp) and running and ... running doctests in and ... Doctest: and ... ok compiling (c) and running anonymousenum ... running doctests in anonymousenum ... Doctest: anonymousenum ... ok compiling (cpp) and running anonymousenum ... running doctests in anonymousenum ... Doctest: anonymousenum ... ok compiling (c) and running append ... running doctests in append ... Doctest: append ... ok compiling (cpp) and running append ... running doctests in append ... Doctest: append ... ok compiling (c) and running arrayassign ... running doctests in arrayassign ... Doctest: arrayassign ... ok compiling (cpp) and running arrayassign ... running doctests in arrayassign ... Doctest: arrayassign ... ok compiling (c) and running ass2cglobal ... running doctests in ass2cglobal ... Doctest: ass2cglobal ... ok compiling (cpp) and running ass2cglobal ... running doctests in ass2cglobal ... Doctest: ass2cglobal ... ok compiling (c) and running ass2global ... running doctests in ass2global ... Doctest: ass2global ... ok compiling (cpp) and running ass2global ... running doctests in ass2global ... Doctest: ass2global ... ok compiling (c) and running ass2local ... running doctests in ass2local ... Doctest: ass2local ... ok compiling (cpp) and running ass2local ... running doctests in ass2local ... Doctest: ass2local ... ok compiling (c) and running assert ... running doctests in assert ... Doctest: assert ... ok compiling (cpp) and running assert ... running doctests in assert ... Doctest: assert ... ok compiling (c) and running attr ... running doctests in attr ... Doctest: attr ... ok compiling (cpp) and running attr ... running doctests in attr ... Doctest: attr ... ok compiling (c) and running baas3 ... running doctests in baas3 ... Doctest: baas3 ... ok compiling (cpp) and running baas3 ... running doctests in baas3 ... Doctest: baas3 ... ok compiling (c) and running backquote ... running doctests in backquote ... Doctest: backquote ... ok compiling (cpp) and running backquote ... running doctests in backquote ... Doctest: backquote ... ok compiling (c) and running behnel1 ... behnel1.c: In function ?__pyx_f_7behnel1_spam?: behnel1.c:337: warning: value computed is not used running doctests in behnel1 ... Doctest: behnel1 ... ok compiling (cpp) and running behnel1 ... running doctests in behnel1 ... Doctest: behnel1 ... ok compiling (c) and running behnel2 ... running doctests in behnel2 ... Doctest: behnel2 ... ok compiling (cpp) and running behnel2 ... running doctests in behnel2 ... Doctest: behnel2 ... ok compiling (c) and running behnel3 ... running doctests in behnel3 ... Doctest: behnel3 ... ok compiling (cpp) and running behnel3 ... running doctests in behnel3 ... Doctest: behnel3 ... ok compiling (c) and running big_indices ... running doctests in big_indices ... Doctest: big_indices ... ok compiling (cpp) and running big_indices ... running doctests in big_indices ... Doctest: big_indices ... ok compiling (c) and running bint ... running doctests in bint ... Doctest: bint ... ok compiling (cpp) and running bint ... running doctests in bint ... Doctest: bint ... ok compiling (c) and running bishop1 ... running doctests in bishop1 ... Doctest: bishop1 ... ok compiling (cpp) and running bishop1 ... running doctests in bishop1 ... Doctest: bishop1 ... ok compiling (c) and running bishop2 ... running doctests in bishop2 ... Doctest: bishop2 ... ok compiling (cpp) and running bishop2 ... running doctests in bishop2 ... Doctest: bishop2 ... ok compiling (c) and running boolop ... running doctests in boolop ... Doctest: boolop ... ok compiling (cpp) and running boolop ... running doctests in boolop ... Doctest: boolop ... ok compiling (c) and running bufaccess ... bufaccess.c: In function ?__pyx_pf_9bufaccess_printbuf?: bufaccess.c:1458: warning: unused variable ?__pyx_boffset_1_buf? bufaccess.c:1457: warning: unused variable ?__pyx_boffset_0_buf? bufaccess.c:1456: warning: unused variable ?__pyx_bshape_1_buf? bufaccess.c:1455: warning: unused variable ?__pyx_bshape_0_buf? bufaccess.c:1454: warning: unused variable ?__pyx_bstride_1_buf? bufaccess.c:1453: warning: unused variable ?__pyx_bstride_0_buf? running doctests in bufaccess ... Doctest: bufaccess.__test__.acquire_failure1 ... ok Doctest: bufaccess.__test__.acquire_failure2 ... ok Doctest: bufaccess.__test__.acquire_failure3 ... ok Doctest: bufaccess.__test__.acquire_failure4 ... ok Doctest: bufaccess.__test__.acquire_failure5 ... ok Doctest: bufaccess.__test__.acquire_nonbuffer1 ... ok Doctest: bufaccess.__test__.acquire_nonbuffer2 ... ok Doctest: bufaccess.__test__.acquire_raise ... ok Doctest: bufaccess.__test__.acquire_release ... ok Doctest: bufaccess.__test__.alignment_string ... ok Doctest: bufaccess.__test__.as_argument ... ok Doctest: bufaccess.__test__.as_argument_defval ... ok Doctest: bufaccess.__test__.assign_temporary_to_object ... ok Doctest: bufaccess.__test__.assign_to_object ... ok Doctest: bufaccess.__test__.basic_struct ... ok Doctest: bufaccess.__test__.bufdefaults1 ... ok Doctest: bufaccess.__test__.buffer_cast ... ok Doctest: bufaccess.__test__.buffer_cast_fails ... ok Doctest: bufaccess.__test__.c_contig ... ok Doctest: bufaccess.__test__.c_contig_2d ... ok Doctest: bufaccess.__test__.cascaded_buffer_assignment ... ok Doctest: bufaccess.__test__.cdef_assignment ... ok Doctest: bufaccess.__test__.complex_struct_dtype ... ok Doctest: bufaccess.__test__.complex_struct_inplace ... ok Doctest: bufaccess.__test__.explicitly_release_buffer ... ok Doctest: bufaccess.__test__.f_contig ... ok Doctest: bufaccess.__test__.f_contig_2d ... ok Doctest: bufaccess.__test__.fmtst1 ... ok Doctest: bufaccess.__test__.fmtst2 ... ok Doctest: bufaccess.__test__.forin_assignment ... ok Doctest: bufaccess.__test__.get_int_2d ... ok Doctest: bufaccess.__test__.get_int_2d_uintindex ... ok Doctest: bufaccess.__test__.inplace_operators ... ok Doctest: bufaccess.__test__.int_and_long_are_same ... ok Doctest: bufaccess.__test__.list_comprehension ... ok Doctest: bufaccess.__test__.mixed_complex_struct_dtype ... ok Doctest: bufaccess.__test__.mixed_get ... ok Doctest: bufaccess.__test__.ndim1 ... ok Doctest: bufaccess.__test__.nested_struct ... ok Doctest: bufaccess.__test__.no_negative_indices ... ok Doctest: bufaccess.__test__.printbuf_float ... ok Doctest: bufaccess.__test__.printbuf_int_2d ... ok Doctest: bufaccess.__test__.printbuf_object ... ok Doctest: bufaccess.__test__.printbuf_td_cy_int ... ok Doctest: bufaccess.__test__.printbuf_td_h_cy_short ... ok Doctest: bufaccess.__test__.printbuf_td_h_double ... ok Doctest: bufaccess.__test__.printbuf_td_h_short ... ok Doctest: bufaccess.__test__.printbuf_td_h_ushort ... ok Doctest: bufaccess.__test__.readonly ... ok Doctest: bufaccess.__test__.safe_get ... ok Doctest: bufaccess.__test__.set_int_2d ... ok Doctest: bufaccess.__test__.strided ... ok Doctest: bufaccess.__test__.tuple_buffer_assignment1 ... ok Doctest: bufaccess.__test__.tuple_buffer_assignment2 ... ok Doctest: bufaccess.__test__.typedbuffer1 ... ok Doctest: bufaccess.__test__.typedbuffer2 ... ok Doctest: bufaccess.__test__.unsafe_get ... ok Doctest: bufaccess.__test__.unsafe_get_nonegative ... ok Doctest: bufaccess.__test__.writable ... ok Doctest: bufaccess.__test__.wrong_string ... ok compiling (cpp) and running bufaccess ... bufaccess.cpp: In function ?PyObject* __pyx_pf_9bufaccess_printbuf(PyObject*, PyObject*)?: bufaccess.cpp:1453: warning: unused variable ?__pyx_bstride_0_buf? bufaccess.cpp:1454: warning: unused variable ?__pyx_bstride_1_buf? bufaccess.cpp:1455: warning: unused variable ?__pyx_bshape_0_buf? bufaccess.cpp:1456: warning: unused variable ?__pyx_bshape_1_buf? bufaccess.cpp:1457: warning: unused variable ?__pyx_boffset_0_buf? bufaccess.cpp:1458: warning: unused variable ?__pyx_boffset_1_buf? bufaccess.cpp: In function ?PyObject* __pyx_pf_9bufaccess_get_int_2d_uintindex(PyObject*, PyObject*, PyObject*)?: bufaccess.cpp:5647: warning: comparison between signed and unsigned integer expressions bufaccess.cpp:5648: warning: comparison between signed and unsigned integer expressions running doctests in bufaccess ... Doctest: bufaccess.__test__.acquire_failure1 ... ok Doctest: bufaccess.__test__.acquire_failure2 ... ok Doctest: bufaccess.__test__.acquire_failure3 ... ok Doctest: bufaccess.__test__.acquire_failure4 ... ok Doctest: bufaccess.__test__.acquire_failure5 ... ok Doctest: bufaccess.__test__.acquire_nonbuffer1 ... ok Doctest: bufaccess.__test__.acquire_nonbuffer2 ... ok Doctest: bufaccess.__test__.acquire_raise ... ok Doctest: bufaccess.__test__.acquire_release ... ok Doctest: bufaccess.__test__.alignment_string ... ok Doctest: bufaccess.__test__.as_argument ... ok Doctest: bufaccess.__test__.as_argument_defval ... ok Doctest: bufaccess.__test__.assign_temporary_to_object ... ok Doctest: bufaccess.__test__.assign_to_object ... ok Doctest: bufaccess.__test__.basic_struct ... ok Doctest: bufaccess.__test__.bufdefaults1 ... ok Doctest: bufaccess.__test__.buffer_cast ... ok Doctest: bufaccess.__test__.buffer_cast_fails ... ok Doctest: bufaccess.__test__.c_contig ... ok Doctest: bufaccess.__test__.c_contig_2d ... ok Doctest: bufaccess.__test__.cascaded_buffer_assignment ... ok Doctest: bufaccess.__test__.cdef_assignment ... ok Doctest: bufaccess.__test__.complex_struct_dtype ... ok Doctest: bufaccess.__test__.complex_struct_inplace ... ok Doctest: bufaccess.__test__.explicitly_release_buffer ... ok Doctest: bufaccess.__test__.f_contig ... ok Doctest: bufaccess.__test__.f_contig_2d ... ok Doctest: bufaccess.__test__.fmtst1 ... ok Doctest: bufaccess.__test__.fmtst2 ... ok Doctest: bufaccess.__test__.forin_assignment ... ok Doctest: bufaccess.__test__.get_int_2d ... ok Doctest: bufaccess.__test__.get_int_2d_uintindex ... ok Doctest: bufaccess.__test__.inplace_operators ... ok Doctest: bufaccess.__test__.int_and_long_are_same ... ok Doctest: bufaccess.__test__.list_comprehension ... ok Doctest: bufaccess.__test__.mixed_complex_struct_dtype ... ok Doctest: bufaccess.__test__.mixed_get ... ok Doctest: bufaccess.__test__.ndim1 ... ok Doctest: bufaccess.__test__.nested_struct ... ok Doctest: bufaccess.__test__.no_negative_indices ... ok Doctest: bufaccess.__test__.printbuf_float ... ok Doctest: bufaccess.__test__.printbuf_int_2d ... ok Doctest: bufaccess.__test__.printbuf_object ... ok Doctest: bufaccess.__test__.printbuf_td_cy_int ... ok Doctest: bufaccess.__test__.printbuf_td_h_cy_short ... ok Doctest: bufaccess.__test__.printbuf_td_h_double ... ok Doctest: bufaccess.__test__.printbuf_td_h_short ... ok Doctest: bufaccess.__test__.printbuf_td_h_ushort ... ok Doctest: bufaccess.__test__.readonly ... ok Doctest: bufaccess.__test__.safe_get ... ok Doctest: bufaccess.__test__.set_int_2d ... ok Doctest: bufaccess.__test__.strided ... ok Doctest: bufaccess.__test__.tuple_buffer_assignment1 ... ok Doctest: bufaccess.__test__.tuple_buffer_assignment2 ... ok Doctest: bufaccess.__test__.typedbuffer1 ... ok Doctest: bufaccess.__test__.typedbuffer2 ... ok Doctest: bufaccess.__test__.unsafe_get ... ok Doctest: bufaccess.__test__.unsafe_get_nonegative ... ok Doctest: bufaccess.__test__.writable ... ok Doctest: bufaccess.__test__.wrong_string ... ok compiling (c) and running buffer ... buffer.c:329: warning: ?__pyx_pf_6buffer_10TestBuffer___getbuffer__? defined but not used buffer.c:478: warning: ?__pyx_pf_6buffer_17TestBufferRelease___releasebuffer__? defined but not used running doctests in buffer ... Doctest: buffer ... ok compiling (cpp) and running buffer ... buffer.cpp:329: warning: ?int __pyx_pf_6buffer_10TestBuffer___getbuffer__(PyObject*, Py_buffer*, int)? defined but not used buffer.cpp:478: warning: ?void __pyx_pf_6buffer_17TestBufferRelease___releasebuffer__(PyObject*, Py_buffer*)? defined but not used running doctests in buffer ... Doctest: buffer ... ok compiling (c) and running builtinnames ... running doctests in builtinnames ... Doctest: builtinnames ... ok compiling (cpp) and running builtinnames ... running doctests in builtinnames ... Doctest: builtinnames ... ok compiling (c) and running call_crash ... call_crash.c: In function ?__pyx_f_10call_crash_first_call?: call_crash.c:443: warning: value computed is not used running doctests in call_crash ... Doctest: call_crash ... ok compiling (cpp) and running call_crash ... running doctests in call_crash ... Doctest: call_crash ... ok compiling (c) and running callargs ... running doctests in callargs ... Doctest: callargs ... ok compiling (cpp) and running callargs ... running doctests in callargs ... Doctest: callargs ... ok compiling (c) and running carrays ... running doctests in carrays ... Doctest: carrays ... ok compiling (cpp) and running carrays ... running doctests in carrays ... Doctest: carrays ... ok compiling (c) and running cdef_opt ... running doctests in cdef_opt ... Doctest: cdef_opt ... ok compiling (cpp) and running cdef_opt ... running doctests in cdef_opt ... Doctest: cdef_opt ... ok compiling (c) and running cdefassign ... running doctests in cdefassign ... Doctest: cdefassign ... ok compiling (cpp) and running cdefassign ... running doctests in cdefassign ... Doctest: cdefassign ... ok compiling (c) and running cdefoptargs ... running doctests in cdefoptargs ... Doctest: cdefoptargs ... ok compiling (cpp) and running cdefoptargs ... running doctests in cdefoptargs ... Doctest: cdefoptargs ... ok compiling (c) and running cfuncdef ... running doctests in cfuncdef ... Doctest: cfuncdef ... ok compiling (cpp) and running cfuncdef ... running doctests in cfuncdef ... Doctest: cfuncdef ... ok compiling (c) and running charencoding ... running doctests in charencoding ... Doctest: charencoding ... FAIL compiling (cpp) and running charencoding ... running doctests in charencoding ... Doctest: charencoding ... FAIL compiling (c) and running charescape ... running doctests in charescape ... Doctest: charescape ... ok compiling (cpp) and running charescape ... running doctests in charescape ... Doctest: charescape ... ok compiling (c) and running cintop ... running doctests in cintop ... Doctest: cintop ... ok compiling (cpp) and running cintop ... running doctests in cintop ... Doctest: cintop ... ok compiling (c) and running classbody_exec ... running doctests in classbody_exec ... Doctest: classbody_exec ... ok compiling (cpp) and running classbody_exec ... running doctests in classbody_exec ... Doctest: classbody_exec ... ok compiling (c) and running classkwonlyargs ... running doctests in classkwonlyargs ... Doctest: classkwonlyargs ... ok compiling (cpp) and running classkwonlyargs ... running doctests in classkwonlyargs ... Doctest: classkwonlyargs ... ok compiling (c) and running classmethod ... running doctests in classmethod ... Doctest: classmethod ... ok compiling (cpp) and running classmethod ... running doctests in classmethod ... Doctest: classmethod ... ok compiling (c) and running classpass ... running doctests in classpass ... Doctest: classpass ... ok compiling (cpp) and running classpass ... running doctests in classpass ... Doctest: classpass ... ok compiling (c) and running compiledef ... running doctests in compiledef ... Doctest: compiledef ... ok compiling (cpp) and running compiledef ... running doctests in compiledef ... Doctest: compiledef ... ok compiling (c) and running concatcstrings ... running doctests in concatcstrings ... Doctest: concatcstrings ... ok compiling (cpp) and running concatcstrings ... running doctests in concatcstrings ... Doctest: concatcstrings ... ok compiling (c) and running consts ... running doctests in consts ... Doctest: consts ... ok compiling (cpp) and running consts ... running doctests in consts ... Doctest: consts ... ok compiling (c) and running cstringmeth ... running doctests in cstringmeth ... Doctest: cstringmeth ... ok compiling (cpp) and running cstringmeth ... running doctests in cstringmeth ... Doctest: cstringmeth ... ok compiling (c) and running cstringmul ... running doctests in cstringmul ... Doctest: cstringmul ... ok compiling (cpp) and running cstringmul ... running doctests in cstringmul ... Doctest: cstringmul ... ok compiling (c) and running cstruct ... running doctests in cstruct ... Doctest: cstruct ... ok compiling (cpp) and running cstruct ... running doctests in cstruct ... Doctest: cstruct ... ok compiling (c) and running ct_DEF ... running doctests in ct_DEF ... Doctest: ct_DEF ... ok compiling (cpp) and running ct_DEF ... running doctests in ct_DEF ... Doctest: ct_DEF ... ok compiling (c) and running ct_IF ... running doctests in ct_IF ... Doctest: ct_IF ... ok compiling (cpp) and running ct_IF ... running doctests in ct_IF ... Doctest: ct_IF ... ok compiling (c) and running cunion ... running doctests in cunion ... Doctest: cunion ... ok compiling (cpp) and running cunion ... running doctests in cunion ... Doctest: cunion ... ok compiling (c) and running cvardef ... running doctests in cvardef ... Doctest: cvardef ... ok compiling (cpp) and running cvardef ... running doctests in cvardef ... Doctest: cvardef ... ok compiling (c) and running decorators ... running doctests in decorators ... Doctest: decorators ... ok compiling (cpp) and running decorators ... running doctests in decorators ... Doctest: decorators ... ok compiling (c) and running dict ... running doctests in dict ... Doctest: dict ... ok compiling (cpp) and running dict ... running doctests in dict ... Doctest: dict ... ok compiling (c) and running dictcomp ... running doctests in dictcomp ... Doctest: dictcomp ... ok compiling (cpp) and running dictcomp ... running doctests in dictcomp ... Doctest: dictcomp ... ok compiling (c) and running dictintindex ... running doctests in dictintindex ... Doctest: dictintindex ... ok compiling (cpp) and running dictintindex ... running doctests in dictintindex ... Doctest: dictintindex ... ok compiling (c) and running dietachmayer1 ... running doctests in dietachmayer1 ... Doctest: dietachmayer1 ... ok compiling (cpp) and running dietachmayer1 ... running doctests in dietachmayer1 ... Doctest: dietachmayer1 ... ok compiling (c) and running embedsignatures ... embedsignatures.c: In function ?__pyx_f_15embedsignatures_3Ext_clone?: embedsignatures.c:1701: warning: value computed is not used running doctests in embedsignatures ... Doctest: embedsignatures ... ok compiling (cpp) and running embedsignatures ... running doctests in embedsignatures ... Doctest: embedsignatures ... ok compiling (c) and running exarkun ... running doctests in exarkun ... Doctest: exarkun ... ok compiling (cpp) and running exarkun ... running doctests in exarkun ... Doctest: exarkun ... ok compiling (c) and running exceptionpropagation ... running doctests in exceptionpropagation ... Doctest: exceptionpropagation ... ok compiling (cpp) and running exceptionpropagation ... running doctests in exceptionpropagation ... Doctest: exceptionpropagation ... ok compiling (c) and running exceptionrefcount ... running doctests in exceptionrefcount ... Doctest: exceptionrefcount ... ok compiling (cpp) and running exceptionrefcount ... running doctests in exceptionrefcount ... Doctest: exceptionrefcount ... ok compiling (c) and running exectest ... running doctests in exectest ... Doctest: exectest ... ok compiling (cpp) and running exectest ... running doctests in exectest ... Doctest: exectest ... ok compiling (c) and running extclassbody ... running doctests in extclassbody ... Doctest: extclassbody ... ok compiling (cpp) and running extclassbody ... running doctests in extclassbody ... Doctest: extclassbody ... ok compiling (c) and running extclasspass ... running doctests in extclasspass ... Doctest: extclasspass ... ok compiling (cpp) and running extclasspass ... running doctests in extclasspass ... Doctest: extclasspass ... ok compiling (c) and running extcmethod ... running doctests in extcmethod ... Doctest: extcmethod ... ok compiling (cpp) and running extcmethod ... running doctests in extcmethod ... Doctest: extcmethod ... ok compiling (c) and running extinherit ... running doctests in extinherit ... Doctest: extinherit ... ok compiling (cpp) and running extinherit ... running doctests in extinherit ... Doctest: extinherit ... ok compiling (c) and running extinstantiate ... running doctests in extinstantiate ... Doctest: extinstantiate ... ok compiling (cpp) and running extinstantiate ... running doctests in extinstantiate ... Doctest: extinstantiate ... ok compiling (c) and running extkwonlyargs ... running doctests in extkwonlyargs ... Doctest: extkwonlyargs ... ok compiling (cpp) and running extkwonlyargs ... running doctests in extkwonlyargs ... Doctest: extkwonlyargs ... ok compiling (c) and running extlen ... running doctests in extlen ... Doctest: extlen ... ok compiling (cpp) and running extlen ... running doctests in extlen ... Doctest: extlen ... ok compiling (c) and running extmember ... running doctests in extmember ... Doctest: extmember ... ok compiling (cpp) and running extmember ... running doctests in extmember ... Doctest: extmember ... ok compiling (c) and running extpropertyref ... running doctests in extpropertyref ... Doctest: extpropertyref ... ok compiling (cpp) and running extpropertyref ... running doctests in extpropertyref ... Doctest: extpropertyref ... ok compiling (c) and running extstarargs ... running doctests in extstarargs ... Doctest: extstarargs ... ok compiling (cpp) and running extstarargs ... running doctests in extstarargs ... Doctest: extstarargs ... ok compiling (c) and running exttype ... running doctests in exttype ... Doctest: exttype ... ok compiling (cpp) and running exttype ... running doctests in exttype ... Doctest: exttype ... ok compiling (c) and running filenames ... running doctests in filenames ... Doctest: filenames ... ok compiling (cpp) and running filenames ... running doctests in filenames ... Doctest: filenames ... ok compiling (c) and running flatin ... running doctests in flatin ... Doctest: flatin ... ok compiling (cpp) and running flatin ... running doctests in flatin ... Doctest: flatin ... ok compiling (c) and running fmod ... running doctests in fmod ... Doctest: fmod ... ok compiling (cpp) and running fmod ... running doctests in fmod ... Doctest: fmod ... ok compiling (c) and running forfrom ... running doctests in forfrom ... Doctest: forfrom ... ok compiling (cpp) and running forfrom ... running doctests in forfrom ... Doctest: forfrom ... ok compiling (c) and running funcexcept ... running doctests in funcexcept ... Doctest: funcexcept ... ok compiling (cpp) and running funcexcept ... running doctests in funcexcept ... Doctest: funcexcept ... ok compiling (c) and running funcexceptchained ... running doctests in funcexceptchained ... Doctest: funcexceptchained ... ok compiling (cpp) and running funcexceptchained ... running doctests in funcexceptchained ... Doctest: funcexceptchained ... ok compiling (c) and running funcexceptcypy ... running doctests in funcexceptcypy ... Doctest: funcexceptcypy ... ok compiling (cpp) and running funcexceptcypy ... running doctests in funcexceptcypy ... Doctest: funcexceptcypy ... ok compiling (c) and running funcexceptreturn ... running doctests in funcexceptreturn ... Doctest: funcexceptreturn ... ok compiling (cpp) and running funcexceptreturn ... running doctests in funcexceptreturn ... Doctest: funcexceptreturn ... ok compiling (c) and running future_division ... running doctests in future_division ... Doctest: future_division ... ok compiling (cpp) and running future_division ... running doctests in future_division ... Doctest: future_division ... ok compiling (c) and running future_unicode_literals ... running doctests in future_unicode_literals ... Doctest: future_unicode_literals ... ok compiling (cpp) and running future_unicode_literals ... running doctests in future_unicode_literals ... Doctest: future_unicode_literals ... ok compiling (c) and running getattr3call ... running doctests in getattr3call ... Doctest: getattr3call ... ok compiling (cpp) and running getattr3call ... running doctests in getattr3call ... Doctest: getattr3call ... ok compiling (c) and running if ... running doctests in if ... Doctest: if ... ok compiling (cpp) and running if ... running doctests in if ... Doctest: if ... ok compiling (c) and running importas ... running doctests in importas ... Doctest: importas ... ok compiling (cpp) and running importas ... running doctests in importas ... Doctest: importas ... ok compiling (c) and running importfrom ... running doctests in importfrom ... Doctest: importfrom ... ok compiling (cpp) and running importfrom ... running doctests in importfrom ... Doctest: importfrom ... ok compiling (c) and running include ... running doctests in include ... Doctest: include ... ok compiling (cpp) and running include ... running doctests in include ... Doctest: include ... ok compiling (c) and running inhcmethcall ... running doctests in inhcmethcall ... Doctest: inhcmethcall ... ok compiling (cpp) and running inhcmethcall ... running doctests in inhcmethcall ... Doctest: inhcmethcall ... ok compiling (c) and running inline ... running doctests in inline ... Doctest: inline ... ok compiling (cpp) and running inline ... running doctests in inline ... Doctest: inline ... ok compiling (c) and running inlinepxd ... running doctests in inlinepxd ... Doctest: inlinepxd ... ok compiling (cpp) and running inlinepxd ... running doctests in inlinepxd ... Doctest: inlinepxd ... ok compiling (c) and running inop ... running doctests in inop ... Doctest: inop ... ok compiling (cpp) and running inop ... running doctests in inop ... Doctest: inop ... ok compiling (c) and running inplace ... running doctests in inplace ... Doctest: inplace ... ok compiling (cpp) and running inplace ... running doctests in inplace ... Doctest: inplace ... ok compiling (c) and running int_literals ... running doctests in int_literals ... Doctest: int_literals ... ok compiling (cpp) and running int_literals ... running doctests in int_literals ... Doctest: int_literals ... ok compiling (c) and running ishimoto2 ... running doctests in ishimoto2 ... Doctest: ishimoto2 ... ok compiling (cpp) and running ishimoto2 ... running doctests in ishimoto2 ... Doctest: ishimoto2 ... ok compiling (c) and running ishimoto3 ... running doctests in ishimoto3 ... Doctest: ishimoto3 ... ok compiling (cpp) and running ishimoto3 ... running doctests in ishimoto3 ... Doctest: ishimoto3 ... ok compiling (c) and running isinstance ... running doctests in isinstance ... Doctest: isinstance ... ok compiling (cpp) and running isinstance ... running doctests in isinstance ... Doctest: isinstance ... ok compiling (c) and running isnonebool ... running doctests in isnonebool ... Doctest: isnonebool ... ok compiling (cpp) and running isnonebool ... running doctests in isnonebool ... Doctest: isnonebool ... ok compiling (c) and running iteratorexception ... running doctests in iteratorexception ... Doctest: iteratorexception ... ok compiling (cpp) and running iteratorexception ... running doctests in iteratorexception ... Doctest: iteratorexception ... ok compiling (c) and running iterdict ... running doctests in iterdict ... Doctest: iterdict ... ok compiling (cpp) and running iterdict ... running doctests in iterdict ... Doctest: iterdict ... ok compiling (c) and running jarausch1 ... running doctests in jarausch1 ... Doctest: jarausch1 ... ok compiling (cpp) and running jarausch1 ... running doctests in jarausch1 ... Doctest: jarausch1 ... ok compiling (c) and running king1 ... running doctests in king1 ... Doctest: king1 ... ok compiling (cpp) and running king1 ... running doctests in king1 ... Doctest: king1 ... ok compiling (c) and running kostyrka ... running doctests in kostyrka ... Doctest: kostyrka ... ok compiling (cpp) and running kostyrka ... running doctests in kostyrka ... Doctest: kostyrka ... ok compiling (c) and running kostyrka2 ... running doctests in kostyrka2 ... Doctest: kostyrka2 ... ok compiling (cpp) and running kostyrka2 ... running doctests in kostyrka2 ... Doctest: kostyrka2 ... ok compiling (c) and running kwargproblems ... running doctests in kwargproblems ... Doctest: kwargproblems ... ok compiling (cpp) and running kwargproblems ... running doctests in kwargproblems ... Doctest: kwargproblems ... ok compiling (c) and running kwonlyargs ... running doctests in kwonlyargs ... Doctest: kwonlyargs ... ok compiling (cpp) and running kwonlyargs ... running doctests in kwonlyargs ... Doctest: kwonlyargs ... ok compiling (c) and running kwonlyargscall ... running doctests in kwonlyargscall ... Doctest: kwonlyargscall ... ok compiling (cpp) and running kwonlyargscall ... running doctests in kwonlyargscall ... Doctest: kwonlyargscall ... ok compiling (c) and running lepage_1 ... running doctests in lepage_1 ... Doctest: lepage_1 ... ok compiling (cpp) and running lepage_1 ... running doctests in lepage_1 ... Doctest: lepage_1 ... ok compiling (c) and running list ... running doctests in list ... Doctest: list ... ok compiling (cpp) and running list ... running doctests in list ... Doctest: list ... ok compiling (c) and running listcomp ... running doctests in listcomp ... Doctest: listcomp ... ok compiling (cpp) and running listcomp ... running doctests in listcomp ... Doctest: listcomp ... ok compiling (c) and running literal_lists ... running doctests in literal_lists ... Doctest: literal_lists ... ok compiling (cpp) and running literal_lists ... running doctests in literal_lists ... Doctest: literal_lists ... ok compiling (c) and running literals ... running doctests in literals ... Doctest: literals ... ok compiling (cpp) and running literals ... running doctests in literals ... Doctest: literals ... ok compiling (c) and running literalslice ... running doctests in literalslice ... Doctest: literalslice ... ok compiling (cpp) and running literalslice ... running doctests in literalslice ... Doctest: literalslice ... ok compiling (c) and running locals ... running doctests in locals ... Doctest: locals ... ok compiling (cpp) and running locals ... running doctests in locals ... Doctest: locals ... ok compiling (c) and running longlongindex ... running doctests in longlongindex ... Doctest: longlongindex ... ok compiling (cpp) and running longlongindex ... running doctests in longlongindex ... Doctest: longlongindex ... ok compiling (c) and running menten1 ... running doctests in menten1 ... Doctest: menten1 ... ok compiling (cpp) and running menten1 ... running doctests in menten1 ... Doctest: menten1 ... ok compiling (c) and running modbody ... running doctests in modbody ... Doctest: modbody ... ok compiling (cpp) and running modbody ... running doctests in modbody ... Doctest: modbody ... ok compiling (c) and running modop ... running doctests in modop ... Doctest: modop ... ok compiling (cpp) and running modop ... running doctests in modop ... Doctest: modop ... ok compiling (c) and running moduletryexcept ... running doctests in moduletryexcept ... Doctest: moduletryexcept ... ok compiling (cpp) and running moduletryexcept ... running doctests in moduletryexcept ... Doctest: moduletryexcept ... ok compiling (c) and running multass ... running doctests in multass ... Doctest: multass ... ok compiling (cpp) and running multass ... running doctests in multass ... Doctest: multass ... ok compiling (c) and running new_style_exceptions ... running doctests in new_style_exceptions ... Doctest: new_style_exceptions ... ok compiling (cpp) and running new_style_exceptions ... running doctests in new_style_exceptions ... Doctest: new_style_exceptions ... ok compiling (c) and running nogil ... running doctests in nogil ... Doctest: nogil ... ok compiling (cpp) and running nogil ... running doctests in nogil ... Doctest: nogil ... ok compiling (c) and running nonecheck ... running doctests in nonecheck ... Doctest: nonecheck ... ok compiling (cpp) and running nonecheck ... running doctests in nonecheck ... Doctest: nonecheck ... ok compiling (c) and running nononetypecheck ... running doctests in nononetypecheck ... Doctest: nononetypecheck ... ok compiling (cpp) and running nononetypecheck ... running doctests in nononetypecheck ... Doctest: nononetypecheck ... ok compiling (c) and running notinop ... running doctests in notinop ... Doctest: notinop ... ok compiling (cpp) and running notinop ... running doctests in notinop ... Doctest: notinop ... ok compiling (c) and running numpy_test ... /usr/include/python2.5/numpy/__multiarray_api.h:959: warning: ?_import_array? defined but not used running doctests in numpy_test ... Doctest: numpy_test ... ok compiling (cpp) and running numpy_test ... /usr/include/python2.5/numpy/__multiarray_api.h:958: warning: ?int _import_array()? defined but not used running doctests in numpy_test ... Doctest: numpy_test ... ok compiling (c) and running or ... running doctests in or ... Doctest: or ... ok compiling (cpp) and running or ... running doctests in or ... Doctest: or ... ok compiling (c) and running pass ... running doctests in pass ... Doctest: pass ... ok compiling (cpp) and running pass ... running doctests in pass ... Doctest: pass ... ok compiling (c) and running pinard5 ... running doctests in pinard5 ... Doctest: pinard5 ... ok compiling (cpp) and running pinard5 ... running doctests in pinard5 ... Doctest: pinard5 ... ok compiling (c) and running pinard6 ... running doctests in pinard6 ... Doctest: pinard6 ... ok compiling (cpp) and running pinard6 ... running doctests in pinard6 ... Doctest: pinard6 ... ok compiling (c) and running pinard7 ... running doctests in pinard7 ... Doctest: pinard7 ... ok compiling (cpp) and running pinard7 ... running doctests in pinard7 ... Doctest: pinard7 ... ok compiling (c) and running pinard8 ... running doctests in pinard8 ... Doctest: pinard8 ... ok compiling (cpp) and running pinard8 ... running doctests in pinard8 ... Doctest: pinard8 ... ok compiling (c) and running powop ... running doctests in powop ... Doctest: powop ... ok compiling (cpp) and running powop ... running doctests in powop ... Doctest: powop ... ok compiling (c) and running print ... running doctests in print ... Doctest: print ... ok compiling (cpp) and running print ... running doctests in print ... Doctest: print ... ok compiling (c) and running pure ... pure.c: In function ?__pyx_pf_4pure_test_declare?: pure.c:555: warning: statement with no effect running doctests in pure ... Doctest: pure ... ok compiling (cpp) and running pure ... pure.cpp: In function ?PyObject* __pyx_pf_4pure_test_sizeof(PyObject*, PyObject*)?: pure.cpp:352: warning: comparison between signed and unsigned integer expressions pure.cpp:362: warning: comparison between signed and unsigned integer expressions pure.cpp: In function ?PyObject* __pyx_pf_4pure_test_declare(PyObject*, PyObject*)?: pure.cpp:555: warning: statement has no effect running doctests in pure ... Doctest: pure ... ok compiling (c) and running pycmp ... running doctests in pycmp ... Doctest: pycmp ... ok compiling (cpp) and running pycmp ... running doctests in pycmp ... Doctest: pycmp ... ok compiling (c) and running pyextattrref ... running doctests in pyextattrref ... Doctest: pyextattrref ... ok compiling (cpp) and running pyextattrref ... running doctests in pyextattrref ... Doctest: pyextattrref ... ok compiling (c) and running pyintop ... running doctests in pyintop ... Doctest: pyintop ... ok compiling (cpp) and running pyintop ... running doctests in pyintop ... Doctest: pyintop ... ok compiling (c) and running pylistsubtype ... running doctests in pylistsubtype ... Doctest: pylistsubtype ... ok compiling (cpp) and running pylistsubtype ... running doctests in pylistsubtype ... Doctest: pylistsubtype ... ok compiling (c) and running pynumop ... running doctests in pynumop ... Doctest: pynumop ... ok compiling (cpp) and running pynumop ... running doctests in pynumop ... Doctest: pynumop ... ok compiling (c) and running r_addint ... running doctests in r_addint ... Doctest: r_addint ... ok compiling (cpp) and running r_addint ... running doctests in r_addint ... Doctest: r_addint ... ok compiling (c) and running r_argdefault ... running doctests in r_argdefault ... Doctest: r_argdefault ... ok compiling (cpp) and running r_argdefault ... running doctests in r_argdefault ... Doctest: r_argdefault ... ok compiling (c) and running r_barbieri1 ... running doctests in r_barbieri1 ... Doctest: r_barbieri1 ... ok compiling (cpp) and running r_barbieri1 ... running doctests in r_barbieri1 ... Doctest: r_barbieri1 ... ok compiling (c) and running r_bishop3 ... running doctests in r_bishop3 ... Doctest: r_bishop3 ... ok compiling (cpp) and running r_bishop3 ... running doctests in r_bishop3 ... Doctest: r_bishop3 ... ok compiling (c) and running r_bowden1 ... running doctests in r_bowden1 ... Doctest: r_bowden1 ... ok compiling (cpp) and running r_bowden1 ... running doctests in r_bowden1 ... Doctest: r_bowden1 ... ok compiling (c) and running r_delgado_1 ... running doctests in r_delgado_1 ... Doctest: r_delgado_1 ... ok compiling (cpp) and running r_delgado_1 ... running doctests in r_delgado_1 ... Doctest: r_delgado_1 ... ok compiling (c) and running r_docstrings ... running doctests in r_docstrings ... Doctest: r_docstrings.__test__.test_docstrings ... ok compiling (cpp) and running r_docstrings ... running doctests in r_docstrings ... Doctest: r_docstrings.__test__.test_docstrings ... ok compiling (c) and running r_extcomplex2 ... running doctests in r_extcomplex2 ... Doctest: r_extcomplex2 ... ok compiling (cpp) and running r_extcomplex2 ... running doctests in r_extcomplex2 ... Doctest: r_extcomplex2 ... ok compiling (c) and running r_extstarargs ... running doctests in r_extstarargs ... Doctest: r_extstarargs ... ok compiling (cpp) and running r_extstarargs ... running doctests in r_extstarargs ... Doctest: r_extstarargs ... ok compiling (c) and running r_forloop ... running doctests in r_forloop ... Doctest: r_forloop ... ok compiling (cpp) and running r_forloop ... running doctests in r_forloop ... Doctest: r_forloop ... ok compiling (c) and running r_hordijk1 ... running doctests in r_hordijk1 ... Doctest: r_hordijk1 ... ok compiling (cpp) and running r_hordijk1 ... running doctests in r_hordijk1 ... Doctest: r_hordijk1 ... ok compiling (c) and running r_huss3 ... running doctests in r_huss3 ... Doctest: r_huss3 ... ok compiling (cpp) and running r_huss3 ... running doctests in r_huss3 ... Doctest: r_huss3 ... ok compiling (c) and running r_jeff_epler_1 ... running doctests in r_jeff_epler_1 ... Doctest: r_jeff_epler_1 ... ok compiling (cpp) and running r_jeff_epler_1 ... running doctests in r_jeff_epler_1 ... Doctest: r_jeff_epler_1 ... ok compiling (c) and running r_jiba1 ... running doctests in r_jiba1 ... Doctest: r_jiba1 ... ok compiling (cpp) and running r_jiba1 ... running doctests in r_jiba1 ... Doctest: r_jiba1 ... ok compiling (c) and running r_lepage_3 ... running doctests in r_lepage_3 ... Doctest: r_lepage_3 ... ok compiling (cpp) and running r_lepage_3 ... running doctests in r_lepage_3 ... Doctest: r_lepage_3 ... ok compiling (c) and running r_mang1 ... running doctests in r_mang1 ... Doctest: r_mang1 ... ok compiling (cpp) and running r_mang1 ... running doctests in r_mang1 ... Doctest: r_mang1 ... ok compiling (c) and running r_mcintyre1 ... running doctests in r_mcintyre1 ... Doctest: r_mcintyre1 ... ok compiling (cpp) and running r_mcintyre1 ... running doctests in r_mcintyre1 ... Doctest: r_mcintyre1 ... ok compiling (c) and running r_mitch_chapman_2 ... running doctests in r_mitch_chapman_2 ... Doctest: r_mitch_chapman_2 ... ok compiling (cpp) and running r_mitch_chapman_2 ... running doctests in r_mitch_chapman_2 ... Doctest: r_mitch_chapman_2 ... ok compiling (c) and running r_primes ... running doctests in r_primes ... Doctest: r_primes ... ok compiling (cpp) and running r_primes ... running doctests in r_primes ... Doctest: r_primes ... ok compiling (c) and running r_print ... running doctests in r_print ... Doctest: r_print ... ok compiling (cpp) and running r_print ... running doctests in r_print ... Doctest: r_print ... ok compiling (c) and running r_pyclass ... running doctests in r_pyclass ... Doctest: r_pyclass ... ok compiling (cpp) and running r_pyclass ... running doctests in r_pyclass ... Doctest: r_pyclass ... ok compiling (c) and running r_pyclassdefault ... running doctests in r_pyclassdefault ... Doctest: r_pyclassdefault ... ok compiling (cpp) and running r_pyclassdefault ... running doctests in r_pyclassdefault ... Doctest: r_pyclassdefault ... ok compiling (c) and running r_pythonapi ... running doctests in r_pythonapi ... Doctest: r_pythonapi ... ok compiling (cpp) and running r_pythonapi ... running doctests in r_pythonapi ... Doctest: r_pythonapi ... ok compiling (c) and running r_spamtype ... running doctests in r_spamtype ... Doctest: r_spamtype ... ok compiling (cpp) and running r_spamtype ... running doctests in r_spamtype ... Doctest: r_spamtype ... ok compiling (c) and running r_starargcall ... running doctests in r_starargcall ... Doctest: r_starargcall ... ok compiling (cpp) and running r_starargcall ... running doctests in r_starargcall ... Doctest: r_starargcall ... ok compiling (c) and running r_starargs ... running doctests in r_starargs ... Doctest: r_starargs ... ok compiling (cpp) and running r_starargs ... running doctests in r_starargs ... Doctest: r_starargs ... ok compiling (c) and running r_starargsonly ... running doctests in r_starargsonly ... Doctest: r_starargsonly ... ok compiling (cpp) and running r_starargsonly ... running doctests in r_starargsonly ... Doctest: r_starargsonly ... ok compiling (c) and running r_toofewargs ... running doctests in r_toofewargs ... Doctest: r_toofewargs ... ok compiling (cpp) and running r_toofewargs ... running doctests in r_toofewargs ... Doctest: r_toofewargs ... ok compiling (c) and running r_typecast ... running doctests in r_typecast ... Doctest: r_typecast ... ok compiling (cpp) and running r_typecast ... running doctests in r_typecast ... Doctest: r_typecast ... ok compiling (c) and running r_uintindex ... running doctests in r_uintindex ... Doctest: r_uintindex ... ok compiling (cpp) and running r_uintindex ... running doctests in r_uintindex ... Doctest: r_uintindex ... ok compiling (c) and running r_vree_1 ... running doctests in r_vree_1 ... Doctest: r_vree_1 ... ok compiling (cpp) and running r_vree_1 ... running doctests in r_vree_1 ... Doctest: r_vree_1 ... ok compiling (c) and running ref2local ... running doctests in ref2local ... Doctest: ref2local ... ok compiling (cpp) and running ref2local ... running doctests in ref2local ... Doctest: ref2local ... ok compiling (c) and running return ... running doctests in return ... Doctest: return ... ok compiling (cpp) and running return ... running doctests in return ... Doctest: return ... ok compiling (c) and running returnparassign ... running doctests in returnparassign ... Doctest: returnparassign ... ok compiling (cpp) and running returnparassign ... running doctests in returnparassign ... Doctest: returnparassign ... ok compiling (c) and running rodriguez_1 ... running doctests in rodriguez_1 ... Doctest: rodriguez_1 ... ok compiling (cpp) and running rodriguez_1 ... running doctests in rodriguez_1 ... Doctest: rodriguez_1 ... ok compiling (c) and running set ... running doctests in set ... Doctest: set ... ok compiling (cpp) and running set ... running doctests in set ... Doctest: set ... ok compiling (c) and running setcomp ... running doctests in setcomp ... Doctest: setcomp ... ok compiling (cpp) and running setcomp ... running doctests in setcomp ... Doctest: setcomp ... ok compiling (c) and running simpcall ... running doctests in simpcall ... Doctest: simpcall ... ok compiling (cpp) and running simpcall ... running doctests in simpcall ... Doctest: simpcall ... ok compiling (c) and running sizeof ... running doctests in sizeof ... Doctest: sizeof ... ok compiling (cpp) and running sizeof ... running doctests in sizeof ... Doctest: sizeof ... ok compiling (c) and running slice2 ... running doctests in slice2 ... Doctest: slice2 ... ok compiling (cpp) and running slice2 ... running doctests in slice2 ... Doctest: slice2 ... ok compiling (c) and running slice3 ... running doctests in slice3 ... Doctest: slice3 ... ok compiling (cpp) and running slice3 ... running doctests in slice3 ... Doctest: slice3 ... ok compiling (c) and running slice_charptr ... running doctests in slice_charptr ... Doctest: slice_charptr ... ok compiling (cpp) and running slice_charptr ... running doctests in slice_charptr ... Doctest: slice_charptr ... ok compiling (c) and running specialfloat ... running doctests in specialfloat ... Doctest: specialfloat ... ok compiling (cpp) and running specialfloat ... running doctests in specialfloat ... Doctest: specialfloat ... ok compiling (c) and running starargs ... running doctests in starargs ... Doctest: starargs ... ok compiling (cpp) and running starargs ... running doctests in starargs ... Doctest: starargs ... ok compiling (c) and running staticmethod ... running doctests in staticmethod ... Doctest: staticmethod ... ok compiling (cpp) and running staticmethod ... running doctests in staticmethod ... Doctest: staticmethod ... ok compiling (c) and running strconstinclass ... running doctests in strconstinclass ... Doctest: strconstinclass ... ok compiling (cpp) and running strconstinclass ... running doctests in strconstinclass ... Doctest: strconstinclass ... ok compiling (c) and running strescapes ... running doctests in strescapes ... Doctest: strescapes ... ok compiling (cpp) and running strescapes ... running doctests in strescapes ... Doctest: strescapes ... ok compiling (c) and running strfunction ... running doctests in strfunction ... Doctest: strfunction ... ok compiling (cpp) and running strfunction ... running doctests in strfunction ... Doctest: strfunction ... ok compiling (c) and running strliterals ... running doctests in strliterals ... Doctest: strliterals ... ok compiling (cpp) and running strliterals ... running doctests in strliterals ... Doctest: strliterals ... ok compiling (c) and running struct_conversion ... running doctests in struct_conversion ... Doctest: struct_conversion ... ok compiling (cpp) and running struct_conversion ... running doctests in struct_conversion ... Doctest: struct_conversion ... ok compiling (c) and running subclasses ... running doctests in subclasses ... Doctest: subclasses ... ok compiling (cpp) and running subclasses ... running doctests in subclasses ... Doctest: subclasses ... ok compiling (c) and running subop ... running doctests in subop ... Doctest: subop ... ok compiling (cpp) and running subop ... running doctests in subop ... Doctest: subop ... ok compiling (c) and running switch ... running doctests in switch ... Doctest: switch ... ok compiling (cpp) and running switch ... running doctests in switch ... Doctest: switch ... ok compiling (c) and running tandemstats ... running doctests in tandemstats ... Doctest: tandemstats ... ok compiling (cpp) and running tandemstats ... running doctests in tandemstats ... Doctest: tandemstats ... ok compiling (c) and running temps_corner1 ... running doctests in temps_corner1 ... Doctest: temps_corner1 ... ok compiling (cpp) and running temps_corner1 ... running doctests in temps_corner1 ... Doctest: temps_corner1 ... ok compiling (c) and running ticket_123 ... running doctests in ticket_123 ... Doctest: ticket_123 ... ok compiling (cpp) and running ticket_123 ... running doctests in ticket_123 ... Doctest: ticket_123 ... ok compiling (c) and running ticket_124 ... running doctests in ticket_124 ... Doctest: ticket_124 ... ok compiling (cpp) and running ticket_124 ... running doctests in ticket_124 ... Doctest: ticket_124 ... ok compiling (c) and running tryfinally ... tryfinally.c: In function ?__pyx_pf_10tryfinally_try_return_temp?: tryfinally.c:436: warning: ?__pyx_r? may be used uninitialized in this function running doctests in tryfinally ... Doctest: tryfinally ... ok compiling (cpp) and running tryfinally ... tryfinally.cpp: In function ?PyObject* __pyx_pf_10tryfinally_try_return_temp(PyObject*, PyObject*)?: tryfinally.cpp:436: warning: ?__pyx_r? may be used uninitialized in this function running doctests in tryfinally ... Doctest: tryfinally ... ok compiling (c) and running tuple ... running doctests in tuple ... Doctest: tuple ... ok compiling (cpp) and running tuple ... running doctests in tuple ... Doctest: tuple ... ok compiling (c) and running tupleassign ... running doctests in tupleassign ... Doctest: tupleassign ... ok compiling (cpp) and running tupleassign ... running doctests in tupleassign ... Doctest: tupleassign ... ok compiling (c) and running tuplereassign ... running doctests in tuplereassign ... Doctest: tuplereassign ... ok compiling (cpp) and running tuplereassign ... running doctests in tuplereassign ... Doctest: tuplereassign ... ok compiling (c) and running unicodefunction ... running doctests in unicodefunction ... Doctest: unicodefunction ... ok compiling (cpp) and running unicodefunction ... running doctests in unicodefunction ... Doctest: unicodefunction ... ok compiling (c) and running unicodeliterals ... running doctests in unicodeliterals ... Doctest: unicodeliterals ... ok compiling (cpp) and running unicodeliterals ... running doctests in unicodeliterals ... Doctest: unicodeliterals ... ok compiling (c) and running unicodeliteralsdefault ... running doctests in unicodeliteralsdefault ... Doctest: unicodeliteralsdefault ... ok compiling (cpp) and running unicodeliteralsdefault ... running doctests in unicodeliteralsdefault ... Doctest: unicodeliteralsdefault ... ok compiling (c) and running unicodeliteralslatin1 ... running doctests in unicodeliteralslatin1 ... Doctest: unicodeliteralslatin1 ... ok compiling (cpp) and running unicodeliteralslatin1 ... running doctests in unicodeliteralslatin1 ... Doctest: unicodeliteralslatin1 ... ok compiling (c) and running unop ... running doctests in unop ... Doctest: unop ... ok compiling (cpp) and running unop ... running doctests in unop ... Doctest: unop ... ok compiling (c) and running unpack ... running doctests in unpack ... Doctest: unpack ... ok compiling (cpp) and running unpack ... running doctests in unpack ... Doctest: unpack ... ok compiling (c) and running unpacklistcomp ... running doctests in unpacklistcomp ... Doctest: unpacklistcomp ... ok compiling (cpp) and running unpacklistcomp ... running doctests in unpacklistcomp ... Doctest: unpacklistcomp ... ok compiling (c) and running unsigned ... running doctests in unsigned ... Doctest: unsigned ... ok compiling (cpp) and running unsigned ... running doctests in unsigned ... Doctest: unsigned ... ok compiling (c) and running varargcall ... running doctests in varargcall ... Doctest: varargcall ... ok compiling (cpp) and running varargcall ... running doctests in varargcall ... Doctest: varargcall ... ok compiling (c) and running varargdecl ... running doctests in varargdecl ... Doctest: varargdecl ... ok compiling (cpp) and running varargdecl ... running doctests in varargdecl ... Doctest: varargdecl ... ok compiling (c) and running watts1 ... running doctests in watts1 ... Doctest: watts1 ... ok compiling (cpp) and running watts1 ... running doctests in watts1 ... Doctest: watts1 ... ok compiling (c) and running withnogil ... running doctests in withnogil ... Doctest: withnogil ... ok compiling (cpp) and running withnogil ... running doctests in withnogil ... Doctest: withnogil ... ok compiling (c) and running withstat ... running doctests in withstat ... Doctest: withstat ... ok compiling (cpp) and running withstat ... running doctests in withstat ... Doctest: withstat ... ok compiling (c) and running wundram1 ... running doctests in wundram1 ... Doctest: wundram1 ... ok compiling (cpp) and running wundram1 ... running doctests in wundram1 ... Doctest: wundram1 ... ok ====================================================================== FAIL: Doctest: charencoding ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.5/doctest.py", line 2128, in runTest raise self.failureException(self.format_failure(new.getvalue())) AssertionError: Failed doctest test for charencoding File "/home/kirr/src/tools/cython/tmp/cython-devel/BUILD/run/c/charencoding.so", line 100, in charencoding ---------------------------------------------------------------------- File "/home/kirr/src/tools/cython/tmp/cython-devel/BUILD/run/c/charencoding.so", line 102, in charencoding Failed example: expected = b''.join([chr(i) for i in range(0x10,0xFF,0x11)] + [chr(0xFF)]) Exception raised: Traceback (most recent call last): File "/usr/lib/python2.5/doctest.py", line 1228, in __run compileflags, 1) in test.globs File "", line 1 expected = b''.join([chr(i) for i in range(0x10,0xFF,0x11)] + [chr(0xFF)]) ^ SyntaxError: invalid syntax ---------------------------------------------------------------------- File "/home/kirr/src/tools/cython/tmp/cython-devel/BUILD/run/c/charencoding.so", line 105, in charencoding Failed example: assert s == expected, repr(s) Exception raised: Traceback (most recent call last): File "/usr/lib/python2.5/doctest.py", line 1228, in __run compileflags, 1) in test.globs File "", line 1, in assert s == expected, repr(s) NameError: name 'expected' is not defined ---------------------------------------------------------------------- File "/home/kirr/src/tools/cython/tmp/cython-devel/BUILD/run/c/charencoding.so", line 108, in charencoding Failed example: assert s == expected, repr(s) Exception raised: Traceback (most recent call last): File "/usr/lib/python2.5/doctest.py", line 1228, in __run compileflags, 1) in test.globs File "", line 1, in assert s == expected, repr(s) NameError: name 'expected' is not defined ====================================================================== FAIL: Doctest: charencoding ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.5/doctest.py", line 2128, in runTest raise self.failureException(self.format_failure(new.getvalue())) AssertionError: Failed doctest test for charencoding File "/home/kirr/src/tools/cython/tmp/cython-devel/BUILD/run/cpp/charencoding.so", line 104, in charencoding ---------------------------------------------------------------------- File "/home/kirr/src/tools/cython/tmp/cython-devel/BUILD/run/cpp/charencoding.so", line 106, in charencoding Failed example: expected = b''.join([chr(i) for i in range(0x10,0xFF,0x11)] + [chr(0xFF)]) Exception raised: Traceback (most recent call last): File "/usr/lib/python2.5/doctest.py", line 1228, in __run compileflags, 1) in test.globs File "", line 1 expected = b''.join([chr(i) for i in range(0x10,0xFF,0x11)] + [chr(0xFF)]) ^ SyntaxError: invalid syntax ---------------------------------------------------------------------- File "/home/kirr/src/tools/cython/tmp/cython-devel/BUILD/run/cpp/charencoding.so", line 109, in charencoding Failed example: assert s == expected, repr(s) Exception raised: Traceback (most recent call last): File "/usr/lib/python2.5/doctest.py", line 1228, in __run compileflags, 1) in test.globs File "", line 1, in assert s == expected, repr(s) NameError: name 'expected' is not defined ---------------------------------------------------------------------- File "/home/kirr/src/tools/cython/tmp/cython-devel/BUILD/run/cpp/charencoding.so", line 112, in charencoding Failed example: assert s == expected, repr(s) Exception raised: Traceback (most recent call last): File "/usr/lib/python2.5/doctest.py", line 1228, in __run compileflags, 1) in test.globs File "", line 1, in assert s == expected, repr(s) NameError: name 'expected' is not defined ---------------------------------------------------------------------- Ran 1372 tests in 270.898s FAILED (failures=2) --- 8< --- Thanks, Kirill