From robertwb at math.washington.edu  Wed Apr  1 01:38:21 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 31 Mar 2009 16:38:21 -0700
Subject: [Cython] Package names
In-Reply-To: <e7ba66e40903311348s75e6b7c2he541bef6e6a1e48a@mail.gmail.com>
References: <49D137CC.8070707@student.matnat.uio.no>
	<e7ba66e40903301600y5f3e167w4795df9ec9ea23a0@mail.gmail.com>
	<49D25E4A.8040309@behnel.de>
	<e7ba66e40903311348s75e6b7c2he541bef6e6a1e48a@mail.gmail.com>
Message-ID: <4207A8C5-E0B4-4223-A614-BB0DABF51E9B@math.washington.edu>

On Mar 31, 2009, at 1:48 PM, Lisandro Dalcin wrote:

> On Tue, Mar 31, 2009 at 3:17 PM, Stefan Behnel  
> <stefan_ml at behnel.de> wrote:
>>
>> Lisandro Dalcin wrote:
>>>> 2) Should all the package names be converted to lowercase?
>>>
>>> Perhaps we should follow PEP 8 ?
>>
>> Sure. When we break stuff, make sure we only have to do it once.  
>> That also
>> includes asking on python-dev what a suitable target package would  
>> be for
>> the stdlib.

+1

>>>> 3) Should one perhaps distribute Cython under two seperate package
>>>> names, so that a standalone version can coexist with the one in  
>>>> stdlib?
>>>> This would facilitate easy installation and usage of non-stdlib  
>>>> Cython
>>>> releases. Can't come up with any good names though.
>>>
>>> 1) cython
>>> 2) cython.core
>>> 3) cython.pyximport
>>
>> Yes, one package for everything. So, what you mean is: rename the  
>> current
>> Cython package to "cython.core"? Would that be for the stdlib or  
>> for the
>> external package?
>>
>
> Both stdlib and external :-) ... but doing 'import cython' should not
> have the side-effect of importing 'cython.core' ... Then we could
> decide that stdlib cython.__version__ is old for a project, and
> hot-fix "sys.path" to get a newer Cython, perhaps from a ZIP file.

One thing I just realized is that we use (by necessity) some absolute  
imports in the cython library, which will make having separate names  
rather a pain (though manageable). Are there any other packages that  
have both an internal and external version that we could learn from?

>>>  and also have cython/run.py for cmd line work with -m
>>> switch, and make __init__py use that for newer Python versions that
>>> let you run packages.
>>
>> Note that those aren't that "new" anymore. What do people think about
>> dropping the "cython" frontend alltogether, and making
>>
>>    python -m cython
>>
>> the official way to start the Cython compiler? I doubt that there  
>> are still
>> a lot of developers who do not have access to Python 2.4 or later  
>> in one
>> way or another.
>
> Stefan, the -m switch is know to be broken in Python 2.4. It does not
> work for packages. See yourself:
>
> $ python2.5 -m distutils # the output is nothing
>
> $ python2.4 -m distutils
> python2.4: module distutils has no associated file
>
> that is, 2.4 does not handle __init__.py as the associated file for  
> a package..

This is motivation to keep the frontend around, though we should  
still try and keep as much of the logic outside of that file as  
possible.

- Robert


From robertwb at math.washington.edu  Wed Apr  1 01:46:40 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 31 Mar 2009 16:46:40 -0700
Subject: [Cython] Naive build system
In-Reply-To: <1f077e770903310820p73c3d014pc7b70ff29f4fb6c7@mail.gmail.com>
References: <1f077e770903301915l47cb09efsc53660b13c0ec6ed@mail.gmail.com>
	<e7ba66e40903310716t436dba00n34ec39f234b38854@mail.gmail.com>
	<1f077e770903310814u14f1c074kf5771f70539a96e3@mail.gmail.com>
	<1f077e770903310820p73c3d014pc7b70ff29f4fb6c7@mail.gmail.com>
Message-ID: <F922B7D2-4655-48E1-B205-A43E3D447B30@math.washington.edu>

On Mar 31, 2009, at 8:20 AM, Nicolas Dumazet wrote:

> 2009/4/1 Nicolas Dumazet <nicdumz at gmail.com>:
>> I looked at ways to avoid the extra setup.py work, it seems  
>> there's no
>> "easy way" here.
>> I thought about extending the distutils build command to build the c
>> files before the module build, but it's definitely not simple: if the
>> C files are correctly generated, one would need to register the C
>> extensions on the fly, once the build process has already started;  
>> and
>> I'm not sure that this is possible.
>> And yes, if it is not possible, one would need to write a complex
>> build system handling the whole compilation chain.
>
> Or... the setup.py behavior could be modified:
> When --no-cython-compile is not provided, if the system is not able to
> generate the .c, it bails out, spitting out an error, instead of
> trying to back out "intelligently" to the pure python implementation
> by itself.
>
> This would make the setup.py implementation much more simple:
> * If --no-cython-compile is here, there's almost nothing to do.
> * If it's not here, add the dependences on the C extensions, extend
> the build_ext distutils command to generate the C files before
> building the cython module, (this is actually easy to do) and...
> voil?! Just exit with an error if you can't generate the C files,
> suggest to try --no-cython-compile instead.

I'm -1 on this idea, because it complicates auto-installation of  
packages (for example, if Cython is downloaded via a package manager,  
especially as a dependance to something else). Regarding Scons, it's  
a nice build system, but I'd rather not make it a dependancy for  
Cython (especially as our build needs really aren't super complicated).

Note also that the implict Cython dependancies (i.e. cimports) are  
not handled by this patch either. Perhaps in our case a simple "if  
the .c files are present and all newer than the .pyx files, skip the  
cythonization step."

- Robert


From dalcinl at gmail.com  Wed Apr  1 02:45:33 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Tue, 31 Mar 2009 21:45:33 -0300
Subject: [Cython] code documentation tool
In-Reply-To: <49D2904E.2020702@gmail.com>
References: <49CFF3CB.2000408@gmail.com>
	<5b8d13220903291934w69a8cac9y1af6e1722eb4ddca@mail.gmail.com>
	<49D2904E.2020702@gmail.com>
Message-ID: <e7ba66e40903311745v1b0629e1s26694140cd3f5fdd@mail.gmail.com>

On Tue, Mar 31, 2009 at 6:51 PM, Martin Gysel <m.gysel at gmail.com> wrote:
>
> AFAIK sphinx gets the info for autodoc from the docstring but how does
> this work for you because my docstrings are removed in the module
> (according the cython doc, that's the normal behavior).
>

What are you talking about, Martin? Cython does not removes any
docstring (with a few exceptions for some special methods). However,
perhaps you need to asks Cython to auto-generate signatures for you,
just add the commented line at the very beginning of your modules:

#cython: embedsignature=True




-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From stefan_ml at behnel.de  Wed Apr  1 08:54:07 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 01 Apr 2009 08:54:07 +0200
Subject: [Cython] Refcount question: SetAttrString
In-Reply-To: <49D27184.9020103@student.matnat.uio.no>
References: <49D27184.9020103@student.matnat.uio.no>
Message-ID: <49D30F8F.7090809@behnel.de>


Dag Sverre Seljebotn wrote:
> While ripping out the old temp system (yay!) ...

Thanks for doing that, BTW. This was sooo long overdue.

Stefan



From dagss at student.matnat.uio.no  Wed Apr  1 22:15:04 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Wed, 1 Apr 2009 22:15:04 +0200 (CEST)
Subject: [Cython] Naive build system
In-Reply-To: <F922B7D2-4655-48E1-B205-A43E3D447B30@math.washington.edu>
References: <1f077e770903301915l47cb09efsc53660b13c0ec6ed@mail.gmail.com>
	<e7ba66e40903310716t436dba00n34ec39f234b38854@mail.gmail.com>
	<1f077e770903310814u14f1c074kf5771f70539a96e3@mail.gmail.com>
	<1f077e770903310820p73c3d014pc7b70ff29f4fb6c7@mail.gmail.com>
	<F922B7D2-4655-48E1-B205-A43E3D447B30@math.washington.edu>
Message-ID: <070974f7c81c88231be5797308c5ce81.squirrel@webmail.uio.no>

Robert Bradshaw wrote:
> On Mar 31, 2009, at 8:20 AM, Nicolas Dumazet wrote:
>
>> 2009/4/1 Nicolas Dumazet <nicdumz at gmail.com>:
>>> I looked at ways to avoid the extra setup.py work, it seems
>>> there's no
>>> "easy way" here.
>>> I thought about extending the distutils build command to build the c
>>> files before the module build, but it's definitely not simple: if the
>>> C files are correctly generated, one would need to register the C
>>> extensions on the fly, once the build process has already started;
>>> and
>>> I'm not sure that this is possible.
>>> And yes, if it is not possible, one would need to write a complex
>>> build system handling the whole compilation chain.
>>
>> Or... the setup.py behavior could be modified:
>> When --no-cython-compile is not provided, if the system is not able to
>> generate the .c, it bails out, spitting out an error, instead of
>> trying to back out "intelligently" to the pure python implementation
>> by itself.
>>
>> This would make the setup.py implementation much more simple:
>> * If --no-cython-compile is here, there's almost nothing to do.
>> * If it's not here, add the dependences on the C extensions, extend
>> the build_ext distutils command to generate the C files before
>> building the cython module, (this is actually easy to do) and...
>> voil?! Just exit with an error if you can't generate the C files,
>> suggest to try --no-cython-compile instead.
>
> I'm -1 on this idea, because it complicates auto-installation of
> packages (for example, if Cython is downloaded via a package manager,
> especially as a dependance to something else). Regarding Scons, it's
> a nice build system, but I'd rather not make it a dependancy for
> Cython (especially as our build needs really aren't super complicated).

About SCons, iI meant that as an apropos and strictly about support for
end users applications. I.e. it's a matter of finding and bundling an
SCons plugin for Cython in the Tools directory, like we ship an emacs
mode. (And perhaps pointing our users towards SCons rather than investing
too much effort in improving Cython builds using the Cython distutils
plugin; again not for Cython itself but for Cython users.)

Dag Sverre


From m.gysel at gmail.com  Wed Apr  1 23:24:17 2009
From: m.gysel at gmail.com (Martin Gysel)
Date: Wed, 01 Apr 2009 23:24:17 +0200
Subject: [Cython] code documentation tool
In-Reply-To: <e7ba66e40903311745v1b0629e1s26694140cd3f5fdd@mail.gmail.com>
References: <49CFF3CB.2000408@gmail.com>	<5b8d13220903291934w69a8cac9y1af6e1722eb4ddca@mail.gmail.com>	<49D2904E.2020702@gmail.com>
	<e7ba66e40903311745v1b0629e1s26694140cd3f5fdd@mail.gmail.com>
Message-ID: <49D3DB81.2070106@gmail.com>

Lisandro Dalcin schrieb:
> On Tue, Mar 31, 2009 at 6:51 PM, Martin Gysel <m.gysel at gmail.com> wrote:
>   
>> AFAIK sphinx gets the info for autodoc from the docstring but how does
>> this work for you because my docstrings are removed in the module
>> (according the cython doc, that's the normal behavior).
>>     
>
> What are you talking about, Martin? Cython does not removes any
> docstring (with a few exceptions for some special methods). However,
> perhaps you need to asks Cython to auto-generate signatures for you,
> just add the commented line at the very beginning of your modules:
>
> #cython: embedsignature=True
>   
thx for the hint, now everything works as expected

/martin


From robertwb at math.washington.edu  Thu Apr  2 00:23:58 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Wed, 1 Apr 2009 15:23:58 -0700
Subject: [Cython] Refcount question: SetAttrString
In-Reply-To: <49D30F8F.7090809@behnel.de>
References: <49D27184.9020103@student.matnat.uio.no>
	<49D30F8F.7090809@behnel.de>
Message-ID: <DDDDB27A-82CC-40E4-AD6B-31F793E909BD@math.washington.edu>

On Mar 31, 2009, at 11:54 PM, Stefan Behnel wrote:

> Dag Sverre Seljebotn wrote:
>> While ripping out the old temp system (yay!) ...
>
> Thanks for doing that, BTW. This was sooo long overdue.

I noticed these changes in the -unstable branch too. Yay!

- Robert


From robertwb at math.washington.edu  Thu Apr  2 00:25:25 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Wed, 1 Apr 2009 15:25:25 -0700
Subject: [Cython] Naive build system
In-Reply-To: <070974f7c81c88231be5797308c5ce81.squirrel@webmail.uio.no>
References: <1f077e770903301915l47cb09efsc53660b13c0ec6ed@mail.gmail.com>
	<e7ba66e40903310716t436dba00n34ec39f234b38854@mail.gmail.com>
	<1f077e770903310814u14f1c074kf5771f70539a96e3@mail.gmail.com>
	<1f077e770903310820p73c3d014pc7b70ff29f4fb6c7@mail.gmail.com>
	<F922B7D2-4655-48E1-B205-A43E3D447B30@math.washington.edu>
	<070974f7c81c88231be5797308c5ce81.squirrel@webmail.uio.no>
Message-ID: <8F086E2D-C82F-4723-A073-52A9F235A01F@math.washington.edu>

On Apr 1, 2009, at 1:15 PM, Dag Sverre Seljebotn wrote:

> Robert Bradshaw wrote:
>> On Mar 31, 2009, at 8:20 AM, Nicolas Dumazet wrote:
>>
>>> 2009/4/1 Nicolas Dumazet <nicdumz at gmail.com>:
>>>> I looked at ways to avoid the extra setup.py work, it seems
>>>> there's no
>>>> "easy way" here.
>>>> I thought about extending the distutils build command to build  
>>>> the c
>>>> files before the module build, but it's definitely not simple:  
>>>> if the
>>>> C files are correctly generated, one would need to register the C
>>>> extensions on the fly, once the build process has already started;
>>>> and
>>>> I'm not sure that this is possible.
>>>> And yes, if it is not possible, one would need to write a complex
>>>> build system handling the whole compilation chain.
>>>
>>> Or... the setup.py behavior could be modified:
>>> When --no-cython-compile is not provided, if the system is not  
>>> able to
>>> generate the .c, it bails out, spitting out an error, instead of
>>> trying to back out "intelligently" to the pure python implementation
>>> by itself.
>>>
>>> This would make the setup.py implementation much more simple:
>>> * If --no-cython-compile is here, there's almost nothing to do.
>>> * If it's not here, add the dependences on the C extensions, extend
>>> the build_ext distutils command to generate the C files before
>>> building the cython module, (this is actually easy to do) and...
>>> voil?! Just exit with an error if you can't generate the C files,
>>> suggest to try --no-cython-compile instead.
>>
>> I'm -1 on this idea, because it complicates auto-installation of
>> packages (for example, if Cython is downloaded via a package manager,
>> especially as a dependance to something else). Regarding Scons, it's
>> a nice build system, but I'd rather not make it a dependancy for
>> Cython (especially as our build needs really aren't super  
>> complicated).
>
> About SCons, iI meant that as an apropos and strictly about support  
> for
> end users applications. I.e. it's a matter of finding and bundling an
> SCons plugin for Cython in the Tools directory, like we ship an emacs
> mode. (And perhaps pointing our users towards SCons rather than  
> investing
> too much effort in improving Cython builds using the Cython distutils
> plugin; again not for Cython itself but for Cython users.)

Ah, thanks for the clarification. Yes, this would be nice to have.

- Robert


From cournape at gmail.com  Thu Apr  2 05:02:17 2009
From: cournape at gmail.com (David Cournapeau)
Date: Thu, 2 Apr 2009 12:02:17 +0900
Subject: [Cython] Naive build system
In-Reply-To: <070974f7c81c88231be5797308c5ce81.squirrel@webmail.uio.no>
References: <1f077e770903301915l47cb09efsc53660b13c0ec6ed@mail.gmail.com>
	<e7ba66e40903310716t436dba00n34ec39f234b38854@mail.gmail.com>
	<1f077e770903310814u14f1c074kf5771f70539a96e3@mail.gmail.com>
	<1f077e770903310820p73c3d014pc7b70ff29f4fb6c7@mail.gmail.com>
	<F922B7D2-4655-48E1-B205-A43E3D447B30@math.washington.edu>
	<070974f7c81c88231be5797308c5ce81.squirrel@webmail.uio.no>
Message-ID: <5b8d13220904012002n7eb55a77u977040a4c62818be@mail.gmail.com>

On Thu, Apr 2, 2009 at 5:15 AM, Dag Sverre Seljebotn

> About SCons, iI meant that as an apropos and strictly about support for
> end users applications. I.e. it's a matter of finding and bundling an
> SCons plugin for Cython in the Tools directory, like we ship an emacs
> mode.

I would not mind giving away my cython tool to cython - the tool in
numscons are almost all independent of numscons, and can be used
directly from scons,

cheers,

David

From kwmsmith at gmail.com  Thu Apr  2 22:55:47 2009
From: kwmsmith at gmail.com (Kurt Smith)
Date: Thu, 2 Apr 2009 15:55:47 -0500
Subject: [Cython] What happened to cython-devel repo?
Message-ID: <c266a7930904021355n6ac0201dk3f03055d563107a1@mail.gmail.com>

Hi,

I tried updating my cython-devel, but no luck:

[506]$ pwd
/home/ksmith/Devel/cython/cython-devel

[507]$ hg pull
pulling from http://hg.cython.org/cython-devel
abort: 'http://hg.cython.org/cython-devel' does not appear to be an hg
repository!

Pulling from cython-unstable works, though, and it seems to be the most current.

Kurt

From robertwb at math.washington.edu  Thu Apr  2 23:02:19 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 2 Apr 2009 14:02:19 -0700
Subject: [Cython] What happened to cython-devel repo?
In-Reply-To: <c266a7930904021355n6ac0201dk3f03055d563107a1@mail.gmail.com>
References: <c266a7930904021355n6ac0201dk3f03055d563107a1@mail.gmail.com>
Message-ID: <BB353352-12B1-4EE4-8863-485D3B2BDF26@math.washington.edu>

In transplanting some patches from -unstable to -devel, I messed  
something up (basically, it was just really messy) and so I took it  
down for a sec to roll it back. Looks like you tried to pull in the  
couple of minutes it was down.

Please try again.

- Robert

On Apr 2, 2009, at 1:55 PM, Kurt Smith wrote:

> Hi,
>
> I tried updating my cython-devel, but no luck:
>
> [506]$ pwd
> /home/ksmith/Devel/cython/cython-devel
>
> [507]$ hg pull
> pulling from http://hg.cython.org/cython-devel
> abort: 'http://hg.cython.org/cython-devel' does not appear to be an hg
> repository!
>
> Pulling from cython-unstable works, though, and it seems to be the  
> most current.
>
> Kurt
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From kwmsmith at gmail.com  Thu Apr  2 23:13:59 2009
From: kwmsmith at gmail.com (Kurt Smith)
Date: Thu, 2 Apr 2009 16:13:59 -0500
Subject: [Cython] What happened to cython-devel repo?
In-Reply-To: <BB353352-12B1-4EE4-8863-485D3B2BDF26@math.washington.edu>
References: <c266a7930904021355n6ac0201dk3f03055d563107a1@mail.gmail.com>
	<BB353352-12B1-4EE4-8863-485D3B2BDF26@math.washington.edu>
Message-ID: <c266a7930904021413y12c9c220t3cb2879b2da4f459@mail.gmail.com>

On Thu, Apr 2, 2009 at 4:02 PM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> In transplanting some patches from -unstable to -devel, I messed
> something up (basically, it was just really messy) and so I took it
> down for a sec to roll it back. Looks like you tried to pull in the
> couple of minutes it was down.
>
> Please try again.

Thanks -- I'll be more patient next time ;-)

From robertwb at math.washington.edu  Thu Apr  2 23:19:15 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 2 Apr 2009 14:19:15 -0700
Subject: [Cython] What happened to cython-devel repo?
In-Reply-To: <c266a7930904021413y12c9c220t3cb2879b2da4f459@mail.gmail.com>
References: <c266a7930904021355n6ac0201dk3f03055d563107a1@mail.gmail.com>
	<BB353352-12B1-4EE4-8863-485D3B2BDF26@math.washington.edu>
	<c266a7930904021413y12c9c220t3cb2879b2da4f459@mail.gmail.com>
Message-ID: <DEF64E94-75B7-45CD-B27E-95A7927A8FB9@math.washington.edu>

On Apr 2, 2009, at 2:13 PM, Kurt Smith wrote:

> On Thu, Apr 2, 2009 at 4:02 PM, Robert Bradshaw
> <robertwb at math.washington.edu> wrote:
>> In transplanting some patches from -unstable to -devel, I messed
>> something up (basically, it was just really messy) and so I took it
>> down for a sec to roll it back. Looks like you tried to pull in the
>> couple of minutes it was down.
>>
>> Please try again.
>
> Thanks -- I'll be more patient next time ;-)

It's very rare for it to be down--last time was due to moving the  
server to a new machine, so yes, please ping the list if you notice  
problems :).

- Robert


From dalcinl at gmail.com  Fri Apr  3 00:43:15 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 2 Apr 2009 19:43:15 -0300
Subject: [Cython] new ISO C99 keywords
Message-ID: <e7ba66e40904021543q707cc419x75629091d5bc6513@mail.gmail.com>

Robert, if you add the list below, we get C99..., and you could rename
"ansi_c_keywords" to "iso_c99_keywords".

_Bool
_Complex
_Imaginary
inline
restrict



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From robertwb at math.washington.edu  Fri Apr  3 00:54:27 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 2 Apr 2009 15:54:27 -0700
Subject: [Cython] new ISO C99 keywords
In-Reply-To: <e7ba66e40904021543q707cc419x75629091d5bc6513@mail.gmail.com>
References: <e7ba66e40904021543q707cc419x75629091d5bc6513@mail.gmail.com>
Message-ID: <31011F93-8203-42FC-A96E-567B7DCDE892@math.washington.edu>

Sure, I could do that.

On Apr 2, 2009, at 3:43 PM, Lisandro Dalcin wrote:

> Robert, if you add the list below, we get C99..., and you could rename
> "ansi_c_keywords" to "iso_c99_keywords".
>
> _Bool
> _Complex
> _Imaginary
> inline
> restrict
>
>
>
> -- 
> Lisandro Dalc?n
> ---------------
> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
> Tel/Fax: +54-(0)342-451.1594
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From dalcinl at gmail.com  Fri Apr  3 01:26:20 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 2 Apr 2009 20:26:20 -0300
Subject: [Cython] PATCH: related to ticket #252, special case __weakref__
Message-ID: <e7ba66e40904021626q46b167d7l42b1f39e2d438334@mail.gmail.com>

Robert, likely you forgot about this detail releated weakrefs. Perhaps
the special-case should be done elsewhere?

diff -r 5cabe56c76ab Cython/Compiler/Symtab.py
--- a/Cython/Compiler/Symtab.py	Wed Mar 25 14:23:29 2009 -0700
+++ b/Cython/Compiler/Symtab.py	Thu Apr 02 20:23:49 2009 -0300
@@ -32,7 +32,10 @@

 def c_safe_identifier(cname):
     # There are some C limitations on struct entry names.
-    if (cname[:2] == '__' and not
cname.startswith(Naming.pyrex_prefix)) or cname in ansi_c_keywords:
+    if ((cname[:2] == '__'
+         and not (cname.startswith(Naming.pyrex_prefix)
+                  or cname == '__weakref__'))
+        or  cname in ansi_c_keywords):
         cname = Naming.pyrex_prefix + cname
     return cname


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dalcinl at gmail.com  Fri Apr  3 01:39:51 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 2 Apr 2009 20:39:51 -0300
Subject: [Cython] new patch for #255 (Py->C for integrals) in the tracker
Message-ID: <e7ba66e40904021639r397e1c5dg6063b0c88176ff54@mail.gmail.com>

- The changes to ModuleNode.py and Utils.py are just quick hacks to
get it working.

- I've tried hard to avoid generation of unreachable code like these
"if (0) {...}" blocks.

- Once this is reviewed, commented and eventually accepted, I'll
continue extending this to extern ctypedef integrals.


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From robertwb at math.washington.edu  Fri Apr  3 01:43:42 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 2 Apr 2009 16:43:42 -0700
Subject: [Cython] PATCH: related to ticket #252, special case __weakref__
In-Reply-To: <e7ba66e40904021626q46b167d7l42b1f39e2d438334@mail.gmail.com>
References: <e7ba66e40904021626q46b167d7l42b1f39e2d438334@mail.gmail.com>
Message-ID: <E70AAD25-536A-49B8-995D-4287724D06D1@math.washington.edu>

On Apr 2, 2009, at 4:26 PM, Lisandro Dalcin wrote:

> Robert, likely you forgot about this detail releated weakrefs. Perhaps
> the special-case should be done elsewhere?
>
> diff -r 5cabe56c76ab Cython/Compiler/Symtab.py
> --- a/Cython/Compiler/Symtab.py	Wed Mar 25 14:23:29 2009 -0700
> +++ b/Cython/Compiler/Symtab.py	Thu Apr 02 20:23:49 2009 -0300
> @@ -32,7 +32,10 @@
>
>  def c_safe_identifier(cname):
>      # There are some C limitations on struct entry names.
> -    if (cname[:2] == '__' and not
> cname.startswith(Naming.pyrex_prefix)) or cname in ansi_c_keywords:
> +    if ((cname[:2] == '__'
> +         and not (cname.startswith(Naming.pyrex_prefix)
> +                  or cname == '__weakref__'))
> +        or  cname in ansi_c_keywords):
>          cname = Naming.pyrex_prefix + cname
>      return cname

Oops, yes. Please push.

- Robert


From robertwb at math.washington.edu  Fri Apr  3 03:52:25 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 2 Apr 2009 18:52:25 -0700
Subject: [Cython] new patch for #255 (Py->C for integrals) in the tracker
In-Reply-To: <e7ba66e40904021639r397e1c5dg6063b0c88176ff54@mail.gmail.com>
References: <e7ba66e40904021639r397e1c5dg6063b0c88176ff54@mail.gmail.com>
Message-ID: <80E1EA88-7452-461E-8EA6-CF788CA4909A@math.washington.edu>

On Apr 2, 2009, at 4:39 PM, Lisandro Dalcin wrote:

> - The changes to ModuleNode.py and Utils.py are just quick hacks to
> get it working.
>
> - I've tried hard to avoid generation of unreachable code like these
> "if (0) {...}" blocks.
>
> - Once this is reviewed, commented and eventually accepted, I'll
> continue extending this to extern ctypedef integrals.

I've taken a look at the code and it looks good to me. You took care  
of every corner case I thought of checking. I haven't tested it under  
Py3, but it works great under Py2. It might be worth noting that I  
improved the utility code templating to take a type since this is  
such a common case: http://hg.cython.org/cython-devel/file/ 
2289ab00261d/Cython/Utils.py

One comment I had is that in your code

static INLINE TYPE __Pyx_PyInt_AsTYPE(PyObject* x) {
     if (sizeof(TYPE) < sizeof(long)) {
         long val = __Pyx_PyInt_AsLong(x);
         if (unlikely(val == -1 && PyErr_Occurred()))
             return (TYPE)-1;
         if (unlikely(val < 0)) {
             PyErr_SetString(PyExc_OverflowError,
                             "can't convert negative value to TYPE");
             return (TYPE)-1;
         }
         if (unlikely(val != (long)(TYPE)val)) {
             PyErr_SetString(PyExc_OverflowError,
                             "value too large to convert to TYPE");
             return (TYPE)-1;
         }
         return (TYPE)val;
     }
     return (TYPE)__Pyx_PyInt_AsUnsignedLong(x);
}

I believe the successful pathway could be shorted by doing

static INLINE TYPE __Pyx_PyInt_AsTYPE(PyObject* x) {
     if (sizeof(TYPE) < sizeof(long)) {
         long val = __Pyx_PyInt_AsLong(x);
         if (unlikely(val != (long)(TYPE)val)) {
             if (unlikely(val == -1 && PyErr_Occurred()));
             else if (val < 0) {
                 PyErr_SetString(PyExc_OverflowError,
                                 "can't convert negative value to  
TYPE");
             }
             else {
                 PyErr_SetString(PyExc_OverflowError,
                                 "value too large to convert to TYPE");
             }
             return (TYPE)-1;
         }
         return (TYPE)val;
     }
     return (TYPE)__Pyx_PyInt_AsUnsignedLong(x);
}

as sizeof(TYPE) < sizeof(long) guarantees val != (long)(TYPE)val for  
negative val on unsigned TYPE and the error does not need to be  
explicitly tested for otherwise (as the calling function will check  
PyErr_Occurred on a return value of -1).

- Robert


From dsurviver at gmail.com  Fri Apr  3 04:57:07 2009
From: dsurviver at gmail.com (Danilo Freitas)
Date: Thu, 2 Apr 2009 23:57:07 -0300
Subject: [Cython] [GSoC Proposal] Improve C++ Support
Message-ID: <45239150904021957l1287e0f5p5e42c7ae7462c991@mail.gmail.com>

Hi, all.

I just wrote a proposal for GSoC, and I'm posting it here to hav
feedbacks about it.

Here is the link with the wiki with my proposal:
http://wiki.cython.org/gsoc09/daniloaf

Fell free to comment about it.

Thanks


Danilo Freitas

From robertwb at math.washington.edu  Fri Apr  3 05:39:22 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 2 Apr 2009 20:39:22 -0700
Subject: [Cython] [GSoC Proposal] Improve C++ Support
In-Reply-To: <45239150904021957l1287e0f5p5e42c7ae7462c991@mail.gmail.com>
References: <45239150904021957l1287e0f5p5e42c7ae7462c991@mail.gmail.com>
Message-ID: <8017B8C9-5681-43CE-9024-886D13281783@math.washington.edu>

On Apr 2, 2009, at 7:57 PM, Danilo Freitas wrote:

> Hi, all.
>
> I just wrote a proposal for GSoC, and I'm posting it here to hav
> feedbacks about it.
>
> Here is the link with the wiki with my proposal:
> http://wiki.cython.org/gsoc09/daniloaf
>
> Fell free to comment about it.

First off, I would like to say I think this would be a very good  
project. I would clarify that the changes made to the Cython language  
are not to make natively it know about STL, but rather extending the  
syntax to more easily wrap polymorphic and templated functions/ 
classes so one can use them from Cython. Being able to wrap all of  
the STL library is a very useful and concrete goal.

Note that the application period is drawing to a close tomorrow, but  
feedback can be taken and the application updated until the close of  
the filtering process.

Hopefully we will have a good, pythonic syntax mostly nailed down for  
doing this before the coding period starts.

- Robert


From dagss at student.matnat.uio.no  Fri Apr  3 09:31:35 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Fri, 3 Apr 2009 09:31:35 +0200 (CEST)
Subject: [Cython] [GSoC Proposal] Improve C++ Support
In-Reply-To: <45239150904021957l1287e0f5p5e42c7ae7462c991@mail.gmail.com>
References: <45239150904021957l1287e0f5p5e42c7ae7462c991@mail.gmail.com>
Message-ID: <fea78156d40597b8a7c8ee69ff9013d0.squirrel@webmail.uio.no>

Danilo Freitas wrote:
> Hi, all.
>
> I just wrote a proposal for GSoC, and I'm posting it here to hav
> feedbacks about it.
>
> Here is the link with the wiki with my proposal:
> http://wiki.cython.org/gsoc09/daniloaf

Hi! Here's some feedback (these are my opinion only and not necesarrily of
others here). As Robert says though, please apply *now* and we can talk
about and edit the application later.

I wrote some notes a year ago on http://wiki.cython.org/enhancements/cpp ,
you could have a look (thopugh they are outdated).

Anyway, I see such a project as a) identifying and b) solving a number of
trouble spots for C++ users. Here's my list. I suppose not all of this
would have to be in a gsoc, it's just some feedback for consideration.

1) Using templates -- you already have this. My preferred syntax is using
[] BTW. As Robert said, it should support any template and not only STL.

2) Operator overloading -- you already have this as well. This should be
made generic, so that all C++ operators which have a corresponding Cython
operator can be used on a type.

3) References. Currently there's a problem (I didn't actually try this,
just assuming this from how I know things work):

If you make this C++ function:

void incargs(int& a, int& b) { a++; b++; }

then it has to be declared in Cython like this:

cdef extern void incargs(int a, int b)

and a call like this would result in:

cdef int a = 3
cdef object b = 3
incargs(a, b)
# now a is 4 and b is 3 !!!

My preferred solution for this dilemma would be to declare C++ references
Cython-side and require pointers to be passed (which are then dereferenced
prior to the call), because that's consistent with Cython semantics. I
find it too confusing syntax-wise that "a" above can change value in
Cython.

4) Calling constructors/destructors.

This is an interesting problem. C++ supports both stack and heap
allocation, while Cython/Python (and e.g. Java) only supports heap
allocation (of objects with constructors/destructors).

For instance, one cannot do this in Cython currently: Instead I would have
this 3-step plan (of which perhaps only the first one is gsoc):

a) Introduce syntax for declaring external C++ classes. Such classes can
only be used as pointers; and can be heap-allocated through new
cython.new/cython.delete magic functions.

b) Add support for this, using the non-pointer syntax for the type:

cdef cpp.std.vectot[int] vec = cpp.std.vectot[int](4) # construct
...
vec = something_else # destruct

This requires reference counting the C++ class. This would result in a
Pythonic feel.

c) Use flow control analysis/live variable analysis to optimize b) and
allocate the variables that can on the stack while preserving the same
semantics.

Dag Sverre


From dagss at student.matnat.uio.no  Fri Apr  3 09:56:15 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Fri, 03 Apr 2009 09:56:15 +0200
Subject: [Cython] [GSoC Proposal] Improve C++ Support
In-Reply-To: <fea78156d40597b8a7c8ee69ff9013d0.squirrel@webmail.uio.no>
References: <45239150904021957l1287e0f5p5e42c7ae7462c991@mail.gmail.com>
	<fea78156d40597b8a7c8ee69ff9013d0.squirrel@webmail.uio.no>
Message-ID: <49D5C11F.3030700@student.matnat.uio.no>

Dag Sverre Seljebotn wrote:
> Danilo Freitas wrote:
>> Hi, all.
>>
>> I just wrote a proposal for GSoC, and I'm posting it here to hav
>> feedbacks about it.
>>
>> Here is the link with the wiki with my proposal:
>> http://wiki.cython.org/gsoc09/daniloaf
> 
> Hi! Here's some feedback (these are my opinion only and not necesarrily of
> others here). As Robert says though, please apply *now* and we can talk
> about and edit the application later.
> 
> I wrote some notes a year ago on http://wiki.cython.org/enhancements/cpp ,
> you could have a look (thopugh they are outdated).
> 
> Anyway, I see such a project as a) identifying and b) solving a number of
> trouble spots for C++ users. Here's my list. I suppose not all of this
> would have to be in a gsoc, it's just some feedback for consideration.
> 
> 1) Using templates -- you already have this. My preferred syntax is using
> [] BTW. As Robert said, it should support any template and not only STL.
> 
> 2) Operator overloading -- you already have this as well. This should be
> made generic, so that all C++ operators which have a corresponding Cython
> operator can be used on a type.
> 
> 3) References. Currently there's a problem (I didn't actually try this,
> just assuming this from how I know things work):
> 
> If you make this C++ function:
> 
> void incargs(int& a, int& b) { a++; b++; }
> 
> then it has to be declared in Cython like this:
> 
> cdef extern void incargs(int a, int b)
> 
> and a call like this would result in:
> 
> cdef int a = 3
> cdef object b = 3
> incargs(a, b)
> # now a is 4 and b is 3 !!!
> 
> My preferred solution for this dilemma would be to declare C++ references
> Cython-side and require pointers to be passed (which are then dereferenced
> prior to the call), because that's consistent with Cython semantics. I
> find it too confusing syntax-wise that "a" above can change value in
> Cython.
> 
> 4) Calling constructors/destructors.
> 
> This is an interesting problem. C++ supports both stack and heap
> allocation, while Cython/Python (and e.g. Java) only supports heap
> allocation (of objects with constructors/destructors).
> 
> For instance, one cannot do this in Cython currently: Instead I would have
> this 3-step plan (of which perhaps only the first one is gsoc):

Something fell out here, sorry about that. Here it is:

For instance, one cannot do this in Cython currently:

cdef my_std_vector_int_iterator it = myvec.begin()

because an STL iterator cannot be constructed without arguments, and 
when that code is Cythonized, it roughly ends up like this:

my_std_vector_int_iterator it; # all declarations at top
...
it = myvec.begin()

(Note that assignment-on-declaration is the same as passing constructor 
arguments in C++).

One solution would be to add a syntax for stack allocation, however that 
interferes with lots of Python rules (variable scoping for instance).

So instead I would have this 3-step plan [...]


-- 
Dag Sverre

From robertwb at math.washington.edu  Fri Apr  3 09:53:47 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Apr 2009 00:53:47 -0700
Subject: [Cython] [GSoC Proposal] Improve C++ Support
In-Reply-To: <fea78156d40597b8a7c8ee69ff9013d0.squirrel@webmail.uio.no>
References: <45239150904021957l1287e0f5p5e42c7ae7462c991@mail.gmail.com>
	<fea78156d40597b8a7c8ee69ff9013d0.squirrel@webmail.uio.no>
Message-ID: <892997BA-44F0-4707-96BD-016239A5540A@math.washington.edu>

On Apr 3, 2009, at 12:31 AM, Dag Sverre Seljebotn wrote:

> Danilo Freitas wrote:
>> Hi, all.
>>
>> I just wrote a proposal for GSoC, and I'm posting it here to hav
>> feedbacks about it.
>>
>> Here is the link with the wiki with my proposal:
>> http://wiki.cython.org/gsoc09/daniloaf
>
> Hi! Here's some feedback (these are my opinion only and not  
> necesarrily of
> others here). As Robert says though, please apply *now* and we can  
> talk
> about and edit the application later.
>
> I wrote some notes a year ago on http://wiki.cython.org/ 
> enhancements/cpp ,
> you could have a look (thopugh they are outdated).
>
> Anyway, I see such a project as a) identifying and b) solving a  
> number of
> trouble spots for C++ users. Here's my list. I suppose not all of this
> would have to be in a gsoc, it's just some feedback for consideration.
>
> 1) Using templates -- you already have this. My preferred syntax is  
> using
> [] BTW. As Robert said, it should support any template and not only  
> STL.
>
> 2) Operator overloading -- you already have this as well. This  
> should be
> made generic, so that all C++ operators which have a corresponding  
> Cython
> operator can be used on a type.
>
> 3) References. Currently there's a problem (I didn't actually try  
> this,
> just assuming this from how I know things work):
>
> If you make this C++ function:
>
> void incargs(int& a, int& b) { a++; b++; }
>
> then it has to be declared in Cython like this:
>
> cdef extern void incargs(int a, int b)
>
> and a call like this would result in:
>
> cdef int a = 3
> cdef object b = 3
> incargs(a, b)
> # now a is 4 and b is 3 !!!
>
> My preferred solution for this dilemma would be to declare C++  
> references
> Cython-side and require pointers to be passed (which are then  
> dereferenced
> prior to the call), because that's consistent with Cython semantics. I
> find it too confusing syntax-wise that "a" above can change value in
> Cython.

+1

> 4) Calling constructors/destructors.
>
> This is an interesting problem. C++ supports both stack and heap
> allocation, while Cython/Python (and e.g. Java) only supports heap
> allocation (of objects with constructors/destructors).
>
> For instance, one cannot do this in Cython currently: Instead I  
> would have
> this 3-step plan (of which perhaps only the first one is gsoc):
>
> a) Introduce syntax for declaring external C++ classes. Such  
> classes can
> only be used as pointers; and can be heap-allocated through new
> cython.new/cython.delete magic functions.
>
> b) Add support for this, using the non-pointer syntax for the type:
>
> cdef cpp.std.vectot[int] vec = cpp.std.vectot[int](4) # construct
> ...
> vec = something_else # destruct
>
> This requires reference counting the C++ class. This would result in a
> Pythonic feel.
>
> c) Use flow control analysis/live variable analysis to optimize b) and
> allocate the variables that can on the stack while preserving the same
> semantics.

Note that the allocate-on-the-stack is not specific to just stacks.  
For example, it is possible to define a C++ class as a member of a  
cdef class, and then its memory is allocated along with the object as  
part of the struct. Currently, one then has to actually initialize  
and destroy manually.

- Robert



From dagss at student.matnat.uio.no  Fri Apr  3 10:59:19 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Fri, 03 Apr 2009 10:59:19 +0200
Subject: [Cython] [GSoC Proposal] Improve C++ Support
In-Reply-To: <892997BA-44F0-4707-96BD-016239A5540A@math.washington.edu>
References: <45239150904021957l1287e0f5p5e42c7ae7462c991@mail.gmail.com>	<fea78156d40597b8a7c8ee69ff9013d0.squirrel@webmail.uio.no>
	<892997BA-44F0-4707-96BD-016239A5540A@math.washington.edu>
Message-ID: <49D5CFE7.6070100@student.matnat.uio.no>

Robert Bradshaw wrote:
> On Apr 3, 2009, at 12:31 AM, Dag Sverre Seljebotn wrote:
> 
>> Danilo Freitas wrote:
>>> Hi, all.
>>>
>>> I just wrote a proposal for GSoC, and I'm posting it here to hav
>>> feedbacks about it.
>>>
>>> Here is the link with the wiki with my proposal:
>>> http://wiki.cython.org/gsoc09/daniloaf
>> Hi! Here's some feedback (these are my opinion only and not  
>> necesarrily of
>> others here). As Robert says though, please apply *now* and we can  
>> talk
>> about and edit the application later.
>>
>> I wrote some notes a year ago on http://wiki.cython.org/ 
>> enhancements/cpp ,
>> you could have a look (thopugh they are outdated).
>>
>> Anyway, I see such a project as a) identifying and b) solving a  
>> number of
>> trouble spots for C++ users. Here's my list. I suppose not all of this
>> would have to be in a gsoc, it's just some feedback for consideration.
>>
>> 1) Using templates -- you already have this. My preferred syntax is  
>> using
>> [] BTW. As Robert said, it should support any template and not only  
>> STL.
>>
>> 2) Operator overloading -- you already have this as well. This  
>> should be
>> made generic, so that all C++ operators which have a corresponding  
>> Cython
>> operator can be used on a type.
>>
>> 3) References. Currently there's a problem (I didn't actually try  
>> this,
>> just assuming this from how I know things work):
>>
>> If you make this C++ function:
>>
>> void incargs(int& a, int& b) { a++; b++; }
>>
>> then it has to be declared in Cython like this:
>>
>> cdef extern void incargs(int a, int b)
>>
>> and a call like this would result in:
>>
>> cdef int a = 3
>> cdef object b = 3
>> incargs(a, b)
>> # now a is 4 and b is 3 !!!
>>
>> My preferred solution for this dilemma would be to declare C++  
>> references
>> Cython-side and require pointers to be passed (which are then  
>> dereferenced
>> prior to the call), because that's consistent with Cython semantics. I
>> find it too confusing syntax-wise that "a" above can change value in
>> Cython.
> 
> +1
> 
>> 4) Calling constructors/destructors.
>>
>> This is an interesting problem. C++ supports both stack and heap
>> allocation, while Cython/Python (and e.g. Java) only supports heap
>> allocation (of objects with constructors/destructors).
>>
>> For instance, one cannot do this in Cython currently: Instead I  
>> would have
>> this 3-step plan (of which perhaps only the first one is gsoc):
>>
>> a) Introduce syntax for declaring external C++ classes. Such  
>> classes can
>> only be used as pointers; and can be heap-allocated through new
>> cython.new/cython.delete magic functions.
>>
>> b) Add support for this, using the non-pointer syntax for the type:
>>
>> cdef cpp.std.vectot[int] vec = cpp.std.vectot[int](4) # construct
>> ...
>> vec = something_else # destruct
>>
>> This requires reference counting the C++ class. This would result in a
>> Pythonic feel.
>>
>> c) Use flow control analysis/live variable analysis to optimize b) and
>> allocate the variables that can on the stack while preserving the same
>> semantics.
> 
> Note that the allocate-on-the-stack is not specific to just stacks.  
> For example, it is possible to define a C++ class as a member of a  
> cdef class, and then its memory is allocated along with the object as  
> part of the struct. Currently, one then has to actually initialize  
> and destroy manually.

Good point. That's actually fairly tricky to do if the class doesn't 
have a no-arg constructor. If you have this:

class MyClass {
    MyClass(int x) {...} // non-empty constructor
};

Then in order to use it in a struct (without going through a pointer) 
one would actually have to do this:

struct foo {
   MyClass myinstance;
public:
   foo() : myinstance(any expression in globals) {}
// or: foo(x,y,z) : myinstance(any expression in x,y,z,globals) {}
}

Getting Cython output code like this is non-trivial, at least without 
directly copying the member-construction-syntax of C++.

-- 
Dag Sverre

From robertwb at math.washington.edu  Fri Apr  3 11:00:47 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Apr 2009 02:00:47 -0700
Subject: [Cython] Cython 0.11.1 alpha
Message-ID: <45493547-5D3E-47BD-B526-7AF0B5610871@math.washington.edu>

In the interest of releasing sooner rather than later, a new alpha is  
up at

http://cython.org/Cython-0.11.1.alpha.tar.gz

There is a good patch by Lisandro that does Python <-> c int  
converstion that I'd like to see make it in, and Sage doesn't yet  
compile (mostly due to the "defined before used" bug catching--should  
this be merely a warning before the next major release?), but at this  
point lets let the -devel branch be primarily focused on resolving  
these issues. I anticipate we could have 0.11.1 out within a week  
with nearly two dozen tickets closed.

- Robert

From psp.reachable at gmail.com  Fri Apr  3 11:10:18 2009
From: psp.reachable at gmail.com (Prajwal Suhas P)
Date: Fri, 3 Apr 2009 14:40:18 +0530
Subject: [Cython] [GSoC Proposal] Control flow analysis for Cython
Message-ID: <fb17e5d0904030210q51e8a855ic531f943aca0da7c@mail.gmail.com>

This is my proposal for gsoc.
http://wiki.cython.org/gsoc/prajwal

Please feel free to give suggestions/comment on it.
-- 
Regards,
Prajwal S.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20090403/de72a058/attachment.htm 

From dagss at student.matnat.uio.no  Fri Apr  3 11:28:58 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Fri, 03 Apr 2009 11:28:58 +0200
Subject: [Cython] Cython 0.11.1 alpha
In-Reply-To: <45493547-5D3E-47BD-B526-7AF0B5610871@math.washington.edu>
References: <45493547-5D3E-47BD-B526-7AF0B5610871@math.washington.edu>
Message-ID: <49D5D6DA.9030201@student.matnat.uio.no>

Robert Bradshaw wrote:
> In the interest of releasing sooner rather than later, a new alpha is  
> up at
> 
> http://cython.org/Cython-0.11.1.alpha.tar.gz
> 
> There is a good patch by Lisandro that does Python <-> c int  
> converstion that I'd like to see make it in, and Sage doesn't yet  
> compile (mostly due to the "defined before used" bug catching--should  
> this be merely a warning before the next major release?), but at this  

+1

-- 
Dag Sverre

From dagss at student.matnat.uio.no  Fri Apr  3 11:31:13 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Fri, 03 Apr 2009 11:31:13 +0200
Subject: [Cython] Cython 0.11.1 alpha
In-Reply-To: <49D5D6DA.9030201@student.matnat.uio.no>
References: <45493547-5D3E-47BD-B526-7AF0B5610871@math.washington.edu>
	<49D5D6DA.9030201@student.matnat.uio.no>
Message-ID: <49D5D761.10900@student.matnat.uio.no>

Dag Sverre Seljebotn wrote:
> Robert Bradshaw wrote:
>> In the interest of releasing sooner rather than later, a new alpha is  
>> up at
>>
>> http://cython.org/Cython-0.11.1.alpha.tar.gz
>>
>> There is a good patch by Lisandro that does Python <-> c int  
>> converstion that I'd like to see make it in, and Sage doesn't yet  
>> compile (mostly due to the "defined before used" bug catching--should  
>> this be merely a warning before the next major release?), but at this  
> 
> +1
> 

Though as the for loop semantics also changed, one could also argue to 
keep it as it is but name it 0.12.

-- 
Dag Sverre

From robertwb at math.washington.edu  Fri Apr  3 11:15:25 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Apr 2009 02:15:25 -0700
Subject: [Cython] [GSoC Proposal] Improve C++ Support
In-Reply-To: <49D5CFE7.6070100@student.matnat.uio.no>
References: <45239150904021957l1287e0f5p5e42c7ae7462c991@mail.gmail.com>	<fea78156d40597b8a7c8ee69ff9013d0.squirrel@webmail.uio.no>
	<892997BA-44F0-4707-96BD-016239A5540A@math.washington.edu>
	<49D5CFE7.6070100@student.matnat.uio.no>
Message-ID: <5FE1DE4C-BB47-4546-8CC5-B10235F6E455@math.washington.edu>

On Apr 3, 2009, at 1:59 AM, Dag Sverre Seljebotn wrote:

> Robert Bradshaw wrote:
>>
>> Note that the allocate-on-the-stack is not specific to just stacks.
>> For example, it is possible to define a C++ class as a member of a
>> cdef class, and then its memory is allocated along with the object as
>> part of the struct. Currently, one then has to actually initialize
>> and destroy manually.
>
> Good point. That's actually fairly tricky to do if the class doesn't
> have a no-arg constructor.

I believe it can be done (IIRC constructors can take a chunk of  
memory to initialize rather than allocating it themselves). I'm no  
expert on this though.

> If you have this:
>
> class MyClass {
>     MyClass(int x) {...} // non-empty constructor
> };
>
> Then in order to use it in a struct (without going through a pointer)
> one would actually have to do this:
>
> struct foo {
>    MyClass myinstance;
> public:
>    foo() : myinstance(any expression in globals) {}
> // or: foo(x,y,z) : myinstance(any expression in x,y,z,globals) {}
> }
>
> Getting Cython output code like this is non-trivial, at least without
> directly copying the member-construction-syntax of C++.

Yes, and perhaps outside the scope of this project.

- Robert


From robertwb at math.washington.edu  Fri Apr  3 11:19:22 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Apr 2009 02:19:22 -0700
Subject: [Cython] [GSoC Proposal] Control flow analysis for Cython
In-Reply-To: <fb17e5d0904030210q51e8a855ic531f943aca0da7c@mail.gmail.com>
References: <fb17e5d0904030210q51e8a855ic531f943aca0da7c@mail.gmail.com>
Message-ID: <9CE9AECB-5E68-4FC1-9AFC-B375546E0A93@math.washington.edu>

On Apr 3, 2009, at 2:10 AM, Prajwal Suhas P wrote:

> This is my proposal for gsoc.
> http://wiki.cython.org/gsoc/prajwal

Thanks.

- Robert



From stefan_ml at behnel.de  Fri Apr  3 11:21:20 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 3 Apr 2009 11:21:20 +0200 (CEST)
Subject: [Cython] Cython 0.11.1 alpha
In-Reply-To: <49D5D761.10900@student.matnat.uio.no>
References: <45493547-5D3E-47BD-B526-7AF0B5610871@math.washington.edu>
	<49D5D6DA.9030201@student.matnat.uio.no>
	<49D5D761.10900@student.matnat.uio.no>
Message-ID: <b82fe26b85789bb4d050ff1ae9624745.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Dag Sverre Seljebotn wrote:
> Dag Sverre Seljebotn wrote:
>> Robert Bradshaw wrote:
>>> In the interest of releasing sooner rather than later, a new alpha is
>>> up at
>>>
>>> http://cython.org/Cython-0.11.1.alpha.tar.gz
>>>
>>> There is a good patch by Lisandro that does Python <-> c int
>>> converstion that I'd like to see make it in, and Sage doesn't yet
>>> compile (mostly due to the "defined before used" bug catching--should
>>> this be merely a warning before the next major release?), but at this
>>
>> +1
>
> Though as the for loop semantics also changed, one could also argue to
> keep it as it is but name it 0.12.

Although, if everyone who needs the bug fixes for 0.11 has to switch to
0.12 anyway, what good is it to not name it 0.11.1? :)

Stefan


From dagss at student.matnat.uio.no  Fri Apr  3 11:39:25 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Fri, 03 Apr 2009 11:39:25 +0200
Subject: [Cython] Cython 0.11.1 alpha
In-Reply-To: <b82fe26b85789bb4d050ff1ae9624745.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
References: <45493547-5D3E-47BD-B526-7AF0B5610871@math.washington.edu>	<49D5D6DA.9030201@student.matnat.uio.no>	<49D5D761.10900@student.matnat.uio.no>
	<b82fe26b85789bb4d050ff1ae9624745.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <49D5D94D.9020108@student.matnat.uio.no>

Stefan Behnel wrote:
> Dag Sverre Seljebotn wrote:
>> Dag Sverre Seljebotn wrote:
>>> Robert Bradshaw wrote:
>>>> In the interest of releasing sooner rather than later, a new alpha is
>>>> up at
>>>>
>>>> http://cython.org/Cython-0.11.1.alpha.tar.gz
>>>>
>>>> There is a good patch by Lisandro that does Python <-> c int
>>>> converstion that I'd like to see make it in, and Sage doesn't yet
>>>> compile (mostly due to the "defined before used" bug catching--should
>>>> this be merely a warning before the next major release?), but at this
>>> +1
>> Though as the for loop semantics also changed, one could also argue to
>> keep it as it is but name it 0.12.
> 
> Although, if everyone who needs the bug fixes for 0.11 has to switch to
> 0.12 anyway, what good is it to not name it 0.11.1? :)

The question is why we have a major/minor naming scheme at all. It could be

a) Depending on how much time's elapsed since the last major. That's 
roughly our current policy :-)

b) Substantial new features means new major. That's one guildeline, 
which speaks for naming this one 0.11.1.

c) New major when backwards compatability is broken in any way. That's 
another, conflicting guideline, which speaks for naming this one 0.12.

Myself I'm -1 on a) and +0 on b) and c).

-- 
Dag Sverre

From robertwb at math.washington.edu  Fri Apr  3 11:24:45 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Apr 2009 02:24:45 -0700
Subject: [Cython] Cython 0.11.1 alpha
In-Reply-To: <49D5D761.10900@student.matnat.uio.no>
References: <45493547-5D3E-47BD-B526-7AF0B5610871@math.washington.edu>
	<49D5D6DA.9030201@student.matnat.uio.no>
	<49D5D761.10900@student.matnat.uio.no>
Message-ID: <056837A0-5F13-43D1-B0A4-A6342CC6746E@math.washington.edu>

On Apr 3, 2009, at 2:31 AM, Dag Sverre Seljebotn wrote:

> Dag Sverre Seljebotn wrote:
>> Robert Bradshaw wrote:
>>> In the interest of releasing sooner rather than later, a new  
>>> alpha is
>>> up at
>>>
>>> http://cython.org/Cython-0.11.1.alpha.tar.gz
>>>
>>> There is a good patch by Lisandro that does Python <-> c int
>>> converstion that I'd like to see make it in, and Sage doesn't yet
>>> compile (mostly due to the "defined before used" bug catching-- 
>>> should
>>> this be merely a warning before the next major release?), but at  
>>> this
>>
>> +1
>>
>
> Though as the for loop semantics also changed, one could also argue to
> keep it as it is but name it 0.12.

For one thing, I'd like to see the temp allocation changes in 0.12,  
and perhaps some other bigger things that will need lots of testing.  
I'm more OK with the loop semantics changing.

- Robert


From dagss at student.matnat.uio.no  Fri Apr  3 11:41:08 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Fri, 03 Apr 2009 11:41:08 +0200
Subject: [Cython] Cython 0.11.1 alpha
In-Reply-To: <056837A0-5F13-43D1-B0A4-A6342CC6746E@math.washington.edu>
References: <45493547-5D3E-47BD-B526-7AF0B5610871@math.washington.edu>	<49D5D6DA.9030201@student.matnat.uio.no>	<49D5D761.10900@student.matnat.uio.no>
	<056837A0-5F13-43D1-B0A4-A6342CC6746E@math.washington.edu>
Message-ID: <49D5D9B4.30002@student.matnat.uio.no>

Robert Bradshaw wrote:
> On Apr 3, 2009, at 2:31 AM, Dag Sverre Seljebotn wrote:
> 
>> Dag Sverre Seljebotn wrote:
>>> Robert Bradshaw wrote:
>>>> In the interest of releasing sooner rather than later, a new  
>>>> alpha is
>>>> up at
>>>>
>>>> http://cython.org/Cython-0.11.1.alpha.tar.gz
>>>>
>>>> There is a good patch by Lisandro that does Python <-> c int
>>>> converstion that I'd like to see make it in, and Sage doesn't yet
>>>> compile (mostly due to the "defined before used" bug catching-- 
>>>> should
>>>> this be merely a warning before the next major release?), but at  
>>>> this
>>> +1
>>>
>> Though as the for loop semantics also changed, one could also argue to
>> keep it as it is but name it 0.12.
> 
> For one thing, I'd like to see the temp allocation changes in 0.12,  
> and perhaps some other bigger things that will need lots of testing.  
> I'm more OK with the loop semantics changing.

Ahh, that adds

d) Increase major when more substantial testing is needed (typically a 
merge of -unstable)

which also seems like a good idea.

-- 
Dag Sverre

From stephane.drouard at st.com  Fri Apr  3 11:28:56 2009
From: stephane.drouard at st.com (Stephane DROUARD)
Date: Fri, 3 Apr 2009 11:28:56 +0200
Subject: [Cython] Cython 0.11.1 alpha
In-Reply-To: <45493547-5D3E-47BD-B526-7AF0B5610871@math.washington.edu>
Message-ID: <008301c9b43e$9e818470$76ac810a@gnb.st.com>

Hello,

The fix for http://trac.cython.org/cython_trac/ticket/265 does not solve the
issue, we get the same error (stdexcept not included).

Cheers,
Stephane 

-----Original Message-----
From: cython-dev-bounces at codespeak.net
[mailto:cython-dev-bounces at codespeak.net] On Behalf Of Robert Bradshaw
Sent: Friday, April 03, 2009 11:01 AM
To: Cython-dev
Subject: [Cython] Cython 0.11.1 alpha

In the interest of releasing sooner rather than later, a new alpha is up at

http://cython.org/Cython-0.11.1.alpha.tar.gz

There is a good patch by Lisandro that does Python <-> c int converstion
that I'd like to see make it in, and Sage doesn't yet compile (mostly due to
the "defined before used" bug catching--should this be merely a warning
before the next major release?), but at this point lets let the -devel
branch be primarily focused on resolving these issues. I anticipate we could
have 0.11.1 out within a week with nearly two dozen tickets closed.

- Robert
_______________________________________________
Cython-dev mailing list
Cython-dev at codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev


From robertwb at math.washington.edu  Fri Apr  3 11:33:58 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Apr 2009 02:33:58 -0700
Subject: [Cython] Cython 0.11.1 alpha
In-Reply-To: <008301c9b43e$9e818470$76ac810a@gnb.st.com>
References: <008301c9b43e$9e818470$76ac810a@gnb.st.com>
Message-ID: <65D2DAA2-A974-4DA5-A818-276A539A8320@math.washington.edu>

On Apr 3, 2009, at 2:28 AM, Stephane DROUARD wrote:

> Hello,
>
> The fix for http://trac.cython.org/cython_trac/ticket/265 does not  
> solve the
> issue, we get the same error (stdexcept not included).
>
> Cheers,
> Stephane

Does a

cdef extern from "stdexcept":
     pass

resolve the issue?

- Robert


From robertwb at math.washington.edu  Fri Apr  3 11:40:26 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Apr 2009 02:40:26 -0700
Subject: [Cython] Cython 0.11.1 alpha
In-Reply-To: <49D5D94D.9020108@student.matnat.uio.no>
References: <45493547-5D3E-47BD-B526-7AF0B5610871@math.washington.edu>	<49D5D6DA.9030201@student.matnat.uio.no>	<49D5D761.10900@student.matnat.uio.no>
	<b82fe26b85789bb4d050ff1ae9624745.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<49D5D94D.9020108@student.matnat.uio.no>
Message-ID: <F169EE31-6CB7-465F-AB5C-3635C5E24E54@math.washington.edu>

On Apr 3, 2009, at 2:39 AM, Dag Sverre Seljebotn wrote:

> Stefan Behnel wrote:
>> Dag Sverre Seljebotn wrote:
>>> Dag Sverre Seljebotn wrote:
>>>> Robert Bradshaw wrote:
>>>>> In the interest of releasing sooner rather than later, a new  
>>>>> alpha is
>>>>> up at
>>>>>
>>>>> http://cython.org/Cython-0.11.1.alpha.tar.gz
>>>>>
>>>>> There is a good patch by Lisandro that does Python <-> c int
>>>>> converstion that I'd like to see make it in, and Sage doesn't yet
>>>>> compile (mostly due to the "defined before used" bug catching-- 
>>>>> should
>>>>> this be merely a warning before the next major release?), but  
>>>>> at this
>>>> +1
>>> Though as the for loop semantics also changed, one could also  
>>> argue to
>>> keep it as it is but name it 0.12.
>>
>> Although, if everyone who needs the bug fixes for 0.11 has to  
>> switch to
>> 0.12 anyway, what good is it to not name it 0.11.1? :)
>
> The question is why we have a major/minor naming scheme at all. It  
> could be
>
> a) Depending on how much time's elapsed since the last major. That's
> roughly our current policy :-)

Well, there's a strong correlation between time elapsed and new  
features.

> b) Substantial new features means new major. That's one guildeline,
> which speaks for naming this one 0.11.1.
>
> c) New major when backwards compatability is broken in any way. That's
> another, conflicting guideline, which speaks for naming this one 0.12.
>
> Myself I'm -1 on a) and +0 on b) and c).

I'm primarily guided by (b), where "substantial new features" may  
include internal re-factoring that requires lots of testing. I think  
(c) is important too, but am not a stickler being pedantically strict  
on this. It should be very safe to do a 0.x.y upgrade, no promises  
that 0.y won't force you to change your code (for the better, though  
it should be avoided--hopefully just making people remove bad/ 
ambiguous/abusive code, or stuff like cdivision).

- Robert


From stefan_ml at behnel.de  Fri Apr  3 11:42:49 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 3 Apr 2009 11:42:49 +0200 (CEST)
Subject: [Cython] [GSoC Proposal] Control flow analysis for Cython
In-Reply-To: <fb17e5d0904030210q51e8a855ic531f943aca0da7c@mail.gmail.com>
References: <fb17e5d0904030210q51e8a855ic531f943aca0da7c@mail.gmail.com>
Message-ID: <77629cb3d9b318f34fdd6e5c48a5f949.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Hi,

Prajwal Suhas P wrote:
> This is my proposal for gsoc.
> http://wiki.cython.org/gsoc/prajwal
>
> Please feel free to give suggestions/comment on it.

My main comment is that your proposal seems to reduce CFA to None check
avoidance. While this is a valid example for CFA application, it does not
quite catch what CFA is by itself. Possible variable values are one thing
that you can *infer* from a control flow graph, possible types are
another.

But the primary goal of this project should be to establish a suitable
representation of the possible execution paths inside a function (and,
less importantly, the global module code). So this really is about finding
(as in reading up) and implementing an efficient way for establishing,
maintaining and querying the control state of a function at each parse
tree node.

Stefan


From stefan_ml at behnel.de  Fri Apr  3 11:48:02 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 3 Apr 2009 11:48:02 +0200 (CEST)
Subject: [Cython] Cython 0.11.1 alpha
In-Reply-To: <49D5D9B4.30002@student.matnat.uio.no>
References: <45493547-5D3E-47BD-B526-7AF0B5610871@math.washington.edu>
	<49D5D6DA.9030201@student.matnat.uio.no>
	<49D5D761.10900@student.matnat.uio.no>
	<056837A0-5F13-43D1-B0A4-A6342CC6746E@math.washington.edu>
	<49D5D9B4.30002@student.matnat.uio.no>
Message-ID: <b65faf794d2cd1a98685a22183ee4b31.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Dag Sverre Seljebotn wrote:
> d) Increase major when more substantial testing is needed (typically a
> merge of -unstable)
>
> which also seems like a good idea.

Hmm, not that good, IMHO. That's what we have alpha and beta releases for.

Stefan


From stephane.drouard at st.com  Fri Apr  3 11:57:15 2009
From: stephane.drouard at st.com (Stephane DROUARD)
Date: Fri, 3 Apr 2009 11:57:15 +0200
Subject: [Cython] Cython 0.11.1 alpha
In-Reply-To: <65D2DAA2-A974-4DA5-A818-276A539A8320@math.washington.edu>
Message-ID: <008501c9b442$91c38180$76ac810a@gnb.st.com>

Yes it solves it.

I confirm that Cython/Compiler/Nodes.py really has patch
http://hg.cython.org/cython-devel/rev/5d12e7cb8dd9, but stdexcept does not
appear in the generated .C file.

Cheers,
Stephane

-----Original Message-----
From: cython-dev-bounces at codespeak.net
[mailto:cython-dev-bounces at codespeak.net] On Behalf Of Robert Bradshaw
Sent: Friday, April 03, 2009 11:34 AM
To: cython-dev at codespeak.net
Subject: Re: [Cython] Cython 0.11.1 alpha

On Apr 3, 2009, at 2:28 AM, Stephane DROUARD wrote:

> Hello,
>
> The fix for http://trac.cython.org/cython_trac/ticket/265 does not 
> solve the issue, we get the same error (stdexcept not included).
>
> Cheers,
> Stephane

Does a

cdef extern from "stdexcept":
     pass

resolve the issue?

- Robert

_______________________________________________
Cython-dev mailing list
Cython-dev at codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev


From stefan_ml at behnel.de  Fri Apr  3 12:24:19 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 3 Apr 2009 12:24:19 +0200 (CEST)
Subject: [Cython] Cython 0.11.1 alpha
In-Reply-To: <F169EE31-6CB7-465F-AB5C-3635C5E24E54@math.washington.edu>
References: <45493547-5D3E-47BD-B526-7AF0B5610871@math.washington.edu>
	<49D5D6DA.9030201@student.matnat.uio.no>
	<49D5D761.10900@student.matnat.uio.no>
	<b82fe26b85789bb4d050ff1ae9624745.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<49D5D94D.9020108@student.matnat.uio.no>
	<F169EE31-6CB7-465F-AB5C-3635C5E24E54@math.washington.edu>
Message-ID: <292c74b8f0dfc4f3218137788f507ae0.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Robert Bradshaw wrote:
> On Apr 3, 2009, at 2:39 AM, Dag Sverre Seljebotn wrote:
>> The question is why we have a major/minor naming scheme at all. It
>> could be
>>
>> a) Depending on how much time's elapsed since the last major. That's
>> roughly our current policy :-)
>
> Well, there's a strong correlation between time elapsed and new
> features.

At least for 0.11, there was also a strong relation between time elapsed
and size of cleanups that needed major testing and consolidation. So the
above doesn't /really/ reflect our current policy.

For the same reason, cython-unstable should become a major 0.xy release.
It changes too many things (not break, just change) to just become a minor
release.


>> b) Substantial new features means new major. That's one guildeline,
>> which speaks for naming this one 0.11.1.
>>
>> c) New major when backwards compatability is broken in any way. That's
>> another, conflicting guideline, which speaks for naming this one 0.12.
>>
>> Myself I'm -1 on a) and +0 on b) and c).
>
> I'm primarily guided by (b), where "substantial new features" may
> include internal re-factoring that requires lots of testing.

+1, even without the "major testing" bit. Substantial code changes do not
belong into a minor release.


> I think
> (c) is important too, but am not a stickler being pedantically strict
> on this. It should be very safe to do a 0.x.y upgrade, no promises
> that 0.y won't force you to change your code (for the better, though
> it should be avoided--hopefully just making people remove bad/
> ambiguous/abusive code, or stuff like cdivision).

That would rather speak for not making major semantic changes before 0.12.
Still, I do consider the loop semantic fixes real bug fixes. The only
reason to go to 0.12 right away would be to say "sorry, 0.11 was a
mistake, it's dead now, please upgrade". I don't think that's true.

Stefan


From psp.reachable at gmail.com  Fri Apr  3 12:25:44 2009
From: psp.reachable at gmail.com (Prajwal Suhas P)
Date: Fri, 3 Apr 2009 15:55:44 +0530
Subject: [Cython] [GSoC Proposal] Control flow analysis for Cython
In-Reply-To: <77629cb3d9b318f34fdd6e5c48a5f949.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
References: <fb17e5d0904030210q51e8a855ic531f943aca0da7c@mail.gmail.com>
	<77629cb3d9b318f34fdd6e5c48a5f949.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <fb17e5d0904030325h7ae07e0ep7a5dfecd7c52cff@mail.gmail.com>

On Fri, Apr 3, 2009 at 3:12 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:

> Hi,
>
> Prajwal Suhas P wrote:
> > This is my proposal for gsoc.
> > http://wiki.cython.org/gsoc/prajwal
> >
> > Please feel free to give suggestions/comment on it.
>
> My main comment is that your proposal seems to reduce CFA to None check
> avoidance. While this is a valid example for CFA application, it does not
> quite catch what CFA is by itself. Possible variable values are one thing
> that you can *infer* from a control flow graph, possible types are
> another.
>
Currently i am looking at the None check case for cython CFA. I haven't
covered other cases for CFA application and i thought of adding them as the
project progresses.

>
> But the primary goal of this project should be to establish a suitable
> representation of the possible execution paths inside a function (and,
> less importantly, the global module code). So this really is about finding
> (as in reading up) and implementing an efficient way for establishing,
> maintaining and querying the control state of a function at each parse
> tree node.
>
> Stefan

I have not gone into the depths of generating a CFG for the possible
execution paths inside a function. I will give more thought into
establishing a CFG and update my app.


> _________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Regards,
Prajwal S.

"There is little difference in people,but this little difference makes a big
difference.The little difference is attitude,the big difference is whether
its positive or negative. "
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20090403/1ec4a79b/attachment.htm 

From robertwb at math.washington.edu  Fri Apr  3 12:49:31 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Apr 2009 03:49:31 -0700
Subject: [Cython] [GSoC Proposal] Control flow analysis for Cython
In-Reply-To: <fb17e5d0904030325h7ae07e0ep7a5dfecd7c52cff@mail.gmail.com>
References: <fb17e5d0904030210q51e8a855ic531f943aca0da7c@mail.gmail.com>
	<77629cb3d9b318f34fdd6e5c48a5f949.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<fb17e5d0904030325h7ae07e0ep7a5dfecd7c52cff@mail.gmail.com>
Message-ID: <82986048-FCAB-47D9-B945-6C61B6426A97@math.washington.edu>

On Apr 3, 2009, at 3:25 AM, Prajwal Suhas P wrote:

> On Fri, Apr 3, 2009 at 3:12 PM, Stefan Behnel <stefan_ml at behnel.de>  
> wrote:
> Hi,
>
> Prajwal Suhas P wrote:
> > This is my proposal for gsoc.
> > http://wiki.cython.org/gsoc/prajwal
> >
> > Please feel free to give suggestions/comment on it.
>
>> My main comment is that your proposal seems to reduce CFA to None  
>> check
>> avoidance. While this is a valid example for CFA application, it  
>> does not
>> quite catch what CFA is by itself. Possible variable values are  
>> one thing
>> that you can *infer* from a control flow graph, possible types are
>> another.
> Currently i am looking at the None check case for cython CFA. I  
> haven't covered other cases for CFA application and i thought of  
> adding them as the project progresses.

I actually suggested the None check case as a concrete goal for the  
midterm evaluation, as I felt that section needed a real milestone  
and measuring stick. Hopefully by that point the framework would be  
in place enough to be useful in this capacity (even if it will need  
further refining).

>> But the primary goal of this project should be to establish a  
>> suitable
>> representation of the possible execution paths inside a function  
>> (and,
>> less importantly, the global module code). So this really is about  
>> finding
>> (as in reading up) and implementing an efficient way for  
>> establishing,
>> maintaining and querying the control state of a function at each  
>> parse
>> tree node.
>>
>> Stefan
>

+1 Something to this effect should be mentioned in the final evaluation.

- Robert


From robertwb at math.washington.edu  Fri Apr  3 12:51:19 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Apr 2009 03:51:19 -0700
Subject: [Cython] Cython 0.11.1 alpha
In-Reply-To: <008501c9b442$91c38180$76ac810a@gnb.st.com>
References: <008501c9b442$91c38180$76ac810a@gnb.st.com>
Message-ID: <01C05F17-A4B3-48AE-9303-70E721C24B9E@math.washington.edu>

On Apr 3, 2009, at 2:57 AM, Stephane DROUARD wrote:

> Yes it solves it.
>
> I confirm that Cython/Compiler/Nodes.py really has patch
> http://hg.cython.org/cython-devel/rev/5d12e7cb8dd9, but stdexcept  
> does not
> appear in the generated .C file.

Hmm... I'll look into this some more.

> Cheers,
> Stephane
>
> -----Original Message-----
> From: cython-dev-bounces at codespeak.net
> [mailto:cython-dev-bounces at codespeak.net] On Behalf Of Robert Bradshaw
> Sent: Friday, April 03, 2009 11:34 AM
> To: cython-dev at codespeak.net
> Subject: Re: [Cython] Cython 0.11.1 alpha
>
> On Apr 3, 2009, at 2:28 AM, Stephane DROUARD wrote:
>
>> Hello,
>>
>> The fix for http://trac.cython.org/cython_trac/ticket/265 does not
>> solve the issue, we get the same error (stdexcept not included).
>>
>> Cheers,
>> Stephane
>
> Does a
>
> cdef extern from "stdexcept":
>      pass
>
> resolve the issue?
>
> - Robert
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From robertwb at math.washington.edu  Fri Apr  3 13:03:39 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Apr 2009 04:03:39 -0700
Subject: [Cython] Cython 0.11.1 alpha
In-Reply-To: <292c74b8f0dfc4f3218137788f507ae0.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
References: <45493547-5D3E-47BD-B526-7AF0B5610871@math.washington.edu>
	<49D5D6DA.9030201@student.matnat.uio.no>
	<49D5D761.10900@student.matnat.uio.no>
	<b82fe26b85789bb4d050ff1ae9624745.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<49D5D94D.9020108@student.matnat.uio.no>
	<F169EE31-6CB7-465F-AB5C-3635C5E24E54@math.washington.edu>
	<292c74b8f0dfc4f3218137788f507ae0.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <928EDE1E-BA8F-422E-A2A9-6B91EEBEFC9D@math.washington.edu>

On Apr 3, 2009, at 3:24 AM, Stefan Behnel wrote:

> Robert Bradshaw wrote:
>> On Apr 3, 2009, at 2:39 AM, Dag Sverre Seljebotn wrote:
>>> The question is why we have a major/minor naming scheme at all. It
>>> could be
>>>
>>> a) Depending on how much time's elapsed since the last major. That's
>>> roughly our current policy :-)
>>
>> Well, there's a strong correlation between time elapsed and new
>> features.
>
> At least for 0.11, there was also a strong relation between time  
> elapsed
> and size of cleanups that needed major testing and consolidation.  
> So the
> above doesn't /really/ reflect our current policy.

Hopefully with many smaller releases this will become less of an  
issue :)

> For the same reason, cython-unstable should become a major 0.xy  
> release.
> It changes too many things (not break, just change) to just become  
> a minor
> release.
>
>
>>> b) Substantial new features means new major. That's one guildeline,
>>> which speaks for naming this one 0.11.1.
>>>
>>> c) New major when backwards compatability is broken in any way.  
>>> That's
>>> another, conflicting guideline, which speaks for naming this one  
>>> 0.12.
>>>
>>> Myself I'm -1 on a) and +0 on b) and c).
>>
>> I'm primarily guided by (b), where "substantial new features" may
>> include internal re-factoring that requires lots of testing.
>
> +1, even without the "major testing" bit. Substantial code changes  
> do not
> belong into a minor release.
>
>
>> I think
>> (c) is important too, but am not a stickler being pedantically strict
>> on this. It should be very safe to do a 0.x.y upgrade, no promises
>> that 0.y won't force you to change your code (for the better, though
>> it should be avoided--hopefully just making people remove bad/
>> ambiguous/abusive code, or stuff like cdivision).
>
> That would rather speak for not making major semantic changes  
> before 0.12.
> Still, I do consider the loop semantic fixes real bug fixes. The only
> reason to go to 0.12 right away would be to say "sorry, 0.11 was a
> mistake, it's dead now, please upgrade". I don't think that's true.


There's a lot of good stuff in 0.11.1, but nothing big that I feel  
warrants a "full version" upgrade. Also, it will be good to push out  
stuff like the cdivision directives and warnings.

- Robert


From robertwb at math.washington.edu  Fri Apr  3 13:12:50 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Apr 2009 04:12:50 -0700
Subject: [Cython] Patches in trac
Message-ID: <296D9E11-7465-42CF-BF92-68A8EE110BFC@math.washington.edu>

I'm planning on setting up a workflow [1] for trac to better keep  
track of tickets that have/need tests/patches. In the meantime, I  
occasionally run into tickets that have patches that I was completely  
unaware of. I hate to let things bitrot, so I'm starting this thread  
to ask if anyone has patches that are sitting in trac (or the email  
archives) that need to be looked at.

- Robert

[1] http://trac.edgewall.org/wiki/WorkFlow

From stefan_ml at behnel.de  Fri Apr  3 13:19:54 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 3 Apr 2009 13:19:54 +0200 (CEST)
Subject: [Cython] Patches in trac
In-Reply-To: <296D9E11-7465-42CF-BF92-68A8EE110BFC@math.washington.edu>
References: <296D9E11-7465-42CF-BF92-68A8EE110BFC@math.washington.edu>
Message-ID: <152284139ca49507df22d43385148d71.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Robert Bradshaw wrote:
> I'm planning on setting up a workflow [1] for trac to better keep
> track of tickets that have/need tests/patches. In the meantime, I
> occasionally run into tickets that have patches that I was completely
> unaware of. I hate to let things bitrot, so I'm starting this thread
> to ask if anyone has patches that are sitting in trac (or the email
> archives) that need to be looked at.

Isn't there a way to write a query for tickets that have attachments?

Stefan


From robertwb at math.washington.edu  Fri Apr  3 13:30:49 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Apr 2009 04:30:49 -0700
Subject: [Cython] Patches in trac
In-Reply-To: <152284139ca49507df22d43385148d71.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
References: <296D9E11-7465-42CF-BF92-68A8EE110BFC@math.washington.edu>
	<152284139ca49507df22d43385148d71.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <AF206718-4A55-4646-8030-1294ECC58040@math.washington.edu>

On Apr 3, 2009, at 4:19 AM, Stefan Behnel wrote:

> Robert Bradshaw wrote:
>> I'm planning on setting up a workflow [1] for trac to better keep
>> track of tickets that have/need tests/patches. In the meantime, I
>> occasionally run into tickets that have patches that I was completely
>> unaware of. I hate to let things bitrot, so I'm starting this thread
>> to ask if anyone has patches that are sitting in trac (or the email
>> archives) that need to be looked at.
>
> Isn't there a way to write a query for tickets that have attachments?

I didn't see one (in my very superficial search) but there likely is.  
Also I'd like to distinguish between patches needing review, patches  
needing work, and patches with positive review (but not yet committed).

- Robert


From stefan_ml at behnel.de  Fri Apr  3 15:40:48 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 3 Apr 2009 15:40:48 +0200 (CEST)
Subject: [Cython] Patches in trac
In-Reply-To: <AF206718-4A55-4646-8030-1294ECC58040@math.washington.edu>
References: <296D9E11-7465-42CF-BF92-68A8EE110BFC@math.washington.edu>
	<152284139ca49507df22d43385148d71.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<AF206718-4A55-4646-8030-1294ECC58040@math.washington.edu>
Message-ID: <7584a7e82fb3a008284fbaa94b5e85d6.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Robert Bradshaw wrote:
> On Apr 3, 2009, at 4:19 AM, Stefan Behnel wrote:
>
>> Robert Bradshaw wrote:
>>> I'm planning on setting up a workflow [1] for trac to better keep
>>> track of tickets that have/need tests/patches. In the meantime, I
>>> occasionally run into tickets that have patches that I was completely
>>> unaware of. I hate to let things bitrot, so I'm starting this thread
>>> to ask if anyone has patches that are sitting in trac (or the email
>>> archives) that need to be looked at.
>>
>> Isn't there a way to write a query for tickets that have attachments?
>
> I didn't see one (in my very superficial search) but there likely is.
> Also I'd like to distinguish between patches needing review, patches
> needing work, and patches with positive review (but not yet committed).

Cool. +1

Stefan


From stefan_ml at behnel.de  Fri Apr  3 15:40:48 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 3 Apr 2009 15:40:48 +0200 (CEST)
Subject: [Cython] Patches in trac
In-Reply-To: <AF206718-4A55-4646-8030-1294ECC58040@math.washington.edu>
References: <296D9E11-7465-42CF-BF92-68A8EE110BFC@math.washington.edu>
	<152284139ca49507df22d43385148d71.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<AF206718-4A55-4646-8030-1294ECC58040@math.washington.edu>
Message-ID: <7584a7e82fb3a008284fbaa94b5e85d6.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Robert Bradshaw wrote:
> On Apr 3, 2009, at 4:19 AM, Stefan Behnel wrote:
>
>> Robert Bradshaw wrote:
>>> I'm planning on setting up a workflow [1] for trac to better keep
>>> track of tickets that have/need tests/patches. In the meantime, I
>>> occasionally run into tickets that have patches that I was completely
>>> unaware of. I hate to let things bitrot, so I'm starting this thread
>>> to ask if anyone has patches that are sitting in trac (or the email
>>> archives) that need to be looked at.
>>
>> Isn't there a way to write a query for tickets that have attachments?
>
> I didn't see one (in my very superficial search) but there likely is.
> Also I'd like to distinguish between patches needing review, patches
> needing work, and patches with positive review (but not yet committed).

Cool. +1

Stefan


From dalcinl at gmail.com  Fri Apr  3 19:21:17 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Fri, 3 Apr 2009 14:21:17 -0300
Subject: [Cython] new patch for #255 (Py->C for integrals) in the tracker
In-Reply-To: <80E1EA88-7452-461E-8EA6-CF788CA4909A@math.washington.edu>
References: <e7ba66e40904021639r397e1c5dg6063b0c88176ff54@mail.gmail.com>
	<80E1EA88-7452-461E-8EA6-CF788CA4909A@math.washington.edu>
Message-ID: <e7ba66e40904031021n6c6915a9p587356057aa89a27@mail.gmail.com>

Rovert, I've uploaded a new version of the patch following your
suggestions (though slightly different).

Please, see also my last comments on the tracker.

Unfortunately, I'll not be able to work further on this until Monday.
Good weekend!


On Thu, Apr 2, 2009 at 10:52 PM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Apr 2, 2009, at 4:39 PM, Lisandro Dalcin wrote:
>
>> - The changes to ModuleNode.py and Utils.py are just quick hacks to
>> get it working.
>>
>> - I've tried hard to avoid generation of unreachable code like these
>> "if (0) {...}" blocks.
>>
>> - Once this is reviewed, commented and eventually accepted, I'll
>> continue extending this to extern ctypedef integrals.
>
> I've taken a look at the code and it looks good to me. You took care
> of every corner case I thought of checking. I haven't tested it under
> Py3, but it works great under Py2. It might be worth noting that I
> improved the utility code templating to take a type since this is
> such a common case: http://hg.cython.org/cython-devel/file/
> 2289ab00261d/Cython/Utils.py
>
> One comment I had is that in your code
>
> static INLINE TYPE __Pyx_PyInt_AsTYPE(PyObject* x) {
> ? ? if (sizeof(TYPE) < sizeof(long)) {
> ? ? ? ? long val = __Pyx_PyInt_AsLong(x);
> ? ? ? ? if (unlikely(val == -1 && PyErr_Occurred()))
> ? ? ? ? ? ? return (TYPE)-1;
> ? ? ? ? if (unlikely(val < 0)) {
> ? ? ? ? ? ? PyErr_SetString(PyExc_OverflowError,
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? "can't convert negative value to TYPE");
> ? ? ? ? ? ? return (TYPE)-1;
> ? ? ? ? }
> ? ? ? ? if (unlikely(val != (long)(TYPE)val)) {
> ? ? ? ? ? ? PyErr_SetString(PyExc_OverflowError,
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? "value too large to convert to TYPE");
> ? ? ? ? ? ? return (TYPE)-1;
> ? ? ? ? }
> ? ? ? ? return (TYPE)val;
> ? ? }
> ? ? return (TYPE)__Pyx_PyInt_AsUnsignedLong(x);
> }
>
> I believe the successful pathway could be shorted by doing
>
> static INLINE TYPE __Pyx_PyInt_AsTYPE(PyObject* x) {
> ? ? if (sizeof(TYPE) < sizeof(long)) {
> ? ? ? ? long val = __Pyx_PyInt_AsLong(x);
> ? ? ? ? if (unlikely(val != (long)(TYPE)val)) {
> ? ? ? ? ? ? if (unlikely(val == -1 && PyErr_Occurred()));
> ? ? ? ? ? ? else if (val < 0) {
> ? ? ? ? ? ? ? ? PyErr_SetString(PyExc_OverflowError,
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "can't convert negative value to
> TYPE");
> ? ? ? ? ? ? }
> ? ? ? ? ? ? else {
> ? ? ? ? ? ? ? ? PyErr_SetString(PyExc_OverflowError,
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "value too large to convert to TYPE");
> ? ? ? ? ? ? }
> ? ? ? ? ? ? return (TYPE)-1;
> ? ? ? ? }
> ? ? ? ? return (TYPE)val;
> ? ? }
> ? ? return (TYPE)__Pyx_PyInt_AsUnsignedLong(x);
> }
>
> as sizeof(TYPE) < sizeof(long) guarantees val != (long)(TYPE)val for
> negative val on unsigned TYPE and the error does not need to be
> explicitly tested for otherwise (as the calling function will check
> PyErr_Occurred on a return value of -1).
>
> - Robert
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From robertwb at math.washington.edu  Fri Apr  3 20:44:23 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Apr 2009 11:44:23 -0700
Subject: [Cython] new patch for #255 (Py->C for integrals) in the tracker
In-Reply-To: <e7ba66e40904031021n6c6915a9p587356057aa89a27@mail.gmail.com>
References: <e7ba66e40904021639r397e1c5dg6063b0c88176ff54@mail.gmail.com>
	<80E1EA88-7452-461E-8EA6-CF788CA4909A@math.washington.edu>
	<e7ba66e40904031021n6c6915a9p587356057aa89a27@mail.gmail.com>
Message-ID: <8C4DB521-BB67-4524-863A-0D271947C787@math.washington.edu>

On Apr 3, 2009, at 10:21 AM, Lisandro Dalcin wrote:

> Rovert, I've uploaded a new version of the patch following your
> suggestions (though slightly different).
>
> Please, see also my last comments on the tracker.

Cool. I'll take a look.

> Unfortunately, I'll not be able to work further on this until Monday.
> Good weekend!

That's fine. I don't know that I'll have time to resolve the Sage  
issues before then either, so I think we can still get this in 0.11.1.

- Robert

>
>
> On Thu, Apr 2, 2009 at 10:52 PM, Robert Bradshaw
> <robertwb at math.washington.edu> wrote:
>> On Apr 2, 2009, at 4:39 PM, Lisandro Dalcin wrote:
>>
>>> - The changes to ModuleNode.py and Utils.py are just quick hacks to
>>> get it working.
>>>
>>> - I've tried hard to avoid generation of unreachable code like these
>>> "if (0) {...}" blocks.
>>>
>>> - Once this is reviewed, commented and eventually accepted, I'll
>>> continue extending this to extern ctypedef integrals.
>>
>> I've taken a look at the code and it looks good to me. You took care
>> of every corner case I thought of checking. I haven't tested it under
>> Py3, but it works great under Py2. It might be worth noting that I
>> improved the utility code templating to take a type since this is
>> such a common case: http://hg.cython.org/cython-devel/file/
>> 2289ab00261d/Cython/Utils.py
>>
>> One comment I had is that in your code
>>
>> static INLINE TYPE __Pyx_PyInt_AsTYPE(PyObject* x) {
>>     if (sizeof(TYPE) < sizeof(long)) {
>>         long val = __Pyx_PyInt_AsLong(x);
>>         if (unlikely(val == -1 && PyErr_Occurred()))
>>             return (TYPE)-1;
>>         if (unlikely(val < 0)) {
>>             PyErr_SetString(PyExc_OverflowError,
>>                             "can't convert negative value to TYPE");
>>             return (TYPE)-1;
>>         }
>>         if (unlikely(val != (long)(TYPE)val)) {
>>             PyErr_SetString(PyExc_OverflowError,
>>                             "value too large to convert to TYPE");
>>             return (TYPE)-1;
>>         }
>>         return (TYPE)val;
>>     }
>>     return (TYPE)__Pyx_PyInt_AsUnsignedLong(x);
>> }
>>
>> I believe the successful pathway could be shorted by doing
>>
>> static INLINE TYPE __Pyx_PyInt_AsTYPE(PyObject* x) {
>>     if (sizeof(TYPE) < sizeof(long)) {
>>         long val = __Pyx_PyInt_AsLong(x);
>>         if (unlikely(val != (long)(TYPE)val)) {
>>             if (unlikely(val == -1 && PyErr_Occurred()));
>>             else if (val < 0) {
>>                 PyErr_SetString(PyExc_OverflowError,
>>                                 "can't convert negative value to
>> TYPE");
>>             }
>>             else {
>>                 PyErr_SetString(PyExc_OverflowError,
>>                                 "value too large to convert to  
>> TYPE");
>>             }
>>             return (TYPE)-1;
>>         }
>>         return (TYPE)val;
>>     }
>>     return (TYPE)__Pyx_PyInt_AsUnsignedLong(x);
>> }
>>
>> as sizeof(TYPE) < sizeof(long) guarantees val != (long)(TYPE)val for
>> negative val on unsigned TYPE and the error does not need to be
>> explicitly tested for otherwise (as the calling function will check
>> PyErr_Occurred on a return value of -1).
>>
>> - Robert
>>
>> _______________________________________________
>> Cython-dev mailing list
>> Cython-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/cython-dev
>>
>
>
>
> -- 
> Lisandro Dalc?n
> ---------------
> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
> Tel/Fax: +54-(0)342-451.1594
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From stefan_ml at behnel.de  Fri Apr  3 20:52:08 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 03 Apr 2009 20:52:08 +0200
Subject: [Cython] [GSoC Proposal] Control flow analysis for Cython
In-Reply-To: <82986048-FCAB-47D9-B945-6C61B6426A97@math.washington.edu>
References: <fb17e5d0904030210q51e8a855ic531f943aca0da7c@mail.gmail.com>	<77629cb3d9b318f34fdd6e5c48a5f949.squirrel@groupware.dvs.informatik.tu-darmstadt.de>	<fb17e5d0904030325h7ae07e0ep7a5dfecd7c52cff@mail.gmail.com>
	<82986048-FCAB-47D9-B945-6C61B6426A97@math.washington.edu>
Message-ID: <49D65AD8.1030705@behnel.de>


Robert Bradshaw wrote:
> On Apr 3, 2009, at 3:25 AM, Prajwal Suhas P wrote:
>> Currently i am looking at the None check case for cython CFA. I  
>> haven't covered other cases for CFA application and i thought of  
>> adding them as the project progresses.
> 
> I actually suggested the None check case as a concrete goal for the  
> midterm evaluation, as I felt that section needed a real milestone  
> and measuring stick. Hopefully by that point the framework would be  
> in place enough to be useful in this capacity (even if it will need  
> further refining).

Oh, I'm perfectly fine with a simple application as a first target. I just
wanted to make sure Prajwal (is that your first name?) understands that
this is really just an application and not the goal by itself. The
implementation will have to be a lot more powerful, and thus less trivial
than the application indicates, and the proposal didn't make it obvious to
me that that was clear.

Stefan

From stefan_ml at behnel.de  Fri Apr  3 20:56:42 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 03 Apr 2009 20:56:42 +0200
Subject: [Cython] [GSoC Proposal] Improve C++ Support
In-Reply-To: <49D5CFE7.6070100@student.matnat.uio.no>
References: <45239150904021957l1287e0f5p5e42c7ae7462c991@mail.gmail.com>	<fea78156d40597b8a7c8ee69ff9013d0.squirrel@webmail.uio.no>	<892997BA-44F0-4707-96BD-016239A5540A@math.washington.edu>
	<49D5CFE7.6070100@student.matnat.uio.no>
Message-ID: <49D65BEA.9040709@behnel.de>


Dag Sverre Seljebotn wrote:
> [some interesting stuff that doesn't currently work (easily) in Cython]

Are these examples in the Wiki somewhere? Would be great to have a page
that just lists stuff that's non-trivial in current Cython and needs
improvement. That makes it easier to focus on those problems that bring the
best relieve.

Stefan

From robertwb at math.washington.edu  Sat Apr  4 01:04:54 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Apr 2009 16:04:54 -0700
Subject: [Cython] Cython 0.11.1 alpha
In-Reply-To: <01C05F17-A4B3-48AE-9303-70E721C24B9E@math.washington.edu>
References: <008501c9b442$91c38180$76ac810a@gnb.st.com>
	<01C05F17-A4B3-48AE-9303-70E721C24B9E@math.washington.edu>
Message-ID: <9A615BC8-3FDA-40C8-8C70-19E8847B7352@math.washington.edu>

On Apr 3, 2009, at 3:51 AM, Robert Bradshaw wrote:

> On Apr 3, 2009, at 2:57 AM, Stephane DROUARD wrote:
>
>> Yes it solves it.
>>
>> I confirm that Cython/Compiler/Nodes.py really has patch
>> http://hg.cython.org/cython-devel/rev/5d12e7cb8dd9, but stdexcept
>> does not
>> appear in the generated .C file.
>
> Hmm... I'll look into this some more.

Thanks for the feedback--turns out I was only taking care of 2 of the  
3 cases. Should be fixed now.

- Robert


From dsurviver at gmail.com  Sat Apr  4 01:49:41 2009
From: dsurviver at gmail.com (Danilo Freitas)
Date: Fri, 3 Apr 2009 20:49:41 -0300
Subject: [Cython] [GSoC Proposal] Improve C++ Support
In-Reply-To: <49D65BEA.9040709@behnel.de>
References: <45239150904021957l1287e0f5p5e42c7ae7462c991@mail.gmail.com>
	<fea78156d40597b8a7c8ee69ff9013d0.squirrel@webmail.uio.no>
	<892997BA-44F0-4707-96BD-016239A5540A@math.washington.edu>
	<49D5CFE7.6070100@student.matnat.uio.no> <49D65BEA.9040709@behnel.de>
Message-ID: <45239150904031649i78d7f645m3ae4517f5d1bcd53@mail.gmail.com>

So, I think one thing I need to change, is that instead of creating a
template syntax, I'll work on improving an already existing syntax
right? And so that it should be useful all templates (including STL)?

And should I also work on those ohter problems? I'm afraid of putting
many things on proposal, and don't finish it in time. :/

2009/4/3 Stefan Behnel <stefan_ml at behnel.de>:
>
> Dag Sverre Seljebotn wrote:
>> [some interesting stuff that doesn't currently work (easily) in Cython]
>
> Are these examples in the Wiki somewhere? Would be great to have a page
> that just lists stuff that's non-trivial in current Cython and needs
> improvement. That makes it easier to focus on those problems that bring the
> best relieve.
>
> Stefan
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>

From dsurviver at gmail.com  Sat Apr  4 01:52:06 2009
From: dsurviver at gmail.com (Danilo Freitas)
Date: Fri, 3 Apr 2009 20:52:06 -0300
Subject: [Cython] [GSoC Proposal] Improve C++ Support
In-Reply-To: <45239150904031649i78d7f645m3ae4517f5d1bcd53@mail.gmail.com>
References: <45239150904021957l1287e0f5p5e42c7ae7462c991@mail.gmail.com>
	<fea78156d40597b8a7c8ee69ff9013d0.squirrel@webmail.uio.no>
	<892997BA-44F0-4707-96BD-016239A5540A@math.washington.edu>
	<49D5CFE7.6070100@student.matnat.uio.no> <49D65BEA.9040709@behnel.de>
	<45239150904031649i78d7f645m3ae4517f5d1bcd53@mail.gmail.com>
Message-ID: <45239150904031652k346c56a7i4215e8242c6d3f97@mail.gmail.com>

Oh... looking now, time is over. :/

I stayed all the evening at university, and got home now. :/

But thanks for the sugestions. If my proposal get accepted, I'll try
to work on the other problems

Thanks for all :D

2009/4/3 Danilo Freitas <dsurviver at gmail.com>:
> So, I think one thing I need to change, is that instead of creating a
> template syntax, I'll work on improving an already existing syntax
> right? And so that it should be useful all templates (including STL)?
>
> And should I also work on those ohter problems? I'm afraid of putting
> many things on proposal, and don't finish it in time. :/
>
> 2009/4/3 Stefan Behnel <stefan_ml at behnel.de>:
>>
>> Dag Sverre Seljebotn wrote:
>>> [some interesting stuff that doesn't currently work (easily) in Cython]
>>
>> Are these examples in the Wiki somewhere? Would be great to have a page
>> that just lists stuff that's non-trivial in current Cython and needs
>> improvement. That makes it easier to focus on those problems that bring the
>> best relieve.
>>
>> Stefan
>> _______________________________________________
>> Cython-dev mailing list
>> Cython-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/cython-dev
>>
>

From robertwb at math.washington.edu  Sat Apr  4 03:10:46 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Apr 2009 18:10:46 -0700
Subject: [Cython] [GSoC Proposal] Improve C++ Support
In-Reply-To: <45239150904031649i78d7f645m3ae4517f5d1bcd53@mail.gmail.com>
References: <45239150904021957l1287e0f5p5e42c7ae7462c991@mail.gmail.com>
	<fea78156d40597b8a7c8ee69ff9013d0.squirrel@webmail.uio.no>
	<892997BA-44F0-4707-96BD-016239A5540A@math.washington.edu>
	<49D5CFE7.6070100@student.matnat.uio.no>
	<49D65BEA.9040709@behnel.de>
	<45239150904031649i78d7f645m3ae4517f5d1bcd53@mail.gmail.com>
Message-ID: <483CA17C-4364-465E-BF62-F87A56815222@math.washington.edu>

On Apr 3, 2009, at 4:49 PM, Danilo Freitas wrote:

> So, I think one thing I need to change, is that instead of creating a
> template syntax, I'll work on improving an already existing syntax
> right?

Probably, yes.

> And so that it should be useful all templates (including STL)?
>
> And should I also work on those ohter problems? I'm afraid of putting
> many things on proposal, and don't finish it in time. :/

As expected, this could turn into a huge wishlist of things that  
would be nice to have, certainly not all feasible for a GSoC  
project :). I would say

(1) Being able to wrap templated code
(2) Polyomorphic functions

would be the priorities, and allow a nearly full wrapping of STL. If  
time permits

(3) operator overloading (we have this for Python classes of course,  
but to allow the + operator, etc. to be used for declared C++ classes)

would be good to.

> 2009/4/3 Stefan Behnel <stefan_ml at behnel.de>:
>>
>> Dag Sverre Seljebotn wrote:
>>> [some interesting stuff that doesn't currently work (easily) in  
>>> Cython]
>>
>> Are these examples in the Wiki somewhere? Would be great to have a  
>> page
>> that just lists stuff that's non-trivial in current Cython and needs
>> improvement. That makes it easier to focus on those problems that  
>> bring the
>> best relieve.
>>
>> Stefan
>> _______________________________________________
>> Cython-dev mailing list
>> Cython-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/cython-dev
>>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From vic.kelson at gmail.com  Sat Apr  4 07:13:05 2009
From: vic.kelson at gmail.com (Vic Kelson)
Date: Sat, 4 Apr 2009 01:13:05 -0400
Subject: [Cython] Status of C complex types
Message-ID: <d5e3120f0904032213nab26d2cl39ff29b196f1bd0c@mail.gmail.com>

Greetings all,
I'm in need of fast computations on complex numbers from Cython. I've been
converting a Python groundwater flow model code to Cython, but I'm reaching
the point where I'd really prefer to have native C complex math. Up to now,
I've used some little C routines that I can call with multiple floats (the C
code converts to/from complex for my purposes). However, I'm nearly to the
point where I'll need to store series coefficients for performance purposes,
and then I'll either need to hide them in float arrays (and that's extremely
nasty), or something like that.

I'd like to say something like



-- 
Vic Kelson
vic.kelson at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20090404/80f69b4e/attachment.htm 

From vic.kelson at gmail.com  Sat Apr  4 07:21:24 2009
From: vic.kelson at gmail.com (Vic Kelson)
Date: Sat, 4 Apr 2009 01:21:24 -0400
Subject: [Cython] Status of C complex types (re-send)
Message-ID: <d5e3120f0904032221v43ee94cdseec8a1fd859f5b63@mail.gmail.com>

Greetings all,
Sorry for the incomplete message, I'm re-sending it.

I'm in need of fast computations on complex numbers from Cython. I've been
converting a Python groundwater flow model code to Cython, but I'm reaching
the point where I'd really prefer to have native C complex math. Up to now,
I've used some little C routines that I can call with multiple floats (the C
code converts to/from complex for my purposes). However, I'm nearly to the
point where I'll need to store series coefficients for performance purposes,
and then I'll either need to hide them in float arrays (and that's extremely
nasty), or something like that.

I'd like to say something like

cdef class Foo:
    cdef cfloat z

    def __cinit__(self, float x, float y):
        self.z = cfloat(x, y)

    cdef cfloat func(Foo self, float x, float y):
        return-some-gnarly-function of x, y, and self.z


Or something like that, where 'cfloat' maps to the appropriate type in the C
compiler, e.g. "double _Complex" in gcc.

I've seen this discussed in the archives. Has it been implemented? Is there
some pre-release code that essentially works? I only need the four math
functions, conjugate, abs, and log.

THANKS! I think Cython will revolutionize my research work!

--vic
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20090404/764215b3/attachment.htm 

From robertwb at math.washington.edu  Sat Apr  4 07:37:48 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Apr 2009 22:37:48 -0700
Subject: [Cython] Status of C complex types (re-send)
In-Reply-To: <d5e3120f0904032221v43ee94cdseec8a1fd859f5b63@mail.gmail.com>
References: <d5e3120f0904032221v43ee94cdseec8a1fd859f5b63@mail.gmail.com>
Message-ID: <488BAB80-5474-4BE8-9ED9-9C33133CFFDE@math.washington.edu>

On Apr 3, 2009, at 10:21 PM, Vic Kelson wrote:

> Greetings all,
>
> Sorry for the incomplete message, I'm re-sending it.
>
> I'm in need of fast computations on complex numbers from Cython.  
> I've been converting a Python groundwater flow model code to  
> Cython, but I'm reaching the point where I'd really prefer to have  
> native C complex math. Up to now, I've used some little C routines  
> that I can call with multiple floats (the C code converts to/from  
> complex for my purposes). However, I'm nearly to the point where  
> I'll need to store series coefficients for performance purposes,  
> and then I'll either need to hide them in float arrays (and that's  
> extremely nasty), or something like that.
>
> I'd like to say something like
>
> cdef class Foo:
>     cdef cfloat z
>
>     def __cinit__(self, float x, float y):
>         self.z = cfloat(x, y)
>
>     cdef cfloat func(Foo self, float x, float y):
>         return-some-gnarly-function of x, y, and self.z
>
>
> Or something like that, where 'cfloat' maps to the appropriate type  
> in the C compiler, e.g. "double _Complex" in gcc.
>
> I've seen this discussed in the archives. Has it been implemented?  
> Is there some pre-release code that essentially works? I only need  
> the four math functions, conjugate, abs, and log.

Yes, I've started this, as it comes up in my research too (number  
theory, computing values of L-functions). I hope to have something  
releasable soon. In the meantime, you can do

cdef extern from "complex.h":
     ctypedef double cdouble "double complex"
     cdef double creal(cdouble)
     cdef double cimag(cdouble)
     cdef cdouble _Complex_I

cdef inline cdouble  new_cdouble(double x, double y):
     return x + _Complex_I*y

Of course, the ctypedef is wrong, it will think it can go back to  
doubles (and Python objects) without any problem, but it works for now.

> THANKS! I think Cython will revolutionize my research work!

That's the goal :) Please cite us if it does.

- Robert


From dagss at student.matnat.uio.no  Sat Apr  4 08:44:10 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Sat, 4 Apr 2009 08:44:10 +0200 (CEST)
Subject: [Cython] [GSoC Proposal] Improve C++ Support
In-Reply-To: <483CA17C-4364-465E-BF62-F87A56815222@math.washington.edu>
References: <45239150904021957l1287e0f5p5e42c7ae7462c991@mail.gmail.com>
	<fea78156d40597b8a7c8ee69ff9013d0.squirrel@webmail.uio.no>
	<892997BA-44F0-4707-96BD-016239A5540A@math.washington.edu>
	<49D5CFE7.6070100@student.matnat.uio.no> <49D65BEA.9040709@behnel.de>
	<45239150904031649i78d7f645m3ae4517f5d1bcd53@mail.gmail.com>
	<483CA17C-4364-465E-BF62-F87A56815222@math.washington.edu>
Message-ID: <bb7be68cea3d7c54707756b339cf9983.squirrel@webmail.uio.no>

Robert Bradshaw wrote:
> On Apr 3, 2009, at 4:49 PM, Danilo Freitas wrote:
>
>> So, I think one thing I need to change, is that instead of creating a
>> template syntax, I'll work on improving an already existing syntax
>> right?
>
> Probably, yes.
>
>> And so that it should be useful all templates (including STL)?
>>
>> And should I also work on those ohter problems? I'm afraid of putting
>> many things on proposal, and don't finish it in time. :/
>
> As expected, this could turn into a huge wishlist of things that
> would be nice to have, certainly not all feasible for a GSoC
> project :). I would say
>
> (1) Being able to wrap templated code

What's needed is
a) Parse new syntax for declaring templates
b) Register that information in PyrexTypes/Symtab
c) Output the type arguments directly to C++.

so it's not very much work (on the same order as complex float support, say).

> (2) Polyomorphic functions
>
> would be the priorities, and allow a nearly full wrapping of STL. If
> time permits

Doesn't polymorphic functions work currently? I'd think that were one of
the few C++ things that *do* work in Cython.

Do you mean "overloaded" functions?

Dag Sverre


From robertwb at math.washington.edu  Sat Apr  4 09:25:03 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 4 Apr 2009 00:25:03 -0700
Subject: [Cython] [GSoC Proposal] Improve C++ Support
In-Reply-To: <bb7be68cea3d7c54707756b339cf9983.squirrel@webmail.uio.no>
References: <45239150904021957l1287e0f5p5e42c7ae7462c991@mail.gmail.com>
	<fea78156d40597b8a7c8ee69ff9013d0.squirrel@webmail.uio.no>
	<892997BA-44F0-4707-96BD-016239A5540A@math.washington.edu>
	<49D5CFE7.6070100@student.matnat.uio.no>
	<49D65BEA.9040709@behnel.de>
	<45239150904031649i78d7f645m3ae4517f5d1bcd53@mail.gmail.com>
	<483CA17C-4364-465E-BF62-F87A56815222@math.washington.edu>
	<bb7be68cea3d7c54707756b339cf9983.squirrel@webmail.uio.no>
Message-ID: <A6F88E16-6A49-418C-A6E8-879105B187A4@math.washington.edu>

On Apr 3, 2009, at 11:44 PM, Dag Sverre Seljebotn wrote:

> Robert Bradshaw wrote:
>> On Apr 3, 2009, at 4:49 PM, Danilo Freitas wrote:
>>
>>> So, I think one thing I need to change, is that instead of  
>>> creating a
>>> template syntax, I'll work on improving an already existing syntax
>>> right?
>>
>> Probably, yes.
>>
>>> And so that it should be useful all templates (including STL)?
>>>
>>> And should I also work on those ohter problems? I'm afraid of  
>>> putting
>>> many things on proposal, and don't finish it in time. :/

BTW, despite being past the deadline, I think the proposal can be  
updated while it's being evaluated.

>> As expected, this could turn into a huge wishlist of things that
>> would be nice to have, certainly not all feasible for a GSoC
>> project :). I would say
>>
>> (1) Being able to wrap templated code
>
> What's needed is
> a) Parse new syntax for declaring templates
> b) Register that information in PyrexTypes/Symtab
> c) Output the type arguments directly to C++.
>
> so it's not very much work (on the same order as complex float  
> support, say).
>
>> (2) Polyomorphic functions
>>
>> would be the priorities, and allow a nearly full wrapping of STL. If
>> time permits
>
> Doesn't polymorphic functions work currently? I'd think that were  
> one of
> the few C++ things that *do* work in Cython.
>
> Do you mean "overloaded" functions?

Yes, that's the more specific term I really meant.

- Robert



From dagss at student.matnat.uio.no  Sat Apr  4 10:34:37 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Sat, 04 Apr 2009 10:34:37 +0200
Subject: [Cython] [GSoC Proposal] Improve C++ Support
In-Reply-To: <A6F88E16-6A49-418C-A6E8-879105B187A4@math.washington.edu>
References: <45239150904021957l1287e0f5p5e42c7ae7462c991@mail.gmail.com>	<fea78156d40597b8a7c8ee69ff9013d0.squirrel@webmail.uio.no>	<892997BA-44F0-4707-96BD-016239A5540A@math.washington.edu>	<49D5CFE7.6070100@student.matnat.uio.no>	<49D65BEA.9040709@behnel.de>	<45239150904031649i78d7f645m3ae4517f5d1bcd53@mail.gmail.com>	<483CA17C-4364-465E-BF62-F87A56815222@math.washington.edu>	<bb7be68cea3d7c54707756b339cf9983.squirrel@webmail.uio.no>
	<A6F88E16-6A49-418C-A6E8-879105B187A4@math.washington.edu>
Message-ID: <49D71B9D.9010601@student.matnat.uio.no>

Robert Bradshaw wrote:
> On Apr 3, 2009, at 11:44 PM, Dag Sverre Seljebotn wrote:
> 
>> Robert Bradshaw wrote:
>>> On Apr 3, 2009, at 4:49 PM, Danilo Freitas wrote:
>>>
>>>> So, I think one thing I need to change, is that instead of  
>>>> creating a
>>>> template syntax, I'll work on improving an already existing syntax
>>>> right?
>>> Probably, yes.
>>>
>>>> And so that it should be useful all templates (including STL)?
>>>>
>>>> And should I also work on those ohter problems? I'm afraid of  
>>>> putting
>>>> many things on proposal, and don't finish it in time. :/
> 
> BTW, despite being past the deadline, I think the proposal can be  
> updated while it's being evaluated.

At least it can be updated on the wiki page, which is just as important.

>>> (2) Polyomorphic functions
>>>
>>> would be the priorities, and allow a nearly full wrapping of STL. If
>>> time permits
>> Doesn't polymorphic functions work currently? I'd think that were  
>> one of
>> the few C++ things that *do* work in Cython.
>>
>> Do you mean "overloaded" functions?
> 
> Yes, that's the more specific term I really meant.

:-) I guess polymorphism means a lot of things. But at least in 
C++-lingo it's a common phrase for "virtual methods", i.e. vtables, 
which made it even more confusing.

-- 
Dag Sverre

From psp.reachable at gmail.com  Sat Apr  4 16:42:58 2009
From: psp.reachable at gmail.com (Prajwal Suhas P)
Date: Sat, 4 Apr 2009 20:12:58 +0530
Subject: [Cython] [GSoC Proposal] Control flow analysis for Cython
In-Reply-To: <49D65AD8.1030705@behnel.de>
References: <fb17e5d0904030210q51e8a855ic531f943aca0da7c@mail.gmail.com>
	<77629cb3d9b318f34fdd6e5c48a5f949.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<fb17e5d0904030325h7ae07e0ep7a5dfecd7c52cff@mail.gmail.com>
	<82986048-FCAB-47D9-B945-6C61B6426A97@math.washington.edu>
	<49D65AD8.1030705@behnel.de>
Message-ID: <fb17e5d0904040742q7cf7b65fi79f90fe955fcc483@mail.gmail.com>

On Sat, Apr 4, 2009 at 12:22 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:

>
> Robert Bradshaw wrote:
> > On Apr 3, 2009, at 3:25 AM, Prajwal Suhas P wrote:
> >> Currently i am looking at the None check case for cython CFA. I
> >> haven't covered other cases for CFA application and i thought of
> >> adding them as the project progresses.
> >
> > I actually suggested the None check case as a concrete goal for the
> > midterm evaluation, as I felt that section needed a real milestone
> > and measuring stick. Hopefully by that point the framework would be
> > in place enough to be useful in this capacity (even if it will need
> > further refining).
>
> Oh, I'm perfectly fine with a simple application as a first target. I just
> wanted to make sure Prajwal (is that your first name?)
>
> Stefan
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>

 Yes, Prajwal is my first name



-- 
Regards,
Prajwal S.

"There is little difference in people,but this little difference makes a big
difference.The little difference is attitude,the big difference is whether
its positive or negative. "
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20090404/25fc033f/attachment-0001.htm 

From vic.kelson at gmail.com  Sat Apr  4 18:09:07 2009
From: vic.kelson at gmail.com (Vic Kelson)
Date: Sat, 4 Apr 2009 12:09:07 -0400
Subject: [Cython] Status of C complex types (re-send)
In-Reply-To: <488BAB80-5474-4BE8-9ED9-9C33133CFFDE@math.washington.edu>
References: <d5e3120f0904032221v43ee94cdseec8a1fd859f5b63@mail.gmail.com>
	<488BAB80-5474-4BE8-9ED9-9C33133CFFDE@math.washington.edu>
Message-ID: <d5e3120f0904040909v69416410vf032ffb155adba56@mail.gmail.com>

Robert,
Thatnks for your helpful note. I can't seem to make it work with gcc
on OS X. I keep getting an error from gcc. I haven't had time to sort
through it yet. I think I'll build a little test problem (I tried
embedding your cod in my existing code and I might have created
another error).
Just to be sure, when I use your code to make a cdouble, the real and
imaginary parts are the same size as a python float, right?
Thanks!
--v


On 4/4/09, Robert Bradshaw <robertwb at math.washington.edu> wrote:
> On Apr 3, 2009, at 10:21 PM, Vic Kelson wrote:
>
>> Greetings all,
>>
>> Sorry for the incomplete message, I'm re-sending it.
>>
>> I'm in need of fast computations on complex numbers from Cython.
>> I've been converting a Python groundwater flow model code to
>> Cython, but I'm reaching the point where I'd really prefer to have
>> native C complex math. Up to now, I've used some little C routines
>> that I can call with multiple floats (the C code converts to/from
>> complex for my purposes). However, I'm nearly to the point where
>> I'll need to store series coefficients for performance purposes,
>> and then I'll either need to hide them in float arrays (and that's
>> extremely nasty), or something like that.
>>
>> I'd like to say something like
>>
>> cdef class Foo:
>>     cdef cfloat z
>>
>>     def __cinit__(self, float x, float y):
>>         self.z = cfloat(x, y)
>>
>>     cdef cfloat func(Foo self, float x, float y):
>>         return-some-gnarly-function of x, y, and self.z
>>
>>
>> Or something like that, where 'cfloat' maps to the appropriate type
>> in the C compiler, e.g. "double _Complex" in gcc.
>>
>> I've seen this discussed in the archives. Has it been implemented?
>> Is there some pre-release code that essentially works? I only need
>> the four math functions, conjugate, abs, and log.
>
> Yes, I've started this, as it comes up in my research too (number
> theory, computing values of L-functions). I hope to have something
> releasable soon. In the meantime, you can do
>
> cdef extern from "complex.h":
>      ctypedef double cdouble "double complex"
>      cdef double creal(cdouble)
>      cdef double cimag(cdouble)
>      cdef cdouble _Complex_I
>
> cdef inline cdouble  new_cdouble(double x, double y):
>      return x + _Complex_I*y
>
> Of course, the ctypedef is wrong, it will think it can go back to
> doubles (and Python objects) without any problem, but it works for now.
>
>> THANKS! I think Cython will revolutionize my research work!
>
> That's the goal :) Please cite us if it does.
>
> - Robert
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>

-- 
Sent from my mobile device

Vic Kelson
vic.kelson at gmail.com

From stefan_ml at behnel.de  Sat Apr  4 20:41:35 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 04 Apr 2009 20:41:35 +0200
Subject: [Cython] [GSoC Proposal] Improve C++ Support
In-Reply-To: <45239150904031649i78d7f645m3ae4517f5d1bcd53@mail.gmail.com>
References: <45239150904021957l1287e0f5p5e42c7ae7462c991@mail.gmail.com>	<fea78156d40597b8a7c8ee69ff9013d0.squirrel@webmail.uio.no>	<892997BA-44F0-4707-96BD-016239A5540A@math.washington.edu>	<49D5CFE7.6070100@student.matnat.uio.no>
	<49D65BEA.9040709@behnel.de>
	<45239150904031649i78d7f645m3ae4517f5d1bcd53@mail.gmail.com>
Message-ID: <49D7A9DF.4080907@behnel.de>

Hi,

please avoid top-posting. Thanks.

Danilo Freitas wrote:
> 2009/4/3 Stefan Behnel <stefan_ml at behnel.de>:
>> Dag Sverre Seljebotn wrote:
>>> [some interesting stuff that doesn't currently work (easily) in Cython]
>> Are these examples in the Wiki somewhere? Would be great to have a page
>> that just lists stuff that's non-trivial in current Cython and needs
>> improvement. That makes it easier to focus on those problems that bring the
>> best relieve.
>
> So, I think one thing I need to change, is that instead of creating a
> template syntax, I'll work on improving an already existing syntax
> right? And so that it should be useful all templates (including STL)?
> 
> And should I also work on those ohter problems? I'm afraid of putting
> many things on proposal, and don't finish it in time. :/

I didn't mean that everything should appear in your proposal. It's
perfectly right to limit the stated goals to those that appear clearly
reachable.

But it would be helpful to have a separate Wiki page dedicated to
describing missing C++ support features in general (whether they get
implemented (and when) or not) just to keep them in sight.

Stefan

From robertwb at math.washington.edu  Sat Apr  4 20:57:16 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 4 Apr 2009 11:57:16 -0700
Subject: [Cython] Status of C complex types (re-send)
In-Reply-To: <d5e3120f0904040909v69416410vf032ffb155adba56@mail.gmail.com>
References: <d5e3120f0904032221v43ee94cdseec8a1fd859f5b63@mail.gmail.com>
	<488BAB80-5474-4BE8-9ED9-9C33133CFFDE@math.washington.edu>
	<d5e3120f0904040909v69416410vf032ffb155adba56@mail.gmail.com>
Message-ID: <29DB5FDB-9FB8-4291-ADC5-5B3234BE4EEF@math.washington.edu>

On Apr 4, 2009, at 9:09 AM, Vic Kelson wrote:

> Robert,
> Thatnks for your helpful note. I can't seem to make it work with gcc
> on OS X.

Hmm... that's the same platform I use.

> I keep getting an error from gcc. I haven't had time to sort
> through it yet. I think I'll build a little test problem (I tried
> embedding your cod in my existing code and I might have created
> another error).

Maybe you could post a snippet of the error you're getting?

> Just to be sure, when I use your code to make a cdouble, the real and
> imaginary parts are the same size as a python float, right?

Yes. Python floats are wrappers of double (and Python complexes wrap  
a pair of doubles).

> Thanks!
> --v
>
>
> On 4/4/09, Robert Bradshaw <robertwb at math.washington.edu> wrote:
>> On Apr 3, 2009, at 10:21 PM, Vic Kelson wrote:
>>
>>> Greetings all,
>>>
>>> Sorry for the incomplete message, I'm re-sending it.
>>>
>>> I'm in need of fast computations on complex numbers from Cython.
>>> I've been converting a Python groundwater flow model code to
>>> Cython, but I'm reaching the point where I'd really prefer to have
>>> native C complex math. Up to now, I've used some little C routines
>>> that I can call with multiple floats (the C code converts to/from
>>> complex for my purposes). However, I'm nearly to the point where
>>> I'll need to store series coefficients for performance purposes,
>>> and then I'll either need to hide them in float arrays (and that's
>>> extremely nasty), or something like that.
>>>
>>> I'd like to say something like
>>>
>>> cdef class Foo:
>>>     cdef cfloat z
>>>
>>>     def __cinit__(self, float x, float y):
>>>         self.z = cfloat(x, y)
>>>
>>>     cdef cfloat func(Foo self, float x, float y):
>>>         return-some-gnarly-function of x, y, and self.z
>>>
>>>
>>> Or something like that, where 'cfloat' maps to the appropriate type
>>> in the C compiler, e.g. "double _Complex" in gcc.
>>>
>>> I've seen this discussed in the archives. Has it been implemented?
>>> Is there some pre-release code that essentially works? I only need
>>> the four math functions, conjugate, abs, and log.
>>
>> Yes, I've started this, as it comes up in my research too (number
>> theory, computing values of L-functions). I hope to have something
>> releasable soon. In the meantime, you can do
>>
>> cdef extern from "complex.h":
>>      ctypedef double cdouble "double complex"
>>      cdef double creal(cdouble)
>>      cdef double cimag(cdouble)
>>      cdef cdouble _Complex_I
>>
>> cdef inline cdouble  new_cdouble(double x, double y):
>>      return x + _Complex_I*y
>>
>> Of course, the ctypedef is wrong, it will think it can go back to
>> doubles (and Python objects) without any problem, but it works for  
>> now.
>>
>>> THANKS! I think Cython will revolutionize my research work!
>>
>> That's the goal :) Please cite us if it does.
>>
>> - Robert
>>
>> _______________________________________________
>> Cython-dev mailing list
>> Cython-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/cython-dev
>>
>
> -- 
> Sent from my mobile device
>
> Vic Kelson
> vic.kelson at gmail.com
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From vic.kelson at gmail.com  Sun Apr  5 02:26:37 2009
From: vic.kelson at gmail.com (Vic Kelson)
Date: Sat, 4 Apr 2009 20:26:37 -0400
Subject: [Cython] Status of C complex types (re-send)
In-Reply-To: <29DB5FDB-9FB8-4291-ADC5-5B3234BE4EEF@math.washington.edu>
References: <d5e3120f0904032221v43ee94cdseec8a1fd859f5b63@mail.gmail.com>
	<488BAB80-5474-4BE8-9ED9-9C33133CFFDE@math.washington.edu>
	<d5e3120f0904040909v69416410vf032ffb155adba56@mail.gmail.com>
	<29DB5FDB-9FB8-4291-ADC5-5B3234BE4EEF@math.washington.edu>
Message-ID: <d5e3120f0904041726ide1e08dxa5638e3cd9c2cd4@mail.gmail.com>

Robert,
Well, as I suspected, it's all my fault! I have  built a tiny module with
your code, a C function and a C class, and it all works! Now to get the
larger code going... Then I'll do some performance testing against some
older Fortran code. If I get 80%, I'm staying with Cython/Python!

THANKS!
--v

On Sat, Apr 4, 2009 at 2:57 PM, Robert Bradshaw <
robertwb at math.washington.edu> wrote:

> On Apr 4, 2009, at 9:09 AM, Vic Kelson wrote:
>
> > Robert,
> > Thatnks for your helpful note. I can't seem to make it work with gcc
> > on OS X.
>
> Hmm... that's the same platform I use.
>
> > I keep getting an error from gcc. I haven't had time to sort
> > through it yet. I think I'll build a little test problem (I tried
> > embedding your cod in my existing code and I might have created
> > another error).
>
> Maybe you could post a snippet of the error you're getting?
>
> > Just to be sure, when I use your code to make a cdouble, the real and
> > imaginary parts are the same size as a python float, right?
>
> Yes. Python floats are wrappers of double (and Python complexes wrap
> a pair of doubles).
>
> > Thanks!
> > --v
> >
> >
> > On 4/4/09, Robert Bradshaw <robertwb at math.washington.edu> wrote:
> >> On Apr 3, 2009, at 10:21 PM, Vic Kelson wrote:
> >>
> >>> Greetings all,
> >>>
> >>> Sorry for the incomplete message, I'm re-sending it.
> >>>
> >>> I'm in need of fast computations on complex numbers from Cython.
> >>> I've been converting a Python groundwater flow model code to
> >>> Cython, but I'm reaching the point where I'd really prefer to have
> >>> native C complex math. Up to now, I've used some little C routines
> >>> that I can call with multiple floats (the C code converts to/from
> >>> complex for my purposes). However, I'm nearly to the point where
> >>> I'll need to store series coefficients for performance purposes,
> >>> and then I'll either need to hide them in float arrays (and that's
> >>> extremely nasty), or something like that.
> >>>
> >>> I'd like to say something like
> >>>
> >>> cdef class Foo:
> >>>     cdef cfloat z
> >>>
> >>>     def __cinit__(self, float x, float y):
> >>>         self.z = cfloat(x, y)
> >>>
> >>>     cdef cfloat func(Foo self, float x, float y):
> >>>         return-some-gnarly-function of x, y, and self.z
> >>>
> >>>
> >>> Or something like that, where 'cfloat' maps to the appropriate type
> >>> in the C compiler, e.g. "double _Complex" in gcc.
> >>>
> >>> I've seen this discussed in the archives. Has it been implemented?
> >>> Is there some pre-release code that essentially works? I only need
> >>> the four math functions, conjugate, abs, and log.
> >>
> >> Yes, I've started this, as it comes up in my research too (number
> >> theory, computing values of L-functions). I hope to have something
> >> releasable soon. In the meantime, you can do
> >>
> >> cdef extern from "complex.h":
> >>      ctypedef double cdouble "double complex"
> >>      cdef double creal(cdouble)
> >>      cdef double cimag(cdouble)
> >>      cdef cdouble _Complex_I
> >>
> >> cdef inline cdouble  new_cdouble(double x, double y):
> >>      return x + _Complex_I*y
> >>
> >> Of course, the ctypedef is wrong, it will think it can go back to
> >> doubles (and Python objects) without any problem, but it works for
> >> now.
> >>
> >>> THANKS! I think Cython will revolutionize my research work!
> >>
> >> That's the goal :) Please cite us if it does.
> >>
> >> - Robert
> >>
> >> _______________________________________________
> >> Cython-dev mailing list
> >> Cython-dev at codespeak.net
> >> http://codespeak.net/mailman/listinfo/cython-dev
> >>
> >
> > --
> > Sent from my mobile device
> >
> > Vic Kelson
> > vic.kelson at gmail.com
> > _______________________________________________
> > Cython-dev mailing list
> > Cython-dev at codespeak.net
> > http://codespeak.net/mailman/listinfo/cython-dev
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Vic Kelson
vic.kelson at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20090404/5827b21d/attachment.htm 

From vic.kelson at gmail.com  Sun Apr  5 06:28:09 2009
From: vic.kelson at gmail.com (Vic Kelson)
Date: Sun, 5 Apr 2009 00:28:09 -0400
Subject: [Cython] Status of C complex types (re-send)
In-Reply-To: <29DB5FDB-9FB8-4291-ADC5-5B3234BE4EEF@math.washington.edu>
References: <d5e3120f0904032221v43ee94cdseec8a1fd859f5b63@mail.gmail.com>
	<488BAB80-5474-4BE8-9ED9-9C33133CFFDE@math.washington.edu>
	<d5e3120f0904040909v69416410vf032ffb155adba56@mail.gmail.com>
	<29DB5FDB-9FB8-4291-ADC5-5B3234BE4EEF@math.washington.edu>
Message-ID: <d5e3120f0904042128p84022d5m296981dbdda8c1eb@mail.gmail.com>

Another question: If I need the constant pi, and I use math.pi or numpy.pi
in a C function (or a C method of a C class), does it add an inefficiency
(Python lookup)?
Just wondering...
--v

On Sat, Apr 4, 2009 at 2:57 PM, Robert Bradshaw <
robertwb at math.washington.edu> wrote:

> On Apr 4, 2009, at 9:09 AM, Vic Kelson wrote:
>
> > Robert,
> > Thatnks for your helpful note. I can't seem to make it work with gcc
> > on OS X.
>
> Hmm... that's the same platform I use.
>
> > I keep getting an error from gcc. I haven't had time to sort
> > through it yet. I think I'll build a little test problem (I tried
> > embedding your cod in my existing code and I might have created
> > another error).
>
> Maybe you could post a snippet of the error you're getting?
>
> > Just to be sure, when I use your code to make a cdouble, the real and
> > imaginary parts are the same size as a python float, right?
>
> Yes. Python floats are wrappers of double (and Python complexes wrap
> a pair of doubles).
>
> > Thanks!
> > --v
> >
> >
> > On 4/4/09, Robert Bradshaw <robertwb at math.washington.edu> wrote:
> >> On Apr 3, 2009, at 10:21 PM, Vic Kelson wrote:
> >>
> >>> Greetings all,
> >>>
> >>> Sorry for the incomplete message, I'm re-sending it.
> >>>
> >>> I'm in need of fast computations on complex numbers from Cython.
> >>> I've been converting a Python groundwater flow model code to
> >>> Cython, but I'm reaching the point where I'd really prefer to have
> >>> native C complex math. Up to now, I've used some little C routines
> >>> that I can call with multiple floats (the C code converts to/from
> >>> complex for my purposes). However, I'm nearly to the point where
> >>> I'll need to store series coefficients for performance purposes,
> >>> and then I'll either need to hide them in float arrays (and that's
> >>> extremely nasty), or something like that.
> >>>
> >>> I'd like to say something like
> >>>
> >>> cdef class Foo:
> >>>     cdef cfloat z
> >>>
> >>>     def __cinit__(self, float x, float y):
> >>>         self.z = cfloat(x, y)
> >>>
> >>>     cdef cfloat func(Foo self, float x, float y):
> >>>         return-some-gnarly-function of x, y, and self.z
> >>>
> >>>
> >>> Or something like that, where 'cfloat' maps to the appropriate type
> >>> in the C compiler, e.g. "double _Complex" in gcc.
> >>>
> >>> I've seen this discussed in the archives. Has it been implemented?
> >>> Is there some pre-release code that essentially works? I only need
> >>> the four math functions, conjugate, abs, and log.
> >>
> >> Yes, I've started this, as it comes up in my research too (number
> >> theory, computing values of L-functions). I hope to have something
> >> releasable soon. In the meantime, you can do
> >>
> >> cdef extern from "complex.h":
> >>      ctypedef double cdouble "double complex"
> >>      cdef double creal(cdouble)
> >>      cdef double cimag(cdouble)
> >>      cdef cdouble _Complex_I
> >>
> >> cdef inline cdouble  new_cdouble(double x, double y):
> >>      return x + _Complex_I*y
> >>
> >> Of course, the ctypedef is wrong, it will think it can go back to
> >> doubles (and Python objects) without any problem, but it works for
> >> now.
> >>
> >>> THANKS! I think Cython will revolutionize my research work!
> >>
> >> That's the goal :) Please cite us if it does.
> >>
> >> - Robert
> >>
> >> _______________________________________________
> >> Cython-dev mailing list
> >> Cython-dev at codespeak.net
> >> http://codespeak.net/mailman/listinfo/cython-dev
> >>
> >
> > --
> > Sent from my mobile device
> >
> > Vic Kelson
> > vic.kelson at gmail.com
> > _______________________________________________
> > Cython-dev mailing list
> > Cython-dev at codespeak.net
> > http://codespeak.net/mailman/listinfo/cython-dev
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Vic Kelson
vic.kelson at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20090405/8adf7d66/attachment.htm 

From wstein at gmail.com  Sun Apr  5 06:53:51 2009
From: wstein at gmail.com (William Stein)
Date: Sat, 4 Apr 2009 21:53:51 -0700
Subject: [Cython] Status of C complex types (re-send)
In-Reply-To: <d5e3120f0904042128p84022d5m296981dbdda8c1eb@mail.gmail.com>
References: <d5e3120f0904032221v43ee94cdseec8a1fd859f5b63@mail.gmail.com> 
	<488BAB80-5474-4BE8-9ED9-9C33133CFFDE@math.washington.edu> 
	<d5e3120f0904040909v69416410vf032ffb155adba56@mail.gmail.com> 
	<29DB5FDB-9FB8-4291-ADC5-5B3234BE4EEF@math.washington.edu> 
	<d5e3120f0904042128p84022d5m296981dbdda8c1eb@mail.gmail.com>
Message-ID: <85e81ba30904042153u562db1cek6d94ead0f939e901@mail.gmail.com>

On Sat, Apr 4, 2009 at 9:28 PM, Vic Kelson <vic.kelson at gmail.com> wrote:
> Another question: If I need the constant pi, and I use math.pi or numpy.pi
> in a C function (or a C method of a C class), does it add an inefficiency
> (Python lookup)?
> Just wondering...

Yes.  Use Cython's annotate option (or just reading the output) shows
that the "k=math.pi" code in the following:

import math
cdef foo():
    cdef double k
    k = math.pi

generates the following C:

    __pyx_1 = __Pyx_GetName(__pyx_m, __pyx_kp_math); if
(unlikely(!__pyx_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 9;
__pyx_clineno = __LINE__; goto __pyx_L1_error;}
  __pyx_2 = PyObject_GetAttr(__pyx_1, __pyx_kp_pi); if
(unlikely(!__pyx_2)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 9;
__pyx_clineno = __LINE__; goto __pyx_L1_error;}
  Py_DECREF(__pyx_1); __pyx_1 = 0;
  __pyx_3 = __pyx_PyFloat_AsDouble(__pyx_2); if
(unlikely(PyErr_Occurred())) {__pyx_filename = __pyx_f[0];
__pyx_lineno = 9; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
  Py_DECREF(__pyx_2); __pyx_2 = 0;
  __pyx_v_k = __pyx_3;


Timing the following two code snippets shows that using math.pi
instead of a hardcoded double constant for pi can easily slow down a
function by a *factor of 100*.

import math
def foo(int n):
    cdef double k = 0
    cdef int i
    for i in range(n):
        k += math.pi
    return k


import math
def foo2(int n):
    cdef double k=0, pi = 3.14159265358979
    cdef int i
    for i in range(n):
        k += pi
    return k


sage: timeit('foo(10**5)')
25 loops, best of 3: 12.5 ms per loop
sage: timeit('foo2(10**5)')
625 loops, best of 3: 114 ??s per loop
sage: 12.5/0.114
109.649122807018

If you want to write very fast code, you might learn and use the -a
option to Cython.

William

 -- William

From dagss at student.matnat.uio.no  Sun Apr  5 18:13:25 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Sun, 5 Apr 2009 18:13:25 +0200 (CEST)
Subject: [Cython] [GSoC Proposal] Improve C++ Support
In-Reply-To: <49D65BEA.9040709@behnel.de>
References: <45239150904021957l1287e0f5p5e42c7ae7462c991@mail.gmail.com>
	<fea78156d40597b8a7c8ee69ff9013d0.squirrel@webmail.uio.no>
	<892997BA-44F0-4707-96BD-016239A5540A@math.washington.edu>
	<49D5CFE7.6070100@student.matnat.uio.no> <49D65BEA.9040709@behnel.de>
Message-ID: <c1d44784e56a49ac2ae45f67f5d7aa9f.squirrel@webmail.uio.no>

>
> Dag Sverre Seljebotn wrote:
>> [some interesting stuff that doesn't currently work (easily) in Cython]
>
> Are these examples in the Wiki somewhere? Would be great to have a page
> that just lists stuff that's non-trivial in current Cython and needs
> improvement. That makes it easier to focus on those problems that bring
> the
> best relieve.

OK I've fleshed it out and found some more points not supported
(especially upcasts!). I'm sure there's more :-)

http://wiki.cython.org/enhancements/cpp

Dag Sverre


From hoytak at cs.ubc.ca  Sun Apr  5 21:26:15 2009
From: hoytak at cs.ubc.ca (Hoyt Koepke)
Date: Sun, 5 Apr 2009 12:26:15 -0700
Subject: [Cython] Py_New still good way to create classes?
Message-ID: <4db580fd0904051226u64d9dfe5n73674e3d258ba677@mail.gmail.com>

Hello,

I'm using an extension class where I need to create a number of
instances quickly, and I'm looking at the section "Can Cython create
objects or apply operators to locally created objects as pure C code?"
on http://wiki.cython.org/FAQ.  Is this information still correct?

In particular, I'm wondering about it in reference to supporting
cyclical garbage collection and creating object via PyObject_GC_New
etc.  IIRC, cython automatically supports cyclical garbage collection
on classes that have python members, and my extension class falls into
this category.  Should I

a) Use Py_NEW, as specified in the FAQ?
b) Replace Py_NEW with Py_Object_GC_New, and do a?
c) Ignore this and just go with standard creation and deletion,
ignoring the fact that I'll be calling the constructors through
python.

Thanks!
--Hoyt

++++++++++++++++++++++++++++++++++++++++++++++++
+ Hoyt Koepke
+ University of Washington Department of Statistics
+ http://www.stat.washington.edu/~hoytak/
+ hoytak at gmail.com
++++++++++++++++++++++++++++++++++++++++++

From stephane.drouard at st.com  Mon Apr  6 09:36:24 2009
From: stephane.drouard at st.com (Stephane DROUARD)
Date: Mon, 6 Apr 2009 09:36:24 +0200
Subject: [Cython] Cython 0.11.1 alpha
In-Reply-To: <9A615BC8-3FDA-40C8-8C70-19E8847B7352@math.washington.edu>
Message-ID: <003001c9b68a$63e71340$76ac810a@gnb.st.com>

On Apr 3, 2009, at 3:51 AM, Robert Bradshaw wrote:

> Thanks for the feedback--turns out I was only taking care of 2 of the
> 3 cases. Should be fixed now.

Applying patch http://hg.cython.org/cython-devel/rev/49523eb3abb5, it solves
the case I reported. Thanks.

Stephane
 


From dalcinl at gmail.com  Mon Apr  6 18:19:46 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 6 Apr 2009 13:19:46 -0300
Subject: [Cython] Status of C complex types (re-send)
In-Reply-To: <d5e3120f0904042128p84022d5m296981dbdda8c1eb@mail.gmail.com>
References: <d5e3120f0904032221v43ee94cdseec8a1fd859f5b63@mail.gmail.com>
	<488BAB80-5474-4BE8-9ED9-9C33133CFFDE@math.washington.edu>
	<d5e3120f0904040909v69416410vf032ffb155adba56@mail.gmail.com>
	<29DB5FDB-9FB8-4291-ADC5-5B3234BE4EEF@math.washington.edu>
	<d5e3120f0904042128p84022d5m296981dbdda8c1eb@mail.gmail.com>
Message-ID: <e7ba66e40904060919v20fa2709y8baac3d98d646b57@mail.gmail.com>

I would do this (at de module-level):

import math
cdef double PI = math.pi

of perhaps

cdef double PI = 3.14...

and just use "PI" on you cdef functions...

On Sun, Apr 5, 2009 at 1:28 AM, Vic Kelson <vic.kelson at gmail.com> wrote:
> Another question: If I need the constant pi, and I use math.pi or numpy.pi
> in a C function (or a C method of a C class), does it add an inefficiency
> (Python lookup)?
> Just wondering...
> --v
>
> On Sat, Apr 4, 2009 at 2:57 PM, Robert Bradshaw
> <robertwb at math.washington.edu> wrote:
>>
>> On Apr 4, 2009, at 9:09 AM, Vic Kelson wrote:
>>
>> > Robert,
>> > Thatnks for your helpful note. I can't seem to make it work with gcc
>> > on OS X.
>>
>> Hmm... that's the same platform I use.
>>
>> > I keep getting an error from gcc. I haven't had time to sort
>> > through it yet. I think I'll build a little test problem (I tried
>> > embedding your cod in my existing code and I might have created
>> > another error).
>>
>> Maybe you could post a snippet of the error you're getting?
>>
>> > Just to be sure, when I use your code to make a cdouble, the real and
>> > imaginary parts are the same size as a python float, right?
>>
>> Yes. Python floats are wrappers of double (and Python complexes wrap
>> a pair of doubles).
>>
>> > Thanks!
>> > --v
>> >
>> >
>> > On 4/4/09, Robert Bradshaw <robertwb at math.washington.edu> wrote:
>> >> On Apr 3, 2009, at 10:21 PM, Vic Kelson wrote:
>> >>
>> >>> Greetings all,
>> >>>
>> >>> Sorry for the incomplete message, I'm re-sending it.
>> >>>
>> >>> I'm in need of fast computations on complex numbers from Cython.
>> >>> I've been converting a Python groundwater flow model code to
>> >>> Cython, but I'm reaching the point where I'd really prefer to have
>> >>> native C complex math. Up to now, I've used some little C routines
>> >>> that I can call with multiple floats (the C code converts to/from
>> >>> complex for my purposes). However, I'm nearly to the point where
>> >>> I'll need to store series coefficients for performance purposes,
>> >>> and then I'll either need to hide them in float arrays (and that's
>> >>> extremely nasty), or something like that.
>> >>>
>> >>> I'd like to say something like
>> >>>
>> >>> cdef class Foo:
>> >>> ? ? cdef cfloat z
>> >>>
>> >>> ? ? def __cinit__(self, float x, float y):
>> >>> ? ? ? ? self.z = cfloat(x, y)
>> >>>
>> >>> ? ? cdef cfloat func(Foo self, float x, float y):
>> >>> ? ? ? ? return-some-gnarly-function of x, y, and self.z
>> >>>
>> >>>
>> >>> Or something like that, where 'cfloat' maps to the appropriate type
>> >>> in the C compiler, e.g. "double _Complex" in gcc.
>> >>>
>> >>> I've seen this discussed in the archives. Has it been implemented?
>> >>> Is there some pre-release code that essentially works? I only need
>> >>> the four math functions, conjugate, abs, and log.
>> >>
>> >> Yes, I've started this, as it comes up in my research too (number
>> >> theory, computing values of L-functions). I hope to have something
>> >> releasable soon. In the meantime, you can do
>> >>
>> >> cdef extern from "complex.h":
>> >> ? ? ?ctypedef double cdouble "double complex"
>> >> ? ? ?cdef double creal(cdouble)
>> >> ? ? ?cdef double cimag(cdouble)
>> >> ? ? ?cdef cdouble _Complex_I
>> >>
>> >> cdef inline cdouble ?new_cdouble(double x, double y):
>> >> ? ? ?return x + _Complex_I*y
>> >>
>> >> Of course, the ctypedef is wrong, it will think it can go back to
>> >> doubles (and Python objects) without any problem, but it works for
>> >> now.
>> >>
>> >>> THANKS! I think Cython will revolutionize my research work!
>> >>
>> >> That's the goal :) Please cite us if it does.
>> >>
>> >> - Robert
>> >>
>> >> _______________________________________________
>> >> Cython-dev mailing list
>> >> Cython-dev at codespeak.net
>> >> http://codespeak.net/mailman/listinfo/cython-dev
>> >>
>> >
>> > --
>> > Sent from my mobile device
>> >
>> > Vic Kelson
>> > vic.kelson at gmail.com
>> > _______________________________________________
>> > Cython-dev mailing list
>> > Cython-dev at codespeak.net
>> > http://codespeak.net/mailman/listinfo/cython-dev
>>
>> _______________________________________________
>> Cython-dev mailing list
>> Cython-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/cython-dev
>
>
>
> --
> Vic Kelson
> vic.kelson at gmail.com
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From vic.kelson at gmail.com  Mon Apr  6 18:53:01 2009
From: vic.kelson at gmail.com (Vic Kelson)
Date: Mon, 6 Apr 2009 12:53:01 -0400
Subject: [Cython] Status of C complex types (re-send)
In-Reply-To: <e7ba66e40904060919v20fa2709y8baac3d98d646b57@mail.gmail.com>
References: <d5e3120f0904032221v43ee94cdseec8a1fd859f5b63@mail.gmail.com>
	<488BAB80-5474-4BE8-9ED9-9C33133CFFDE@math.washington.edu>
	<d5e3120f0904040909v69416410vf032ffb155adba56@mail.gmail.com>
	<29DB5FDB-9FB8-4291-ADC5-5B3234BE4EEF@math.washington.edu>
	<d5e3120f0904042128p84022d5m296981dbdda8c1eb@mail.gmail.com>
	<e7ba66e40904060919v20fa2709y8baac3d98d646b57@mail.gmail.com>
Message-ID: <d5e3120f0904060953n3dae636bxda39c40485bbd27c@mail.gmail.com>

I did the first one. I just wanted to make sure that it was resolved at
compile time.

THANKS!
--v

On Mon, Apr 6, 2009 at 12:19 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:

> I would do this (at de module-level):
>
> import math
> cdef double PI = math.pi
>
> of perhaps
>
> cdef double PI = 3.14...
>
> and just use "PI" on you cdef functions...
>
> On Sun, Apr 5, 2009 at 1:28 AM, Vic Kelson <vic.kelson at gmail.com> wrote:
> > Another question: If I need the constant pi, and I use math.pi or
> numpy.pi
> > in a C function (or a C method of a C class), does it add an inefficiency
> > (Python lookup)?
> > Just wondering...
> > --v
> >
> > On Sat, Apr 4, 2009 at 2:57 PM, Robert Bradshaw
> > <robertwb at math.washington.edu> wrote:
> >>
> >> On Apr 4, 2009, at 9:09 AM, Vic Kelson wrote:
> >>
> >> > Robert,
> >> > Thatnks for your helpful note. I can't seem to make it work with gcc
> >> > on OS X.
> >>
> >> Hmm... that's the same platform I use.
> >>
> >> > I keep getting an error from gcc. I haven't had time to sort
> >> > through it yet. I think I'll build a little test problem (I tried
> >> > embedding your cod in my existing code and I might have created
> >> > another error).
> >>
> >> Maybe you could post a snippet of the error you're getting?
> >>
> >> > Just to be sure, when I use your code to make a cdouble, the real and
> >> > imaginary parts are the same size as a python float, right?
> >>
> >> Yes. Python floats are wrappers of double (and Python complexes wrap
> >> a pair of doubles).
> >>
> >> > Thanks!
> >> > --v
> >> >
> >> >
> >> > On 4/4/09, Robert Bradshaw <robertwb at math.washington.edu> wrote:
> >> >> On Apr 3, 2009, at 10:21 PM, Vic Kelson wrote:
> >> >>
> >> >>> Greetings all,
> >> >>>
> >> >>> Sorry for the incomplete message, I'm re-sending it.
> >> >>>
> >> >>> I'm in need of fast computations on complex numbers from Cython.
> >> >>> I've been converting a Python groundwater flow model code to
> >> >>> Cython, but I'm reaching the point where I'd really prefer to have
> >> >>> native C complex math. Up to now, I've used some little C routines
> >> >>> that I can call with multiple floats (the C code converts to/from
> >> >>> complex for my purposes). However, I'm nearly to the point where
> >> >>> I'll need to store series coefficients for performance purposes,
> >> >>> and then I'll either need to hide them in float arrays (and that's
> >> >>> extremely nasty), or something like that.
> >> >>>
> >> >>> I'd like to say something like
> >> >>>
> >> >>> cdef class Foo:
> >> >>>     cdef cfloat z
> >> >>>
> >> >>>     def __cinit__(self, float x, float y):
> >> >>>         self.z = cfloat(x, y)
> >> >>>
> >> >>>     cdef cfloat func(Foo self, float x, float y):
> >> >>>         return-some-gnarly-function of x, y, and self.z
> >> >>>
> >> >>>
> >> >>> Or something like that, where 'cfloat' maps to the appropriate type
> >> >>> in the C compiler, e.g. "double _Complex" in gcc.
> >> >>>
> >> >>> I've seen this discussed in the archives. Has it been implemented?
> >> >>> Is there some pre-release code that essentially works? I only need
> >> >>> the four math functions, conjugate, abs, and log.
> >> >>
> >> >> Yes, I've started this, as it comes up in my research too (number
> >> >> theory, computing values of L-functions). I hope to have something
> >> >> releasable soon. In the meantime, you can do
> >> >>
> >> >> cdef extern from "complex.h":
> >> >>      ctypedef double cdouble "double complex"
> >> >>      cdef double creal(cdouble)
> >> >>      cdef double cimag(cdouble)
> >> >>      cdef cdouble _Complex_I
> >> >>
> >> >> cdef inline cdouble  new_cdouble(double x, double y):
> >> >>      return x + _Complex_I*y
> >> >>
> >> >> Of course, the ctypedef is wrong, it will think it can go back to
> >> >> doubles (and Python objects) without any problem, but it works for
> >> >> now.
> >> >>
> >> >>> THANKS! I think Cython will revolutionize my research work!
> >> >>
> >> >> That's the goal :) Please cite us if it does.
> >> >>
> >> >> - Robert
> >> >>
> >> >> _______________________________________________
> >> >> Cython-dev mailing list
> >> >> Cython-dev at codespeak.net
> >> >> http://codespeak.net/mailman/listinfo/cython-dev
> >> >>
> >> >
> >> > --
> >> > Sent from my mobile device
> >> >
> >> > Vic Kelson
> >> > vic.kelson at gmail.com
> >> > _______________________________________________
> >> > Cython-dev mailing list
> >> > Cython-dev at codespeak.net
> >> > http://codespeak.net/mailman/listinfo/cython-dev
> >>
> >> _______________________________________________
> >> Cython-dev mailing list
> >> Cython-dev at codespeak.net
> >> http://codespeak.net/mailman/listinfo/cython-dev
> >
> >
> >
> > --
> > Vic Kelson
> > vic.kelson at gmail.com
> >
> > _______________________________________________
> > Cython-dev mailing list
> > Cython-dev at codespeak.net
> > http://codespeak.net/mailman/listinfo/cython-dev
> >
> >
>
>
>
> --
> Lisandro Dalc?n
> ---------------
> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
> Tel/Fax: +54-(0)342-451.1594
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Vic Kelson
vic.kelson at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20090406/72fb4ae3/attachment-0001.htm 

From dalcinl at gmail.com  Mon Apr  6 18:58:53 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 6 Apr 2009 13:58:53 -0300
Subject: [Cython] updated patch for #255 (Py->C for integrals) in the tracker
Message-ID: <e7ba66e40904060958m1c376150gd7b2af5cb805b55a@mail.gmail.com>

I've uploaded an updated version of the patch for #255.

It is an attempt to reimplement __Pyx_PyNumber_Int() in an smarter
way. Moreover, I've tried to mimic the error messages in core CPython.

Additionally, I've extended the testcase to test for user classes
defining __int__()/__long__(). So far, this test pass on Linux 32 with
Python 2.3 to 2.7 (recent trunk checkout) and 3.0.1

-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From stefan_ml at behnel.de  Mon Apr  6 19:04:10 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 06 Apr 2009 19:04:10 +0200
Subject: [Cython] Status of C complex types (re-send)
In-Reply-To: <d5e3120f0904060953n3dae636bxda39c40485bbd27c@mail.gmail.com>
References: <d5e3120f0904032221v43ee94cdseec8a1fd859f5b63@mail.gmail.com>	<488BAB80-5474-4BE8-9ED9-9C33133CFFDE@math.washington.edu>	<d5e3120f0904040909v69416410vf032ffb155adba56@mail.gmail.com>	<29DB5FDB-9FB8-4291-ADC5-5B3234BE4EEF@math.washington.edu>	<d5e3120f0904042128p84022d5m296981dbdda8c1eb@mail.gmail.com>	<e7ba66e40904060919v20fa2709y8baac3d98d646b57@mail.gmail.com>
	<d5e3120f0904060953n3dae636bxda39c40485bbd27c@mail.gmail.com>
Message-ID: <49DA360A.9010601@behnel.de>

Hi,

please try to avoid top-posting.


Vic Kelson wrote:
> On Mon, Apr 6, 2009 at 12:19 PM, Lisandro Dalcin wrote:
> 
>> I would do this (at de module-level):
>>
>> import math
>> cdef double PI = math.pi
>>
>> of perhaps
>>
>> cdef double PI = 3.14...
>>
>> and just use "PI" on you cdef functions...
>>
> I did the first one. I just wanted to make sure that it was resolved at
> compile time.

Well, not really. The value will be resolved and assigned at *runtime*, or
rather module initialisation time. However, *using* the PI variable in your
code will not involve any Python API calls, as it's a plain C variable.

You can also take the value from math.h (cdef extern from ...), which will
avoid any loss in precision due to the type conversion.

Stefan

From dalcinl at gmail.com  Mon Apr  6 19:13:50 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 6 Apr 2009 14:13:50 -0300
Subject: [Cython] pushed fix for __Pyx_SetItemInt_Fast
Message-ID: <e7ba66e40904061013y480938c3q4b994e9c7ee12c1f@mail.gmail.com>

Robert, I've just pushed this fix... please review... (remember "v is
L[i]" can be True)

# HG changeset patch
# User Lisandro Dalcin <dalcinl at gmail.com>
# Date 1239037693 10800
# Node ID ea5b7e6e9db7dcd2f07affbd3f985e8b53b5b335
# Parent  49523eb3abb51b4b783676dc0484b3b1f413e9f8
fix bug in __Pyx_SetItemInt_Fast

diff -r 49523eb3abb5 -r ea5b7e6e9db7 Cython/Compiler/ExprNodes.py
--- a/Cython/Compiler/ExprNodes.py	Fri Apr 03 15:59:21 2009 -0700
+++ b/Cython/Compiler/ExprNodes.py	Mon Apr 06 14:08:13 2009 -0300
@@ -5500,8 +5500,8 @@

 static INLINE int __Pyx_SetItemInt_Fast(PyObject *o, Py_ssize_t i,
PyObject *v, int fits_long) {
     if (PyList_CheckExact(o) && ((0 <= i) & (i < PyList_GET_SIZE(o)))) {
+        Py_INCREF(v);
         Py_DECREF(PyList_GET_ITEM(o, i));
-        Py_INCREF(v);
         PyList_SET_ITEM(o, i, v);
         return 1;
     }


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From stefan_ml at behnel.de  Mon Apr  6 19:15:38 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 06 Apr 2009 19:15:38 +0200
Subject: [Cython] constant folding for floats?
Message-ID: <49DA38BA.5070704@behnel.de>

Hi,

should we enable constant folding for floats by using the decimal module?

This would not work on Py2.3, but I wouldn't mind just generating less
optimal code when you run Cython under a long out-of-maintenance Python
version.

Stefan

From carl.witty at gmail.com  Mon Apr  6 19:24:14 2009
From: carl.witty at gmail.com (Carl Witty)
Date: Mon, 6 Apr 2009 10:24:14 -0700
Subject: [Cython] constant folding for floats?
In-Reply-To: <49DA38BA.5070704@behnel.de>
References: <49DA38BA.5070704@behnel.de>
Message-ID: <ac1e4c420904061024v73713efai6eea28c87dee9870@mail.gmail.com>

On Mon, Apr 6, 2009 at 10:15 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Hi,
>
> should we enable constant folding for floats by using the decimal module?

I don't think so.

1) Is there any advantage to doing constant folding in Cython instead
of letting the C compiler do it?

2) If I write a constant floating-point expression, I expect it to be
evaluated according to IEEE rounding rules.  (So I would prefer doing
the constant folding using Python floats, which will usually match the
computations performed in the target, rather than the decimal module,
which won't match the target.)

Carl

From stefan_ml at behnel.de  Mon Apr  6 19:38:21 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 06 Apr 2009 19:38:21 +0200
Subject: [Cython] constant folding for floats?
In-Reply-To: <ac1e4c420904061024v73713efai6eea28c87dee9870@mail.gmail.com>
References: <49DA38BA.5070704@behnel.de>
	<ac1e4c420904061024v73713efai6eea28c87dee9870@mail.gmail.com>
Message-ID: <49DA3E0D.90504@behnel.de>


Carl Witty wrote:
> On Mon, Apr 6, 2009 at 10:15 AM, Stefan Behnel wrote:
>> should we enable constant folding for floats by using the decimal module?
> 
> I don't think so.
> 
> 1) Is there any advantage to doing constant folding in Cython instead
> of letting the C compiler do it?

Well, yes. In many situations, Cython can use values that are known at
compile time to generate simpler code, e.g. avoid Python type conversions
or move code sections into C completely. Loops are a great example. Cython
0.12 will bring another couple of enhancements here.

Not sure if this really applies to float values, so we might end up
postponing this discussion until there is a concrete reason to enable this.


> 2) If I write a constant floating-point expression, I expect it to be
> evaluated according to IEEE rounding rules.  (So I would prefer doing
> the constant folding using Python floats, which will usually match the
> computations performed in the target, rather than the decimal module,
> which won't match the target.)

Makes sense to me.

Stefan

From robertwb at math.washington.edu  Mon Apr  6 20:24:31 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 6 Apr 2009 11:24:31 -0700
Subject: [Cython] constant folding for floats?
In-Reply-To: <49DA3E0D.90504@behnel.de>
References: <49DA38BA.5070704@behnel.de>
	<ac1e4c420904061024v73713efai6eea28c87dee9870@mail.gmail.com>
	<49DA3E0D.90504@behnel.de>
Message-ID: <4D98C3BE-B202-432B-8827-DCAFC868F865@math.washington.edu>

On Apr 6, 2009, at 10:38 AM, Stefan Behnel wrote:

> Carl Witty wrote:
>> On Mon, Apr 6, 2009 at 10:15 AM, Stefan Behnel wrote:
>>> should we enable constant folding for floats by using the decimal  
>>> module?
>>
>> I don't think so.
>>
>> 1) Is there any advantage to doing constant folding in Cython instead
>> of letting the C compiler do it?
>
> Well, yes. In many situations, Cython can use values that are known at
> compile time to generate simpler code, e.g. avoid Python type  
> conversions
> or move code sections into C completely. Loops are a great example.  
> Cython
> 0.12 will bring another couple of enhancements here.
>
> Not sure if this really applies to float values, so we might end up
> postponing this discussion until there is a concrete reason to  
> enable this.
>
>
>> 2) If I write a constant floating-point expression, I expect it to be
>> evaluated according to IEEE rounding rules.  (So I would prefer doing
>> the constant folding using Python floats, which will usually match  
>> the
>> computations performed in the target, rather than the decimal module,
>> which won't match the target.)
>
> Makes sense to me.

+1 Using the decimals module here seems strange. Note that one needs  
to use repr rather than str to get enough digits to accurately  
represent constants in the source.

- Robert


From dalcinl at gmail.com  Mon Apr  6 21:44:15 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 6 Apr 2009 16:44:15 -0300
Subject: [Cython] PATCH: the right way to use PyErr_Format with
	Py_ssize_t arguments
In-Reply-To: <2E671135-5B33-4551-82FB-183486183E88@math.washington.edu>
References: <e7ba66e40903110731h65be202ep8b6bed072e9fa9ea@mail.gmail.com>
	<49B7CDF6.3020401@student.matnat.uio.no>
	<e7ba66e40903110901u3c68118fj6dd64441b3f852@mail.gmail.com>
	<2E671135-5B33-4551-82FB-183486183E88@math.washington.edu>
Message-ID: <e7ba66e40904061244s39fd91c0q1595ae0c540da711@mail.gmail.com>

Just opened ticket, http://trac.cython.org/cython_trac/attachment/ticket/274/

The patch is two lines, obviously trivial (for any one with some
knowledge of Python C-API). I've spent more time opening the ticket
and deciding the right Type & Priority tags than working on the fix.
And of course, sum up the time to write this comment ;-). Do we really
need to be so strict about this patch requires ticket policy?


On Wed, Mar 11, 2009 at 8:17 PM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Mar 11, 2009, at 9:01 AM, Lisandro Dalcin wrote:
>
>> On Wed, Mar 11, 2009 at 12:43 PM, Dag Sverre Seljebotn
>> <dagss at student.matnat.uio.no> wrote:
>>> Lisandro Dalcin wrote:
>>>> Likely Dag should review this... If you agree on the patch, please
>>>> push ... IMHO, this fix should go to 0.11
>
> The patch looks good, can you make a ticket for it? Also, can you use
> mercurial to output diffs so we get your name and a comment with it
> as well?
>
>>> a) Saying that this should go into 0.11 means saying that this
>>> specific
>>> bug is more important than everything else that sits in milestone
>>> 0.11.1
>>> in Trac. We probably put out 0.10 with the same bug and the Cython
>>> project survived! We need to have some disipline and get things
>>> released
>>> -- release early, release often.
>
> Yes, we need to return to the "release early, release often"
> schedule. We can start putting patches up and getting reviews for
> them on trac for 0.11.1 right away. I've added milestone 0.11.2,
> everything that doesn't have a patch or easy fix for 0.11.1 will get
> moved there so we can release 0.11.1 shortly.
>
>> I understand that... I just wanted to point that this fix likely does
>> not have high impact and fix 64 bits issues, more or less like recent
>> Robert's pushes did.
>
>
> The last push was to facilitate getting cython 0.11 into the rpm
> repo, though in retrospect I guess it's lower priority than some
> other stuff. At this point, unless there's a major bug, the only
> thing I can see adding is perhaps testcases.
>
> - Robert
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From robertwb at math.washington.edu  Tue Apr  7 20:40:31 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 7 Apr 2009 11:40:31 -0700
Subject: [Cython] PATCH: the right way to use PyErr_Format with
	Py_ssize_t arguments
In-Reply-To: <e7ba66e40904061244s39fd91c0q1595ae0c540da711@mail.gmail.com>
References: <e7ba66e40903110731h65be202ep8b6bed072e9fa9ea@mail.gmail.com>
	<49B7CDF6.3020401@student.matnat.uio.no>
	<e7ba66e40903110901u3c68118fj6dd64441b3f852@mail.gmail.com>
	<2E671135-5B33-4551-82FB-183486183E88@math.washington.edu>
	<e7ba66e40904061244s39fd91c0q1595ae0c540da711@mail.gmail.com>
Message-ID: <9EF7AF6E-B848-4CC4-8A69-AF9EE8775D4E@math.washington.edu>

On Apr 6, 2009, at 12:44 PM, Lisandro Dalcin wrote:

> Just opened ticket, http://trac.cython.org/cython_trac/attachment/ 
> ticket/274/
>
> The patch is two lines, obviously trivial (for any one with some
> knowledge of Python C-API). I've spent more time opening the ticket
> and deciding the right Type & Priority tags than working on the fix.
> And of course, sum up the time to write this comment ;-). Do we really
> need to be so strict about this patch requires ticket policy?

I feel, at this stage of the project, there are patches simple enough  
to not need a ticket. An obvious example would be fixing typos. This  
is right on the borderline. The trac site serves many purposes, and  
one of them is to be a record of what has been fixed/done (as the  
changelogs are getting lengthy enough that it takes work to extract a  
clear summary from them).

Personally, for a ticket like this, I would probably open a quick  
ticket, not worry to much about the type ("code generation") or  
priority (just leave it at the default) and post a link to the patch  
in the repo. If I found other places where this should be changed, I  
would simply push them (probably adding a link to the already created  
ticket).

- Robert


From bblais at gmail.com  Tue Apr  7 20:48:51 2009
From: bblais at gmail.com (Brian Blais)
Date: Tue, 7 Apr 2009 14:48:51 -0400
Subject: [Cython] preferable way to replace a class
Message-ID: <6F6D1E65-2A4A-4A2F-A8FD-77CAB7D10291@gmail.com>

Hello,

I am writing a tool to do some neural simulation, where I have  
several different neuron groups, which are defined as python  
classes.  Is there a preferred way to write a cython-only replacement  
such that the end-user sees no difference?  What I mean is that they  
should be able to do:

import sim

n1=sim.NeuronGroup1(5)

sim.run_sim([n1])

and if the cython version exists, use that one (for both the neuron  
group and the function to run the simulation), otherwise use the  
python one.


		thanks,

			Brian Blais

--
Brian Blais
bblais at gmail.com
http://web.bryant.edu/~bblais

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20090407/e3a31e91/attachment.htm 

From lev at columbia.edu  Tue Apr  7 21:05:18 2009
From: lev at columbia.edu (Lev Givon)
Date: Tue, 7 Apr 2009 15:05:18 -0400
Subject: [Cython] preferable way to replace a class
In-Reply-To: <6F6D1E65-2A4A-4A2F-A8FD-77CAB7D10291@gmail.com>
References: <6F6D1E65-2A4A-4A2F-A8FD-77CAB7D10291@gmail.com>
Message-ID: <20090407190518.GJ957@avicenna.ee.columbia.edu>

Received from Brian Blais on Tue, Apr 07, 2009 at 02:48:51PM EDT:
> Hello,
>
> I am writing a tool to do some neural simulation, where I have several 
> different neuron groups, which are defined as python classes.  Is there a 
> preferred way to write a cython-only replacement such that the end-user 
> sees no difference?  What I mean is that they should be able to do:
>
> import sim
>
> n1=sim.NeuronGroup1(5)
>
> sim.run_sim([n1])
>
> and if the cython version exists, use that one (for both the neuron  
> group and the function to run the simulation), otherwise use the python 
> one.
>
> thanks,
> Brian Blais

You could give the cython and python modules slightly different names
(e.g., sim_cython and sim_python) and create a wrapper module (called
sim) that contains the following code:

try:
    from sim_cython import *
except ImportError:
    from sim_python import *

							L.G.



From bblais at gmail.com  Tue Apr  7 21:29:11 2009
From: bblais at gmail.com (Brian Blais)
Date: Tue, 7 Apr 2009 15:29:11 -0400
Subject: [Cython] preferable way to replace a class
In-Reply-To: <20090407190518.GJ957@avicenna.ee.columbia.edu>
References: <6F6D1E65-2A4A-4A2F-A8FD-77CAB7D10291@gmail.com>
	<20090407190518.GJ957@avicenna.ee.columbia.edu>
Message-ID: <D91B9982-C562-4046-95AC-95EB6AFFE580@gmail.com>

On Apr 7, 2009, at 15:05 , Lev Givon wrote:

> Received from Brian Blais on Tue, Apr 07, 2009 at 02:48:51PM EDT:
>> I am writing a tool to do some neural simulation, where I have  
>> several
>> different neuron groups, which are defined as python classes.  Is  
>> there a
>> preferred way to write a cython-only replacement such that the end- 
>> user
>> sees no difference?  What I mean is that they should be able to do:
>>
>> import sim
>>
>> n1=sim.NeuronGroup1(5)
>>
>> sim.run_sim([n1])
>>
>> and if the cython version exists, use that one (for both the neuron
>> group and the function to run the simulation), otherwise use the  
>> python
>> one.
>>
>>
> You could give the cython and python modules slightly different names
> (e.g., sim_cython and sim_python) and create a wrapper module (called
> sim) that contains the following code:
>
> try:
>     from sim_cython import *
> except ImportError:
>     from sim_python import *
>

Thanks!  Now, if not every class is implemented in sim_cython, should  
I do:

from sim_python import *
try:
     from sim_cython import *
except ImportError:
     pass


		thanks,

			bb

--
Brian Blais
bblais at gmail.com
http://web.bryant.edu/~bblais

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20090407/1950955d/attachment.htm 

From robertwb at math.washington.edu  Tue Apr  7 21:33:57 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 7 Apr 2009 12:33:57 -0700
Subject: [Cython] preferable way to replace a class
In-Reply-To: <D91B9982-C562-4046-95AC-95EB6AFFE580@gmail.com>
References: <6F6D1E65-2A4A-4A2F-A8FD-77CAB7D10291@gmail.com>
	<20090407190518.GJ957@avicenna.ee.columbia.edu>
	<D91B9982-C562-4046-95AC-95EB6AFFE580@gmail.com>
Message-ID: <1EF296C2-624C-4D5A-9FE1-C38EA4696AEA@math.washington.edu>

On Apr 7, 2009, at 12:29 PM, Brian Blais wrote:

> On Apr 7, 2009, at 15:05 , Lev Givon wrote:
>
>> Received from Brian Blais on Tue, Apr 07, 2009 at 02:48:51PM EDT:
>>> I am writing a tool to do some neural simulation, where I have  
>>> several
>>> different neuron groups, which are defined as python classes.  Is  
>>> there a
>>> preferred way to write a cython-only replacement such that the  
>>> end-user
>>> sees no difference?  What I mean is that they should be able to do:
>>>
>>> import sim
>>>
>>> n1=sim.NeuronGroup1(5)
>>>
>>> sim.run_sim([n1])
>>>
>>> and if the cython version exists, use that one (for both the neuron
>>> group and the function to run the simulation), otherwise use the  
>>> python
>>> one.
>>>
>>>
>> You could give the cython and python modules slightly different names
>> (e.g., sim_cython and sim_python) and create a wrapper module (called
>> sim) that contains the following code:
>>
>> try:
>>     from sim_cython import *
>> except ImportError:
>>     from sim_python import *
>>
>
> Thanks!  Now, if not every class is implemented in sim_cython,  
> should I do:
>
> from sim_python import *
> try:
>     from sim_cython import *
> except ImportError:
>     pass

Yes, that would work well.

You might also be interested in http://wiki.cython.org/pure which may  
allow you to keep a single codebase for the compiled and interpreted  
implementations.

- Robert


From lev at columbia.edu  Wed Apr  8 00:59:22 2009
From: lev at columbia.edu (Lev Givon)
Date: Tue, 7 Apr 2009 18:59:22 -0400
Subject: [Cython] preferable way to replace a class
In-Reply-To: <D91B9982-C562-4046-95AC-95EB6AFFE580@gmail.com>
References: <6F6D1E65-2A4A-4A2F-A8FD-77CAB7D10291@gmail.com>
	<20090407190518.GJ957@avicenna.ee.columbia.edu>
	<D91B9982-C562-4046-95AC-95EB6AFFE580@gmail.com>
Message-ID: <20090407225922.GM957@avicenna.ee.columbia.edu>

Received from Brian Blais on Tue, Apr 07, 2009 at 03:29:11PM EDT:

(snip)

>> You could give the cython and python modules slightly different names
>> (e.g., sim_cython and sim_python) and create a wrapper module (called
>> sim) that contains the following code:
>>
>> try:
>>     from sim_cython import *
>> except ImportError:
>>     from sim_python import *
>>
>
> Thanks!  Now, if not every class is implemented in sim_cython, should I 
> do:
>
> from sim_python import *
> try:
>     from sim_cython import *
> except ImportError:
>     pass
>
> thanks,
> bb

Yes, if you intend to replace the symbols imported into the current
namespace from sim_python with their analogues in sim_cython.

						L.G.

From dalcinl at gmail.com  Wed Apr  8 04:15:27 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Tue, 7 Apr 2009 23:15:27 -0300
Subject: [Cython] fix for "exec" statement (ticket #275)
Message-ID: <e7ba66e40904071915j6904a6c3i58efbcea5db02d70@mail.gmail.com>

Just opened ticket and patch attached (tested in Py2.6 and Py3.0 on
Win32 with MSVC compiler, likely ready for push)

http://trac.cython.org/cython_trac/ticket/275

-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dalcinl at gmail.com  Wed Apr  8 04:22:29 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Tue, 7 Apr 2009 23:22:29 -0300
Subject: [Cython] How to fix Win32 linker errors when running tests?
Message-ID: <e7ba66e40904071922v3d09f0a7v6ff900e9355792fa@mail.gmail.com>

I'm getting a few errors like the following:

coercearraytoptr.obj : error LNK2019: unresolved external symbol _spam
referenced in function ___pyx_f_16coercearraytoptr_eggs

Likely including a fake C header with "cdef extern" will solve some of
the issues. Where should I put such headers?

-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dalcinl at gmail.com  Wed Apr  8 04:44:55 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Tue, 7 Apr 2009 23:44:55 -0300
Subject: [Cython] How to fix Win32 linker errors when running tests?
In-Reply-To: <e7ba66e40904071922v3d09f0a7v6ff900e9355792fa@mail.gmail.com>
References: <e7ba66e40904071922v3d09f0a7v6ff900e9355792fa@mail.gmail.com>
Message-ID: <e7ba66e40904071944v7cf866b1jc2c4716c2ccacfb1@mail.gmail.com>

It seems this issue have an open ticket

http://trac.cython.org/cython_trac/ticket/106

On Tue, Apr 7, 2009 at 11:22 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
> I'm getting a few errors like the following:
>
> coercearraytoptr.obj : error LNK2019: unresolved external symbol _spam
> referenced in function ___pyx_f_16coercearraytoptr_eggs
>
> Likely including a fake C header with "cdef extern" will solve some of
> the issues. Where should I put such headers?
>
> --
> Lisandro Dalc?n
> ---------------
> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
> Tel/Fax: +54-(0)342-451.1594
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dagss at student.matnat.uio.no  Wed Apr  8 15:58:36 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Wed, 8 Apr 2009 15:58:36 +0200 (CEST)
Subject: [Cython] MPI and Cython?
Message-ID: <8fb9f12447b21bacc1fc304859ef7b91.squirrel@webmail.uio.no>

What's the best way of using MPI from Cython? Should one use the C library
directly (has anyone written a pxd?), or is there an MPI wrapper library
with the same Python and Cyhthon interfaces?

Dag Sverre


From dalcinl at gmail.com  Wed Apr  8 16:57:46 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 8 Apr 2009 11:57:46 -0300
Subject: [Cython] MPI and Cython?
In-Reply-To: <8fb9f12447b21bacc1fc304859ef7b91.squirrel@webmail.uio.no>
References: <8fb9f12447b21bacc1fc304859ef7b91.squirrel@webmail.uio.no>
Message-ID: <e7ba66e40904080757s70955328hb613bda1e1f4dec8@mail.gmail.com>

Well, I would say you should try mpi4py. However, mpi4py hgih-level,
object oriented API is targeted to be used from Python and not Cython
(i.e. no cpdef methods at all). However, you can easily extract the
MPI C handles from the Python objects and call any MPI C API
functions, as you can see here:

http://code.google.com/p/mpi4py/source/browse/trunk/demo/cython/helloworld.pyx

This esample is a bit of mess, because it shows many different ways of
doing things... If you want to make fast MPI calls, you will be
interested in the last part.

Note that "from mpi4py.mpi_c cimport *" should also work (instead of
'include mpi4py/mpi.pxi'), though I've not tested that. That example
was written long ago in a time where likely "cimport *" did not work.



On Wed, Apr 8, 2009 at 10:58 AM, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:
> What's the best way of using MPI from Cython? Should one use the C library
> directly (has anyone written a pxd?), or is there an MPI wrapper library
> with the same Python and Cyhthon interfaces?
>
> Dag Sverre
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dalcinl at gmail.com  Wed Apr  8 18:22:12 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 8 Apr 2009 13:22:12 -0300
Subject: [Cython] moving #255 to milestone 0.11.1
Message-ID: <e7ba66e40904080922r909e786p5b7e156ee675d7df@mail.gmail.com>

What do you think about this?

-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From jasone at canonware.com  Wed Apr  8 19:51:09 2009
From: jasone at canonware.com (Jason Evans)
Date: Wed, 08 Apr 2009 10:51:09 -0700
Subject: [Cython] MPI and Cython?
In-Reply-To: <8fb9f12447b21bacc1fc304859ef7b91.squirrel@webmail.uio.no>
References: <8fb9f12447b21bacc1fc304859ef7b91.squirrel@webmail.uio.no>
Message-ID: <49DCE40D.1010606@canonware.com>

Dag Sverre Seljebotn wrote:
> What's the best way of using MPI from Cython? Should one use the C library
> directly (has anyone written a pxd?), or is there an MPI wrapper library
> with the same Python and Cyhthon interfaces?

I'm using mpi4py in Crux (http://canonware.com/cgi-bin/hg_crux), and am 
very pleased with how it works.  In most places I'm using the cdef 
bindings, but the Python bindings have also been useful in pure-Python 
scripts.

Jason

From robertwb at math.washington.edu  Wed Apr  8 20:15:30 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Wed, 8 Apr 2009 11:15:30 -0700
Subject: [Cython] moving #255 to milestone 0.11.1
In-Reply-To: <e7ba66e40904080922r909e786p5b7e156ee675d7df@mail.gmail.com>
References: <e7ba66e40904080922r909e786p5b7e156ee675d7df@mail.gmail.com>
Message-ID: <ECA17B71-E3BD-4CAA-9B32-FA3CC8719943@math.washington.edu>

+1. I've been busy so I haven't had a chance to work on Cython much  
the last couple of days, but was planning on seeing if I can get Sage  
to compile tonight and take a look at this. BTW, just apply the last  
patch?

- Robert

On Apr 8, 2009, at 9:22 AM, Lisandro Dalcin wrote:

> What do you think about this?
>
> -- 
> Lisandro Dalc?n
> ---------------
> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
> Tel/Fax: +54-(0)342-451.1594
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From dalcinl at gmail.com  Wed Apr  8 22:27:56 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 8 Apr 2009 17:27:56 -0300
Subject: [Cython] moving #255 to milestone 0.11.1
In-Reply-To: <ECA17B71-E3BD-4CAA-9B32-FA3CC8719943@math.washington.edu>
References: <e7ba66e40904080922r909e786p5b7e156ee675d7df@mail.gmail.com>
	<ECA17B71-E3BD-4CAA-9B32-FA3CC8719943@math.washington.edu>
Message-ID: <e7ba66e40904081327h1025f223xafe1468a8b29fe46@mail.gmail.com>

On Wed, Apr 8, 2009 at 3:15 PM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> +1. I've been busy so I haven't had a chance to work on Cython much
> the last couple of days, but was planning on seeing if I can get Sage
> to compile tonight and take a look at this. BTW, just apply the last
> patch?

Yes, just the c-int-types.diff (it contains the testcase for the ticket).

>
> - Robert
>
> On Apr 8, 2009, at 9:22 AM, Lisandro Dalcin wrote:
>
>> What do you think about this?
>>
>> --
>> Lisandro Dalc?n
>> ---------------
>> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
>> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
>> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
>> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
>> Tel/Fax: +54-(0)342-451.1594
>> _______________________________________________
>> Cython-dev mailing list
>> Cython-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/cython-dev
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From robertwb at math.washington.edu  Thu Apr  9 19:36:23 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 9 Apr 2009 10:36:23 -0700
Subject: [Cython] Cython 0.11.1 alpha
In-Reply-To: <003001c9b68a$63e71340$76ac810a@gnb.st.com>
References: <003001c9b68a$63e71340$76ac810a@gnb.st.com>
Message-ID: <B1B5470D-8AD1-48A3-811E-F2E41939E980@math.washington.edu>

Just a status update on this--Sage still doesn't compile, but as soon  
as it does I'll push a beta.

- Robert


From robertwb at math.washington.edu  Fri Apr 10 09:19:06 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 10 Apr 2009 00:19:06 -0700
Subject: [Cython] Cython 0.11.1 beta
Message-ID: <943E3D18-3037-4DE7-8684-92D42F420467@math.washington.edu>

Please try it out. http://cython.org/Cython-0.11.1.beta.tar.gz

There are still two blockers:

Exceptions in __Pyx_InitGlobals() lead to a crash
http://trac.cython.org/cython_trac/ticket/216

Sage segfaults
http://trac.cython.org/cython_trac/ticket/278

I'll be working on the second one.

- Robert


From stefan_ml at behnel.de  Fri Apr 10 10:57:25 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 10 Apr 2009 10:57:25 +0200
Subject: [Cython] Cython 0.11.1 beta
In-Reply-To: <943E3D18-3037-4DE7-8684-92D42F420467@math.washington.edu>
References: <943E3D18-3037-4DE7-8684-92D42F420467@math.washington.edu>
Message-ID: <49DF09F5.3060507@behnel.de>


Robert Bradshaw wrote:
> Please try it out. http://cython.org/Cython-0.11.1.beta.tar.gz
> 
> There are still two blockers:
> 
> Exceptions in __Pyx_InitGlobals() lead to a crash
> http://trac.cython.org/cython_trac/ticket/216

I know, I set this to critical (because it's incorrect code that can lead
to a crash), but I don't really think it's that worrisome. The only obvious
reason I can see why there should be an exception in that function is that
we are running out of memory. Rather unlikely for a module import, as it
usually happens at program initialisation time.

Since the fix is not trivial (although not complex either), I'm fine with
moving it to a later milestone (say, 0.12, although 0.11.2 should be ok).

Note that there are still some tests in cython-devel that fail at the C
compiler level:

- cpp_exceptions_T265.cpp seems to lack a file "foo.h"

- bad_c_struct_T252 says:

bad_c_struct_T252.c: In function '__pyx_f_17bad_c_struct_T252_f':
bad_c_struct_T252.c:344: error: expected identifier before 'void'

ERROR

- The test for #232 is also failing, I'll look into that.

Stefan

From stefan_ml at behnel.de  Fri Apr 10 17:29:36 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 10 Apr 2009 17:29:36 +0200
Subject: [Cython] Cython 0.11.1 beta
In-Reply-To: <49DF09F5.3060507@behnel.de>
References: <943E3D18-3037-4DE7-8684-92D42F420467@math.washington.edu>
	<49DF09F5.3060507@behnel.de>
Message-ID: <49DF65E0.10306@behnel.de>


Stefan Behnel wrote:
> - The test for #232 is also failing, I'll look into that.

My bad, that was fixed in cython-unstable, not cython-devel. I just
accidentally copied the test case over into tests/run/ on my side. So this
won't go into 0.11.1 anyway.

Stefan

From riccardo.murri at gmail.com  Mon Apr 13 23:14:32 2009
From: riccardo.murri at gmail.com (Riccardo Murri)
Date: Mon, 13 Apr 2009 23:14:32 +0200
Subject: [Cython] very simple iterator crashing Cython-0.11
Message-ID: <1c34ba170904131414s42ea8446o7c06b5bbbd876705@mail.gmail.com>

Hello,

I have started using Cython recently to spped up some Python code; it
works very well and stably, but today I managed to crash Cython 0.11
while trying to compile the following pure Python code:

  # iter.py
  class Iter(object):

      """A docstring."""

      def __iter__(self):
          return self

      def __next__(self):
          raise StopIteration

I'm augmenting the pure Python code with the following .pxd file::

  # iter.pxd
  cpdef class Iter(object)

The Cython-0.11 invocation "cython iter.py" dies with the following backtrace::

  $ cython iter.py

  Error converting Pyrex file to C:
  ------------------------------------------------------------
  ...
  class Iter(object):

      """A docstring."""

      def __iter__(self):
     ^
  ------------------------------------------------------------

  /tmp/iter.py:5:4: Compiler crash in AlignFunctionDefinitions

  ModuleNode.body = StatListNode(iter.py:1:0)
  StatListNode.stats[0] = PyClassDefNode(iter.py:1:0,
      doc = 'A docstring.',
      name = u'Iter')
  CClassDefNode.body = StatListNode(iter.py:5:4)
  StatListNode.stats[0] = DefNode(iter.py:5:4,
      name = u'__iter__',
      num_required_args = 1,
      reqd_kw_flags_cname = '0')

  Compiler crash traceback from this point on:
    File "Visitor.py", line 149, in Cython.Compiler.Visitor.visitchild
(/home/rmurri/phd.cython/src/Cython-0.11/Cython/Compiler/Visitor.c:2814)
    File "Visitor.py", line 45, in
Cython.Compiler.Visitor.BasicVisitor.visit
(/home/rmurri/phd.cython/src/Cython-0.11/Cython/Compiler/Visitor.c:1356)
    File "/home/rmurri/phd.cython/src/Cython-0.11/Cython/Compiler/ParseTreeTransforms.py",
line 788, in visit_DefNode
      pxd_def = self.scope.lookup(node.name)
  AttributeError: 'NoneType' object has no attribute 'lookup'


Compilation is fine if I merge the .pxd and the .py into a .pyx::

  cpdef class Iter(object):

      """A docstring."""

      def __iter__(self):
          return self

      def __next__(self):
          raise StopIteration


Best regards,
Riccardo


-- 
Riccardo Murri, via Ross?e 17, 6987 Caslano (CH)

From dalcinl at gmail.com  Tue Apr 14 01:17:30 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 13 Apr 2009 20:17:30 -0300
Subject: [Cython] very simple iterator crashing Cython-0.11
In-Reply-To: <1c34ba170904131414s42ea8446o7c06b5bbbd876705@mail.gmail.com>
References: <1c34ba170904131414s42ea8446o7c06b5bbbd876705@mail.gmail.com>
Message-ID: <e7ba66e40904131617n3d464c8ct2dd1dae693f651f1@mail.gmail.com>

What happens if you write "cdef class ..." instead of "cpdef class
..." on your pxd?

On Mon, Apr 13, 2009 at 6:14 PM, Riccardo Murri
<riccardo.murri at gmail.com> wrote:
> Hello,
>
> I have started using Cython recently to spped up some Python code; it
> works very well and stably, but today I managed to crash Cython 0.11
> while trying to compile the following pure Python code:
>
> ?# iter.py
> ?class Iter(object):
>
> ? ? ?"""A docstring."""
>
> ? ? ?def __iter__(self):
> ? ? ? ? ?return self
>
> ? ? ?def __next__(self):
> ? ? ? ? ?raise StopIteration
>
> I'm augmenting the pure Python code with the following .pxd file::
>
> ?# iter.pxd
> ?cpdef class Iter(object)
>
> The Cython-0.11 invocation "cython iter.py" dies with the following backtrace::
>
> ?$ cython iter.py
>
> ?Error converting Pyrex file to C:
> ?------------------------------------------------------------
> ?...
> ?class Iter(object):
>
> ? ? ?"""A docstring."""
>
> ? ? ?def __iter__(self):
> ? ? ^
> ?------------------------------------------------------------
>
> ?/tmp/iter.py:5:4: Compiler crash in AlignFunctionDefinitions
>
> ?ModuleNode.body = StatListNode(iter.py:1:0)
> ?StatListNode.stats[0] = PyClassDefNode(iter.py:1:0,
> ? ? ?doc = 'A docstring.',
> ? ? ?name = u'Iter')
> ?CClassDefNode.body = StatListNode(iter.py:5:4)
> ?StatListNode.stats[0] = DefNode(iter.py:5:4,
> ? ? ?name = u'__iter__',
> ? ? ?num_required_args = 1,
> ? ? ?reqd_kw_flags_cname = '0')
>
> ?Compiler crash traceback from this point on:
> ? ?File "Visitor.py", line 149, in Cython.Compiler.Visitor.visitchild
> (/home/rmurri/phd.cython/src/Cython-0.11/Cython/Compiler/Visitor.c:2814)
> ? ?File "Visitor.py", line 45, in
> Cython.Compiler.Visitor.BasicVisitor.visit
> (/home/rmurri/phd.cython/src/Cython-0.11/Cython/Compiler/Visitor.c:1356)
> ? ?File "/home/rmurri/phd.cython/src/Cython-0.11/Cython/Compiler/ParseTreeTransforms.py",
> line 788, in visit_DefNode
> ? ? ?pxd_def = self.scope.lookup(node.name)
> ?AttributeError: 'NoneType' object has no attribute 'lookup'
>
>
> Compilation is fine if I merge the .pxd and the .py into a .pyx::
>
> ?cpdef class Iter(object):
>
> ? ? ?"""A docstring."""
>
> ? ? ?def __iter__(self):
> ? ? ? ? ?return self
>
> ? ? ?def __next__(self):
> ? ? ? ? ?raise StopIteration
>
>
> Best regards,
> Riccardo
>
>
> --
> Riccardo Murri, via Ross?e 17, 6987 Caslano (CH)
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From riccardo.murri at gmail.com  Tue Apr 14 10:01:14 2009
From: riccardo.murri at gmail.com (Riccardo Murri)
Date: Tue, 14 Apr 2009 10:01:14 +0200
Subject: [Cython] very simple iterator crashing Cython-0.11
In-Reply-To: <e7ba66e40904131617n3d464c8ct2dd1dae693f651f1@mail.gmail.com>
References: <1c34ba170904131414s42ea8446o7c06b5bbbd876705@mail.gmail.com>
	<e7ba66e40904131617n3d464c8ct2dd1dae693f651f1@mail.gmail.com>
Message-ID: <1c34ba170904140101l37100db6k7805277f5c50db9d@mail.gmail.com>

On Tue, Apr 14, 2009 at 1:17 AM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
> What happens if you write "cdef class ..." instead of "cpdef class
> ..." on your pxd?
>

Same error.

-- 
Riccardo Murri, via Ross?e 17, 6987 Caslano (CH)

From robertwb at math.washington.edu  Tue Apr 14 10:09:14 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 14 Apr 2009 01:09:14 -0700
Subject: [Cython] very simple iterator crashing Cython-0.11
In-Reply-To: <1c34ba170904140101l37100db6k7805277f5c50db9d@mail.gmail.com>
References: <1c34ba170904131414s42ea8446o7c06b5bbbd876705@mail.gmail.com>
	<e7ba66e40904131617n3d464c8ct2dd1dae693f651f1@mail.gmail.com>
	<1c34ba170904140101l37100db6k7805277f5c50db9d@mail.gmail.com>
Message-ID: <530BC1C3-5A3C-404B-AD51-DD7B1DD5A61B@math.washington.edu>

On Apr 14, 2009, at 1:01 AM, Riccardo Murri wrote:

> On Tue, Apr 14, 2009 at 1:17 AM, Lisandro Dalcin  
> <dalcinl at gmail.com> wrote:
>> What happens if you write "cdef class ..." instead of "cpdef class
>> ..." on your pxd?
>>
>
> Same error.

I'm surprised that "cpdef class" works at all (it shouldn't). In any  
case, this is most certainly a bug, we are tracking it at http:// 
trac.cython.org/cython_trac/ticket/279 .

- Robert



From dagss at student.matnat.uio.no  Tue Apr 14 14:41:25 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 14 Apr 2009 14:41:25 +0200
Subject: [Cython] Poll: Buffer access exception under nogil
Message-ID: <49E48475.9090406@student.matnat.uio.no>

Cython currently segfaults when using a buffer in nogil and accessing 
out of bounds (which triggers an IndexError). This is a bug, however I 
would like to do more than disallow it.

I.e. what should happen here:

     cdef object[int] buf = ... # buffer of length < 6
     with nogil:
         buf[5] = 0

Option 1) Reacquire the GIL temporarily to construct the IndexError 
instance. This seems to work wonderfully in my current patch; but is it 
too unintuitive to raise an exception from an operation in a nogil 
section? Will any other operation trigger an exception within a nogil 
section?

Option 2) Only allow buffer use in nogil if @cython.boundscheck(False). 
This has some usability problems; it would be a pain to turn on/off 
nogil for debugging.

Option 3) Print to standard output using printf instead of raising 
exception.



-- 
Dag Sverre

From stefan_ml at behnel.de  Tue Apr 14 15:30:44 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 14 Apr 2009 15:30:44 +0200 (CEST)
Subject: [Cython] Poll: Buffer access exception under nogil
In-Reply-To: <49E48475.9090406@student.matnat.uio.no>
References: <49E48475.9090406@student.matnat.uio.no>
Message-ID: <c37ebdbdf4fe28e78fbe0036ccb602c3.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Dag Sverre Seljebotn wrote:
> Cython currently segfaults when using a buffer in nogil and accessing
> out of bounds (which triggers an IndexError). This is a bug, however I
> would like to do more than disallow it.
>
> I.e. what should happen here:
>
>      cdef object[int] buf = ... # buffer of length < 6
>      with nogil:
>          buf[5] = 0
>
> Option 1) Reacquire the GIL temporarily to construct the IndexError
> instance. This seems to work wonderfully in my current patch; but is it
> too unintuitive to raise an exception from an operation in a nogil
> section?

Well, exceptions happen to be the error reporting mechanism in Python. In
any case, the above must exit the nogil block immediately and should not
crash (as long as we can prevent it in a reasonable way). To me, this
sounds like an exception is the right thing.

However, it's quite possible that this kind of control flow makes error
handling and C-level cleanup measures a nightmare. How would such a nogil
block interact with try-except or try-finally, for example? We can't allow
try-except /inside/ the block, so you'd have to anticipate all errors and
wrap separate sections of the nogil block in their own try-except or
try-finally statements to make sure the cleanup works as required.


> Will any other operation trigger an exception within a nogil
> section?

You can have exceptions that get raised in callbacks that re-acquire the
GIL, so you can't be sure that nogil blocks do not raise exceptions
anyway.

Think of code like this (which does not currently work):

    cdef extern from *:
        # some external C library function
        void call_into_c(...) nogil except *

    cdef void callback(void* data) with gil except *:
        ((...*)data).failed = 1   # cause abort in calling C library
        raise IndexError

    try:
        with nogil:
            call_into_c(callback, &some_data)
            printf("we never get here!")
    except IndexError:
        print "holy moly!"

To support this, Cython would have to generate code to re-acquire the GIL
after the call to "call_into_c()", check for an error and then either jump
out of the failed nogil block or release the GIL and continue.

If we start raising exceptions inside nogil blocks, this would be the next
obvious step. I have not made up my mind yet if this is a good thing. At
least, I know a couple of cases in lxml where I could use it, but this
will certainly need some prior exploration before deciding to make this an
official language feature.


> Option 2) Only allow buffer use in nogil if @cython.boundscheck(False).
> This has some usability problems; it would be a pain to turn on/off
> nogil for debugging.

Sounds like a viable alternative, though. Should be followed if exception
raising is not considered accurate from inside nogil blocks.


> Option 3) Print to standard output using printf instead of raising
> exception.

-1 This is a clear error case that should not pass silently.

Stefan


From dagss at student.matnat.uio.no  Tue Apr 14 18:17:28 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 14 Apr 2009 18:17:28 +0200
Subject: [Cython] C int division by zero
Message-ID: <49E4B718.5020502@student.matnat.uio.no>

I was just thinking about this:

     cdef int a = 10, b = 0
     print a // b

which currently gives a C exception terminating the Python interpreter.

On one hand: We have already changed division to Python behaviour (with 
a speed penalty); so using the same arguments (which I won't reiterate) 
it would make sense to raise a Python exception instead here.

It seems to make sense to couple this as well to the cdivision directive 
and cython.cdiv function.

On the other hand: Well, all the arguments that was against changing // 
to Python behaviour.

But in addition:

This is coupled with the "exception under nogil" issue that I raised in 
another thread today, as one should definitely be allowed to do C 
division within a nogil section.

One solution would be to work on accepting exceptions under nogil, 
another one would be to require use of cdivision directive/cdiv to do 
division in nogil sections.

-- 
Dag Sverre

From dagss at student.matnat.uio.no  Tue Apr 14 20:19:40 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 14 Apr 2009 20:19:40 +0200
Subject: [Cython] Poll: Buffer access exception under nogil
In-Reply-To: <c37ebdbdf4fe28e78fbe0036ccb602c3.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
References: <49E48475.9090406@student.matnat.uio.no>
	<c37ebdbdf4fe28e78fbe0036ccb602c3.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <49E4D3BC.8040809@student.matnat.uio.no>

Thanks for the feedback. I'm quite keen to get this resolved myself so a 
patch is sure to follow once I can get the semantics part sorted.

Stefan Behnel wrote:
> Well, exceptions happen to be the error reporting mechanism in Python. In
> any case, the above must exit the nogil block immediately and should not
> crash (as long as we can prevent it in a reasonable way). To me, this
> sounds like an exception is the right thing.
> 
> However, it's quite possible that this kind of control flow makes error
> handling and C-level cleanup measures a nightmare. How would such a nogil
> block interact with try-except or try-finally, for example? We can't allow
> try-except /inside/ the block, so you'd have to anticipate all errors and
> wrap separate sections of the nogil block in their own try-except or
> try-finally statements to make sure the cleanup works as required.

I'm not sure I get what you mean here; but anyway:

Looking at generated C source, it seems that "with nogil" sets up a 
plain "try/finally" block, with an acquisition of the GIL in the finally 
part. So even as it is it is perfectly possible to raise an exception 
the normal way; labels are set correctly and the exception-raising-goto 
path is resumed after acquisition of the GIL.

However I don't think any code makes use of this (?), and if it is, it 
raises an issue with the (lack of) thread-safety of the 
__pyx_filename/lineno/clineno variables (which in other places are 
protected by the GIL).

How about this:

Change things so that instead of a GIL block setting up a "try/finally" 
to reacquire the GIL, any exception-raising code within a GIL block is 
responsible for reacquiring the GIL. (Basically, code.funcstate.in_nogil 
is maintained and if nodes want to raise an exception they reacquire the 
GIL as part of doing that.)

>> Option 2) Only allow buffer use in nogil if @cython.boundscheck(False).
>> This has some usability problems; it would be a pain to turn on/off
>> nogil for debugging.
> 
> Sounds like a viable alternative, though. Should be followed if exception
> raising is not considered accurate from inside nogil blocks.

I might end up doing this for 0.11 and await input for the scheme above 
for 0.12 (if I decide to bother with 0.11).

-- 
Dag Sverre

From stefan_ml at behnel.de  Tue Apr 14 20:44:40 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 14 Apr 2009 20:44:40 +0200
Subject: [Cython] very simple iterator crashing Cython-0.11
In-Reply-To: <530BC1C3-5A3C-404B-AD51-DD7B1DD5A61B@math.washington.edu>
References: <1c34ba170904131414s42ea8446o7c06b5bbbd876705@mail.gmail.com>	<e7ba66e40904131617n3d464c8ct2dd1dae693f651f1@mail.gmail.com>	<1c34ba170904140101l37100db6k7805277f5c50db9d@mail.gmail.com>
	<530BC1C3-5A3C-404B-AD51-DD7B1DD5A61B@math.washington.edu>
Message-ID: <49E4D998.1030205@behnel.de>

Hi,

Robert Bradshaw wrote:
> On Apr 14, 2009, at 1:01 AM, Riccardo Murri wrote:
> 
>> On Tue, Apr 14, 2009 at 1:17 AM, Lisandro Dalcin  
>> <dalcinl at gmail.com> wrote:
>>> What happens if you write "cdef class ..." instead of "cpdef class
>>> ..." on your pxd?
>>>
>> Same error.
> 
> I'm surprised that "cpdef class" works at all (it shouldn't). In any  
> case, this is most certainly a bug, we are tracking it at http:// 
> trac.cython.org/cython_trac/ticket/279 .

The syntax of the cdef statement is waaay too forgiving. It even allows you
to write

	cdef pass

for a "pass" statement. I'll try to fix it up.

Stefan

From robertwb at math.washington.edu  Tue Apr 14 20:53:17 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 14 Apr 2009 11:53:17 -0700
Subject: [Cython] C int division by zero
In-Reply-To: <49E4B718.5020502@student.matnat.uio.no>
References: <49E4B718.5020502@student.matnat.uio.no>
Message-ID: <67E81388-686E-4B1E-A878-EBBD3B8A3662@math.washington.edu>

On Apr 14, 2009, at 9:17 AM, Dag Sverre Seljebotn wrote:

> I was just thinking about this:
>
>      cdef int a = 10, b = 0
>      print a // b
>
> which currently gives a C exception terminating the Python  
> interpreter.
>
> On one hand: We have already changed division to Python behaviour  
> (with
> a speed penalty); so using the same arguments (which I won't  
> reiterate)
> it would make sense to raise a Python exception instead here.
>
> It seems to make sense to couple this as well to the cdivision  
> directive
> and cython.cdiv function.
>
> On the other hand: Well, all the arguments that was against  
> changing //
> to Python behaviour.

Since we changed it, we should chang it all the way. It will be  
somewhat uglier as this means when writing a / (c-c) one needs to  
store the denominator in a temp and check it rather than simply  
letting the expression through to C (I already did this for warnings).

> But in addition:
>
> This is coupled with the "exception under nogil" issue that I  
> raised in
> another thread today, as one should definitely be allowed to do C
> division within a nogil section.
>
> One solution would be to work on accepting exceptions under nogil,
> another one would be to require use of cdivision directive/cdiv to do
> division in nogil sections.

I don't think exceptions are safe without the gil. With cdivision on  
it won't raise anything.

- Robert


From collette at physics.ucla.edu  Tue Apr 14 21:04:46 2009
From: collette at physics.ucla.edu (Andrew Collette)
Date: Tue, 14 Apr 2009 12:04:46 -0700
Subject: [Cython] Poll: Buffer access exception under nogil
In-Reply-To: <49E4D3BC.8040809@student.matnat.uio.no>
References: <49E48475.9090406@student.matnat.uio.no>
	<c37ebdbdf4fe28e78fbe0036ccb602c3.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<49E4D3BC.8040809@student.matnat.uio.no>
Message-ID: <f125f5e0904141204s1626d871sf88a949388d82717@mail.gmail.com>

Just for another data point w.r.t. C callbacks and the GIL, I actually
use this technique in my project HDF5 for Python
(h5py.googlecode.com).  The HDF5 library allows you to set an
arbitrary callback function, so I set it to (abridged):

void err_callback(...) with gil:
    raise SomeException("Whoops!" )

The library functions I care about are all imported with an exception
clause as e.g:

void HDF5_read_data(...) except -1

so that when I call into HDF5, if an error occurs, a Python exception
is constructed automatically.  In particular, the code for reading
data is wrapped in a nogil block:

try:
    with nogil:
        HDF5_read_data(...)
finally:
    ... some cleanup ...

In practice this seems to work just fine, although I haven't
extensively tested it with threads.  However, all Cython functions in
this module are protected by (the same) reentrant lock as HDF5 isn't
thread safe, so maybe this has been helping me avoid trouble.

Importantly, if the function HDF5_read_data is declared "except *",
Cython immediately crashes if an exception is raised; apparently it's
not valid to check for an exception without holding the GIL.
Declaring with "except <n>" seems to work just fine.

The code in question is at:
http://code.google.com/p/h5py/source/browse/trunk/h5py/h5.pyx (line 651)
and
http://code.google.com/p/h5py/source/browse/trunk/h5py/h5d.pyx (line 161)

Andrew Collette

On Tue, Apr 14, 2009 at 11:19 AM, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:
> Thanks for the feedback. I'm quite keen to get this resolved myself so a
> patch is sure to follow once I can get the semantics part sorted.
>
> Stefan Behnel wrote:
>> Well, exceptions happen to be the error reporting mechanism in Python. In
>> any case, the above must exit the nogil block immediately and should not
>> crash (as long as we can prevent it in a reasonable way). To me, this
>> sounds like an exception is the right thing.
>>
>> However, it's quite possible that this kind of control flow makes error
>> handling and C-level cleanup measures a nightmare. How would such a nogil
>> block interact with try-except or try-finally, for example? We can't allow
>> try-except /inside/ the block, so you'd have to anticipate all errors and
>> wrap separate sections of the nogil block in their own try-except or
>> try-finally statements to make sure the cleanup works as required.
>
> I'm not sure I get what you mean here; but anyway:
>
> Looking at generated C source, it seems that "with nogil" sets up a
> plain "try/finally" block, with an acquisition of the GIL in the finally
> part. So even as it is it is perfectly possible to raise an exception
> the normal way; labels are set correctly and the exception-raising-goto
> path is resumed after acquisition of the GIL.
>
> However I don't think any code makes use of this (?), and if it is, it
> raises an issue with the (lack of) thread-safety of the
> __pyx_filename/lineno/clineno variables (which in other places are
> protected by the GIL).
>
> How about this:
>
> Change things so that instead of a GIL block setting up a "try/finally"
> to reacquire the GIL, any exception-raising code within a GIL block is
> responsible for reacquiring the GIL. (Basically, code.funcstate.in_nogil
> is maintained and if nodes want to raise an exception they reacquire the
> GIL as part of doing that.)
>
>>> Option 2) Only allow buffer use in nogil if @cython.boundscheck(False).
>>> This has some usability problems; it would be a pain to turn on/off
>>> nogil for debugging.
>>
>> Sounds like a viable alternative, though. Should be followed if exception
>> raising is not considered accurate from inside nogil blocks.
>
> I might end up doing this for 0.11 and await input for the scheme above
> for 0.12 (if I decide to bother with 0.11).
>
> --
> Dag Sverre
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>
>

From stefan_ml at behnel.de  Tue Apr 14 21:13:19 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 14 Apr 2009 21:13:19 +0200
Subject: [Cython] Poll: Buffer access exception under nogil
In-Reply-To: <f125f5e0904141204s1626d871sf88a949388d82717@mail.gmail.com>
References: <49E48475.9090406@student.matnat.uio.no>	<c37ebdbdf4fe28e78fbe0036ccb602c3.squirrel@groupware.dvs.informatik.tu-darmstadt.de>	<49E4D3BC.8040809@student.matnat.uio.no>
	<f125f5e0904141204s1626d871sf88a949388d82717@mail.gmail.com>
Message-ID: <49E4E04F.603@behnel.de>

Hi,

just a quick note on this:

Andrew Collette wrote:
> void HDF5_read_data(...) except -1
> [...]
> try:
>     with nogil:
>         HDF5_read_data(...)
> finally:
>     ... some cleanup ...
>
> Importantly, if the function HDF5_read_data is declared "except *",
> Cython immediately crashes if an exception is raised

It shouldn't allow you to do that at all. I assume you declared the
function "nogil" to call it in a "with nogil" block. Combining "nogil" with
exception handling should raise an error at compile time.

There may be cases where this rule could potentially be relaxed, but the
compiler should catch this early in any case.

Stefan


From dagss at student.matnat.uio.no  Tue Apr 14 21:18:55 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 14 Apr 2009 21:18:55 +0200 (CEST)
Subject: [Cython] C int division by zero
In-Reply-To: <67E81388-686E-4B1E-A878-EBBD3B8A3662@math.washington.edu>
References: <49E4B718.5020502@student.matnat.uio.no>
	<67E81388-686E-4B1E-A878-EBBD3B8A3662@math.washington.edu>
Message-ID: <8ffacc54543ae4591accfdb1a58bce93.squirrel@webmail.uio.no>

Robert Bradshaw wrote:
> On Apr 14, 2009, at 9:17 AM, Dag Sverre Seljebotn wrote:
>
>> I was just thinking about this:
>>
>>      cdef int a = 10, b = 0
>>      print a // b
>>
>> which currently gives a C exception terminating the Python
>> interpreter.
>>
>> On one hand: We have already changed division to Python behaviour
>> (with
>> a speed penalty); so using the same arguments (which I won't
>> reiterate)
>> it would make sense to raise a Python exception instead here.
>
> Since we changed it, we should chang it all the way. It will be
> somewhat uglier as this means when writing a / (c-c) one needs to
> store the denominator in a temp and check it rather than simply
> letting the expression through to C (I already did this for warnings).

We're already in beta, but still: Can we get a warning cycle for this as
well into 0.11.1 so that people only have to deal with C int semantic
changes once? I can probably make it tomorrow but I'll have a higher
learning curve as you did the previous change.

>> One solution would be to work on accepting exceptions under nogil,
>> another one would be to require use of cdivision directive/cdiv to do
>> division in nogil sections.
>
> I don't think exceptions are safe without the gil. With cdivision on
> it won't raise anything.

This is an exceptional case; I think it is OK to acquire the GIL
temporarily to construct the exception if the exception hits, as long as
non-exception paths doesn't require waiting for a GIL lock?

See also the other thread on this.

Dag Sverre


From stefan_ml at behnel.de  Tue Apr 14 21:24:40 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 14 Apr 2009 21:24:40 +0200
Subject: [Cython] very simple iterator crashing Cython-0.11
In-Reply-To: <49E4D998.1030205@behnel.de>
References: <1c34ba170904131414s42ea8446o7c06b5bbbd876705@mail.gmail.com>	<e7ba66e40904131617n3d464c8ct2dd1dae693f651f1@mail.gmail.com>	<1c34ba170904140101l37100db6k7805277f5c50db9d@mail.gmail.com>	<530BC1C3-5A3C-404B-AD51-DD7B1DD5A61B@math.washington.edu>
	<49E4D998.1030205@behnel.de>
Message-ID: <49E4E2F8.5040907@behnel.de>


Stefan Behnel wrote:
> The syntax of the cdef statement is waaay too forgiving. It even allows you
> to write
> 
> 	cdef pass
> 
> for a "pass" statement.

I now wonder why "pass" appears inside the "cdef" parsing function at all.
I don't mean using "pass" inside of a cdef block, just right in the
declaration itself. Is there any combination of "cdef" with "pass" that
makes any sense? Any kind of cdef-extern, struct/enum/union or class
declaration line?

Stefan

From robertwb at math.washington.edu  Tue Apr 14 21:33:51 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 14 Apr 2009 12:33:51 -0700
Subject: [Cython] C int division by zero
In-Reply-To: <8ffacc54543ae4591accfdb1a58bce93.squirrel@webmail.uio.no>
References: <49E4B718.5020502@student.matnat.uio.no>
	<67E81388-686E-4B1E-A878-EBBD3B8A3662@math.washington.edu>
	<8ffacc54543ae4591accfdb1a58bce93.squirrel@webmail.uio.no>
Message-ID: <AF86289D-B18E-478D-89E5-0BEB5DFC2C05@math.washington.edu>

On Apr 14, 2009, at 12:18 PM, Dag Sverre Seljebotn wrote:

> Robert Bradshaw wrote:
>> On Apr 14, 2009, at 9:17 AM, Dag Sverre Seljebotn wrote:
>>
>>> I was just thinking about this:
>>>
>>>      cdef int a = 10, b = 0
>>>      print a // b
>>>
>>> which currently gives a C exception terminating the Python
>>> interpreter.
>>>
>>> On one hand: We have already changed division to Python behaviour
>>> (with
>>> a speed penalty); so using the same arguments (which I won't
>>> reiterate)
>>> it would make sense to raise a Python exception instead here.
>>
>> Since we changed it, we should chang it all the way. It will be
>> somewhat uglier as this means when writing a / (c-c) one needs to
>> store the denominator in a temp and check it rather than simply
>> letting the expression through to C (I already did this for  
>> warnings).
>
> We're already in beta, but still: Can we get a warning cycle for  
> this as
> well into 0.11.1 so that people only have to deal with C int semantic
> changes once? I can probably make it tomorrow but I'll have a higher
> learning curve as you did the previous change.

Yes, I'll do this. I think segfault -> ZeroDivisionError exception  
will be an easy migration path for most people :).

>>> One solution would be to work on accepting exceptions under nogil,
>>> another one would be to require use of cdivision directive/cdiv  
>>> to do
>>> division in nogil sections.
>>
>> I don't think exceptions are safe without the gil. With cdivision on
>> it won't raise anything.
>
> This is an exceptional case; I think it is OK to acquire the GIL
> temporarily to construct the exception if the exception hits, as  
> long as
> non-exception paths doesn't require waiting for a GIL lock?
>
> See also the other thread on this.

Hmm... I don't have a definite opinion on this yet, but that might be  
reasonable. Propagating an exception requires the gil too...it  
dilutes the pureness of "no gil."

- Robert



From dagss at student.matnat.uio.no  Tue Apr 14 22:19:24 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 14 Apr 2009 22:19:24 +0200
Subject: [Cython] Exceptions under nogil
In-Reply-To: <c37ebdbdf4fe28e78fbe0036ccb602c3.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
References: <49E48475.9090406@student.matnat.uio.no>
	<c37ebdbdf4fe28e78fbe0036ccb602c3.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <49E4EFCC.30402@student.matnat.uio.no>

Stefan Behnel wrote:
> Dag Sverre Seljebotn wrote:
>> Cython currently segfaults when using a buffer in nogil and accessing
>> out of bounds (which triggers an IndexError). This is a bug, however I
>> would like to do more than disallow it.
>>
>> I.e. what should happen here:
>>
>>      cdef object[int] buf = ... # buffer of length < 6
>>      with nogil:
>>          buf[5] = 0
>>
>> Option 1) Reacquire the GIL temporarily to construct the IndexError
>> instance. This seems to work wonderfully in my current patch; but is it
>> too unintuitive to raise an exception from an operation in a nogil
>> section?
> 
> Well, exceptions happen to be the error reporting mechanism in Python. In
> any case, the above must exit the nogil block immediately and should not
> crash (as long as we can prevent it in a reasonable way). To me, this
> sounds like an exception is the right thing.

However if the disadvantages outweighs the benefits... also, as Robert 
says, exception control flow would "water out" the nogil.

I'm OK with waiting for a GIL if an exception is raised, but on a mere 
function call seems much worse.

How about

  a) only flag for exceptions while in nogil, using a thread-local variable
  b) upon exiting the nogil section a 
ExceptionRaisedWhileInNoGilSectionError can be raised?
  c) Possibly add a function cython.nogil_exception_occurred so that 
exceptions can be checked for in nogil sections.

Try/except cannot ever be used in nogil sections after all.

After all, requiring compiler directives explictly silencing exceptions 
doesn't seem better than getting the exception a little late :-)

BTW, does anybody know if there's perhaps a way of querying 
PyErr_Occurred without the GIL, by Python-interpreter-dependant hacks if 
necesarry? It's just a read of a flag which is likely pointed to by a 
thread-local variable after all.

If this could be done I'm all for exception propagation in nogil 
sections (and a GIL is automatically acquired if exception must be 
constructed).

-- 
Dag Sverre

From carl.witty at gmail.com  Tue Apr 14 22:04:05 2009
From: carl.witty at gmail.com (Carl Witty)
Date: Tue, 14 Apr 2009 13:04:05 -0700
Subject: [Cython] C int division by zero
In-Reply-To: <67E81388-686E-4B1E-A878-EBBD3B8A3662@math.washington.edu>
References: <49E4B718.5020502@student.matnat.uio.no>
	<67E81388-686E-4B1E-A878-EBBD3B8A3662@math.washington.edu>
Message-ID: <ac1e4c420904141304o20e87d71tfd0fc5d3ced9a681@mail.gmail.com>

On Tue, Apr 14, 2009 at 11:53 AM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Apr 14, 2009, at 9:17 AM, Dag Sverre Seljebotn wrote:
>
>> I was just thinking about this:
>>
>> ? ? ?cdef int a = 10, b = 0
>> ? ? ?print a // b
>>
>> which currently gives a C exception terminating the Python
>> interpreter.
>>
>> On one hand: We have already changed division to Python behaviour
>> (with
>> a speed penalty); so using the same arguments (which I won't
>> reiterate)
>> it would make sense to raise a Python exception instead here.
>>
>> It seems to make sense to couple this as well to the cdivision
>> directive
>> and cython.cdiv function.
>>
>> On the other hand: Well, all the arguments that was against
>> changing //
>> to Python behaviour.
>
> Since we changed it, we should chang it all the way. It will be
> somewhat uglier as this means when writing a / (c-c) one needs to
> store the denominator in a temp and check it rather than simply
> letting the expression through to C (I already did this for warnings).

If we're going through and listing special cases for division, there's
one more to consider (I think this is the last one):

On x86 processors, dividing -2**31 by -1 gives a processor signal;
under Linux, this results in exiting the process with "Floating point
exception".  (Even though it's not floating point.)  (Presumably, the
same thing happens when dividing -2**63 by -1 on a 64-bit processor; I
haven't tested.)

Carl

From collette at physics.ucla.edu  Tue Apr 14 22:35:17 2009
From: collette at physics.ucla.edu (Andrew Collette)
Date: Tue, 14 Apr 2009 13:35:17 -0700
Subject: [Cython] Poll: Buffer access exception under nogil
In-Reply-To: <49E4E04F.603@behnel.de>
References: <49E48475.9090406@student.matnat.uio.no>
	<c37ebdbdf4fe28e78fbe0036ccb602c3.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<49E4D3BC.8040809@student.matnat.uio.no>
	<f125f5e0904141204s1626d871sf88a949388d82717@mail.gmail.com>
	<49E4E04F.603@behnel.de>
Message-ID: <f125f5e0904141335u2711af22qdb11781ab3d63a28@mail.gmail.com>

Hi,

Well, the function signature in question is (in the referred file h5d.pyx):

"cdef int H5PY_H5Dread(<stuff>) nogil except -1"

which compiles just fine... is this going to stop working?  Is there a
better way to do it?  It seems like "except -1" simply  translates to
"after this function is called, if the return code is -1, a Python
exception happened in this thread".  It seems like the GIL should be
reacquired, the nogil block exited immediately, and the appropriate
branches of any try...except...finally statement should be executed.
If -1 isn't returned, the GIL isn't touched.

Whatever happens, I need to be able to run nogil code (reading in
data) which can possibly acquire the GIL and set an exception (via a
callback), and then be able to detect and propagate the exception back
to Python.   I am certainly open to refactoring the code if there's a
better way to do this in Cython.

Andrew Collette


On Tue, Apr 14, 2009 at 12:13 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Hi,
>
> just a quick note on this:
>
> Andrew Collette wrote:
>> void HDF5_read_data(...) except -1
>> [...]
>> try:
>> ? ? with nogil:
>> ? ? ? ? HDF5_read_data(...)
>> finally:
>> ? ? ... some cleanup ...
>>
>> Importantly, if the function HDF5_read_data is declared "except *",
>> Cython immediately crashes if an exception is raised
>
> It shouldn't allow you to do that at all. I assume you declared the
> function "nogil" to call it in a "with nogil" block. Combining "nogil" with
> exception handling should raise an error at compile time.
>
> There may be cases where this rule could potentially be relaxed, but the
> compiler should catch this early in any case.
>
> Stefan
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>
>

From stefan_ml at behnel.de  Tue Apr 14 22:47:17 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 14 Apr 2009 22:47:17 +0200
Subject: [Cython] Poll: Buffer access exception under nogil
In-Reply-To: <f125f5e0904141335u2711af22qdb11781ab3d63a28@mail.gmail.com>
References: <49E48475.9090406@student.matnat.uio.no>	<c37ebdbdf4fe28e78fbe0036ccb602c3.squirrel@groupware.dvs.informatik.tu-darmstadt.de>	<49E4D3BC.8040809@student.matnat.uio.no>	<f125f5e0904141204s1626d871sf88a949388d82717@mail.gmail.com>	<49E4E04F.603@behnel.de>
	<f125f5e0904141335u2711af22qdb11781ab3d63a28@mail.gmail.com>
Message-ID: <49E4F655.8080001@behnel.de>

Hi,

please don't top-post.

Andrew Collette wrote:
> Well, the function signature in question is (in the referred file h5d.pyx):
> 
> "cdef int H5PY_H5Dread(<stuff>) nogil except -1"
> 
> which compiles just fine... is this going to stop working?

I think this is the one case where exception propagation will easily work
in a nogil block. If we get the exception propagation semantics right here,
we can leave this in. But it's not entirely clear (at least to me) if this
currently behaves 'correctly'.

It *might* be possible to enable the same for "except *" somehow, but until
then, Cython should raise an error at compile time rather than letting the
code crash at runtime.

Stefan


From stefan_ml at behnel.de  Tue Apr 14 23:14:49 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 14 Apr 2009 23:14:49 +0200
Subject: [Cython] very simple iterator crashing Cython-0.11
In-Reply-To: <49E4D998.1030205@behnel.de>
References: <1c34ba170904131414s42ea8446o7c06b5bbbd876705@mail.gmail.com>	<e7ba66e40904131617n3d464c8ct2dd1dae693f651f1@mail.gmail.com>	<1c34ba170904140101l37100db6k7805277f5c50db9d@mail.gmail.com>	<530BC1C3-5A3C-404B-AD51-DD7B1DD5A61B@math.washington.edu>
	<49E4D998.1030205@behnel.de>
Message-ID: <49E4FCC9.80704@behnel.de>


Stefan Behnel wrote:
> The syntax of the cdef statement is waaay too forgiving. It even allows you
> to write
> 
> 	cdef pass
> 
> for a "pass" statement. I'll try to fix it up.

Patch is up at

http://trac.cython.org/cython_trac/ticket/280

Not sure if this is safe to go into 0.11.1, so I'd be happy to see it
tested on some real code.

Stefan

From collette at physics.ucla.edu  Tue Apr 14 23:56:23 2009
From: collette at physics.ucla.edu (Andrew Collette)
Date: Tue, 14 Apr 2009 14:56:23 -0700
Subject: [Cython] Poll: Buffer access exception under nogil
In-Reply-To: <49E4F655.8080001@behnel.de>
References: <49E48475.9090406@student.matnat.uio.no>
	<c37ebdbdf4fe28e78fbe0036ccb602c3.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<49E4D3BC.8040809@student.matnat.uio.no>
	<f125f5e0904141204s1626d871sf88a949388d82717@mail.gmail.com>
	<49E4E04F.603@behnel.de>
	<f125f5e0904141335u2711af22qdb11781ab3d63a28@mail.gmail.com>
	<49E4F655.8080001@behnel.de>
Message-ID: <f125f5e0904141456k1c81d0bbl6e88e30b96c8ddf1@mail.gmail.com>

> please don't top-post.

Well excuse me. :) I didn't notice Gmail was stuffing the whole thing
under my reply.

> I think this is the one case where exception propagation will easily work
> in a nogil block. If we get the exception propagation semantics right here,
> we can leave this in. But it's not entirely clear (at least to me) if this
> currently behaves 'correctly'.

OK, it sounds like I should rework this so that it fits better with
the current defined behavior of Cython, to avoid unexpected breakage.

> It *might* be possible to enable the same for "except *" somehow, but until
> then, Cython should raise an error at compile time rather than letting the
> code crash at runtime.

I agree, this shouldn't even compile with "except *"; re-acquiring the
GIL inside a nogil block every time an "except-*" declared function
returns seems like it's too surprising, especially in the (common)
case of infrequently raised exceptions.  Nogil code shouldn't touch
the GIL unless it really, really has to.

Thanks,
Andrew Collette

From robertwb at math.washington.edu  Wed Apr 15 00:27:08 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 14 Apr 2009 15:27:08 -0700
Subject: [Cython] C int division by zero
In-Reply-To: <ac1e4c420904141304o20e87d71tfd0fc5d3ced9a681@mail.gmail.com>
References: <49E4B718.5020502@student.matnat.uio.no>
	<67E81388-686E-4B1E-A878-EBBD3B8A3662@math.washington.edu>
	<ac1e4c420904141304o20e87d71tfd0fc5d3ced9a681@mail.gmail.com>
Message-ID: <9411A55A-C607-4303-A7A7-64F259CC8E21@math.washington.edu>

On Apr 14, 2009, at 1:04 PM, Carl Witty wrote:

> On Tue, Apr 14, 2009 at 11:53 AM, Robert Bradshaw
> <robertwb at math.washington.edu> wrote:
>> On Apr 14, 2009, at 9:17 AM, Dag Sverre Seljebotn wrote:
>>
>>> I was just thinking about this:
>>>
>>>      cdef int a = 10, b = 0
>>>      print a // b
>>>
>>> which currently gives a C exception terminating the Python
>>> interpreter.
>>>
>>> On one hand: We have already changed division to Python behaviour
>>> (with
>>> a speed penalty); so using the same arguments (which I won't
>>> reiterate)
>>> it would make sense to raise a Python exception instead here.
>>>
>>> It seems to make sense to couple this as well to the cdivision
>>> directive
>>> and cython.cdiv function.
>>>
>>> On the other hand: Well, all the arguments that was against
>>> changing //
>>> to Python behaviour.
>>
>> Since we changed it, we should chang it all the way. It will be
>> somewhat uglier as this means when writing a / (c-c) one needs to
>> store the denominator in a temp and check it rather than simply
>> letting the expression through to C (I already did this for  
>> warnings).

OK, pushed. http://hg.cython.org/cython-devel/rev/a09a309febdc

> If we're going through and listing special cases for division, there's
> one more to consider (I think this is the last one):
>
> On x86 processors, dividing -2**31 by -1 gives a processor signal;
> under Linux, this results in exiting the process with "Floating point
> exception".  (Even though it's not floating point.)  (Presumably, the
> same thing happens when dividing -2**63 by -1 on a 64-bit processor; I
> haven't tested.)

Interesting corner case. This seems more like an overflow issue, and  
we're currently not trying to protect the user from those.

- Robert


From robertwb at math.washington.edu  Wed Apr 15 01:14:39 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 14 Apr 2009 16:14:39 -0700
Subject: [Cython] very simple iterator crashing Cython-0.11
In-Reply-To: <49E4FCC9.80704@behnel.de>
References: <1c34ba170904131414s42ea8446o7c06b5bbbd876705@mail.gmail.com>	<e7ba66e40904131617n3d464c8ct2dd1dae693f651f1@mail.gmail.com>	<1c34ba170904140101l37100db6k7805277f5c50db9d@mail.gmail.com>	<530BC1C3-5A3C-404B-AD51-DD7B1DD5A61B@math.washington.edu>
	<49E4D998.1030205@behnel.de> <49E4FCC9.80704@behnel.de>
Message-ID: <08592E04-036D-464C-94B2-32AF1473663F@math.washington.edu>

On Apr 14, 2009, at 2:14 PM, Stefan Behnel wrote:

>
> Stefan Behnel wrote:
>> The syntax of the cdef statement is waaay too forgiving. It even  
>> allows you
>> to write
>>
>> 	cdef pass
>>
>> for a "pass" statement. I'll try to fix it up.
>
> Patch is up at
>
> http://trac.cython.org/cython_trac/ticket/280
>
> Not sure if this is safe to go into 0.11.1, so I'd be happy to see it
> tested on some real code.

I don't see anything wrong with it, but better safe than sorry.

- Robert


From robertwb at math.washington.edu  Wed Apr 15 01:21:23 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 14 Apr 2009 16:21:23 -0700
Subject: [Cython] Cython 0.11.1 release candidate
Message-ID: <27D772B7-E8B6-4A8A-9701-49D7C10A9DA0@math.washington.edu>

Unless something is horribly broken, I will release in the next day  
or two.

http://cython.org/Cython-0.11.1.rc1.tar.gz

- Robert

From dagss at student.matnat.uio.no  Wed Apr 15 08:44:19 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Wed, 15 Apr 2009 08:44:19 +0200
Subject: [Cython] C int division by zero
In-Reply-To: <9411A55A-C607-4303-A7A7-64F259CC8E21@math.washington.edu>
References: <49E4B718.5020502@student.matnat.uio.no>	<67E81388-686E-4B1E-A878-EBBD3B8A3662@math.washington.edu>	<ac1e4c420904141304o20e87d71tfd0fc5d3ced9a681@mail.gmail.com>
	<9411A55A-C607-4303-A7A7-64F259CC8E21@math.washington.edu>
Message-ID: <49E58243.6090704@student.matnat.uio.no>

Robert Bradshaw wrote:
> On Apr 14, 2009, at 1:04 PM, Carl Witty wrote:
>> On x86 processors, dividing -2**31 by -1 gives a processor signal;
>> under Linux, this results in exiting the process with "Floating point
>> exception".  (Even though it's not floating point.)  (Presumably, the
>> same thing happens when dividing -2**63 by -1 on a 64-bit processor; I
>> haven't tested.)
> 
> Interesting corner case. This seems more like an overflow issue, and  
> we're currently not trying to protect the user from those.

Well, yes, but usually with overflows one just gets the overflowed value 
silently. With division overflow one crashes the application, which is 
IMO much worse even if it is a very rare corner case.

I think we should raise OverflowError here under the same circumstances 
as ZeroDivisionError.

-- 
Dag Sverre

From dagss at student.matnat.uio.no  Wed Apr 15 08:49:27 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Wed, 15 Apr 2009 08:49:27 +0200
Subject: [Cython] C int division by zero
In-Reply-To: <9411A55A-C607-4303-A7A7-64F259CC8E21@math.washington.edu>
References: <49E4B718.5020502@student.matnat.uio.no>	<67E81388-686E-4B1E-A878-EBBD3B8A3662@math.washington.edu>	<ac1e4c420904141304o20e87d71tfd0fc5d3ced9a681@mail.gmail.com>
	<9411A55A-C607-4303-A7A7-64F259CC8E21@math.washington.edu>
Message-ID: <49E58377.8020103@student.matnat.uio.no>

Robert Bradshaw wrote:
> 
> OK, pushed. http://hg.cython.org/cython-devel/rev/a09a309febdc

Note that this now segfaults:

@cython.cdivision(False)
def f():
     cdef int a=10, b=0
     with nogil:
         a = a // b
     return a

-- 
Dag Sverre

From stefan_ml at behnel.de  Wed Apr 15 08:35:12 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 15 Apr 2009 08:35:12 +0200 (CEST)
Subject: [Cython] C int division by zero
In-Reply-To: <9411A55A-C607-4303-A7A7-64F259CC8E21@math.washington.edu>
References: <49E4B718.5020502@student.matnat.uio.no>
	<67E81388-686E-4B1E-A878-EBBD3B8A3662@math.washington.edu>
	<ac1e4c420904141304o20e87d71tfd0fc5d3ced9a681@mail.gmail.com>
	<9411A55A-C607-4303-A7A7-64F259CC8E21@math.washington.edu>
Message-ID: <4d3dda8bda5f2d8cd9b626b526cac84f.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Robert Bradshaw wrote:
>>> On Apr 14, 2009, at 9:17 AM, Dag Sverre Seljebotn wrote:
>>>
>>>> I was just thinking about this:
>>>>
>>>>      cdef int a = 10, b = 0
>>>>      print a // b
>>>>
>>>> which currently gives a C exception terminating the Python
>>>> interpreter.
>>>
>>> Since we changed it, we should chang it all the way. It will be
>>> somewhat uglier as this means when writing a / (c-c) one needs to
>>> store the denominator in a temp and check it rather than simply
>>> letting the expression through to C (I already did this for
>>> warnings).
>
> OK, pushed. http://hg.cython.org/cython-devel/rev/a09a309febdc

Note that this requires a GIL check now (just like the warning case),
haven't checked if this case is currently handled.

Stefan


From robertwb at math.washington.edu  Wed Apr 15 08:41:34 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 14 Apr 2009 23:41:34 -0700
Subject: [Cython] C int division by zero
In-Reply-To: <49E58377.8020103@student.matnat.uio.no>
References: <49E4B718.5020502@student.matnat.uio.no>	<67E81388-686E-4B1E-A878-EBBD3B8A3662@math.washington.edu>	<ac1e4c420904141304o20e87d71tfd0fc5d3ced9a681@mail.gmail.com>
	<9411A55A-C607-4303-A7A7-64F259CC8E21@math.washington.edu>
	<49E58377.8020103@student.matnat.uio.no>
Message-ID: <B21DBA68-07FD-4C89-B28F-91985AE1DF6C@math.washington.edu>

On Apr 14, 2009, at 11:49 PM, Dag Sverre Seljebotn wrote:

> Robert Bradshaw wrote:
>>
>> OK, pushed. http://hg.cython.org/cython-devel/rev/a09a309febdc
>
> Note that this now segfaults:
>
> @cython.cdivision(False)
> def f():
>      cdef int a=10, b=0
>      with nogil:
>          a = a // b
>      return a

Ah, because raising the exception requires the GIL. I'll have to add  
a check in this case.

- Robert


From robertwb at math.washington.edu  Wed Apr 15 08:42:40 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 14 Apr 2009 23:42:40 -0700
Subject: [Cython] C int division by zero
In-Reply-To: <49E58243.6090704@student.matnat.uio.no>
References: <49E4B718.5020502@student.matnat.uio.no>	<67E81388-686E-4B1E-A878-EBBD3B8A3662@math.washington.edu>	<ac1e4c420904141304o20e87d71tfd0fc5d3ced9a681@mail.gmail.com>
	<9411A55A-C607-4303-A7A7-64F259CC8E21@math.washington.edu>
	<49E58243.6090704@student.matnat.uio.no>
Message-ID: <0EB5D934-2764-464E-A7E8-38D614FAC5A9@math.washington.edu>

On Apr 14, 2009, at 11:44 PM, Dag Sverre Seljebotn wrote:

> Robert Bradshaw wrote:
>> On Apr 14, 2009, at 1:04 PM, Carl Witty wrote:
>>> On x86 processors, dividing -2**31 by -1 gives a processor signal;
>>> under Linux, this results in exiting the process with "Floating  
>>> point
>>> exception".  (Even though it's not floating point.)  (Presumably,  
>>> the
>>> same thing happens when dividing -2**63 by -1 on a 64-bit  
>>> processor; I
>>> haven't tested.)
>>
>> Interesting corner case. This seems more like an overflow issue, and
>> we're currently not trying to protect the user from those.
>
> Well, yes, but usually with overflows one just gets the overflowed  
> value
> silently. With division overflow one crashes the application, which is
> IMO much worse even if it is a very rare corner case.
>
> I think we should raise OverflowError here under the same  
> circumstances
> as ZeroDivisionError.

Ah, I hadn't realized it segfaults (well, it does on my machine). OK,  
another check coming right up.

- Robert



From stefan_ml at behnel.de  Wed Apr 15 08:43:00 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 15 Apr 2009 08:43:00 +0200 (CEST)
Subject: [Cython] C int division by zero
In-Reply-To: <49E58243.6090704@student.matnat.uio.no>
References: <49E4B718.5020502@student.matnat.uio.no>
	<67E81388-686E-4B1E-A878-EBBD3B8A3662@math.washington.edu>
	<ac1e4c420904141304o20e87d71tfd0fc5d3ced9a681@mail.gmail.com>
	<9411A55A-C607-4303-A7A7-64F259CC8E21@math.washington.edu>
	<49E58243.6090704@student.matnat.uio.no>
Message-ID: <998260cf7038809d556b704c199bb10e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Dag Sverre Seljebotn wrote:
> usually with overflows one just gets the overflowed value
> silently. With division overflow one crashes the application, which is
> IMO much worse even if it is a very rare corner case.
>
> I think we should raise OverflowError here under the same circumstances
> as ZeroDivisionError.

+1, this matches the patch Lisandro provided.

BTW, isn't there a way to catch these signals in a way that would allow us
to raise an exception instead? Catching signals isn't the hard part, I
guess, but we'd have to extract the current state and code position from
the signal handler, so that we can jump back to the right label and
continue.

As long as they don't occur, signals are a lot cheaper than doing runtime
checks all over the place.

Stefan


From robertwb at math.washington.edu  Wed Apr 15 08:53:38 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 14 Apr 2009 23:53:38 -0700
Subject: [Cython] C int division by zero
In-Reply-To: <998260cf7038809d556b704c199bb10e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
References: <49E4B718.5020502@student.matnat.uio.no>
	<67E81388-686E-4B1E-A878-EBBD3B8A3662@math.washington.edu>
	<ac1e4c420904141304o20e87d71tfd0fc5d3ced9a681@mail.gmail.com>
	<9411A55A-C607-4303-A7A7-64F259CC8E21@math.washington.edu>
	<49E58243.6090704@student.matnat.uio.no>
	<998260cf7038809d556b704c199bb10e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <71603C95-E8C0-4933-BA96-CBA9005BFE97@math.washington.edu>

On Apr 14, 2009, at 11:43 PM, Stefan Behnel wrote:

> Dag Sverre Seljebotn wrote:
>> usually with overflows one just gets the overflowed value
>> silently. With division overflow one crashes the application,  
>> which is
>> IMO much worse even if it is a very rare corner case.
>>
>> I think we should raise OverflowError here under the same  
>> circumstances
>> as ZeroDivisionError.
>
> +1, this matches the patch Lisandro provided.
>
> BTW, isn't there a way to catch these signals in a way that would  
> allow us
> to raise an exception instead? Catching signals isn't the hard part, I
> guess, but we'd have to extract the current state and code position  
> from
> the signal handler, so that we can jump back to the right label and
> continue.
>
> As long as they don't occur, signals are a lot cheaper than doing  
> runtime
> checks all over the place.

Yes, this would be much better. I just am not familiar enough with  
signals to know how to trap them. Also, are there platform  
compatibility issues? (We may be able to use macros which use signals  
when available, and runtime tests if not.)

In any case, I'm going to push a patch to get the behavior right the  
"easy way" for now.

- Robert


From dagss at student.matnat.uio.no  Wed Apr 15 09:14:40 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Wed, 15 Apr 2009 09:14:40 +0200
Subject: [Cython] Poll: Buffer access exception under nogil
In-Reply-To: <49E4F655.8080001@behnel.de>
References: <49E48475.9090406@student.matnat.uio.no>	<c37ebdbdf4fe28e78fbe0036ccb602c3.squirrel@groupware.dvs.informatik.tu-darmstadt.de>	<49E4D3BC.8040809@student.matnat.uio.no>	<f125f5e0904141204s1626d871sf88a949388d82717@mail.gmail.com>	<49E4E04F.603@behnel.de>	<f125f5e0904141335u2711af22qdb11781ab3d63a28@mail.gmail.com>
	<49E4F655.8080001@behnel.de>
Message-ID: <49E58960.7090802@student.matnat.uio.no>

Stefan Behnel wrote:
> I think this is the one case where exception propagation will easily work
> in a nogil block. If we get the exception propagation semantics right here,
> we can leave this in. But it's not entirely clear (at least to me) if this
> currently behaves 'correctly'.
> 
> It *might* be possible to enable the same for "except *" somehow, but until
> then, Cython should raise an error at compile time rather than letting the
> code crash at runtime.

I think we can pull of except * without the GIL:

When releasing the GIL, we get a PyThreadState* pointing to our current 
thread state, in _save.

When nested functions reacquire the GIL in order to construct and raise 
an exception, they should get the same PyThreadState* object.

So back in our original function, we can simply directly inspect 
_save->exc_type rather than use PyErr_Occurred.

All in all I propose this:
  - It is possible to raise exceptions from a nogil context, by fetching 
the GIL temporarily to raise the exception, release the GIL, and return 
the normal exceptional way.
  - In order to propagate the exception, nogil code can look at 
_save->exc_type, which is very fast.
  - If returning from a nogil function with an exception set, the 
function will acquire the GIL, push exception stack information, and 
release the GIL.

This will mean that
a) Normally nogil code does not acquire the GIL.
b) If an exception is raised, the GIL is still acquired/reacquired, once 
per stack level.

-- 
Dag Sverre

From robertwb at math.washington.edu  Wed Apr 15 09:31:01 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Wed, 15 Apr 2009 00:31:01 -0700
Subject: [Cython] C int division by zero
In-Reply-To: <71603C95-E8C0-4933-BA96-CBA9005BFE97@math.washington.edu>
References: <49E4B718.5020502@student.matnat.uio.no>
	<67E81388-686E-4B1E-A878-EBBD3B8A3662@math.washington.edu>
	<ac1e4c420904141304o20e87d71tfd0fc5d3ced9a681@mail.gmail.com>
	<9411A55A-C607-4303-A7A7-64F259CC8E21@math.washington.edu>
	<49E58243.6090704@student.matnat.uio.no>
	<998260cf7038809d556b704c199bb10e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<71603C95-E8C0-4933-BA96-CBA9005BFE97@math.washington.edu>
Message-ID: <CB38D686-3918-45A1-91E5-CB3872FF7CCA@math.washington.edu>

On Apr 14, 2009, at 11:53 PM, Robert Bradshaw wrote:

> In any case, I'm going to push a patch to get the behavior right the
> "easy way" for now.

OK, http://hg.cython.org/cython-devel/rev/bbf007cb50f5 . Thanks Carl  
and Dag for pointing these bugs out.

- Robert


From stefan_ml at behnel.de  Wed Apr 15 10:49:45 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 15 Apr 2009 10:49:45 +0200 (CEST)
Subject: [Cython] Poll: Buffer access exception under nogil
In-Reply-To: <49E58960.7090802@student.matnat.uio.no>
References: <49E48475.9090406@student.matnat.uio.no>
	<c37ebdbdf4fe28e78fbe0036ccb602c3.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<49E4D3BC.8040809@student.matnat.uio.no>
	<f125f5e0904141204s1626d871sf88a949388d82717@mail.gmail.com>
	<49E4E04F.603@behnel.de>
	<f125f5e0904141335u2711af22qdb11781ab3d63a28@mail.gmail.com>
	<49E4F655.8080001@behnel.de> <49E58960.7090802@student.matnat.uio.no>
Message-ID: <30350dc4037cf40ac415a1b6685c91fe.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Dag Sverre Seljebotn wrote:
> So back in our original function, we can simply directly inspect
> _save->exc_type rather than use PyErr_Occurred.

Just for correctness, that would be _save->curexc_type.

Stefan


From stefan_ml at behnel.de  Wed Apr 15 10:53:52 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 15 Apr 2009 10:53:52 +0200 (CEST)
Subject: [Cython] C int division by zero
In-Reply-To: <CB38D686-3918-45A1-91E5-CB3872FF7CCA@math.washington.edu>
References: <49E4B718.5020502@student.matnat.uio.no>
	<67E81388-686E-4B1E-A878-EBBD3B8A3662@math.washington.edu>
	<ac1e4c420904141304o20e87d71tfd0fc5d3ced9a681@mail.gmail.com>
	<9411A55A-C607-4303-A7A7-64F259CC8E21@math.washington.edu>
	<49E58243.6090704@student.matnat.uio.no>
	<998260cf7038809d556b704c199bb10e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<71603C95-E8C0-4933-BA96-CBA9005BFE97@math.washington.edu>
	<CB38D686-3918-45A1-91E5-CB3872FF7CCA@math.washington.edu>
Message-ID: <2a94782f6aed546bd32424fcb11fd68e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Robert Bradshaw wrote:
> On Apr 14, 2009, at 11:53 PM, Robert Bradshaw wrote:
>
>> In any case, I'm going to push a patch to get the behavior right the
>> "easy way" for now.
>
> OK, http://hg.cython.org/cython-devel/rev/bbf007cb50f5

Just a quick reminder that the GIL checks work different in
cython-unstable, where we have a dedicated tree visitor to do this. So
this is ok for cython-devel, but should be moved to the appropriate method
in cython-unstable.

Stefan


From stefan_ml at behnel.de  Wed Apr 15 11:04:46 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 15 Apr 2009 11:04:46 +0200 (CEST)
Subject: [Cython] Poll: Buffer access exception under nogil
In-Reply-To: <49E58960.7090802@student.matnat.uio.no>
References: <49E48475.9090406@student.matnat.uio.no>
	<c37ebdbdf4fe28e78fbe0036ccb602c3.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<49E4D3BC.8040809@student.matnat.uio.no>
	<f125f5e0904141204s1626d871sf88a949388d82717@mail.gmail.com>
	<49E4E04F.603@behnel.de>
	<f125f5e0904141335u2711af22qdb11781ab3d63a28@mail.gmail.com>
	<49E4F655.8080001@behnel.de> <49E58960.7090802@student.matnat.uio.no>
Message-ID: <88afe4ab5f4012a00253e6add6830ac4.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Dag Sverre Seljebotn wrote:
> When releasing the GIL, we get a PyThreadState* pointing to our current
> thread state, in _save.

That is true for the nogil-block case, but not for nogil functions.

I don't see a way to retrieve the original thread state from a nogil
function without acquiring the GIL first (maybe I'm missing something). So
how would calling a function that raises an exception work from within a
nogil function?

Stefan


From robertwb at math.washington.edu  Wed Apr 15 11:16:55 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Wed, 15 Apr 2009 02:16:55 -0700
Subject: [Cython] Poll: Buffer access exception under nogil
In-Reply-To: <49E58960.7090802@student.matnat.uio.no>
References: <49E48475.9090406@student.matnat.uio.no>	<c37ebdbdf4fe28e78fbe0036ccb602c3.squirrel@groupware.dvs.informatik.tu-darmstadt.de>	<49E4D3BC.8040809@student.matnat.uio.no>	<f125f5e0904141204s1626d871sf88a949388d82717@mail.gmail.com>	<49E4E04F.603@behnel.de>	<f125f5e0904141335u2711af22qdb11781ab3d63a28@mail.gmail.com>
	<49E4F655.8080001@behnel.de>
	<49E58960.7090802@student.matnat.uio.no>
Message-ID: <566293ED-750B-4737-B57E-AAC3123128BF@math.washington.edu>

On Apr 15, 2009, at 12:14 AM, Dag Sverre Seljebotn wrote:

> Stefan Behnel wrote:
>> I think this is the one case where exception propagation will  
>> easily work
>> in a nogil block. If we get the exception propagation semantics  
>> right here,
>> we can leave this in. But it's not entirely clear (at least to me)  
>> if this
>> currently behaves 'correctly'.
>>
>> It *might* be possible to enable the same for "except *" somehow,  
>> but until
>> then, Cython should raise an error at compile time rather than  
>> letting the
>> code crash at runtime.
>
> I think we can pull of except * without the GIL:
>
> When releasing the GIL, we get a PyThreadState* pointing to our  
> current
> thread state, in _save.
>
> When nested functions reacquire the GIL in order to construct and  
> raise
> an exception, they should get the same PyThreadState* object.
>
> So back in our original function, we can simply directly inspect
> _save->exc_type rather than use PyErr_Occurred.
>
> All in all I propose this:
>   - It is possible to raise exceptions from a nogil context, by  
> fetching
> the GIL temporarily to raise the exception, release the GIL, and  
> return
> the normal exceptional way.
>   - In order to propagate the exception, nogil code can look at
> _save->exc_type, which is very fast.
>   - If returning from a nogil function with an exception set, the
> function will acquire the GIL, push exception stack information, and
> release the GIL.
>
> This will mean that
> a) Normally nogil code does not acquire the GIL.
> b) If an exception is raised, the GIL is still acquired/reacquired,  
> once
> per stack level.

In other words, you're changing the definition of nogil to be "gets  
the GIL if it needs" rather than "doesn't use (or need) the GIL."  
Granted, exceptions are exceptional, but I'm still a bit wary.  
Specifically, can implicitly acquiring the GIL for this cause block  
and/or lead to deadlock/race conditions?

- Robert



From robertwb at math.washington.edu  Wed Apr 15 11:31:42 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Wed, 15 Apr 2009 02:31:42 -0700
Subject: [Cython] Py_New still good way to create classes?
In-Reply-To: <4db580fd0904051226u64d9dfe5n73674e3d258ba677@mail.gmail.com>
References: <4db580fd0904051226u64d9dfe5n73674e3d258ba677@mail.gmail.com>
Message-ID: <1228653C-157E-48C6-A305-E9D2CFE27AB0@math.washington.edu>

On Apr 5, 2009, at 12:26 PM, Hoyt Koepke wrote:

> Hello,
>
> I'm using an extension class where I need to create a number of
> instances quickly, and I'm looking at the section "Can Cython create
> objects or apply operators to locally created objects as pure C code?"
> on http://wiki.cython.org/FAQ.  Is this information still correct?
>
> In particular, I'm wondering about it in reference to supporting
> cyclical garbage collection and creating object via PyObject_GC_New
> etc.  IIRC, cython automatically supports cyclical garbage collection
> on classes that have python members, and my extension class falls into
> this category.  Should I
>
> a) Use Py_NEW, as specified in the FAQ?

Yes, if one has an absolute need for speed. Cyclic garbage collection  
is supported via a flag on the type.

> b) Replace Py_NEW with Py_Object_GC_New, and do a?

Py_Object_GC_New will allocate the space, but will not call the  
__cinit__ functions, and in particular will not initialize the base  
fields, so don't use this unless you really know what you're doing.

> c) Ignore this and just go with standard creation and deletion,
> ignoring the fact that I'll be calling the constructors through
> python.

This is what I would do, at least until I determined object creation  
was a large bottleneck. PY_NEW as stated on the wiki calls the  
__cinit__ functions, but avoids calling the __init__ functions, so  
depending on your usecase it can be a savings. Also, it would be  
desirable and feasible to potentially call __init__ via a C call  
rather than a Python call (maybe, if the signature is simple enough,  
using the cpdef trick to avoid the pythonic argument packing/ 
unpacking overhead.)

- Robert


From dagss at student.matnat.uio.no  Wed Apr 15 12:14:32 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Wed, 15 Apr 2009 12:14:32 +0200
Subject: [Cython] Poll: Buffer access exception under nogil
In-Reply-To: <566293ED-750B-4737-B57E-AAC3123128BF@math.washington.edu>
References: <49E48475.9090406@student.matnat.uio.no>	<c37ebdbdf4fe28e78fbe0036ccb602c3.squirrel@groupware.dvs.informatik.tu-darmstadt.de>	<49E4D3BC.8040809@student.matnat.uio.no>	<f125f5e0904141204s1626d871sf88a949388d82717@mail.gmail.com>	<49E4E04F.603@behnel.de>	<f125f5e0904141335u2711af22qdb11781ab3d63a28@mail.gmail.com>	<49E4F655.8080001@behnel.de>	<49E58960.7090802@student.matnat.uio.no>
	<566293ED-750B-4737-B57E-AAC3123128BF@math.washington.edu>
Message-ID: <49E5B388.8000408@student.matnat.uio.no>

Robert Bradshaw wrote:
>> This will mean that
>> a) Normally nogil code does not acquire the GIL.
>> b) If an exception is raised, the GIL is still acquired/reacquired,  
>> once
>> per stack level.
> 
> In other words, you're changing the definition of nogil to be "gets  
> the GIL if it needs" rather than "doesn't use (or need) the GIL."  
> Granted, exceptions are exceptional, but I'm still a bit wary.  
> Specifically, can implicitly acquiring the GIL for this cause block  
> and/or lead to deadlock/race conditions?

Thanks, that was very helpful to my thoughts. I'll be clearer.

This is actually two orthogonal issues:

I) Exception propagation in nogil.

The following currently works and is fully supported:

cdef int raise_some_error() except -1 with gil:
     raise ValueError("test")

def f():
     cdef int a
     try:
         with nogil:
             a = 0
             if 1 + 1 == 2: # very exceptional case
                 raise_some_error()
             a = 1
     except ValueError:
         a = 3
     print a # prints 3

However "except *" is not supported, which is a limitation to this.

II) Whether division and buffer access is allowed to imply a hidden 
"with gil" section in their exceptional cases.

To answer your question: The only thing I can think of is if you start 
spawning new threads using C functions; then reacquiring the GIL will 
fail using those threads, and calling functions declared "with gil" is 
perhaps more explicit and can more easily be avoided. (But that's a 
rather expert case and there's always compiler directives.)

-- 
Dag Sverre

From dagss at student.matnat.uio.no  Wed Apr 15 13:37:53 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Wed, 15 Apr 2009 13:37:53 +0200
Subject: [Cython] Poll: Buffer access exception under nogil
In-Reply-To: <88afe4ab5f4012a00253e6add6830ac4.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
References: <49E48475.9090406@student.matnat.uio.no>	<c37ebdbdf4fe28e78fbe0036ccb602c3.squirrel@groupware.dvs.informatik.tu-darmstadt.de>	<49E4D3BC.8040809@student.matnat.uio.no>	<f125f5e0904141204s1626d871sf88a949388d82717@mail.gmail.com>	<49E4E04F.603@behnel.de>	<f125f5e0904141335u2711af22qdb11781ab3d63a28@mail.gmail.com>	<49E4F655.8080001@behnel.de>
	<49E58960.7090802@student.matnat.uio.no>
	<88afe4ab5f4012a00253e6add6830ac4.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <49E5C711.8040800@student.matnat.uio.no>

Stefan Behnel wrote:
> Dag Sverre Seljebotn wrote:
>> When releasing the GIL, we get a PyThreadState* pointing to our current
>> thread state, in _save.
> 
> That is true for the nogil-block case, but not for nogil functions.
> 
> I don't see a way to retrieve the original thread state from a nogil
> function without acquiring the GIL first (maybe I'm missing something). So
> how would calling a function that raises an exception work from within a
> nogil function?

No, you appear to be right, I didn't consider that :-(

However as there is already a way of propagating exceptions [1], using 
return values, it appears to be orthogonal to the question of whether 
buffer access or int division should imply an automatic "with gil" 
section raising an exception.

(We could try to get a patch into Python for PyErr_OccurredWithoutGIL, 
don't know if such a thing would be accepted -- it would be easy to 
write. And then only acquire a GIL in the current Python versions. After 
all, it would only happen for something which is currently not allowed.)

[1] From what I can see it works ok, except for a highly unlikely race 
condition on the __pyx_filename/lineno/... which can corrupt the 
exception stack description.

-- 
Dag Sverre

From dagss at student.matnat.uio.no  Wed Apr 15 16:24:33 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Wed, 15 Apr 2009 16:24:33 +0200
Subject: [Cython] Cython 0.11.1 release candidate
In-Reply-To: <27D772B7-E8B6-4A8A-9701-49D7C10A9DA0@math.washington.edu>
References: <27D772B7-E8B6-4A8A-9701-49D7C10A9DA0@math.washington.edu>
Message-ID: <49E5EE21.80505@student.matnat.uio.no>

Robert Bradshaw wrote:
> Unless something is horribly broken, I will release in the next day  
> or two.
> 
> http://cython.org/Cython-0.11.1.rc1.tar.gz

I know it is late (sorry!), but I just remembered some changes to 
directives behaviour which I'd like to get in; emitting a warning for 
change of behaviour in 0.12. I think they're rather safe, but I'll 
definitely take no for an answer.

Attaching a patch as I don't want to disturb the freeze (tell me and 
I'll push).

-- 
Dag Sverre


# HG changeset patch
# User Dag Sverre Seljebotn <dagss at student.matnat.uio.no>
# Date 1239804937 -7200
# Node ID c8d2f8143120ecc3efd1e20d675b26a55e5c8684
# Parent  d22204d89cfdbdfa05e2f191b7a1b33d2aaa6dc2
Add warning in situations where command line directive behaviour will 
change in 0.12

diff -r d22204d89cfd -r c8d2f8143120 Cython/Compiler/ParseTreeTransforms.py
--- a/Cython/Compiler/ParseTreeTransforms.py    Wed Apr 15 15:53:26 2009 
+0200
+++ b/Cython/Compiler/ParseTreeTransforms.py    Wed Apr 15 16:15:37 2009 
+0200
@@ -332,13 +332,20 @@ class InterpretCompilerDirectives(Cython

      def __init__(self, context, compilation_option_overrides):
          super(InterpretCompilerDirectives, self).__init__(context)
-        self.compilation_option_overrides = compilation_option_overrides
+        self.compilation_option_overrides = {}
+        for key, value in compilation_option_overrides.iteritems():
+            self.compilation_option_overrides[unicode(key)] = value
          self.cython_module_names = set()
          self.option_names = {}

      # Set up processing and handle the cython: comments.
      def visit_ModuleNode(self, node):
          options = copy.copy(Options.option_defaults)
+        for key, value in self.compilation_option_overrides.iteritems():
+            if key in node.option_comments and 
node.option_comments[key] != value:
+                warning(node.pos, "Compiler directive differs between 
environment and file header; this will change "
+                        "in Cython 0.12. See 
http://article.gmane.org/gmane.comp.python.cython.devel/5233", 2)
+                break
          options.update(node.option_comments)
          options.update(self.compilation_option_overrides)
          self.options = options
# HG changeset patch
# User Dag Sverre Seljebotn <dagss at student.matnat.uio.no>
# Date 1239803606 -7200
# Node ID d22204d89cfdbdfa05e2f191b7a1b33d2aaa6dc2
# Parent  bbf007cb50f57bcf0c010661e66b883191958aa3
Make -X cmdline switch apply to directives, as help text says

diff -r bbf007cb50f5 -r d22204d89cfd Cython/Compiler/CmdLine.py
--- a/Cython/Compiler/CmdLine.py        Wed Apr 15 00:27:57 2009 -0700
+++ b/Cython/Compiler/CmdLine.py        Wed Apr 15 15:53:26 2009 +0200
@@ -79,7 +79,7 @@ def parse_command_line(args):
                  options.use_listing_file = 1
              elif option in ("-C", "--compile"):
                  options.c_only = 0
-            elif option in ("-X", "--link"):
+            elif option in ("--link"):
                  options.c_only = 0
                  options.obj_only = 0
              elif option in ("-+", "--cplus"):




-------------- next part --------------
A non-text attachment was scrubbed...
Name: directives.diff
Type: text/x-diff
Size: 2598 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20090415/4918b6fe/attachment-0001.bin 

From bpederse at gmail.com  Wed Apr 15 19:39:01 2009
From: bpederse at gmail.com (Brent Pedersen)
Date: Wed, 15 Apr 2009 10:39:01 -0700
Subject: [Cython] Py_New still good way to create classes?
In-Reply-To: <1228653C-157E-48C6-A305-E9D2CFE27AB0@math.washington.edu>
References: <4db580fd0904051226u64d9dfe5n73674e3d258ba677@mail.gmail.com>
	<1228653C-157E-48C6-A305-E9D2CFE27AB0@math.washington.edu>
Message-ID: <e183a99d0904151039r7e33bf7aj97044f4fe4285844@mail.gmail.com>

On Wed, Apr 15, 2009 at 2:31 AM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Apr 5, 2009, at 12:26 PM, Hoyt Koepke wrote:
>
>> Hello,
>>
>> I'm using an extension class where I need to create a number of
>> instances quickly, and I'm looking at the section "Can Cython create
>> objects or apply operators to locally created objects as pure C code?"
>> on http://wiki.cython.org/FAQ. ?Is this information still correct?
>>
>> In particular, I'm wondering about it in reference to supporting
>> cyclical garbage collection and creating object via PyObject_GC_New
>> etc. ?IIRC, cython automatically supports cyclical garbage collection
>> on classes that have python members, and my extension class falls into
>> this category. ?Should I
>>
>> a) Use Py_NEW, as specified in the FAQ?
>
> Yes, if one has an absolute need for speed. Cyclic garbage collection
> is supported via a flag on the type.
>
>> b) Replace Py_NEW with Py_Object_GC_New, and do a?
>
> Py_Object_GC_New will allocate the space, but will not call the
> __cinit__ functions, and in particular will not initialize the base
> fields, so don't use this unless you really know what you're doing.
>
>> c) Ignore this and just go with standard creation and deletion,
>> ignoring the fact that I'll be calling the constructors through
>> python.
>
> This is what I would do, at least until I determined object creation
> was a large bottleneck. PY_NEW as stated on the wiki calls the
> __cinit__ functions, but avoids calling the __init__ functions, so
> depending on your usecase it can be a savings. Also, it would be
> desirable and feasible to potentially call __init__ via a C call
> rather than a Python call (maybe, if the signature is simple enough,
> using the cpdef trick to avoid the pythonic argument packing/
> unpacking overhead.)
>
> - Robert
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>

hi, i have a use-case where i'm creating lots of objects as well. so i
wrote up a quick test to see the speed of various methods:
http://gist.github.com/95916

assuming i haven't done anything stupid, results in seconds for that case are:

PY_NEW on Cython class 0.652909994125
__init__ on Python class 1.51110291481
batch PY_NEW 0.434900999069
__init__ on Cython class 0.673034906387
batch __init__ on Cython class 0.532235145569

-b

From stefan_ml at behnel.de  Wed Apr 15 20:25:44 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 15 Apr 2009 20:25:44 +0200
Subject: [Cython] Py_New still good way to create classes?
In-Reply-To: <e183a99d0904151039r7e33bf7aj97044f4fe4285844@mail.gmail.com>
References: <4db580fd0904051226u64d9dfe5n73674e3d258ba677@mail.gmail.com>	<1228653C-157E-48C6-A305-E9D2CFE27AB0@math.washington.edu>
	<e183a99d0904151039r7e33bf7aj97044f4fe4285844@mail.gmail.com>
Message-ID: <49E626A8.4000608@behnel.de>

Hi,

thanks for sharing that.

Brent Pedersen wrote:
> assuming i haven't done anything stupid

No, nothing stupid, but something that can reduce the comparability of the
timings. You are creating a 1000000 item list on each benchmark, using a
call to range() in some cases and a list comprehension in others.

It's usually better to move initialisations out of the timings, e.g. by
creating a large range() object once and re-using it. That reduces the
impact of unrelated operations on the absolute numbers.

Stefan


From robertwb at math.washington.edu  Wed Apr 15 20:25:23 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Wed, 15 Apr 2009 11:25:23 -0700
Subject: [Cython] Cython 0.11.1 release candidate
In-Reply-To: <49E5EE21.80505@student.matnat.uio.no>
References: <27D772B7-E8B6-4A8A-9701-49D7C10A9DA0@math.washington.edu>
	<49E5EE21.80505@student.matnat.uio.no>
Message-ID: <27887959-53F9-491B-8991-7EC5535CD251@math.washington.edu>

On Apr 15, 2009, at 7:24 AM, Dag Sverre Seljebotn wrote:

> Robert Bradshaw wrote:
>> Unless something is horribly broken, I will release in the next  
>> day  or two.
>> http://cython.org/Cython-0.11.1.rc1.tar.gz
>
> I know it is late (sorry!), but I just remembered some changes to  
> directives behaviour which I'd like to get in; emitting a warning  
> for change of behaviour in 0.12. I think they're rather safe, but  
> I'll definitely take no for an answer.

The warning looks like it'd be fine to slip in at the last moment,  
but I'm not as sure about removing the -X option--perhaps a warning  
about that would be in order though.

> Attaching a patch as I don't want to disturb the freeze (tell me  
> and I'll push).
>
> -- 
> Dag Sverre
>
>
> # HG changeset patch
> # User Dag Sverre Seljebotn <dagss at student.matnat.uio.no>
> # Date 1239804937 -7200
> # Node ID c8d2f8143120ecc3efd1e20d675b26a55e5c8684
> # Parent  d22204d89cfdbdfa05e2f191b7a1b33d2aaa6dc2
> Add warning in situations where command line directive behaviour  
> will change in 0.12
>
> diff -r d22204d89cfd -r c8d2f8143120 Cython/Compiler/ 
> ParseTreeTransforms.py
> --- a/Cython/Compiler/ParseTreeTransforms.py    Wed Apr 15 15:53:26  
> 2009 +0200
> +++ b/Cython/Compiler/ParseTreeTransforms.py    Wed Apr 15 16:15:37  
> 2009 +0200
> @@ -332,13 +332,20 @@ class InterpretCompilerDirectives(Cython
>
>      def __init__(self, context, compilation_option_overrides):
>          super(InterpretCompilerDirectives, self).__init__(context)
> -        self.compilation_option_overrides =  
> compilation_option_overrides
> +        self.compilation_option_overrides = {}
> +        for key, value in compilation_option_overrides.iteritems():
> +            self.compilation_option_overrides[unicode(key)] = value
>          self.cython_module_names = set()
>          self.option_names = {}
>
>      # Set up processing and handle the cython: comments.
>      def visit_ModuleNode(self, node):
>          options = copy.copy(Options.option_defaults)
> +        for key, value in  
> self.compilation_option_overrides.iteritems():
> +            if key in node.option_comments and node.option_comments 
> [key] != value:
> +                warning(node.pos, "Compiler directive differs  
> between environment and file header; this will change "
> +                        "in Cython 0.12. See http:// 
> article.gmane.org/gmane.comp.python.cython.devel/5233", 2)
> +                break
>          options.update(node.option_comments)
>          options.update(self.compilation_option_overrides)
>          self.options = options
> # HG changeset patch
> # User Dag Sverre Seljebotn <dagss at student.matnat.uio.no>
> # Date 1239803606 -7200
> # Node ID d22204d89cfdbdfa05e2f191b7a1b33d2aaa6dc2
> # Parent  bbf007cb50f57bcf0c010661e66b883191958aa3
> Make -X cmdline switch apply to directives, as help text says
>
> diff -r bbf007cb50f5 -r d22204d89cfd Cython/Compiler/CmdLine.py
> --- a/Cython/Compiler/CmdLine.py        Wed Apr 15 00:27:57 2009 -0700
> +++ b/Cython/Compiler/CmdLine.py        Wed Apr 15 15:53:26 2009 +0200
> @@ -79,7 +79,7 @@ def parse_command_line(args):
>                  options.use_listing_file = 1
>              elif option in ("-C", "--compile"):
>                  options.c_only = 0
> -            elif option in ("-X", "--link"):
> +            elif option in ("--link"):
>                  options.c_only = 0
>                  options.obj_only = 0
>              elif option in ("-+", "--cplus"):
>
>
>
>
> # HG changeset patch
> # User Dag Sverre Seljebotn <dagss at student.matnat.uio.no>
> # Date 1239804937 -7200
> # Node ID c8d2f8143120ecc3efd1e20d675b26a55e5c8684
> # Parent  d22204d89cfdbdfa05e2f191b7a1b33d2aaa6dc2
> Add warning in situations where command line directive behaviour  
> will change in 0.12
>
> diff -r d22204d89cfd -r c8d2f8143120 Cython/Compiler/ 
> ParseTreeTransforms.py
> --- a/Cython/Compiler/ParseTreeTransforms.py	Wed Apr 15 15:53:26  
> 2009 +0200
> +++ b/Cython/Compiler/ParseTreeTransforms.py	Wed Apr 15 16:15:37  
> 2009 +0200
> @@ -332,13 +332,20 @@ class InterpretCompilerDirectives(Cython
>
>      def __init__(self, context, compilation_option_overrides):
>          super(InterpretCompilerDirectives, self).__init__(context)
> -        self.compilation_option_overrides =  
> compilation_option_overrides
> +        self.compilation_option_overrides = {}
> +        for key, value in compilation_option_overrides.iteritems():
> +            self.compilation_option_overrides[unicode(key)] = value
>          self.cython_module_names = set()
>          self.option_names = {}
>
>      # Set up processing and handle the cython: comments.
>      def visit_ModuleNode(self, node):
>          options = copy.copy(Options.option_defaults)
> +        for key, value in  
> self.compilation_option_overrides.iteritems():
> +            if key in node.option_comments and node.option_comments 
> [key] != value:
> +                warning(node.pos, "Compiler directive differs  
> between environment and file header; this will change "
> +                        "in Cython 0.12. See http:// 
> article.gmane.org/gmane.comp.python.cython.devel/5233", 2)
> +                break
>          options.update(node.option_comments)
>          options.update(self.compilation_option_overrides)
>          self.options = options
> # HG changeset patch
> # User Dag Sverre Seljebotn <dagss at student.matnat.uio.no>
> # Date 1239803606 -7200
> # Node ID d22204d89cfdbdfa05e2f191b7a1b33d2aaa6dc2
> # Parent  bbf007cb50f57bcf0c010661e66b883191958aa3
> Make -X cmdline switch apply to directives, as help text says
>
> diff -r bbf007cb50f5 -r d22204d89cfd Cython/Compiler/CmdLine.py
> --- a/Cython/Compiler/CmdLine.py	Wed Apr 15 00:27:57 2009 -0700
> +++ b/Cython/Compiler/CmdLine.py	Wed Apr 15 15:53:26 2009 +0200
> @@ -79,7 +79,7 @@ def parse_command_line(args):
>                  options.use_listing_file = 1
>              elif option in ("-C", "--compile"):
>                  options.c_only = 0
> -            elif option in ("-X", "--link"):
> +            elif option in ("--link"):
>                  options.c_only = 0
>                  options.obj_only = 0
>              elif option in ("-+", "--cplus"):
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From dagss at student.matnat.uio.no  Wed Apr 15 22:14:00 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Wed, 15 Apr 2009 22:14:00 +0200
Subject: [Cython] Cython 0.11.1 release candidate
In-Reply-To: <27887959-53F9-491B-8991-7EC5535CD251@math.washington.edu>
References: <27D772B7-E8B6-4A8A-9701-49D7C10A9DA0@math.washington.edu>	<49E5EE21.80505@student.matnat.uio.no>
	<27887959-53F9-491B-8991-7EC5535CD251@math.washington.edu>
Message-ID: <49E64008.4000203@student.matnat.uio.no>

Robert Bradshaw wrote:
> On Apr 15, 2009, at 7:24 AM, Dag Sverre Seljebotn wrote:
> 
>> Robert Bradshaw wrote:
>>> Unless something is horribly broken, I will release in the next  
>>> day  or two.
>>> http://cython.org/Cython-0.11.1.rc1.tar.gz
>> I know it is late (sorry!), but I just remembered some changes to  
>> directives behaviour which I'd like to get in; emitting a warning  
>> for change of behaviour in 0.12. I think they're rather safe, but  
>> I'll definitely take no for an answer.
> 
> The warning looks like it'd be fine to slip in at the last moment,  
> but I'm not as sure about removing the -X option--perhaps a warning  
> about that would be in order though.

OK, I've pushed changes to this effect. This also fixes the help text 
which is currently WRONG in this area.

In the list of patches I should have done earlier, there's also

http://trac.cython.org/cython_trac/attachment/ticket/210/buffernogil.patch

Again I'd be really surprised to have this break anything, and it is 
something there's been expressed a need for in the NumPy community.

I know how irritating this must be though :-) If you say no I won't 
complain.

-- 
Dag Sverre
-------------- next part --------------
A non-text attachment was scrubbed...
Name: deprecateX.patch
Type: text/x-diff
Size: 1665 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20090415/8586ce36/attachment.bin 

From robertwb at math.washington.edu  Wed Apr 15 22:14:15 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Wed, 15 Apr 2009 13:14:15 -0700
Subject: [Cython] Cython 0.11.1 release candidate
In-Reply-To: <49E64008.4000203@student.matnat.uio.no>
References: <27D772B7-E8B6-4A8A-9701-49D7C10A9DA0@math.washington.edu>	<49E5EE21.80505@student.matnat.uio.no>
	<27887959-53F9-491B-8991-7EC5535CD251@math.washington.edu>
	<49E64008.4000203@student.matnat.uio.no>
Message-ID: <E7EF79BB-55A9-4671-A298-ED280E969F04@math.washington.edu>

On Apr 15, 2009, at 1:14 PM, Dag Sverre Seljebotn wrote:

> Robert Bradshaw wrote:
>> On Apr 15, 2009, at 7:24 AM, Dag Sverre Seljebotn wrote:
>>> Robert Bradshaw wrote:
>>>> Unless something is horribly broken, I will release in the next   
>>>> day  or two.
>>>> http://cython.org/Cython-0.11.1.rc1.tar.gz
>>> I know it is late (sorry!), but I just remembered some changes  
>>> to  directives behaviour which I'd like to get in; emitting a  
>>> warning  for change of behaviour in 0.12. I think they're rather  
>>> safe, but  I'll definitely take no for an answer.
>> The warning looks like it'd be fine to slip in at the last  
>> moment,  but I'm not as sure about removing the -X option--perhaps  
>> a warning  about that would be in order though.
>
> OK, I've pushed changes to this effect. This also fixes the help  
> text which is currently WRONG in this area.

Thanks.

>
> In the list of patches I should have done earlier, there's also
>
> http://trac.cython.org/cython_trac/attachment/ticket/210/ 
> buffernogil.patch
>
> Again I'd be really surprised to have this break anything, and it  
> is something there's been expressed a need for in the NumPy community.
>
> I know how irritating this must be though :-) If you say no I won't  
> complain.

No. First thing for 0.11.2 (or 0.12) though. Unless a major bug is  
discovered, 0.11.1 will be the current tip of cython-devel http:// 
hg.cython.org/cython-devel/shortlog/53e4979c9b84 , released tonight.

- Robert


From bpederse at gmail.com  Thu Apr 16 04:31:29 2009
From: bpederse at gmail.com (Brent Pedersen)
Date: Wed, 15 Apr 2009 19:31:29 -0700
Subject: [Cython] Py_New still good way to create classes?
In-Reply-To: <49E626A8.4000608@behnel.de>
References: <4db580fd0904051226u64d9dfe5n73674e3d258ba677@mail.gmail.com>
	<1228653C-157E-48C6-A305-E9D2CFE27AB0@math.washington.edu>
	<e183a99d0904151039r7e33bf7aj97044f4fe4285844@mail.gmail.com>
	<49E626A8.4000608@behnel.de>
Message-ID: <e183a99d0904151931l7b3e5997id566dbb511b39cda@mail.gmail.com>

On Wed, Apr 15, 2009 at 11:25 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Hi,
>
> thanks for sharing that.
>
> Brent Pedersen wrote:
>> assuming i haven't done anything stupid
>
> No, nothing stupid, but something that can reduce the comparability of the
> timings. You are creating a 1000000 item list on each benchmark, using a
> call to range() in some cases and a list comprehension in others.
>
> It's usually better to move initialisations out of the timings, e.g. by
> creating a large range() object once and re-using it. That reduces the
> impact of unrelated operations on the absolute numbers.
>

ah, i see, updated that. fixing that makes the python constructor look
even slower.
now it assumes that creating a list comprehension without assgning to
a variable is
the same as calling a function that returns an array--also without assigning.

here are the new timings:

PY_NEW on Cython class: 1.137
__init__ on Python class: 28.468
__init__ on Python class with slots: 9.936
batch PY_NEW total: 0.821 , interval only: 0.363
batch __init__ on Cython class total 0.975 , interval_only: 0.524
__init__ on Cython class 1.154

so for this case using PY_NEW macro actual doesnt improve speed that
much over a cdef'ed class.
especially if using a "batch" method is applicable (as it is for my use-case).

i didnt realize slots would affect on object creation time that much
-- in this case it's 3x faster with slots!
(and then another 6-10x with a cdef'ed class)

here's my updated gist:
http://gist.github.com/95916


> Stefan
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>

From greg.ewing at canterbury.ac.nz  Thu Apr 16 02:56:55 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 16 Apr 2009 12:56:55 +1200
Subject: [Cython] Poll: Buffer access exception under nogil
In-Reply-To: <566293ED-750B-4737-B57E-AAC3123128BF@math.washington.edu>
References: <49E48475.9090406@student.matnat.uio.no>
	<c37ebdbdf4fe28e78fbe0036ccb602c3.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<49E4D3BC.8040809@student.matnat.uio.no>
	<f125f5e0904141204s1626d871sf88a949388d82717@mail.gmail.com>
	<49E4E04F.603@behnel.de>
	<f125f5e0904141335u2711af22qdb11781ab3d63a28@mail.gmail.com>
	<49E4F655.8080001@behnel.de> <49E58960.7090802@student.matnat.uio.no>
	<566293ED-750B-4737-B57E-AAC3123128BF@math.washington.edu>
Message-ID: <49E68257.7020205@canterbury.ac.nz>

Robert Bradshaw wrote:

> In other words, you're changing the definition of nogil to be "gets  
> the GIL if it needs" rather than "doesn't use (or need) the GIL."  

But it has never meant that, at least as far as
Pyrex is concerned. It's always meant "this function
can be called without the gil" and no more. So
there's hing wrong with this in principle if you
can make it work.

-- 
Greg

From stefan_ml at behnel.de  Thu Apr 16 07:05:13 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 16 Apr 2009 07:05:13 +0200
Subject: [Cython] Py_New still good way to create classes?
In-Reply-To: <e183a99d0904151931l7b3e5997id566dbb511b39cda@mail.gmail.com>
References: <4db580fd0904051226u64d9dfe5n73674e3d258ba677@mail.gmail.com>	<1228653C-157E-48C6-A305-E9D2CFE27AB0@math.washington.edu>	<e183a99d0904151039r7e33bf7aj97044f4fe4285844@mail.gmail.com>	<49E626A8.4000608@behnel.de>
	<e183a99d0904151931l7b3e5997id566dbb511b39cda@mail.gmail.com>
Message-ID: <49E6BC89.7020400@behnel.de>

Hi,

now, those are interesting timings. Which versions of Cython and Python did
you use?

Brent Pedersen wrote:
> here are the new timings:
> 
> PY_NEW on Cython class: 1.137
> __init__ on Cython class 1.154

You're right, that's almost negligible. However, the __init__ benchmark
involves the overhead of a Python call to create_interval, which uses the
same Python calling convention as the constructor. So it's more or less
doing the same thing in both cases, just additionally calling __init__
through a C struct member dereference in the second case (but without
repacking the arguments).

Also note that it can actually be a feature that PY_NEW doesn't call
__init__(). I happily misuse that in lxml.etree to assign different
behaviour to a user class instantiation and an internal proxy creation.


> batch PY_NEW total: 0.821 , interval only: 0.363
> batch __init__ on Cython class total 0.975 , interval_only: 0.524

As expected, the difference is a lot larger here. Ignoring the setup
overhead, that's some 30% faster, as PY_NEW does the field initialisation
inside a plain C call, whereas the call to __init__ requires the Python
calling convention (i.e. tuple packing and unpacking).


> __init__ on Python class: 28.468
> __init__ on Python class with slots: 9.936

Ok, that's a completely different performance league. The first benchmark
requires setting up a dictionary for the instance and putting the
attributes there, whereas the second happily assigns the attributes to
predefined slots, without the overhead of a dict.

To sum it up, using PY_NEW from the beginning looks like a premature
optimisation to me. If you notice that class creation becomes a bottleneck
later on, *and* your classes are instantiated internally in the relevant
cases, *and* the setup involves working with plain C field values (such as
the 'start' and 'end' ints in your case), switching to PY_NEW can give you
about 30% in plain class instantiation time (YMWV, don't forget to
benchmark your own code).

Thanks for sharing,

Stefan

From robertwb at math.washington.edu  Thu Apr 16 10:36:42 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 16 Apr 2009 01:36:42 -0700
Subject: [Cython] Cython 0.11.1 released
Message-ID: <36C7FFE0-FBE8-4FC5-ADDA-AFDF8ACDA034@math.washington.edu>

Cython 0.11.1 is released and can be downloaded from http:// 
cython.org/. The major changes in this release are better conversion  
between c integer and Python types, better error reporting, and more  
correct (and consistent) for-from and for-in loops. More python  
constructions are supported (from package import module and integer  
powering) and several bugs were fixed. CEP 516 (http:// 
wiki.cython.org/enhancements/division) was also implemented,  
providing Python semantics for division, but is off by default until  
0.12. See

http://trac.cython.org/cython_trac/query? 
group=component&milestone=0.11.1

for a list of all tickets closed.

- Robert


From ndbecker2 at gmail.com  Thu Apr 16 13:49:48 2009
From: ndbecker2 at gmail.com (Neal Becker)
Date: Thu, 16 Apr 2009 07:49:48 -0400
Subject: [Cython] Cython 0.11.1 released
References: <36C7FFE0-FBE8-4FC5-ADDA-AFDF8ACDA034@math.washington.edu>
Message-ID: <gs760s$43o$2@ger.gmane.org>

Where can I find release notes?  (Need to point to it for the fedora update)



From stefan_ml at behnel.de  Thu Apr 16 14:46:09 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 16 Apr 2009 14:46:09 +0200 (CEST)
Subject: [Cython] Cython 0.11.1 released
In-Reply-To: <gs760s$43o$2@ger.gmane.org>
References: <36C7FFE0-FBE8-4FC5-ADDA-AFDF8ACDA034@math.washington.edu>
	<gs760s$43o$2@ger.gmane.org>
Message-ID: <58cffd5f7d955299f62535b329d9ad77.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Neal Becker wrote:
> Where can I find release notes?  (Need to point to it for the fedora
> update)

You can just point to the trac query page that Robert sent a link for.

Here is another query that works for me:

http://trac.cython.org/cython_trac/query?status=closed&order=type&col=id&col=summary&col=type&col=owner&milestone=0.11.1

You can also link to the corresponding RSS feed if you prefer.

Stefan


From bpederse at gmail.com  Thu Apr 16 17:04:19 2009
From: bpederse at gmail.com (Brent Pedersen)
Date: Thu, 16 Apr 2009 08:04:19 -0700
Subject: [Cython] Py_New still good way to create classes?
In-Reply-To: <49E6BC89.7020400@behnel.de>
References: <4db580fd0904051226u64d9dfe5n73674e3d258ba677@mail.gmail.com>
	<1228653C-157E-48C6-A305-E9D2CFE27AB0@math.washington.edu>
	<e183a99d0904151039r7e33bf7aj97044f4fe4285844@mail.gmail.com>
	<49E626A8.4000608@behnel.de>
	<e183a99d0904151931l7b3e5997id566dbb511b39cda@mail.gmail.com>
	<49E6BC89.7020400@behnel.de>
Message-ID: <e183a99d0904160804i75517678p92ea2ceb1750e47f@mail.gmail.com>

On Wed, Apr 15, 2009 at 10:05 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Hi,
>
> now, those are interesting timings. Which versions of Cython and Python did
> you use?

hi,
i am using python 2.5 with cython .10 and .11 -- which both give same ballpark.
thanks for the explanation.
-brent

>
> Brent Pedersen wrote:
>> here are the new timings:
>>
>> PY_NEW on Cython class: 1.137
>> __init__ on Cython class 1.154
>
> You're right, that's almost negligible. However, the __init__ benchmark
> involves the overhead of a Python call to create_interval, which uses the
> same Python calling convention as the constructor. So it's more or less
> doing the same thing in both cases, just additionally calling __init__
> through a C struct member dereference in the second case (but without
> repacking the arguments).
>
> Also note that it can actually be a feature that PY_NEW doesn't call
> __init__(). I happily misuse that in lxml.etree to assign different
> behaviour to a user class instantiation and an internal proxy creation.
>
>
>> batch PY_NEW total: 0.821 , interval only: 0.363
>> batch __init__ on Cython class total 0.975 , interval_only: 0.524
>
> As expected, the difference is a lot larger here. Ignoring the setup
> overhead, that's some 30% faster, as PY_NEW does the field initialisation
> inside a plain C call, whereas the call to __init__ requires the Python
> calling convention (i.e. tuple packing and unpacking).
>
>
>> __init__ on Python class: 28.468
>> __init__ on Python class with slots: 9.936
>
> Ok, that's a completely different performance league. The first benchmark
> requires setting up a dictionary for the instance and putting the
> attributes there, whereas the second happily assigns the attributes to
> predefined slots, without the overhead of a dict.
>
> To sum it up, using PY_NEW from the beginning looks like a premature
> optimisation to me. If you notice that class creation becomes a bottleneck
> later on, *and* your classes are instantiated internally in the relevant
> cases, *and* the setup involves working with plain C field values (such as
> the 'start' and 'end' ints in your case), switching to PY_NEW can give you
> about 30% in plain class instantiation time (YMWV, don't forget to
> benchmark your own code).
>
> Thanks for sharing,
>
> Stefan
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>

From dalcinl at gmail.com  Thu Apr 16 17:27:13 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 16 Apr 2009 12:27:13 -0300
Subject: [Cython] Why the tarball release includes '.hg' directory?
Message-ID: <e7ba66e40904160827t5ebcac9ated6ab21916294fb5@mail.gmail.com>

Do we really need this?

$ tar -zxf Cython-0.11.1.tar.gz
$ du -sh Cython-0.11.1
11M	Cython-0.11.1

$ du -sh Cython-0.11.1/.hg
6.2M	Cython-0.11.1/.hg

$ rm -rf Cython-0.11.1/.hg
$ du -sh Cython-0.11.1
4.5M	Cython-0.11.1

-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dalcinl at gmail.com  Thu Apr 16 17:38:21 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 16 Apr 2009 12:38:21 -0300
Subject: [Cython] moving some tests to the same place
Message-ID: <e7ba66e40904160838r30fb85a6j653bd7c63071bf9@mail.gmail.com>

See this...

$ find Cython -name 'Test*'
Cython/TestUtils.py
Cython/Compiler/Tests
Cython/Compiler/Tests/TestParseTreeTransforms.py
Cython/Compiler/Tests/TestTreeFragment.py
Cython/Compiler/Tests/TestBuffer.py
Cython/Compiler/Tests/TestDecorators.py
Cython/Tests
Cython/Tests/TestCodeWriter.py

What do you think about moving ALL that inside "Cython/Tests"...

BTW, Why setup.py lists  'Cython.Tests' and 'Cython.Compiler.Tests' as
packages? Do a final user-targeted installation really need these?


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From hoytak at cs.ubc.ca  Thu Apr 16 19:24:34 2009
From: hoytak at cs.ubc.ca (Hoyt Koepke)
Date: Thu, 16 Apr 2009 10:24:34 -0700
Subject: [Cython] Py_New still good way to create classes?
In-Reply-To: <4db580fd0904161022w5d1d4fd1g72377f97ef806df4@mail.gmail.com>
References: <4db580fd0904051226u64d9dfe5n73674e3d258ba677@mail.gmail.com> 
	<1228653C-157E-48C6-A305-E9D2CFE27AB0@math.washington.edu> 
	<e183a99d0904151039r7e33bf7aj97044f4fe4285844@mail.gmail.com> 
	<49E626A8.4000608@behnel.de>
	<e183a99d0904151931l7b3e5997id566dbb511b39cda@mail.gmail.com> 
	<49E6BC89.7020400@behnel.de>
	<e183a99d0904160804i75517678p92ea2ceb1750e47f@mail.gmail.com> 
	<4db580fd0904161022w5d1d4fd1g72377f97ef806df4@mail.gmail.com>
Message-ID: <4db580fd0904161024l30136b0cxba4457a3919212e2@mail.gmail.com>

Thanks everyone, this discussion is quite informative.  My classes all
fall in the potential-30% class, and I've already written it in, so I
think I'll keep things in.

-- Hoyt

++++++++++++++++++++++++++++++++++++++++++++++++
+ Hoyt Koepke
+ University of Washington Department of Statistics
+ http://www.stat.washington.edu/~hoytak/
+ hoytak at gmail.com
++++++++++++++++++++++++++++++++++++++++++

From dagss at student.matnat.uio.no  Thu Apr 16 20:32:09 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Thu, 16 Apr 2009 20:32:09 +0200
Subject: [Cython] moving some tests to the same place
In-Reply-To: <e7ba66e40904160838r30fb85a6j653bd7c63071bf9@mail.gmail.com>
References: <e7ba66e40904160838r30fb85a6j653bd7c63071bf9@mail.gmail.com>
Message-ID: <49E779A9.5070105@student.matnat.uio.no>

Lisandro Dalcin wrote:
> See this...
> 
> $ find Cython -name 'Test*'
> Cython/TestUtils.py

This is a collection of utilities for writing tests, and is so not in 
the same category as the rest...

> Cython/Compiler/Tests
> Cython/Compiler/Tests/TestParseTreeTransforms.py
> Cython/Compiler/Tests/TestTreeFragment.py
> Cython/Compiler/Tests/TestBuffer.py
> Cython/Compiler/Tests/TestDecorators.py
> Cython/Tests
> Cython/Tests/TestCodeWriter.py
> 
> What do you think about moving ALL that inside "Cython/Tests"...

Why?

These are unit tests (and NOT integration tests, like tests/*). I think 
it is very uncommon to remove tests that far from the package they are 
testing. The idea is that the Cython.Compiler package has its unit tests 
in Cython.Compiler.Tests, and the Cython package has its unit tests in 
Cython.Tests.

(The reason it is very uncommon to do what you suggest is that they test 
1:1 with the file they refer to. What do you do with your scheme if both 
Cython/Foo.py and Cython/Compiler/Foo.py exists?)

If anything, I think all the test files should be moved alongside the 
other files; but since unit tests are so little in use that would be a 
bit intrusive..

Another common convention is to put the unit tests in the same script 
files as they are testing (perhaps as doctests), but that is


> BTW, Why setup.py lists  'Cython.Tests' and 'Cython.Compiler.Tests' as
> packages? Do a final user-targeted installation really need these?

I don't mind either way, but it's rather common for a Python package to 
package its test (e.g. with NumPy there's a method you can call to run 
the test suite). But in that case, tests/* would be much more important 
anyway, so removing these seems more consistent.

-- 
Dag Sverre

From dagss at student.matnat.uio.no  Thu Apr 16 20:38:20 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Thu, 16 Apr 2009 20:38:20 +0200
Subject: [Cython] Why the tarball release includes '.hg' directory?
In-Reply-To: <e7ba66e40904160827t5ebcac9ated6ab21916294fb5@mail.gmail.com>
References: <e7ba66e40904160827t5ebcac9ated6ab21916294fb5@mail.gmail.com>
Message-ID: <49E77B1C.2090009@student.matnat.uio.no>

Lisandro Dalcin wrote:
> Do we really need this?
> 
> $ tar -zxf Cython-0.11.1.tar.gz
> $ du -sh Cython-0.11.1
> 11M	Cython-0.11.1
> 
> $ du -sh Cython-0.11.1/.hg
> 6.2M	Cython-0.11.1/.hg
> 
> $ rm -rf Cython-0.11.1/.hg
> $ du -sh Cython-0.11.1
> 4.5M	Cython-0.11.1
> 

Well, 11M \approx 4.5M \approx 0M :-)

An argument in favor is that end-users can very easily contribute a 
bugfix. If users suddenly encounter a bug they can fix then the fix they 
do is very likely to happen in the downloaded release directory, and 
having done that then it's a lot less effort to submit a patch if it's a 
working repository.

Especially with Cython, whose end-users are all developers, this is a 
nice idea.

I don't really mind though. But I have a feeling that the time of making 
a reliable fix for doing releases without .hg, or even writing this 
email, far surpass the download time of those few extra MBs...

-- 
Dag Sverre

From robertwb at math.washington.edu  Thu Apr 16 20:51:49 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 16 Apr 2009 11:51:49 -0700
Subject: [Cython] Why the tarball release includes '.hg' directory?
In-Reply-To: <49E77B1C.2090009@student.matnat.uio.no>
References: <e7ba66e40904160827t5ebcac9ated6ab21916294fb5@mail.gmail.com>
	<49E77B1C.2090009@student.matnat.uio.no>
Message-ID: <013DBDD2-DFB9-4961-B21C-93625D2A42EA@math.washington.edu>

On Apr 16, 2009, at 11:38 AM, Dag Sverre Seljebotn wrote:

> Lisandro Dalcin wrote:
>> Do we really need this?
>>
>> $ tar -zxf Cython-0.11.1.tar.gz
>> $ du -sh Cython-0.11.1
>> 11M	Cython-0.11.1
>>
>> $ du -sh Cython-0.11.1/.hg
>> 6.2M	Cython-0.11.1/.hg
>>
>> $ rm -rf Cython-0.11.1/.hg
>> $ du -sh Cython-0.11.1
>> 4.5M	Cython-0.11.1
>>
>
> Well, 11M \approx 4.5M \approx 0M :-)
>
> An argument in favor is that end-users can very easily contribute a
> bugfix. If users suddenly encounter a bug they can fix then the fix  
> they
> do is very likely to happen in the downloaded release directory, and
> having done that then it's a lot less effort to submit a patch if  
> it's a
> working repository.
>
> Especially with Cython, whose end-users are all developers, this is a
> nice idea.

This is exactly why we do it. Sage ships this way and I think it's  
been a significant factor in attracting developers.

> I don't really mind though. But I have a feeling that the time of  
> making
> a reliable fix for doing releases without .hg, or even writing this
> email, far surpass the download time of those few extra MBs...

This is a good point too.

- Robert


From robertwb at math.washington.edu  Thu Apr 16 21:30:50 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 16 Apr 2009 12:30:50 -0700
Subject: [Cython] test/bugs dirctory
Message-ID: <6307A74B-D2C1-46F2-9339-E3697BCDE602@math.washington.edu>

I like being able to create testfiles for specific tickets and easily  
test them, but having to move tickets back and forth between bugs and  
error/compile/run is cumbersome. Is anyone opposed to putting the  
tests directly in the appropriate test directories, and then having a  
"outstanding bugs" list in runtests.py that it uses to skip over the  
Txxx ones that are still unresolved?

- Robert


From dagss at student.matnat.uio.no  Thu Apr 16 22:02:36 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Thu, 16 Apr 2009 22:02:36 +0200
Subject: [Cython] test/bugs dirctory
In-Reply-To: <6307A74B-D2C1-46F2-9339-E3697BCDE602@math.washington.edu>
References: <6307A74B-D2C1-46F2-9339-E3697BCDE602@math.washington.edu>
Message-ID: <49E78EDC.8030108@student.matnat.uio.no>

Robert Bradshaw wrote:
> I like being able to create testfiles for specific tickets and easily  
> test them, but having to move tickets back and forth between bugs and  
> error/compile/run is cumbersome. Is anyone opposed to putting the  
> tests directly in the appropriate test directories, and then having a  
> "outstanding bugs" list in runtests.py that it uses to skip over the  
> Txxx ones that are still unresolved?

+1, though I'd like that to be a seperate unsolved.txt rather than part 
of runtests.py (prettier commit history).

-- 
Dag Sverre

From robertwb at math.washington.edu  Thu Apr 16 22:07:03 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 16 Apr 2009 13:07:03 -0700
Subject: [Cython] test/bugs dirctory
In-Reply-To: <49E78EDC.8030108@student.matnat.uio.no>
References: <6307A74B-D2C1-46F2-9339-E3697BCDE602@math.washington.edu>
	<49E78EDC.8030108@student.matnat.uio.no>
Message-ID: <A5E2F57E-6514-4796-9FF1-ADBC2C5D9A8A@math.washington.edu>

On Apr 16, 2009, at 1:02 PM, Dag Sverre Seljebotn wrote:

> Robert Bradshaw wrote:
>> I like being able to create testfiles for specific tickets and easily
>> test them, but having to move tickets back and forth between bugs and
>> error/compile/run is cumbersome. Is anyone opposed to putting the
>> tests directly in the appropriate test directories, and then having a
>> "outstanding bugs" list in runtests.py that it uses to skip over the
>> Txxx ones that are still unresolved?
>
> +1, though I'd like that to be a seperate unsolved.txt rather than  
> part
> of runtests.py (prettier commit history).

Sure.

- Robert


From dagss at student.matnat.uio.no  Thu Apr 16 22:28:55 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Thu, 16 Apr 2009 22:28:55 +0200
Subject: [Cython] test/bugs dirctory
In-Reply-To: <49E78EDC.8030108@student.matnat.uio.no>
References: <6307A74B-D2C1-46F2-9339-E3697BCDE602@math.washington.edu>
	<49E78EDC.8030108@student.matnat.uio.no>
Message-ID: <49E79507.3090602@student.matnat.uio.no>

Dag Sverre Seljebotn wrote:
> Robert Bradshaw wrote:
>> I like being able to create testfiles for specific tickets and easily  
>> test them, but having to move tickets back and forth between bugs and  
>> error/compile/run is cumbersome. Is anyone opposed to putting the  
>> tests directly in the appropriate test directories, and then having a  
>> "outstanding bugs" list in runtests.py that it uses to skip over the  
>> Txxx ones that are still unresolved?
> 
> +1, though I'd like that to be a seperate unsolved.txt rather than part 
> of runtests.py (prettier commit history).
> 

I'd now prefer something slighly more ambituous though: Mark each 
testcase as known to fail in the testcase itself (e.g. as a comment on 
the first line).




Going on an utopic rant:

If I really had my wish without anyone spending time to implement it 
:-), I would actually prefer to structure all testcases in a free-form 
hierarchy, and instead have a header which

  a) indicate whether it is run/compile/error
  b) which ticket(s) it is related to
  c) module requirements
  d) some kind of tag system for the feature which is tested -- tags 
could be e.g. "IndexNode", "temps"


-- 
Dag Sverre

From robertwb at math.washington.edu  Thu Apr 16 22:22:38 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 16 Apr 2009 13:22:38 -0700
Subject: [Cython] test/bugs dirctory
In-Reply-To: <49E79507.3090602@student.matnat.uio.no>
References: <6307A74B-D2C1-46F2-9339-E3697BCDE602@math.washington.edu>
	<49E78EDC.8030108@student.matnat.uio.no>
	<49E79507.3090602@student.matnat.uio.no>
Message-ID: <E1004CEC-4D9B-4557-B0DA-087CAFE0958A@math.washington.edu>

On Apr 16, 2009, at 1:28 PM, Dag Sverre Seljebotn wrote:

> Dag Sverre Seljebotn wrote:
>> Robert Bradshaw wrote:
>>> I like being able to create testfiles for specific tickets and  
>>> easily
>>> test them, but having to move tickets back and forth between bugs  
>>> and
>>> error/compile/run is cumbersome. Is anyone opposed to putting the
>>> tests directly in the appropriate test directories, and then  
>>> having a
>>> "outstanding bugs" list in runtests.py that it uses to skip over the
>>> Txxx ones that are still unresolved?
>>
>> +1, though I'd like that to be a seperate unsolved.txt rather than  
>> part
>> of runtests.py (prettier commit history).
>>
>
> I'd now prefer something slighly more ambituous though: Mark each
> testcase as known to fail in the testcase itself (e.g. as a comment on
> the first line).

I think it's nice to have this kind of data external, as it makes it  
much easier to quickly at a glance see what's "still bad" and not  
have untested files fall through the cracks.

> Going on an utopic rant:
>
> If I really had my wish without anyone spending time to implement it
> :-), I would actually prefer to structure all testcases in a free-form
> hierarchy, and instead have a header which
>
>   a) indicate whether it is run/compile/error
>   b) which ticket(s) it is related to
>   c) module requirements
>   d) some kind of tag system for the feature which is tested -- tags
> could be e.g. "IndexNode", "temps"

Maybe someday, but one think I like about the current system is that  
it's insanely simplicity.

- Robert


From m.gysel at gmail.com  Thu Apr 16 23:15:46 2009
From: m.gysel at gmail.com (Martin Gysel)
Date: Thu, 16 Apr 2009 23:15:46 +0200
Subject: [Cython] subclassing builtin
Message-ID: <49E7A002.50002@gmail.com>

Hi

I wanna subclass the builtin dict. I read through the doc but
unfortunately I haven't got it...
I think I need to do something like:

cdef extern from "dictobject.h":
    ctypedef class __builtin__.dict [object PyDictObject]

but then what else, how do I define it after declaration?

thx
martin

From dalcinl at gmail.com  Thu Apr 16 23:43:58 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 16 Apr 2009 18:43:58 -0300
Subject: [Cython] subclassing builtin
In-Reply-To: <49E7A002.50002@gmail.com>
References: <49E7A002.50002@gmail.com>
Message-ID: <e7ba66e40904161443n34129729t2ef0ffce7c9c62f6@mail.gmail.com>

On Thu, Apr 16, 2009 at 6:15 PM, Martin Gysel <m.gysel at gmail.com> wrote:
> Hi
>
> I wanna subclass the builtin dict. I read through the doc but
> unfortunately I haven't got it...
> I think I need to do something like:
>
> cdef extern from "dictobject.h":
> ? ?ctypedef class __builtin__.dict [object PyDictObject]
>

I woud just use "Python.h" ... and add ": pass" at the end (perhaps
add a newline after ":")


> but then what else, how do I define it after declaration?

cdef class MyDict(dict):
   # implement me !!


>
> thx
> martin
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dalcinl at gmail.com  Thu Apr 16 23:49:29 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 16 Apr 2009 18:49:29 -0300
Subject: [Cython] make __pyx_modulename have static storage
Message-ID: <e7ba66e40904161449x1dcc7d41t20e3dedb823a10b4@mail.gmail.com>

Just noticed this line (reported by Intel compiler)

const char *__pyx_modulename = "mpi4py.MPI";

Any good reason to not make it have "static" storage?

-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From kwmsmith at gmail.com  Fri Apr 17 05:28:34 2009
From: kwmsmith at gmail.com (Kurt Smith)
Date: Thu, 16 Apr 2009 22:28:34 -0500
Subject: [Cython] cdef list variable raises exception on SetItem
Message-ID: <c266a7930904162028q23e5156do975e1bf8ff17f596@mail.gmail.com>

Hi,

The first function below "in_range" works fine (the last line
generates a call to PyObject_SetItem) but the "out_of_range" function
does not.  The last line in it generates a call to PyList_SetItem as
it should, but the 'ob' PyObject is not converted to a Py_ssize_t.

[590]$ cat _out_of_range.pyx
def in_range():
    lst = range(11)
    ob = 10
    lst[ob] = -10

def out_of_range():
    cdef list lst = range(11)
    ob = 10
    lst[ob] = -10


Compiling raises a warning, as it should:

[591]$ python setup.py build_ext --inplace
running build_ext
cythoning _out_of_range.pyx to _out_of_range.c
building '_out_of_range' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
-Wstrict-prototypes -fPIC -I/usr/include/python2.5 -c _out_of_range.c
-o build/temp.linux-x86_64-2.5/_out_of_range.o
_out_of_range.c: In function ?__pyx_pf_13_out_of_range_out_of_range?:
_out_of_range.c:429: warning: passing argument 2 of ?PyList_SetItem?
makes integer from pointer without a cast
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions
build/temp.linux-x86_64-2.5/_out_of_range.o -o _out_of_range.so

And calling it raises the exception:

[592]$ python out_of_range.py
after in_range
Traceback (most recent call last):
  File "out_of_range.py", line 5, in <module>
    _out_of_range.out_of_range()
  File "_out_of_range.pyx", line 9, in _out_of_range.out_of_range
(_out_of_range.c:429)
    lst[ob] = -10
IndexError: list assignment index out of range

[593]$ cat out_of_range.py
import _out_of_range

_out_of_range.in_range()
print "after in_range"
_out_of_range.out_of_range()
print "after out_of_range"

From kwmsmith at gmail.com  Fri Apr 17 06:54:29 2009
From: kwmsmith at gmail.com (Kurt Smith)
Date: Thu, 16 Apr 2009 23:54:29 -0500
Subject: [Cython] cdef list variable raises exception on SetItem
In-Reply-To: <c266a7930904162028q23e5156do975e1bf8ff17f596@mail.gmail.com>
References: <c266a7930904162028q23e5156do975e1bf8ff17f596@mail.gmail.com>
Message-ID: <c266a7930904162154q7da19f8bi1e0bec1a853d86ec@mail.gmail.com>

On Thu, Apr 16, 2009 at 10:28 PM, Kurt Smith <kwmsmith at gmail.com> wrote:
> Hi,
>
> The first function below "in_range" works fine (the last line
> generates a call to PyObject_SetItem) but the "out_of_range" function
> does not. ?The last line in it generates a call to PyList_SetItem as
> it should, but the 'ob' PyObject is not converted to a Py_ssize_t.
>

See

http://trac.cython.org/cython_trac/ticket/284

From kwmsmith at gmail.com  Fri Apr 17 07:21:06 2009
From: kwmsmith at gmail.com (Kurt Smith)
Date: Fri, 17 Apr 2009 00:21:06 -0500
Subject: [Cython] cdef list variable raises exception on SetItem
In-Reply-To: <c266a7930904162154q7da19f8bi1e0bec1a853d86ec@mail.gmail.com>
References: <c266a7930904162028q23e5156do975e1bf8ff17f596@mail.gmail.com>
	<c266a7930904162154q7da19f8bi1e0bec1a853d86ec@mail.gmail.com>
Message-ID: <c266a7930904162221g4bfbabd3g499965591476278a@mail.gmail.com>

On Thu, Apr 16, 2009 at 11:54 PM, Kurt Smith <kwmsmith at gmail.com> wrote:


>
> See
>
> http://trac.cython.org/cython_trac/ticket/284
>

Here's a testcase.

diff -r 6fc636aebe32 -r 1097ad309db6 tests/run/cdef_setitem_T284.pyx
--- /dev/null   Thu Jan 01 00:00:00 1970 +0000
+++ b/tests/run/cdef_setitem_T284.pyx   Fri Apr 17 00:15:48 2009 -0500
@@ -0,0 +1,17 @@
+__doc__ = u'''
+>>> no_cdef()
+>>> with_cdef()
+'''
+def no_cdef():
+    lst = range(11)
+    ob = 10L
+    lst[ob] = -10
+    dd = {}
+    dd[ob] = -10
+
+def with_cdef():
+    cdef list lst = range(11)
+    ob = 10L
+    lst[ob] = -10
+    cdef dict dd = {}
+    dd[ob] = -10

And here's a trivial fix -- it seems hacky and if there's a better way
to do it please say so...

diff -r 6fc636aebe32 -r 1097ad309db6 Cython/Compiler/ExprNodes.py
--- a/Cython/Compiler/ExprNodes.py      Thu Apr 16 19:10:51 2009 -0700
+++ b/Cython/Compiler/ExprNodes.py      Fri Apr 17 00:15:48 2009 -0500
@@ -1858,6 +1858,7 @@ class IndexNode(ExprNode):
                 function = "PyDict_SetItem"
             elif self.base.type is list_type:
                 function = "PyList_SetItem"
+                index_code = "PyInt_AsSsize_t(%s)" % index_code
             # don't use PyTuple_SetItem(), as we'd normally get a
             # TypeError when changing a tuple, while PyTuple_SetItem()
             # would allow updates

From stefan_ml at behnel.de  Fri Apr 17 07:39:06 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 17 Apr 2009 07:39:06 +0200
Subject: [Cython] make __pyx_modulename have static storage
In-Reply-To: <e7ba66e40904161449x1dcc7d41t20e3dedb823a10b4@mail.gmail.com>
References: <e7ba66e40904161449x1dcc7d41t20e3dedb823a10b4@mail.gmail.com>
Message-ID: <49E815FA.1030504@behnel.de>


Lisandro Dalcin wrote:
> Just noticed this line (reported by Intel compiler)
> 
> const char *__pyx_modulename = "mpi4py.MPI";
> 
> Any good reason to not make it have "static" storage?

No, it shouldn't be used anywhere else anyway.

Stefan

From robertwb at math.washington.edu  Fri Apr 17 07:58:59 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 16 Apr 2009 22:58:59 -0700
Subject: [Cython] cdef list variable raises exception on SetItem
In-Reply-To: <c266a7930904162221g4bfbabd3g499965591476278a@mail.gmail.com>
References: <c266a7930904162028q23e5156do975e1bf8ff17f596@mail.gmail.com>
	<c266a7930904162154q7da19f8bi1e0bec1a853d86ec@mail.gmail.com>
	<c266a7930904162221g4bfbabd3g499965591476278a@mail.gmail.com>
Message-ID: <72AA5609-31A3-454C-8916-E33AA5CB3093@math.washington.edu>

On Apr 16, 2009, at 10:21 PM, Kurt Smith wrote:

> On Thu, Apr 16, 2009 at 11:54 PM, Kurt Smith <kwmsmith at gmail.com>  
> wrote:
>
>
>>
>> See
>>
>> http://trac.cython.org/cython_trac/ticket/284
>>
>
> Here's a testcase.
>
> diff -r 6fc636aebe32 -r 1097ad309db6 tests/run/cdef_setitem_T284.pyx
> --- /dev/null   Thu Jan 01 00:00:00 1970 +0000
> +++ b/tests/run/cdef_setitem_T284.pyx   Fri Apr 17 00:15:48 2009 -0500
> @@ -0,0 +1,17 @@
> +__doc__ = u'''
> +>>> no_cdef()
> +>>> with_cdef()
> +'''
> +def no_cdef():
> +    lst = range(11)
> +    ob = 10L
> +    lst[ob] = -10
> +    dd = {}
> +    dd[ob] = -10
> +
> +def with_cdef():
> +    cdef list lst = range(11)
> +    ob = 10L
> +    lst[ob] = -10
> +    cdef dict dd = {}
> +    dd[ob] = -10
>

Thanks.

> And here's a trivial fix -- it seems hacky and if there's a better way
> to do it please say so...
>
> diff -r 6fc636aebe32 -r 1097ad309db6 Cython/Compiler/ExprNodes.py
> --- a/Cython/Compiler/ExprNodes.py      Thu Apr 16 19:10:51 2009 -0700
> +++ b/Cython/Compiler/ExprNodes.py      Fri Apr 17 00:15:48 2009 -0500
> @@ -1858,6 +1858,7 @@ class IndexNode(ExprNode):
>                  function = "PyDict_SetItem"
>              elif self.base.type is list_type:
>                  function = "PyList_SetItem"
> +                index_code = "PyInt_AsSsize_t(%s)" % index_code
>              # don't use PyTuple_SetItem(), as we'd normally get a
>              # TypeError when changing a tuple, while  
> PyTuple_SetItem()
>              # would allow updates

This will actually raise the wrong kind of error for non-int objects-- 
perhaps we should just remove the entire line? Optimized setting/ 
getting still happens with int indices.

- Robert


From riccardo.murri at gmail.com  Fri Apr 17 12:54:30 2009
From: riccardo.murri at gmail.com (Riccardo Murri)
Date: Fri, 17 Apr 2009 12:54:30 +0200
Subject: [Cython] cannot compile derived class? (cython 0.11.1)
Message-ID: <1c34ba170904170354j576ba8a8i58e1ed246d3ecc9b@mail.gmail.com>

Hello,

I cannot get Cython 0.11.1 to compile the following simple example::

  $ cat a.pxd
  cdef class A(object)
  cdef class B(A)

  $ cat a.pyx
  cdef class A(object):
       pass

  cdef class B(A):
       pass

  $ cat c.pyx
  from a cimport B

  cdef class C(B):
       pass

File "a.pyx" compiles correctly, but Cython chockes on "c.pxd"::

  $ cython -v a.pyx c.pyx
  Compiling /tmp/a.pyx
  Compiling /tmp/c.pyx
  Traceback (most recent call last):
    File "/home/rmurri/sw/Cython-0.11.1/bin/cython", line 8, in <module>
      main(command_line = 1)
    File "/home/rmurri/sw/Cython-0.11.1/Cython/Compiler/Main.py", line
734, in main
      result = compile(sources, options)
    File "/home/rmurri/sw/Cython-0.11.1/Cython/Compiler/Main.py", line
711, in compile
      return compile_multiple(source, options)
    File "/home/rmurri/sw/Cython-0.11.1/Cython/Compiler/Main.py", line
681, in compile_multiple
      result = run_pipeline(source, options)
    File "/home/rmurri/sw/Cython-0.11.1/Cython/Compiler/Main.py", line
543, in run_pipeline
      err, enddata = context.run_pipeline(pipeline, source)
    File "/home/rmurri/sw/Cython-0.11.1/Cython/Compiler/Main.py", line
198, in run_pipeline
      data = phase(data)
    File "/home/rmurri/sw/Cython-0.11.1/Cython/Compiler/Main.py", line
139, in generate_pyx_code
      module_node.process_implementation(options, result)
    File "/home/rmurri/sw/Cython-0.11.1/Cython/Compiler/ModuleNode.py",
line 71, in process_implementation
      self.generate_c_code(env, options, result)
    File "/home/rmurri/sw/Cython-0.11.1/Cython/Compiler/ModuleNode.py",
line 269, in generate_c_code
      self.generate_typeobj_definitions(env, code)
    File "/home/rmurri/sw/Cython-0.11.1/Cython/Compiler/ModuleNode.py",
line 825, in generate_typeobj_definitions
      self.generate_new_function(scope, code)
    File "/home/rmurri/sw/Cython-0.11.1/Cython/Compiler/ModuleNode.py",
line 885, in generate_new_function
      tp_new = TypeSlots.get_base_slot_function(scope, tp_slot)
    File "/home/rmurri/sw/Cython-0.11.1/Cython/Compiler/TypeSlots.py",
line 429, in get_base_slot_function
      if scope.parent_scope is base_type.scope.parent_scope:
  AttributeError: 'NoneType' object has no attribute 'parent_scope'

  $ cython --version
  Cython version 0.11.1

Same error if I change "class A(object)" to "class A".

Is this a bug or am I doing something wrong?

Best regards,
Riccardo

-- 
Riccardo Murri, via Ross?e 17, 6987 Caslano (CH)

From riccardo.murri at gmail.com  Fri Apr 17 14:50:47 2009
From: riccardo.murri at gmail.com (Riccardo Murri)
Date: Fri, 17 Apr 2009 14:50:47 +0200
Subject: [Cython] incorrect C code generated on method assignment (cython
	0.11.1)
Message-ID: <1c34ba170904170550jc9a800cue5c1dad48d30e256@mail.gmail.com>

Hello,

while converting a class from pure Python to Cython, I've come across
the following:

  $ cat r.pyx
  cdef class r(object):
       cdef long long n
       cdef long long d

       def __init__(self, long long n, long long d):
           self.n = n
           self.d = d

       def __int__(self):
           return (self.n // self.d)

       __long__ = __int__

  $ cython -v r.pyx
  Compiling /tmp/r.pyx

  $ gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
-Wstrict-prototypes -fPIC -I/usr/include/python2.5 -c r.c -o r.o
  r.c:487: error: ?None? undeclared here (not in a function)

  $ sed -e '484,490!d;488i^^^^^^^^^^^^^^^^^^^^^^^^ up here' r.c
    0, /*nb_coerce*/
    #endif
    __pyx_pf_1r_1r___int__, /*nb_int*/
    None, /*nb_long*/
  ^^^^^^^^^^^^^^^^^^^^^^^^ up here
    0, /*nb_float*/
    #if PY_MAJOR_VERSION < 3
    0, /*nb_oct*/

I understand that "__long__ = __int__" is not the correct way to do
method aliasing in Cython, but the compiler should probably emit a
warning -- generating incorrect C code looks like a bug.

Best regards,
Riccardo

-- 
Riccardo Murri, via Ross?e 17, 6987 Caslano (CH)

From stefan_ml at behnel.de  Fri Apr 17 15:09:02 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 17 Apr 2009 15:09:02 +0200 (CEST)
Subject: [Cython] incorrect C code generated on method assignment
 (cython 0.11.1)
In-Reply-To: <1c34ba170904170550jc9a800cue5c1dad48d30e256@mail.gmail.com>
References: <1c34ba170904170550jc9a800cue5c1dad48d30e256@mail.gmail.com>
Message-ID: <09c458c88760dc5ea641d522c6ec939e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Hi,

thanks for the report.

Riccardo Murri wrote:
> while converting a class from pure Python to Cython, I've come across
> the following:
>
>   $ cat r.pyx
>   cdef class r(object):
>        cdef long long n
>        cdef long long d
>
>        def __init__(self, long long n, long long d):
>            self.n = n
>            self.d = d
>
>        def __int__(self):
>            return (self.n // self.d)
>
>        __long__ = __int__
>
>   $ cython -v r.pyx
>   Compiling /tmp/r.pyx
>
>   $ gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
> -Wstrict-prototypes -fPIC -I/usr/include/python2.5 -c r.c -o r.o
>   r.c:487: error: ???None??? undeclared here (not in a function)
>
>   $ sed -e '484,490!d;488i^^^^^^^^^^^^^^^^^^^^^^^^ up here' r.c
>     0, /*nb_coerce*/
>     #endif
>     __pyx_pf_1r_1r___int__, /*nb_int*/
>     None, /*nb_long*/
>   ^^^^^^^^^^^^^^^^^^^^^^^^ up here
>     0, /*nb_float*/
>     #if PY_MAJOR_VERSION < 3
>     0, /*nb_oct*/

Yes, that's a bug. I assume that the above code does not make Cython set
up the C level function name.

Raising an error here would work for now.


> I understand that "__long__ = __int__" is not the correct way to do
> method aliasing in Cython

... and that's a feature request, it would be nice to make that work.
Could you create two tickets for those?

http://trac.cython.org/cython_trac/

If you don't have a trac account yet, please send a htpasswd password line
to Robert Bradshaw.

Thanks!

Stefan


From dalcinl at gmail.com  Fri Apr 17 15:33:15 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Fri, 17 Apr 2009 10:33:15 -0300
Subject: [Cython] make __pyx_modulename have static storage
In-Reply-To: <49E815FA.1030504@behnel.de>
References: <e7ba66e40904161449x1dcc7d41t20e3dedb823a10b4@mail.gmail.com>
	<49E815FA.1030504@behnel.de>
Message-ID: <e7ba66e40904170633g2fa55eb3rd6392ac2725282ba@mail.gmail.com>

Sorry, I did not understad you comment, perhaps I was not clear
enough... This is a global var, not one loca to function, then I'm
asking to emit the C code like this:

static const char *__pyx_modulename = "mpi4py.MPI";

See this:

$ grep __pyx_modulename src/mpi4py.MPI.c
const char *__pyx_modulename = "mpi4py.MPI";

$ nm ~/lib/python/mpi4py/MPI.so | grep __pyx_modulename
000c0fa0 D __pyx_modulename

What's the use case for that symbol to be dynamic??

All this was introduced by Robert when implementing the new C division stuff

changeset:   1916:413a75b92fc0
user:        Robert Bradshaw <robertwb at math.washington.edu>
date:        Sat Mar 28 02:22:07 2009 -0700
summary:     Use warnings framework for c division.


On Fri, Apr 17, 2009 at 2:39 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
>
> Lisandro Dalcin wrote:
>> Just noticed this line (reported by Intel compiler)
>>
>> const char *__pyx_modulename = "mpi4py.MPI";
>>
>> Any good reason to not make it have "static" storage?
>
> No, it shouldn't be used anywhere else anyway.
>
> Stefan
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From riccardo.murri at gmail.com  Fri Apr 17 15:51:42 2009
From: riccardo.murri at gmail.com (Riccardo Murri)
Date: Fri, 17 Apr 2009 15:51:42 +0200
Subject: [Cython] subclassing builtin
In-Reply-To: <e7ba66e40904161443n34129729t2ef0ffce7c9c62f6@mail.gmail.com>
References: <49E7A002.50002@gmail.com>
	<e7ba66e40904161443n34129729t2ef0ffce7c9c62f6@mail.gmail.com>
Message-ID: <1c34ba170904170651i3a2cf1cbkb9f1b4ea6906de14@mail.gmail.com>

Hi,

I cannot get the documented method to work::

  $ cat t.pyx
  cdef extern from "tupleobject.h":
      ctypedef class __builtin__.tuple [object PyTupleObject]:
          pass

  cdef class MyTuple(tuple):
      pass

  $ cython t.pyx
  warning: /tmp/t.pyx:2:4: tuple already a builtin Cython type
  $ gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
-Wstrict-prototypes -fPIC -I/usr/include/python2.5 -c t.c -o t.o
  $ gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions t.o -o t.so

  $ python
  Python 2.5.2 (r252:60911, Oct  5 2008, 19:24:49)
  [GCC 4.3.2] on linux2
  Type "help", "copyright", "credits" or "license" for more information.
  >>> import t
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "t.pyx", line 2, in t (t.c:610)
      ctypedef class __builtin__.tuple [object PyTupleObject]:
  ValueError: __builtin__.tuple does not appear to be the correct type object

Relevant lines in t.c are::

  $ sed -e '547,551!d' t.c
    /*--- Function export code ---*/
    /*--- Type init code ---*/
    __pyx_ptype_1t_tuple = __Pyx_ImportType(__Pyx_BUILTIN_MODULE_NAME,
"tuple", sizeof(PyTupleObject)); if (unlikely(!__pyx_ptype_1t_tuple))
{__pyx_filename = __pyx_f[0]; __pyx_lineno = 2; __pyx_clineno =
__LINE__; goto __pyx_L1_error;}
    __pyx_type_1t_MyTuple.tp_base = __pyx_ptype_1t_tuple;
    if (PyType_Ready(&__pyx_type_1t_MyTuple) < 0) {__pyx_filename =
__pyx_f[0]; __pyx_lineno = 5; __pyx_clineno = __LINE__; goto
__pyx_L1_error;}

I'm using Cython 0.11.1; thanks for any help!

Best regards,
Riccardo

-- 
Riccardo Murri, via Ross?e 17, 6987 Caslano (CH)

From dalcinl at gmail.com  Fri Apr 17 16:01:09 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Fri, 17 Apr 2009 11:01:09 -0300
Subject: [Cython] incorrect C code generated on method assignment
	(cython 0.11.1)
In-Reply-To: <09c458c88760dc5ea641d522c6ec939e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
References: <1c34ba170904170550jc9a800cue5c1dad48d30e256@mail.gmail.com>
	<09c458c88760dc5ea641d522c6ec939e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <e7ba66e40904170701h4c6af104m7f6d0ab9328c83cf@mail.gmail.com>

On Fri, Apr 17, 2009 at 10:09 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
>
> ... and that's a feature request, it would be nice to make that work.
> Could you create two tickets for those?
>

Mmm... we will have to have care here... In Py3k, nb_long was renamed
to nb_reserved (moreover, now it is a bare void* pointer). So, In
Py3k, if a class only defines one of __int__() or __long__(), Cython
should generate code filling the 'nb_int' slot, and put a NULL in
'nb_reserved'. What I do not know is what to do if a class defines
both special methods...

Also note that the in implementation of __Pyx_PyNumber_Int() (from my
patch for C-int conversion), "nb_long" takes precedence over "nb_int"
in Python 2. IMHO, that was the best thing to do...



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dalcinl at gmail.com  Fri Apr 17 16:13:04 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Fri, 17 Apr 2009 11:13:04 -0300
Subject: [Cython] subclassing builtin
In-Reply-To: <1c34ba170904170651i3a2cf1cbkb9f1b4ea6906de14@mail.gmail.com>
References: <49E7A002.50002@gmail.com>
	<e7ba66e40904161443n34129729t2ef0ffce7c9c62f6@mail.gmail.com>
	<1c34ba170904170651i3a2cf1cbkb9f1b4ea6906de14@mail.gmail.com>
Message-ID: <e7ba66e40904170713i3572d04dv8dfc47691b210bda@mail.gmail.com>

On Fri, Apr 17, 2009 at 10:51 AM, Riccardo Murri
<riccardo.murri at gmail.com> wrote:
> Hi,
>
> I cannot get the documented method to work::
>
> ?$ cat t.pyx
> ?cdef extern from "tupleobject.h":
> ? ? ?ctypedef class __builtin__.tuple [object PyTupleObject]:
> ? ? ? ? ?pass
>
> ?cdef class MyTuple(tuple):
> ? ? ?pass
>

Tuples are a different beast... They are PyObject_VAR_HEAD structures
(like strings in Py2, and perhaps in Py3 in the future, no right
now)...

typedef struct {
    PyObject_VAR_HEAD
    PyObject *ob_item[1];
} PyTupleObject;

Cython cannot currently handle this...

Perhaps the error message could be made more explicit in this case (by
checking if tp_itemsize != 0 ) ... ?


> ?$ cython t.pyx
> ?warning: /tmp/t.pyx:2:4: tuple already a builtin Cython type
> ?$ gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
> -Wstrict-prototypes -fPIC -I/usr/include/python2.5 -c t.c -o t.o
> ?$ gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions t.o -o t.so
>
> ?$ python
> ?Python 2.5.2 (r252:60911, Oct ?5 2008, 19:24:49)
> ?[GCC 4.3.2] on linux2
> ?Type "help", "copyright", "credits" or "license" for more information.
> ?>>> import t
> ?Traceback (most recent call last):
> ? ?File "<stdin>", line 1, in <module>
> ? ?File "t.pyx", line 2, in t (t.c:610)
> ? ? ?ctypedef class __builtin__.tuple [object PyTupleObject]:
> ?ValueError: __builtin__.tuple does not appear to be the correct type object
>
> Relevant lines in t.c are::
>
> ?$ sed -e '547,551!d' t.c
> ? ?/*--- Function export code ---*/
> ? ?/*--- Type init code ---*/
> ? ?__pyx_ptype_1t_tuple = __Pyx_ImportType(__Pyx_BUILTIN_MODULE_NAME,
> "tuple", sizeof(PyTupleObject)); if (unlikely(!__pyx_ptype_1t_tuple))
> {__pyx_filename = __pyx_f[0]; __pyx_lineno = 2; __pyx_clineno =
> __LINE__; goto __pyx_L1_error;}
> ? ?__pyx_type_1t_MyTuple.tp_base = __pyx_ptype_1t_tuple;
> ? ?if (PyType_Ready(&__pyx_type_1t_MyTuple) < 0) {__pyx_filename =
> __pyx_f[0]; __pyx_lineno = 5; __pyx_clineno = __LINE__; goto
> __pyx_L1_error;}
>
> I'm using Cython 0.11.1; thanks for any help!
>
> Best regards,
> Riccardo
>
> --
> Riccardo Murri, via Ross?e 17, 6987 Caslano (CH)
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From riccardo.murri at gmail.com  Fri Apr 17 16:32:13 2009
From: riccardo.murri at gmail.com (Riccardo Murri)
Date: Fri, 17 Apr 2009 16:32:13 +0200
Subject: [Cython] subclassing builtin
In-Reply-To: <e7ba66e40904170713i3572d04dv8dfc47691b210bda@mail.gmail.com>
References: <49E7A002.50002@gmail.com>
	<e7ba66e40904161443n34129729t2ef0ffce7c9c62f6@mail.gmail.com>
	<1c34ba170904170651i3a2cf1cbkb9f1b4ea6906de14@mail.gmail.com>
	<e7ba66e40904170713i3572d04dv8dfc47691b210bda@mail.gmail.com>
Message-ID: <1c34ba170904170732k245bd6c3q3d619204aa914607@mail.gmail.com>

On Fri, Apr 17, 2009 at 4:13 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
> On Fri, Apr 17, 2009 at 10:51 AM, Riccardo Murri <riccardo.murri at gmail.com> wrote:
>> I cannot get the documented method to work::
>>
>> ?$ cat t.pyx
>> ?cdef extern from "tupleobject.h":
>> ? ? ?ctypedef class __builtin__.tuple [object PyTupleObject]:
>> ? ? ? ? ?pass
>>
>> ?cdef class MyTuple(tuple):
>> ? ? ?pass
>>
>
> Tuples are a different beast... They are PyObject_VAR_HEAD structures
> (like strings in Py2, and perhaps in Py3 in the future, no right
> now)...
>
> typedef struct {
> ? ?PyObject_VAR_HEAD
> ? ?PyObject *ob_item[1];
> } PyTupleObject;
>
> Cython cannot currently handle this...
>

Thank you very much!  I have updated the Wiki page, stating that not
only strings but also tuples cannot be subclassed:

http://wiki.cython.org/FAQ/cdef_derive#preview

Riccardo

-- 
Riccardo Murri, via Ross?e 17, 6987 Caslano (CH)

From stefan_ml at behnel.de  Fri Apr 17 16:39:04 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 17 Apr 2009 16:39:04 +0200 (CEST)
Subject: [Cython] subclassing builtin
In-Reply-To: <e7ba66e40904170713i3572d04dv8dfc47691b210bda@mail.gmail.com>
References: <49E7A002.50002@gmail.com>
	<e7ba66e40904161443n34129729t2ef0ffce7c9c62f6@mail.gmail.com>
	<1c34ba170904170651i3a2cf1cbkb9f1b4ea6906de14@mail.gmail.com>
	<e7ba66e40904170713i3572d04dv8dfc47691b210bda@mail.gmail.com>
Message-ID: <867910dd5a852ea84f8c11706783366b.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Lisandro Dalcin wrote:
> On Fri, Apr 17, 2009 at 10:51 AM, Riccardo Murri wrote:
>> I cannot get the documented method to work::
>>
>> ?$ cat t.pyx
>> ?cdef extern from "tupleobject.h":
>> ? ? ?ctypedef class __builtin__.tuple [object PyTupleObject]:
>> ? ? ? ? ?pass
>>
>> ?cdef class MyTuple(tuple):
>> ? ? ?pass
>>
>
> Tuples are a different beast... They are PyObject_VAR_HEAD structures

Yep, that's a missing feature. It's not trivial to implement as this
requires a special case in the way Cython deals with subtypes, likely with
a major code impact.

http://trac.cython.org/cython_trac/ticket/152

It's usually best to use a normal Python class to subtype str (Py2.x) and
tuple, instead of a cdef class.

Stefan


From stefan_ml at behnel.de  Fri Apr 17 16:49:47 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 17 Apr 2009 16:49:47 +0200 (CEST)
Subject: [Cython] make __pyx_modulename have static storage
In-Reply-To: <e7ba66e40904170633g2fa55eb3rd6392ac2725282ba@mail.gmail.com>
References: <e7ba66e40904161449x1dcc7d41t20e3dedb823a10b4@mail.gmail.com>
	<49E815FA.1030504@behnel.de>
	<e7ba66e40904170633g2fa55eb3rd6392ac2725282ba@mail.gmail.com>
Message-ID: <df4cd3fdb1630b6f453aa61b0dac1992.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Lisandro Dalcin wrote:
> Sorry, I did not understad you comment, perhaps I was not clear
> enough... This is a global var, not one loca to function, then I'm
> asking to emit the C code like this:
>
> static const char *__pyx_modulename = "mpi4py.MPI";

Yes, that's what I meant. It's not supposed to be used anywhere outside of
the module, so "static" is correct here. I assume that Robert added the
constant because utility functions are not currently module specific.

Stefan


From dalcinl at gmail.com  Fri Apr 17 16:53:27 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Fri, 17 Apr 2009 11:53:27 -0300
Subject: [Cython] make __pyx_modulename have static storage
In-Reply-To: <df4cd3fdb1630b6f453aa61b0dac1992.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
References: <e7ba66e40904161449x1dcc7d41t20e3dedb823a10b4@mail.gmail.com>
	<49E815FA.1030504@behnel.de>
	<e7ba66e40904170633g2fa55eb3rd6392ac2725282ba@mail.gmail.com>
	<df4cd3fdb1630b6f453aa61b0dac1992.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <e7ba66e40904170753r5d54eb44pd1ad1d57ad7d9abf@mail.gmail.com>

On Fri, Apr 17, 2009 at 11:49 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Lisandro Dalcin wrote:
>>
>> static const char *__pyx_modulename = "mpi4py.MPI";
>
> Yes, that's what I meant.

OK, then I'll push a fix in cython-devel.


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From stefan_ml at behnel.de  Fri Apr 17 17:06:39 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 17 Apr 2009 17:06:39 +0200 (CEST)
Subject: [Cython] incorrect C code generated on method assignment
 (cython 0.11.1)
In-Reply-To: <e7ba66e40904170701h4c6af104m7f6d0ab9328c83cf@mail.gmail.com>
References: <1c34ba170904170550jc9a800cue5c1dad48d30e256@mail.gmail.com>
	<09c458c88760dc5ea641d522c6ec939e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<e7ba66e40904170701h4c6af104m7f6d0ab9328c83cf@mail.gmail.com>
Message-ID: <63f6aed263dda2ed3b6d431bad1e93c6.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Lisandro Dalcin wrote:
> On Fri, Apr 17, 2009 at 10:09 AM, Stefan Behnel wrote:
>> ... and that's a feature request, it would be nice to make that work.
>> Could you create two tickets for those?

What I meant was: add support for setting special methods by assignment,
at least in some cases. I didn't specifically mean "__int__" and
"__long__".


> Mmm... we will have to have care here... In Py3k, nb_long was renamed
> to nb_reserved (moreover, now it is a bare void* pointer).

Yes, they removed all sorts of stuff (and I lobbied for removing all sorts
of dead meat), but they forgot to remove that one in time. It's still in
3.1a1.


> So, In
> Py3k, if a class only defines one of __int__() or __long__(), Cython
> should generate code filling the 'nb_int' slot,

Sure.


> and put a NULL in 'nb_reserved'.

Always in Py3.


> What I do not know is what to do
> if a class defines both special methods...

Good question. I think if __long__ is defined, that's what best resembles
nb_int in Py3. People are more likely to expect C-int compatibility in
__int__, so the only reason why one would define __long__ is to pass
larger numbers.

If both are defined, I'd vote for using __long__ in Py3.


> Also note that the in implementation of __Pyx_PyNumber_Int() (from my
> patch for C-int conversion), "nb_long" takes precedence over "nb_int"
> in Python 2. IMHO, that was the best thing to do...

What's CPython doing here?

Stefan


From dalcinl at gmail.com  Fri Apr 17 18:04:58 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Fri, 17 Apr 2009 13:04:58 -0300
Subject: [Cython] incorrect C code generated on method assignment
	(cython 0.11.1)
In-Reply-To: <63f6aed263dda2ed3b6d431bad1e93c6.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
References: <1c34ba170904170550jc9a800cue5c1dad48d30e256@mail.gmail.com>
	<09c458c88760dc5ea641d522c6ec939e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<e7ba66e40904170701h4c6af104m7f6d0ab9328c83cf@mail.gmail.com>
	<63f6aed263dda2ed3b6d431bad1e93c6.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <e7ba66e40904170904w54c3fe86j29dc2f08cac7195a@mail.gmail.com>

On Fri, Apr 17, 2009 at 12:06 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Lisandro Dalcin wrote:
>> On Fri, Apr 17, 2009 at 10:09 AM, Stefan Behnel wrote:
>>> ... and that's a feature request, it would be nice to make that work.
>>> Could you create two tickets for those?
>
> What I meant was: add support for setting special methods by assignment,
> at least in some cases. I didn't specifically mean "__int__" and
> "__long__".
>
>
>> Mmm... we will have to have care here... In Py3k, nb_long was renamed
>> to nb_reserved (moreover, now it is a bare void* pointer).
>
> Yes, they removed all sorts of stuff (and I lobbied for removing all sorts
> of dead meat), but they forgot to remove that one in time. It's still in
> 3.1a1.
>
>
>> So, In
>> Py3k, if a class only defines one of __int__() or __long__(), Cython
>> should generate code filling the 'nb_int' slot,
>
> Sure.
>
>
>> and put a NULL in 'nb_reserved'.
>
> Always in Py3.
>
>
>> What I do not know is what to do
>> if a class defines both special methods...
>
> Good question. I think if __long__ is defined, that's what best resembles
> nb_int in Py3. People are more likely to expect C-int compatibility in
> __int__, so the only reason why one would define __long__ is to pass
> larger numbers.
>
> If both are defined, I'd vote for using __long__ in Py3.
>

I agree in all above points with you.

>
>> Also note that the in implementation of __Pyx_PyNumber_Int() (from my
>> patch for C-int conversion), "nb_long" takes precedence over "nb_int"
>> in Python 2. IMHO, that was the best thing to do...
>
> What's CPython doing here?
>

CPython is a real mess here... At the price of manually inspecting the
existence of nb_int/nb_long slots, __Pyx_PyNumber_Int(ob) is my best
attempt for making all work as expected for Cython. I've worked hard
to make to avoid the PyInt/PyLong distinction. So if you do "cdef
short j = obj" it will work for PyInt/PyLong in all supported Python
runtimes.

The only possible issue is that if a integral-like type defines both
nb_int/nb_long and they return Python integrals with different values
(in the == sense), then the return of "nb_long" will be used and
"nb_int" will not be ever called. However, what would be the use case
for such implementations of __int__() and __long__() ???. However,
this work on Python.

In [1]: class A(object):
   ...:     def __int__(self): return 2
   ...:     def __long__(self): return 3
   ...:
   ...:

In [2]:

In [3]: int(A())
Out[3]: 2

In [4]: long(A())
Out[4]: 3





-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From kwmsmith at gmail.com  Fri Apr 17 18:40:13 2009
From: kwmsmith at gmail.com (Kurt Smith)
Date: Fri, 17 Apr 2009 11:40:13 -0500
Subject: [Cython] cdef list variable raises exception on SetItem
In-Reply-To: <72AA5609-31A3-454C-8916-E33AA5CB3093@math.washington.edu>
References: <c266a7930904162028q23e5156do975e1bf8ff17f596@mail.gmail.com>
	<c266a7930904162154q7da19f8bi1e0bec1a853d86ec@mail.gmail.com>
	<c266a7930904162221g4bfbabd3g499965591476278a@mail.gmail.com>
	<72AA5609-31A3-454C-8916-E33AA5CB3093@math.washington.edu>
Message-ID: <c266a7930904170940h2c1e549ye23d8b44d74e50b9@mail.gmail.com>

On Fri, Apr 17, 2009 at 12:58 AM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:

>
> This will actually raise the wrong kind of error for non-int objects--
> perhaps we should just remove the entire line? Optimized setting/
> getting still happens with int indices.

Here's a patch that comments out the lines and adds a comment.

diff -r fc73225aaea1 Cython/Compiler/ExprNodes.py
--- a/Cython/Compiler/ExprNodes.py	Fri Apr 17 09:11:16 2009 +0200
+++ b/Cython/Compiler/ExprNodes.py	Fri Apr 17 11:37:44 2009 -0500
@@ -1856,8 +1856,12 @@
             index_code = self.index.py_result()
             if self.base.type is dict_type:
                 function = "PyDict_SetItem"
-            elif self.base.type is list_type:
-                function = "PyList_SetItem"
+            # PyList_SetItem() requires the index to be converted to a
+            # Py_ssize_t, so we fall through to PyObject_SetItem instead.
+            #
+            #elif self.base.type is list_type:
+            #    function = "PyList_SetItem"
+
             # don't use PyTuple_SetItem(), as we'd normally get a
             # TypeError when changing a tuple, while PyTuple_SetItem()
             # would allow updates

Kurt

From stefan_ml at behnel.de  Fri Apr 17 18:52:48 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 17 Apr 2009 18:52:48 +0200
Subject: [Cython] incorrect C code generated on method
 assignment	(cython 0.11.1)
In-Reply-To: <e7ba66e40904170904w54c3fe86j29dc2f08cac7195a@mail.gmail.com>
References: <1c34ba170904170550jc9a800cue5c1dad48d30e256@mail.gmail.com>	<09c458c88760dc5ea641d522c6ec939e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>	<e7ba66e40904170701h4c6af104m7f6d0ab9328c83cf@mail.gmail.com>	<63f6aed263dda2ed3b6d431bad1e93c6.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<e7ba66e40904170904w54c3fe86j29dc2f08cac7195a@mail.gmail.com>
Message-ID: <49E8B3E0.2050403@behnel.de>


Lisandro Dalcin wrote:
> On Fri, Apr 17, 2009 at 12:06 PM, Stefan Behnel wrote:
>> Lisandro Dalcin wrote:
>>> What I do not know is what to do
>>> if a class defines both special methods...
>> Good question. I think if __long__ is defined, that's what best resembles
>> nb_int in Py3. People are more likely to expect C-int compatibility in
>> __int__, so the only reason why one would define __long__ is to pass
>> larger numbers.

BTW, in Py3, PyNumber_Int() was replaced in favour of PyNumber_Long(). That
makes it a bit funny, the C layer uses "long", whereas the Python layer
uses "int".


>>> Also note that the in implementation of __Pyx_PyNumber_Int() (from my
>>> patch for C-int conversion), "nb_long" takes precedence over "nb_int"
>>> in Python 2. IMHO, that was the best thing to do...
>> What's CPython doing here?
> 
> CPython is a real mess here... At the price of manually inspecting the
> existence of nb_int/nb_long slots, __Pyx_PyNumber_Int(ob) is my best
> attempt for making all work as expected for Cython. I've worked hard
> to make to avoid the PyInt/PyLong distinction. So if you do "cdef
> short j = obj" it will work for PyInt/PyLong in all supported Python
> runtimes.
> 
> The only possible issue is that if a integral-like type defines both
> nb_int/nb_long and they return Python integrals with different values
> (in the == sense), then the return of "nb_long" will be used and
> "nb_int" will not be ever called. However, what would be the use case
> for such implementations of __int__() and __long__() ???. However,
> this work on Python.

I don't see a major use case either, but calling nb_int when we convert to
a C-int or a smaller type might still make sense in Py2. The function might
actually do some work (as opposed to returning a constant value). I'd
suspect an __int__() function to be potentially more optimised by users
than something that already knows that it has to return a PyLong.

Note that Py2 allows __int__() to return a PyLong if the value doesn't fit
into a PyInt, while __long__() must return a PyLong:

http://docs.python.org/c-api/number.html#PyNumber_Int

"""
PyObject* PyNumber_Int(PyObject *o)
    Return value: New reference.

    Returns the o converted to an integer object on success, or NULL on
failure. If the argument is outside the integer range a long object will be
returned instead. This is the equivalent of the Python expression int(o).

PyObject* PyNumber_Long(PyObject *o)
    Return value: New reference.

    Returns the o converted to a long integer object on success, or NULL on
failure. This is the equivalent of the Python expression long(o).
"""

Stefan

From dalcinl at gmail.com  Fri Apr 17 19:19:28 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Fri, 17 Apr 2009 14:19:28 -0300
Subject: [Cython] incorrect C code generated on method assignment
	(cython 0.11.1)
In-Reply-To: <49E8B3E0.2050403@behnel.de>
References: <1c34ba170904170550jc9a800cue5c1dad48d30e256@mail.gmail.com>
	<09c458c88760dc5ea641d522c6ec939e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<e7ba66e40904170701h4c6af104m7f6d0ab9328c83cf@mail.gmail.com>
	<63f6aed263dda2ed3b6d431bad1e93c6.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<e7ba66e40904170904w54c3fe86j29dc2f08cac7195a@mail.gmail.com>
	<49E8B3E0.2050403@behnel.de>
Message-ID: <e7ba66e40904171019u6e3b48f7tc8e8d3535a9d9d87@mail.gmail.com>

On Fri, Apr 17, 2009 at 1:52 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
>
> BTW, in Py3, PyNumber_Int() was replaced in favour of PyNumber_Long(). That
> makes it a bit funny, the C layer uses "long", whereas the Python layer
> uses "int".
>

Of course... They should have to remove nb_int, and mantain nb_long...
I remember there were some recent discussions about this
inconsistencies on Python-Dev...

>
> I don't see a major use case either, but calling nb_int when we convert to
> a C-int or a smaller type might still make sense in Py2.

Well, my implementation first check if nb_long exists, if not, it
checks nb_int. So all this work just fine in Py2.

> Note that Py2 allows __int__() to return a PyLong if the value doesn't fit
> into a PyInt, while __long__() must return a PyLong:
>

Then I would vote for getting rid of "nb_long" at all, and always fill
"nb_int" with these rule:

1) if "__int__()" is implemented, always use it and ignore
"__long__()" if present, then fill the "nb_int" slot in Py2/Py3, and
set "nb_long"/"nb_reserved" to NULL.

2) if only "__long__()" is implemented, then fill the "nb_int" slot in
Py2/Py3, and set "nb_long"/"nb_reserved" to NULL.

BTW, filling only the "nb_long" slot has issues, see for example the
implementation "PyNumber_Check()" in Python 2. A  cdef class only
implementing "__long__()" will likely fail that check, right?


What do you think?


> http://docs.python.org/c-api/number.html#PyNumber_Int
>
> """
> PyObject* PyNumber_Int(PyObject *o)
> ? ?Return value: New reference.
>
> ? ?Returns the o converted to an integer object on success, or NULL on
> failure. If the argument is outside the integer range a long object will be
> returned instead. This is the equivalent of the Python expression int(o).
>
> PyObject* PyNumber_Long(PyObject *o)
> ? ?Return value: New reference.
>
> ? ?Returns the o converted to a long integer object on success, or NULL on
> failure. This is the equivalent of the Python expression long(o).
> """
>
> Stefan
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From robertwb at math.washington.edu  Fri Apr 17 19:56:14 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 17 Apr 2009 10:56:14 -0700
Subject: [Cython] make __pyx_modulename have static storage
In-Reply-To: <df4cd3fdb1630b6f453aa61b0dac1992.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
References: <e7ba66e40904161449x1dcc7d41t20e3dedb823a10b4@mail.gmail.com>
	<49E815FA.1030504@behnel.de>
	<e7ba66e40904170633g2fa55eb3rd6392ac2725282ba@mail.gmail.com>
	<df4cd3fdb1630b6f453aa61b0dac1992.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <665B9775-8C01-4EBA-9A08-31021C3E8259@math.washington.edu>

On Apr 17, 2009, at 7:49 AM, Stefan Behnel wrote:

> Lisandro Dalcin wrote:
>> Sorry, I did not understad you comment, perhaps I was not clear
>> enough... This is a global var, not one loca to function, then I'm
>> asking to emit the C code like this:
>>
>> static const char *__pyx_modulename = "mpi4py.MPI";
>
> Yes, that's what I meant. It's not supposed to be used anywhere  
> outside of
> the module, so "static" is correct here. I assume that Robert added  
> the
> constant because utility functions are not currently module specific.

Yep, IIRC, I needed to provide the module name in the warning. The  
above should be totally fine.

- Robert



From robertwb at math.washington.edu  Fri Apr 17 20:01:00 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 17 Apr 2009 11:01:00 -0700
Subject: [Cython] cannot compile derived class? (cython 0.11.1)
In-Reply-To: <1c34ba170904170354j576ba8a8i58e1ed246d3ecc9b@mail.gmail.com>
References: <1c34ba170904170354j576ba8a8i58e1ed246d3ecc9b@mail.gmail.com>
Message-ID: <991E4AA8-4F7C-42DB-B070-F9ECC1B5A867@math.washington.edu>

On Apr 17, 2009, at 3:54 AM, Riccardo Murri wrote:

> Hello,
>
> I cannot get Cython 0.11.1 to compile the following simple example::
>
>   $ cat a.pxd
>   cdef class A(object)
>   cdef class B(A)
>
>   $ cat a.pyx
>   cdef class A(object):
>        pass
>
>   cdef class B(A):
>        pass
>
>   $ cat c.pyx
>   from a cimport B
>
>   cdef class C(B):
>        pass

[...]

> Is this a bug or am I doing something wrong?

No, I think this is a bug. However, does it work if you write

   cdef class A(object):
        pass

   cdef class B(A):
        pass

in your .pxd?

- Robert


From dalcinl at gmail.com  Fri Apr 17 21:04:28 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Fri, 17 Apr 2009 16:04:28 -0300
Subject: [Cython] make __pyx_modulename have static storage
In-Reply-To: <665B9775-8C01-4EBA-9A08-31021C3E8259@math.washington.edu>
References: <e7ba66e40904161449x1dcc7d41t20e3dedb823a10b4@mail.gmail.com>
	<49E815FA.1030504@behnel.de>
	<e7ba66e40904170633g2fa55eb3rd6392ac2725282ba@mail.gmail.com>
	<df4cd3fdb1630b6f453aa61b0dac1992.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<665B9775-8C01-4EBA-9A08-31021C3E8259@math.washington.edu>
Message-ID: <e7ba66e40904171204x797720fdtd8aa712ca740be31@mail.gmail.com>

Well, if we add "static" we start getting unused GCC warns .. So I've
decided to change this to a #define'd macro... Just in case, I've not
removed the definition in Naming.py.

On Fri, Apr 17, 2009 at 2:56 PM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Apr 17, 2009, at 7:49 AM, Stefan Behnel wrote:
>
>> Lisandro Dalcin wrote:
>>> Sorry, I did not understad you comment, perhaps I was not clear
>>> enough... This is a global var, not one loca to function, then I'm
>>> asking to emit the C code like this:
>>>
>>> static const char *__pyx_modulename = "mpi4py.MPI";
>>
>> Yes, that's what I meant. It's not supposed to be used anywhere
>> outside of
>> the module, so "static" is correct here. I assume that Robert added
>> the
>> constant because utility functions are not currently module specific.
>
> Yep, IIRC, I needed to provide the module name in the warning. The
> above should be totally fine.
>
> - Robert
>
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From stefan_ml at behnel.de  Fri Apr 17 21:14:07 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 17 Apr 2009 21:14:07 +0200
Subject: [Cython] incorrect C code generated on method assignment
 (cython 0.11.1)
In-Reply-To: <e7ba66e40904171019u6e3b48f7tc8e8d3535a9d9d87@mail.gmail.com>
References: <1c34ba170904170550jc9a800cue5c1dad48d30e256@mail.gmail.com>	<09c458c88760dc5ea641d522c6ec939e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>	<e7ba66e40904170701h4c6af104m7f6d0ab9328c83cf@mail.gmail.com>	<63f6aed263dda2ed3b6d431bad1e93c6.squirrel@groupware.dvs.informatik.tu-darmstadt.de>	<e7ba66e40904170904w54c3fe86j29dc2f08cac7195a@mail.gmail.com>	<49E8B3E0.2050403@behnel.de>
	<e7ba66e40904171019u6e3b48f7tc8e8d3535a9d9d87@mail.gmail.com>
Message-ID: <49E8D4FF.1050003@behnel.de>


Lisandro Dalcin wrote:
> On Fri, Apr 17, 2009 at 1:52 PM, Stefan Behnel wrote:
>> I don't see a major use case either, but calling nb_int when we convert to
>> a C-int or a smaller type might still make sense in Py2.
> 
> Well, my implementation first check if nb_long exists, if not, it
> checks nb_int. So all this work just fine in Py2.

What I meant was: if we convert to a type that fits into a C int, we should
prefer nb_int over nb_long, not the other way round.


>> Note that Py2 allows __int__() to return a PyLong if the value doesn't fit
>> into a PyInt, while __long__() must return a PyLong:
> 
> Then I would vote for getting rid of "nb_long" at all, and always fill
> "nb_int" with these rule:
> 
> 1) if "__int__()" is implemented, always use it and ignore
> "__long__()" if present, then fill the "nb_int" slot in Py2/Py3, and
> set "nb_long"/"nb_reserved" to NULL.
> 
> 2) if only "__long__()" is implemented, then fill the "nb_int" slot in
> Py2/Py3, and set "nb_long"/"nb_reserved" to NULL.

I think you say this because it follows the Py3 rules. I'm fine with that,
but I do think that we should use both methods in Py2 if defined. So 1)
should only apply to Py3.

Given that there is no obvious use case for __long__, I also think that
defining it should emit a deprecation warning in general, not only in case 2)

Stefan


From riccardo.murri at gmail.com  Fri Apr 17 21:17:18 2009
From: riccardo.murri at gmail.com (Riccardo Murri)
Date: Fri, 17 Apr 2009 21:17:18 +0200
Subject: [Cython] cannot compile derived class? (cython 0.11.1)
In-Reply-To: <991E4AA8-4F7C-42DB-B070-F9ECC1B5A867@math.washington.edu>
References: <1c34ba170904170354j576ba8a8i58e1ed246d3ecc9b@mail.gmail.com>
	<991E4AA8-4F7C-42DB-B070-F9ECC1B5A867@math.washington.edu>
Message-ID: <1c34ba170904171217m1f6b193ckc5065667c70d409b@mail.gmail.com>

On Fri, Apr 17, 2009 at 8:01 PM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> No, I think this is a bug. However, does it work if you write
>
> ? cdef class A(object):
> ? ? ? ?pass
>
> ? cdef class B(A):
> ? ? ? ?pass
>
> in your .pxd?
>

Yes, it does.

Riccardo

-- 
Riccardo Murri, via Ross?e 17, 6987 Caslano (CH)

From dalcinl at gmail.com  Fri Apr 17 22:36:03 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Fri, 17 Apr 2009 17:36:03 -0300
Subject: [Cython] incorrect C code generated on method assignment
	(cython 0.11.1)
In-Reply-To: <49E8D4FF.1050003@behnel.de>
References: <1c34ba170904170550jc9a800cue5c1dad48d30e256@mail.gmail.com>
	<09c458c88760dc5ea641d522c6ec939e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<e7ba66e40904170701h4c6af104m7f6d0ab9328c83cf@mail.gmail.com>
	<63f6aed263dda2ed3b6d431bad1e93c6.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<e7ba66e40904170904w54c3fe86j29dc2f08cac7195a@mail.gmail.com>
	<49E8B3E0.2050403@behnel.de>
	<e7ba66e40904171019u6e3b48f7tc8e8d3535a9d9d87@mail.gmail.com>
	<49E8D4FF.1050003@behnel.de>
Message-ID: <e7ba66e40904171336q5cb44517jf3ba8c57f06b5378@mail.gmail.com>

On Fri, Apr 17, 2009 at 4:14 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
>
> Lisandro Dalcin wrote:
>> On Fri, Apr 17, 2009 at 1:52 PM, Stefan Behnel wrote:
>>> I don't see a major use case either, but calling nb_int when we convert to
>>> a C-int or a smaller type might still make sense in Py2.
>>
>> Well, my implementation first check if nb_long exists, if not, it
>> checks nb_int. So all this work just fine in Py2.
>
> What I meant was: if we convert to a type that fits into a C int, we should
> prefer nb_int over nb_long, not the other way round.
>

Well, to do it the right way, nb_int it should take precedence if the
type fits in a C int, or if it fits in a "signed long" but not
"unsigned long". It seems that I was lazy, though I've actually tried
to maintain the implementation of _Pyx_PyNumber_Int() simple and make
it take a single argument (like the one in core CPython). I'll try to
review my work and provide a patch, though I'm not sure if further
work will pay that much... For core CPython types, the only case where
all this can have some impact in for float -> int conversion.

>
>>> Note that Py2 allows __int__() to return a PyLong if the value doesn't fit
>>> into a PyInt, while __long__() must return a PyLong:
>>
>> Then I would vote for getting rid of "nb_long" at all, and always fill
>> "nb_int" with these rule:
>>
>> 1) if "__int__()" is implemented, always use it and ignore
>> "__long__()" if present, then fill the "nb_int" slot in Py2/Py3, and
>> set "nb_long"/"nb_reserved" to NULL.
>>
>> 2) if only "__long__()" is implemented, then fill the "nb_int" slot in
>> Py2/Py3, and set "nb_long"/"nb_reserved" to NULL.
>
> I think you say this because it follows the Py3 rules. I'm fine with that,
> but I do think that we should use both methods in Py2 if defined. So 1)
> should only apply to Py3.
>

As I've started to work on a patch, let put it clear case by case...

Python 2
----------
a) if only __int__(), fill as "nb_int = __int__" and "nb_long = NULL"
b) if only __long__(), fill as "nb_int = __long__" and "nb_long = NULL"
c) if __int__() AND __long__(): fill as "nb_int = __int__" and
"nb_long = __long__"

Python 3
----------
a) if only __int__(), fill as "nb_int = __int__" and "nb_reserved = NULL"
b) if only __long__(), fill as "nb_int = __long__" and "nb_reserved = NULL"
c) if __int__() AND __long__(): fill as "nb_int = __int__" and
"nb_reserved = NULL"

So only cases (c) makes a difference for Py2/Py3...

>
> Given that there is no obvious use case for __long__, I also think that
> defining it should emit a deprecation warning in general, not only in case 2)
>

I'm fine with that, but I'll likely ask you to do the work :-)



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From kwmsmith at gmail.com  Fri Apr 17 22:45:00 2009
From: kwmsmith at gmail.com (Kurt Smith)
Date: Fri, 17 Apr 2009 15:45:00 -0500
Subject: [Cython] endianness bug fix in numpy_test.pyx
Message-ID: <c266a7930904171345s7fe30968g1fa469dba070ee7d@mail.gmail.com>

The current numpy_test.pyx file fails for PowerPC macs due to
endianness issues in the dtype.  This is a small fix to make it work
(and make all tests pass on my machine).  It also adds an explicit
big-endian test to the doctest.

diff -r fc73225aaea1 tests/run/numpy_test.pyx
--- a/tests/run/numpy_test.pyx	Fri Apr 17 09:11:16 2009 +0200
+++ b/tests/run/numpy_test.pyx	Fri Apr 17 15:43:11 2009 -0500
@@ -132,13 +132,20 @@
     >>> test_recordarray()

     >>> test_nested_dtypes(np.zeros((3,), dtype=np.dtype([\
-            ('a', np.dtype('i,i')),\
-            ('b', np.dtype('i,i'))\
+            ('a', np.dtype('<i,<i')),\
+            ('b', np.dtype('<i,<i'))\
         ])))
     array([((0, 0), (0, 0)), ((1, 2), (1, 4)), ((1, 2), (1, 4))],
           dtype=[('a', [('f0', '<i4'), ('f1', '<i4')]), ('b', [('f0',
'<i4'), ('f1', '<i4')])])

     >>> test_nested_dtypes(np.zeros((3,), dtype=np.dtype([\
+            ('a', np.dtype('>i,>i')),\
+            ('b', np.dtype('>i,>i'))\
+        ])))
+    array([((0, 0), (0, 0)), ((1, 2), (1, 4)), ((1, 2), (1, 4))],
+          dtype=[('a', [('f0', '>i4'), ('f1', '>i4')]), ('b', [('f0',
'>i4'), ('f1', '>i4')])])
+
+    >>> test_nested_dtypes(np.zeros((3,), dtype=np.dtype([\
             ('a', np.dtype('i,f')),\
             ('b', np.dtype('i,i'))\
         ])))
@@ -297,6 +304,10 @@
     arr[1].b.x = arr[0].a.y + 1
     arr[1].b.y = 4
     arr[2] = arr[1]
+    if not arr.dtype.isnative:
+        # required to get the endianness
+        # indicator (< or >) correct in output.
+        arr.byteswap(True)
     return arr

 def test_bad_nested_dtypes():

From robertwb at math.washington.edu  Fri Apr 17 23:48:28 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 17 Apr 2009 14:48:28 -0700
Subject: [Cython] endianness bug fix in numpy_test.pyx
In-Reply-To: <c266a7930904171345s7fe30968g1fa469dba070ee7d@mail.gmail.com>
References: <c266a7930904171345s7fe30968g1fa469dba070ee7d@mail.gmail.com>
Message-ID: <1D04E878-D6F2-4279-98E3-B727916FF58D@math.washington.edu>

On Apr 17, 2009, at 1:45 PM, Kurt Smith wrote:

> The current numpy_test.pyx file fails for PowerPC macs due to
> endianness issues in the dtype.  This is a small fix to make it work
> (and make all tests pass on my machine).  It also adds an explicit
> big-endian test to the doctest.

Hmm... This looks like a bug in the code itself, not just the test.

>
> diff -r fc73225aaea1 tests/run/numpy_test.pyx
> --- a/tests/run/numpy_test.pyx	Fri Apr 17 09:11:16 2009 +0200
> +++ b/tests/run/numpy_test.pyx	Fri Apr 17 15:43:11 2009 -0500
> @@ -132,13 +132,20 @@
>>>> test_recordarray()
>
>>>> test_nested_dtypes(np.zeros((3,), dtype=np.dtype([\
> -            ('a', np.dtype('i,i')),\
> -            ('b', np.dtype('i,i'))\
> +            ('a', np.dtype('<i,<i')),\
> +            ('b', np.dtype('<i,<i'))\
>          ])))
>      array([((0, 0), (0, 0)), ((1, 2), (1, 4)), ((1, 2), (1, 4))],
>            dtype=[('a', [('f0', '<i4'), ('f1', '<i4')]), ('b', [('f0',
> '<i4'), ('f1', '<i4')])])
>
>>>> test_nested_dtypes(np.zeros((3,), dtype=np.dtype([\
> +            ('a', np.dtype('>i,>i')),\
> +            ('b', np.dtype('>i,>i'))\
> +        ])))
> +    array([((0, 0), (0, 0)), ((1, 2), (1, 4)), ((1, 2), (1, 4))],
> +          dtype=[('a', [('f0', '>i4'), ('f1', '>i4')]), ('b', [('f0',
> '>i4'), ('f1', '>i4')])])
> +
> +    >>> test_nested_dtypes(np.zeros((3,), dtype=np.dtype([\
>              ('a', np.dtype('i,f')),\
>              ('b', np.dtype('i,i'))\
>          ])))
> @@ -297,6 +304,10 @@
>      arr[1].b.x = arr[0].a.y + 1
>      arr[1].b.y = 4
>      arr[2] = arr[1]
> +    if not arr.dtype.isnative:
> +        # required to get the endianness
> +        # indicator (< or >) correct in output.
> +        arr.byteswap(True)
>      return arr
>
>  def test_bad_nested_dtypes():
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From dalcinl at gmail.com  Sat Apr 18 00:12:19 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Fri, 17 Apr 2009 19:12:19 -0300
Subject: [Cython] setting cdef members of cdef classes in nogil functions
Message-ID: <e7ba66e40904171512w5769dd28ha3d3aed52c862ed2@mail.gmail.com>

I'm working on providing support in mpi4py for complex types, and then
I did more or les this:

cdef class Datatype: # cdef class
    MPI_Datatype ob_mpi

cdef Datatype COMPLEX = Datatype() # cdef global

cdef void setup_complex_types() nogil: # nogil function
   cdef MPI_Datatype newtype
   create_complex_type(&newtype) # implemented elsewhere
   COMPLEX.ob_mpi = newtype # Look HERE  !

As you see, I'm touching a C field of the object structure inside a
"nogil" function.

 I'm very hapy it worked!! . AFAIK, there are no GIL issues in this
code. Anyway, I would like to know your opinion on this.

-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From robertwb at math.washington.edu  Sat Apr 18 02:59:01 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 17 Apr 2009 17:59:01 -0700
Subject: [Cython] setting cdef members of cdef classes in nogil functions
In-Reply-To: <e7ba66e40904171512w5769dd28ha3d3aed52c862ed2@mail.gmail.com>
References: <e7ba66e40904171512w5769dd28ha3d3aed52c862ed2@mail.gmail.com>
Message-ID: <A1498618-5FE3-41F5-B7E2-B1E8EAE9AC13@math.washington.edu>

On Apr 17, 2009, at 3:12 PM, Lisandro Dalcin wrote:

> I'm working on providing support in mpi4py for complex types, and then
> I did more or les this:
>
> cdef class Datatype: # cdef class
>     MPI_Datatype ob_mpi
>
> cdef Datatype COMPLEX = Datatype() # cdef global
>
> cdef void setup_complex_types() nogil: # nogil function
>    cdef MPI_Datatype newtype
>    create_complex_type(&newtype) # implemented elsewhere
>    COMPLEX.ob_mpi = newtype # Look HERE  !
>
> As you see, I'm touching a C field of the object structure inside a
> "nogil" function.
>
>  I'm very hapy it worked!! . AFAIK, there are no GIL issues in this
> code. Anyway, I would like to know your opinion on this.

Yes, there's no reason at all that this should need the GIL.

- Robert



From cournape at gmail.com  Sat Apr 18 07:30:42 2009
From: cournape at gmail.com (David Cournapeau)
Date: Sat, 18 Apr 2009 14:30:42 +0900
Subject: [Cython] endianness bug fix in numpy_test.pyx
In-Reply-To: <1D04E878-D6F2-4279-98E3-B727916FF58D@math.washington.edu>
References: <c266a7930904171345s7fe30968g1fa469dba070ee7d@mail.gmail.com>
	<1D04E878-D6F2-4279-98E3-B727916FF58D@math.washington.edu>
Message-ID: <5b8d13220904172230h44c1a86fg41dd4493366a3047@mail.gmail.com>

On Sat, Apr 18, 2009 at 6:48 AM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Apr 17, 2009, at 1:45 PM, Kurt Smith wrote:
>
>> The current numpy_test.pyx file fails for PowerPC macs due to
>> endianness issues in the dtype. ?This is a small fix to make it work
>> (and make all tests pass on my machine). ?It also adds an explicit
>> big-endian test to the doctest.
>
> Hmm... This looks like a bug in the code itself, not just the test.

This may be related to a python 2.6 bug on ppc:

http://bugs.python.org/issue4728

There is a workaround in numpy 1.3 to bypass the problem altogether -
but python 2.6 + ppc + numpy < 1.3 will not work,

David

From stefan_ml at behnel.de  Sat Apr 18 08:26:11 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 18 Apr 2009 08:26:11 +0200
Subject: [Cython] incorrect C code generated on method assignment
 (cython 0.11.1)
In-Reply-To: <e7ba66e40904171336q5cb44517jf3ba8c57f06b5378@mail.gmail.com>
References: <1c34ba170904170550jc9a800cue5c1dad48d30e256@mail.gmail.com>	<09c458c88760dc5ea641d522c6ec939e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>	<e7ba66e40904170701h4c6af104m7f6d0ab9328c83cf@mail.gmail.com>	<63f6aed263dda2ed3b6d431bad1e93c6.squirrel@groupware.dvs.informatik.tu-darmstadt.de>	<e7ba66e40904170904w54c3fe86j29dc2f08cac7195a@mail.gmail.com>	<49E8B3E0.2050403@behnel.de>	<e7ba66e40904171019u6e3b48f7tc8e8d3535a9d9d87@mail.gmail.com>	<49E8D4FF.1050003@behnel.de>
	<e7ba66e40904171336q5cb44517jf3ba8c57f06b5378@mail.gmail.com>
Message-ID: <49E97283.9000505@behnel.de>


Lisandro Dalcin wrote:
> Well, to do it the right way, nb_int it should take precedence if the
> type fits in a C int, or if it fits in a "signed long" but not
> "unsigned long". It seems that I was lazy, though I've actually tried
> to maintain the implementation of _Pyx_PyNumber_Int() simple and make
> it take a single argument (like the one in core CPython). I'll try to
> review my work and provide a patch

Thanks!


> though I'm not sure if further
> work will pay that much... For core CPython types, the only case where
> all this can have some impact in for float -> int conversion.

... plus, all of this will go away in Py3 anyway. So, don't invest too much
time, but if you can get it right, I think it would make things more robust
and consistent.


> As I've started to work on a patch, let put it clear case by case...
> 
> Python 2
> ----------
> a) if only __int__(), fill as "nb_int = __int__" and "nb_long = NULL"
> b) if only __long__(), fill as "nb_int = __long__" and "nb_long = NULL"
> c) if __int__() AND __long__(): fill as "nb_int = __int__" and
> "nb_long = __long__"
> 
> Python 3
> ----------
> a) if only __int__(), fill as "nb_int = __int__" and "nb_reserved = NULL"
> b) if only __long__(), fill as "nb_int = __long__" and "nb_reserved = NULL"

Fine so far.


> c) if __int__() AND __long__(): fill as "nb_int = __int__" and
> "nb_reserved = NULL"

I'm still unsure whether Py3 should prefer __long__ or __int__. I guess it
would be more consistent to use __int__ for nb_int in both Py2 and Py3, but
since nb_int *must* return a PyLong in Py3, it might be cleaner (and
easier?) to use __long__ instead.

If we use __int__ in Py3, I think it would make sense to raise a warning
only for case b). That would help users to get their code right. That might
be the best solution after all: use __int__ when available, support
__long__ only in Py2, and raise a warning when someone forgets to define
__int__ next to it.

Stefan


From robertwb at math.washington.edu  Sat Apr 18 10:03:48 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 18 Apr 2009 01:03:48 -0700
Subject: [Cython] cdef list variable raises exception on SetItem
In-Reply-To: <c266a7930904162221g4bfbabd3g499965591476278a@mail.gmail.com>
References: <c266a7930904162028q23e5156do975e1bf8ff17f596@mail.gmail.com>
	<c266a7930904162154q7da19f8bi1e0bec1a853d86ec@mail.gmail.com>
	<c266a7930904162221g4bfbabd3g499965591476278a@mail.gmail.com>
Message-ID: <32A99FF0-83AB-44E6-BC64-BF7F082CDE7E@math.washington.edu>

On Apr 16, 2009, at 10:21 PM, Kurt Smith wrote:

> On Thu, Apr 16, 2009 at 11:54 PM, Kurt Smith <kwmsmith at gmail.com>  
> wrote:
>
>> See
>>
>> http://trac.cython.org/cython_trac/ticket/284

Fixed. Thanks for tracking this down.

> Here's a testcase.
>
> diff -r 6fc636aebe32 -r 1097ad309db6 tests/run/cdef_setitem_T284.pyx
> --- /dev/null   Thu Jan 01 00:00:00 1970 +0000
> +++ b/tests/run/cdef_setitem_T284.pyx   Fri Apr 17 00:15:48 2009 -0500

BTW, do you know how to output mercurial diffs?

- Robert

From dagss at student.matnat.uio.no  Sat Apr 18 10:22:30 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Sat, 18 Apr 2009 10:22:30 +0200
Subject: [Cython] endianness bug fix in numpy_test.pyx
In-Reply-To: <1D04E878-D6F2-4279-98E3-B727916FF58D@math.washington.edu>
References: <c266a7930904171345s7fe30968g1fa469dba070ee7d@mail.gmail.com>
	<1D04E878-D6F2-4279-98E3-B727916FF58D@math.washington.edu>
Message-ID: <49E98DC6.2040803@student.matnat.uio.no>

Robert Bradshaw wrote:
> On Apr 17, 2009, at 1:45 PM, Kurt Smith wrote:
> 
>> The current numpy_test.pyx file fails for PowerPC macs due to
>> endianness issues in the dtype.  This is a small fix to make it work
>> (and make all tests pass on my machine).  It also adds an explicit
>> big-endian test to the doctest.
> 
> Hmm... This looks like a bug in the code itself, not just the test.

Thanks for bringing this up Kurt.

Using typed buffers in Cython only support the native endianness, so 
what one should get is an exception on one of the platforms.

It seems that the problem is that numpy.pxd does not include endianness 
in the formatting string; in fact it looks like this wasn't considered 
at all.

http://trac.cython.org/cython_trac/ticket/285

When it comes to the testcase, one should have two tests:

a) With native endianness, which should succeed. I think the best way is 
to modify the result:

x = ...
print(repr(x).replace(">","<")) # then follows little endian result

b) With non-native endianness, which should result in an exception from 
buffer acquisition.


> 
>> diff -r fc73225aaea1 tests/run/numpy_test.pyx
>> --- a/tests/run/numpy_test.pyx	Fri Apr 17 09:11:16 2009 +0200
>> +++ b/tests/run/numpy_test.pyx	Fri Apr 17 15:43:11 2009 -0500
>> @@ -132,13 +132,20 @@
>>>>> test_recordarray()
>>>>> test_nested_dtypes(np.zeros((3,), dtype=np.dtype([\
>> -            ('a', np.dtype('i,i')),\
>> -            ('b', np.dtype('i,i'))\
>> +            ('a', np.dtype('<i,<i')),\
>> +            ('b', np.dtype('<i,<i'))\
>>          ])))
>>      array([((0, 0), (0, 0)), ((1, 2), (1, 4)), ((1, 2), (1, 4))],
>>            dtype=[('a', [('f0', '<i4'), ('f1', '<i4')]), ('b', [('f0',
>> '<i4'), ('f1', '<i4')])])
>>
>>>>> test_nested_dtypes(np.zeros((3,), dtype=np.dtype([\
>> +            ('a', np.dtype('>i,>i')),\
>> +            ('b', np.dtype('>i,>i'))\
>> +        ])))
>> +    array([((0, 0), (0, 0)), ((1, 2), (1, 4)), ((1, 2), (1, 4))],
>> +          dtype=[('a', [('f0', '>i4'), ('f1', '>i4')]), ('b', [('f0',
>> '>i4'), ('f1', '>i4')])])
>> +
>> +    >>> test_nested_dtypes(np.zeros((3,), dtype=np.dtype([\
>>              ('a', np.dtype('i,f')),\
>>              ('b', np.dtype('i,i'))\
>>          ])))
>> @@ -297,6 +304,10 @@
>>      arr[1].b.x = arr[0].a.y + 1
>>      arr[1].b.y = 4
>>      arr[2] = arr[1]
>> +    if not arr.dtype.isnative:
>> +        # required to get the endianness
>> +        # indicator (< or >) correct in output.
>> +        arr.byteswap(True)
>>      return arr
>>
>>  def test_bad_nested_dtypes():
>> _______________________________________________
>> Cython-dev mailing list
>> Cython-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/cython-dev
> 
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


-- 
Dag Sverre

From robertwb at math.washington.edu  Sat Apr 18 10:15:29 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 18 Apr 2009 01:15:29 -0700
Subject: [Cython] Py_New still good way to create classes?
In-Reply-To: <e183a99d0904151931l7b3e5997id566dbb511b39cda@mail.gmail.com>
References: <4db580fd0904051226u64d9dfe5n73674e3d258ba677@mail.gmail.com>
	<1228653C-157E-48C6-A305-E9D2CFE27AB0@math.washington.edu>
	<e183a99d0904151039r7e33bf7aj97044f4fe4285844@mail.gmail.com>
	<49E626A8.4000608@behnel.de>
	<e183a99d0904151931l7b3e5997id566dbb511b39cda@mail.gmail.com>
Message-ID: <C270C8B3-7EE2-4FA1-A59A-6E5D1956D2F7@math.washington.edu>

On Apr 15, 2009, at 7:31 PM, Brent Pedersen wrote:

> On Wed, Apr 15, 2009 at 11:25 AM, Stefan Behnel  
> <stefan_ml at behnel.de> wrote:
>> Hi,
>>
>> thanks for sharing that.
>>
>> Brent Pedersen wrote:
>>> assuming i haven't done anything stupid
>>
>> No, nothing stupid, but something that can reduce the  
>> comparability of the
>> timings. You are creating a 1000000 item list on each benchmark,  
>> using a
>> call to range() in some cases and a list comprehension in others.
>>
>> It's usually better to move initialisations out of the timings,  
>> e.g. by
>> creating a large range() object once and re-using it. That reduces  
>> the
>> impact of unrelated operations on the absolute numbers.
>>
>
> ah, i see, updated that. fixing that makes the python constructor look
> even slower.
> now it assumes that creating a list comprehension without assgning to
> a variable is
> the same as calling a function that returns an array--also without  
> assigning.
>
> here are the new timings:
>
> PY_NEW on Cython class: 1.137
> __init__ on Python class: 28.468
> __init__ on Python class with slots: 9.936
> batch PY_NEW total: 0.821 , interval only: 0.363
> batch __init__ on Cython class total 0.975 , interval_only: 0.524
> __init__ on Cython class 1.154
>
> so for this case using PY_NEW macro actual doesnt improve speed that
> much over a cdef'ed class.
> especially if using a "batch" method is applicable (as it is for my  
> use-case).

Here's some more data points, specific to Sage (Python 2.5, Cython 0.11)

-------------------

%cython

from sage.rings.integer cimport Integer

cdef class A:
     pass

def time_py_new(long N):
     cdef long i
     for i from 0 <= i < N:
         z = PY_NEW(Integer)

def time_py_init(long N):
     cdef long i
     for i from 0 <= i < N:
         z = Integer()

def time_py_new_A(long N):
     cdef long i
     for i from 0 <= i < N:
         z = PY_NEW(A)

def time_py_init_A(long N):
     cdef long i
     for i from 0 <= i < N:
         z = A()

--------------------

sage: time time_py_init(10**7)
Time: CPU 0.67 s, Wall: 0.68 s
sage: time time_py_new(10**7)
Time: CPU 0.16 s, Wall: 0.17 s
sage: time time_py_init_A(10**7)
Time: CPU 0.66 s, Wall: 0.67 s
sage: time time_py_new_A(10**7)
Time: CPU 0.47 s, Wall: 0.48 s

Note that I have an empty constructor in both cases. In summary,  
Integer has a custom tp_new slot with a pool to avoid allocation/ 
deallocation overhead, and PY_NEW here saves quite a bit. The 30%  
speed difference for a standard cdef class seems to hold about right-- 
probably two thirds of the time is probably in allocating memory from  
the heap (and also releasing it, in my test).

- Robert


From dagss at student.matnat.uio.no  Sat Apr 18 10:39:51 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Sat, 18 Apr 2009 10:39:51 +0200
Subject: [Cython] endianness bug fix in numpy_test.pyx
In-Reply-To: <c266a7930904171345s7fe30968g1fa469dba070ee7d@mail.gmail.com>
References: <c266a7930904171345s7fe30968g1fa469dba070ee7d@mail.gmail.com>
Message-ID: <49E991D7.5000809@student.matnat.uio.no>

Kurt Smith wrote:
> The current numpy_test.pyx file fails for PowerPC macs due to
> endianness issues in the dtype.  This is a small fix to make it work
> (and make all tests pass on my machine).  It also adds an explicit
> big-endian test to the doctest.
> 
> diff -r fc73225aaea1 tests/run/numpy_test.pyx
> --- a/tests/run/numpy_test.pyx	Fri Apr 17 09:11:16 2009 +0200
> +++ b/tests/run/numpy_test.pyx	Fri Apr 17 15:43:11 2009 -0500
> @@ -132,13 +132,20 @@
>      >>> test_recordarray()
> 
>      >>> test_nested_dtypes(np.zeros((3,), dtype=np.dtype([\
> -            ('a', np.dtype('i,i')),\
> -            ('b', np.dtype('i,i'))\
> +            ('a', np.dtype('<i,<i')),\
> +            ('b', np.dtype('<i,<i'))\
>          ])))
>      array([((0, 0), (0, 0)), ((1, 2), (1, 4)), ((1, 2), (1, 4))],
>            dtype=[('a', [('f0', '<i4'), ('f1', '<i4')]), ('b', [('f0',
> '<i4'), ('f1', '<i4')])])

Any ideas on why this particular case actually passes on your machine? 
The Cython code alwayws works with big-endian on your machine, so I 
wouldn't expect to see ((1, 2), (1, 4)) there, but the byteswapped 
versions of those numbers...

-- 
Dag Sverre

From dagss at student.matnat.uio.no  Sat Apr 18 14:57:05 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Sat, 18 Apr 2009 14:57:05 +0200
Subject: [Cython] endianness bug fix in numpy_test.pyx
In-Reply-To: <49E991D7.5000809@student.matnat.uio.no>
References: <c266a7930904171345s7fe30968g1fa469dba070ee7d@mail.gmail.com>
	<49E991D7.5000809@student.matnat.uio.no>
Message-ID: <49E9CE21.8040105@student.matnat.uio.no>

Dag Sverre Seljebotn wrote:
> Kurt Smith wrote:
>> The current numpy_test.pyx file fails for PowerPC macs due to
>> endianness issues in the dtype.  This is a small fix to make it work
>> (and make all tests pass on my machine).  It also adds an explicit
>> big-endian test to the doctest.
>>
>> diff -r fc73225aaea1 tests/run/numpy_test.pyx
>> --- a/tests/run/numpy_test.pyx	Fri Apr 17 09:11:16 2009 +0200
>> +++ b/tests/run/numpy_test.pyx	Fri Apr 17 15:43:11 2009 -0500
>> @@ -132,13 +132,20 @@
>>      >>> test_recordarray()
>>
>>      >>> test_nested_dtypes(np.zeros((3,), dtype=np.dtype([\
>> -            ('a', np.dtype('i,i')),\
>> -            ('b', np.dtype('i,i'))\
>> +            ('a', np.dtype('<i,<i')),\
>> +            ('b', np.dtype('<i,<i'))\
>>          ])))
>>      array([((0, 0), (0, 0)), ((1, 2), (1, 4)), ((1, 2), (1, 4))],
>>            dtype=[('a', [('f0', '<i4'), ('f1', '<i4')]), ('b', [('f0',
>> '<i4'), ('f1', '<i4')])])
> 
> Any ideas on why this particular case actually passes on your machine? 
> The Cython code alwayws works with big-endian on your machine, so I 
> wouldn't expect to see ((1, 2), (1, 4)) there, but the byteswapped 
> versions of those numbers...
> 

Ahh right; the only value read from the input array (in the test 
function) is zero, which is the same in both endians, and you added 
explicit byteswap on the output.

-- 
Dag Sverre

From m.gysel at gmail.com  Sat Apr 18 15:15:48 2009
From: m.gysel at gmail.com (Martin Gysel)
Date: Sat, 18 Apr 2009 15:15:48 +0200
Subject: [Cython] subclassing builtin
In-Reply-To: <e7ba66e40904161443n34129729t2ef0ffce7c9c62f6@mail.gmail.com>
References: <49E7A002.50002@gmail.com>
	<e7ba66e40904161443n34129729t2ef0ffce7c9c62f6@mail.gmail.com>
Message-ID: <49E9D284.10905@gmail.com>

Lisandro Dalcin schrieb:
> On Thu, Apr 16, 2009 at 6:15 PM, Martin Gysel <m.gysel at gmail.com> wrote:
>   
>> cdef extern from "dictobject.h":
>>    ctypedef class __builtin__.dict [object PyDictObject]
>>     
> I woud just use "Python.h" ... and add ": pass" at the end (perhaps
> add a newline after ":")
>   
thx for the hint with the ': pass'

cheers
martin


From kwmsmith at gmail.com  Sat Apr 18 19:11:35 2009
From: kwmsmith at gmail.com (Kurt Smith)
Date: Sat, 18 Apr 2009 12:11:35 -0500
Subject: [Cython] endianness bug fix in numpy_test.pyx
In-Reply-To: <49E9CE21.8040105@student.matnat.uio.no>
References: <c266a7930904171345s7fe30968g1fa469dba070ee7d@mail.gmail.com>
	<49E991D7.5000809@student.matnat.uio.no>
	<49E9CE21.8040105@student.matnat.uio.no>
Message-ID: <c266a7930904181011t32fce0b9u7495655efbf48059@mail.gmail.com>

On Sat, Apr 18, 2009 at 7:57 AM, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:
> Dag Sverre Seljebotn wrote:
>> Kurt Smith wrote:
>>> The current numpy_test.pyx file fails for PowerPC macs due to
>>> endianness issues in the dtype. ?This is a small fix to make it work
>>> (and make all tests pass on my machine). ?It also adds an explicit
>>> big-endian test to the doctest.
>>>
>>> diff -r fc73225aaea1 tests/run/numpy_test.pyx
>>> --- a/tests/run/numpy_test.pyx ? ? ? Fri Apr 17 09:11:16 2009 +0200
>>> +++ b/tests/run/numpy_test.pyx ? ? ? Fri Apr 17 15:43:11 2009 -0500
>>> @@ -132,13 +132,20 @@
>>> ? ? ?>>> test_recordarray()
>>>
>>> ? ? ?>>> test_nested_dtypes(np.zeros((3,), dtype=np.dtype([\
>>> - ? ? ? ? ? ?('a', np.dtype('i,i')),\
>>> - ? ? ? ? ? ?('b', np.dtype('i,i'))\
>>> + ? ? ? ? ? ?('a', np.dtype('<i,<i')),\
>>> + ? ? ? ? ? ?('b', np.dtype('<i,<i'))\
>>> ? ? ? ? ?])))
>>> ? ? ?array([((0, 0), (0, 0)), ((1, 2), (1, 4)), ((1, 2), (1, 4))],
>>> ? ? ? ? ? ?dtype=[('a', [('f0', '<i4'), ('f1', '<i4')]), ('b', [('f0',
>>> '<i4'), ('f1', '<i4')])])
>>
>> Any ideas on why this particular case actually passes on your machine?
>> The Cython code alwayws works with big-endian on your machine, so I
>> wouldn't expect to see ((1, 2), (1, 4)) there, but the byteswapped
>> versions of those numbers...
>>
>
> Ahh right; the only value read from the input array (in the test
> function) is zero, which is the same in both endians, and you added
> explicit byteswap on the output.

Sorry -- can't devote the time to address these questions right now
but I'll get to them as soon as I can (Monday?).

Thanks,

Kurt

From dagss at student.matnat.uio.no  Sat Apr 18 20:36:10 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Sat, 18 Apr 2009 20:36:10 +0200
Subject: [Cython] endianness bug fix in numpy_test.pyx
In-Reply-To: <c266a7930904181011t32fce0b9u7495655efbf48059@mail.gmail.com>
References: <c266a7930904171345s7fe30968g1fa469dba070ee7d@mail.gmail.com>	<49E991D7.5000809@student.matnat.uio.no>	<49E9CE21.8040105@student.matnat.uio.no>
	<c266a7930904181011t32fce0b9u7495655efbf48059@mail.gmail.com>
Message-ID: <49EA1D9A.1040103@student.matnat.uio.no>

Kurt Smith wrote:
> On Sat, Apr 18, 2009 at 7:57 AM, Dag Sverre Seljebotn
> <dagss at student.matnat.uio.no> wrote:
>> Dag Sverre Seljebotn wrote:
>>> Kurt Smith wrote:
>>>> The current numpy_test.pyx file fails for PowerPC macs due to
>>>> endianness issues in the dtype.  This is a small fix to make it work
>>>> (and make all tests pass on my machine).  It also adds an explicit
>>>> big-endian test to the doctest.
>>>>
>>>> diff -r fc73225aaea1 tests/run/numpy_test.pyx
>>>> --- a/tests/run/numpy_test.pyx       Fri Apr 17 09:11:16 2009 +0200
>>>> +++ b/tests/run/numpy_test.pyx       Fri Apr 17 15:43:11 2009 -0500
>>>> @@ -132,13 +132,20 @@
>>>>      >>> test_recordarray()
>>>>
>>>>      >>> test_nested_dtypes(np.zeros((3,), dtype=np.dtype([\
>>>> -            ('a', np.dtype('i,i')),\
>>>> -            ('b', np.dtype('i,i'))\
>>>> +            ('a', np.dtype('<i,<i')),\
>>>> +            ('b', np.dtype('<i,<i'))\
>>>>          ])))
>>>>      array([((0, 0), (0, 0)), ((1, 2), (1, 4)), ((1, 2), (1, 4))],
>>>>            dtype=[('a', [('f0', '<i4'), ('f1', '<i4')]), ('b', [('f0',
>>>> '<i4'), ('f1', '<i4')])])
>>> Any ideas on why this particular case actually passes on your machine?
>>> The Cython code alwayws works with big-endian on your machine, so I
>>> wouldn't expect to see ((1, 2), (1, 4)) there, but the byteswapped
>>> versions of those numbers...
>>>
>> Ahh right; the only value read from the input array (in the test
>> function) is zero, which is the same in both endians, and you added
>> explicit byteswap on the output.
> 
> Sorry -- can't devote the time to address these questions right now
> but I'll get to them as soon as I can (Monday?).

I wasn't implying that you should fix this :-) just that the problem is 
larger than the patch.

Actually struct buffer access isn't fully worked out yet (the 
rudimentary support that is there now was added some months ago and 
contains some weaknesses). I now actually think some rather involved 
fixes on "struct packing" (native vs. packed) must be done as well to 
support most common numpy dtypes; and those are very connected, so may 
do those at the same time.

If you like we can do it together. We'll see; my work on it must happen 
sporadically over the course of some weeks anyway.

-- 
Dag Sverre

From riccardo.murri at gmail.com  Sun Apr 19 09:39:01 2009
From: riccardo.murri at gmail.com (Riccardo Murri)
Date: Sun, 19 Apr 2009 09:39:01 +0200
Subject: [Cython] cannot compile derived class? (cython 0.11.1)
In-Reply-To: <991E4AA8-4F7C-42DB-B070-F9ECC1B5A867@math.washington.edu>
References: <1c34ba170904170354j576ba8a8i58e1ed246d3ecc9b@mail.gmail.com>
	<991E4AA8-4F7C-42DB-B070-F9ECC1B5A867@math.washington.edu>
Message-ID: <1c34ba170904190039v7722b723o58e0028741c6aec4@mail.gmail.com>

On Fri, Apr 17, 2009 at 8:01 PM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Apr 17, 2009, at 3:54 AM, Riccardo Murri wrote:
>
>> I cannot get Cython 0.11.1 to compile the following simple example::
>>
>> ? $ cat a.pxd
>> ? cdef class A(object)
>> ? cdef class B(A)
>>
>> ? $ cat a.pyx
>> ? cdef class A(object):
>> ? ? ? ?pass
>>
>> ? cdef class B(A):
>> ? ? ? ?pass
>>
>> ? $ cat c.pyx
>> ? from a cimport B
>>
>> ? cdef class C(B):
>> ? ? ? ?pass
>
> [...]
>
>> Is this a bug or am I doing something wrong?
>
> No, I think this is a bug.

Reported in ticket #286: http://trac.cython.org/cython_trac/ticket/286

Riccardo

-- 
Riccardo Murri, via Ross?e 17, 6987 Caslano (CH)

From riccardo.murri at gmail.com  Sun Apr 19 10:02:12 2009
From: riccardo.murri at gmail.com (Riccardo Murri)
Date: Sun, 19 Apr 2009 10:02:12 +0200
Subject: [Cython] incorrect C code generated on method assignment
	(cython 0.11.1)
In-Reply-To: <09c458c88760dc5ea641d522c6ec939e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
References: <1c34ba170904170550jc9a800cue5c1dad48d30e256@mail.gmail.com>
	<09c458c88760dc5ea641d522c6ec939e.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <1c34ba170904190102g5d60ca61ve9045fcb204a93b6@mail.gmail.com>

On Fri, Apr 17, 2009 at 3:09 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Riccardo Murri wrote:
>> while converting a class from pure Python to Cython, I've come across
>> the following:
>>
>> ? $ cat r.pyx
>> ? cdef class r(object):
>> ? ? ? ?cdef long long n
>> ? ? ? ?cdef long long d
>>
>> ? ? ? ?def __init__(self, long long n, long long d):
>> ? ? ? ? ? ?self.n = n
>> ? ? ? ? ? ?self.d = d
>>
>> ? ? ? ?def __int__(self):
>> ? ? ? ? ? ?return (self.n // self.d)
>>
>> ? ? ? ?__long__ = __int__
>>
>> ? $ cython -v r.pyx
>> ? Compiling /tmp/r.pyx
>>
>> ? $ gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
>> -Wstrict-prototypes -fPIC -I/usr/include/python2.5 -c r.c -o r.o
>> ? r.c:487: error: ???None??? undeclared here (not in a function)
>>
>> ? $ sed -e '484,490!d;488i^^^^^^^^^^^^^^^^^^^^^^^^ up here' r.c
>> ? ? 0, /*nb_coerce*/
>> ? ? #endif
>> ? ? __pyx_pf_1r_1r___int__, /*nb_int*/
>> ? ? None, /*nb_long*/
>> ? ^^^^^^^^^^^^^^^^^^^^^^^^ up here
>> ? ? 0, /*nb_float*/
>> ? ? #if PY_MAJOR_VERSION < 3
>> ? ? 0, /*nb_oct*/
>
> Yes, that's a bug. I assume that the above code does not make Cython set
> up the C level function name.
>
> Raising an error here would work for now.
>
>
>> I understand that "__long__ = __int__" is not the correct way to do
>> method aliasing in Cython
>
> ... and that's a feature request, it would be nice to make that work.
> Could you create two tickets for those?
>

Done: tickets #287 and #288

http://trac.cython.org/cython_trac/ticket/287
http://trac.cython.org/cython_trac/ticket/288

Thank you!

Riccardo

-- 
Riccardo Murri, via Ross?e 17, 6987 Caslano (CH)

From lrhazi at gmail.com  Sun Apr 19 12:56:22 2009
From: lrhazi at gmail.com (Mohamed Lrhazi)
Date: Sun, 19 Apr 2009 06:56:22 -0400
Subject: [Cython] Cython just to avoid coding C?
Message-ID: <c4ab18df0904190356w6298837boec858b2e18034567@mail.gmail.com>

Hello,

Please excuse these naive questions of mine...

Would it make sense to use Cython merely to avoid writing C? That is,
am I missing something in thinking that I could do that?

Assume I wrote a foo.pyx, which uses an external C lib. foo calls
functions on the C lib and defines callbacks the latter lib will be
calling. I then compile the foo.pyx into foo.so, then write a shell
script that runs: python -c 'import foo'

Could someone explain, in OS level terms, what would be happening when
this shell script is running? what would the Python VM be doing? would
it be idle all the time, while the Cython generated code is all that's
running? is there a way to avoid that by compiling the foo.pyx
directly into native executable?

It's late and am having trouble following my own thoughts. I hope
someone out there can detect the source of my confusion and help me
clear it out.

I am looking at using Cython because I need to use Python libs as
well, but started wondering, if I did not need any Python libs at
all.. Could I just use Cython for its pythonic syntax and semantics?

Thanks a lot,
Mohamed.

From dagss at student.matnat.uio.no  Sun Apr 19 13:48:42 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Sun, 19 Apr 2009 13:48:42 +0200 (CEST)
Subject: [Cython] Cython just to avoid coding C?
In-Reply-To: <c4ab18df0904190356w6298837boec858b2e18034567@mail.gmail.com>
References: <c4ab18df0904190356w6298837boec858b2e18034567@mail.gmail.com>
Message-ID: <f512b545782c02ba410bd78bcfcddf8d.squirrel@webmail.uio.no>

Mohamed Lrhazi wrote:
> Would it make sense to use Cython merely to avoid writing C? That is,
> am I missing something in thinking that I could do that?
>
> Assume I wrote a foo.pyx, which uses an external C lib. foo calls
> functions on the C lib and defines callbacks the latter lib will be
> calling. I then compile the foo.pyx into foo.so, then write a shell
> script that runs: python -c 'import foo'
>
> Could someone explain, in OS level terms, what would be happening when
> this shell script is running? what would the Python VM be doing? would
> it be idle all the time, while the Cython generated code is all that's
> running? is there a way to avoid that by compiling the foo.pyx
> directly into native executable?

Well, look at the Python executable as two parts:
 a) A library with lots of objects, interfaces etc. for strings, numbers,
classes etc
 b) An interpreter for Python code

The interpreter part would hardly be run at all -- it would interpret
"import foo" and then wait on that. But that is just some code sitting in
memory and having it around doesn't hurt.

You need part a) for Cython programs, and that answers your last question
("no").

All in all, what happens is that the interpreter part of Python does some
things, then does a call into the C code Cython generates which doesn't
return until your application closes.

Bottom line: Unless the memory usage or startup time is a problem, don't
worry about this, just do it :-)

(Hmm.. perhaps someone could comment on any other threads that are running
or reference cycle collection -- but I wouldn't expect it to be a
problem.)

Dag Sverre


From stefan_ml at behnel.de  Sun Apr 19 13:58:43 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 19 Apr 2009 13:58:43 +0200
Subject: [Cython] Cython just to avoid coding C?
In-Reply-To: <c4ab18df0904190356w6298837boec858b2e18034567@mail.gmail.com>
References: <c4ab18df0904190356w6298837boec858b2e18034567@mail.gmail.com>
Message-ID: <49EB11F3.1080908@behnel.de>


Mohamed Lrhazi wrote:
> Would it make sense to use Cython merely to avoid writing C? That is,
> am I missing something in thinking that I could do that?
> 
> Assume I wrote a foo.pyx, which uses an external C lib. foo calls
> functions on the C lib and defines callbacks the latter lib will be
> calling. I then compile the foo.pyx into foo.so, then write a shell
> script that runs: python -c 'import foo'

That should work. It's basically the same as starting a Python module
manually. You have to keep the interpreter alive while the C lib does its
work, though. That usually means calling into some kind of event loop of
the C library that does not return but only calls predefined callbacks (GUI
or networking toolkits come to mind).

Note that you can use pyximport to build the module for you, i.e.

    $ ls
    foo.pyx
    $ export PYTHONPATH=$PWD
    $ python -c 'import pyximport; pyximport.install(); import foo'

At least during development, this proves very helpful, as you safe the
extra "run Cython and build the .so" steps.

BTW, it does not seem to work to just run

    $ python -m foo

for a "foo.so", the error being "No code object available for foo". Not
sure if there is anything we can do about that...


> Could someone explain, in OS level terms, what would be happening when
> this shell script is running? what would the Python VM be doing? would
> it be idle all the time, while the Cython generated code is all that's
> running?

The Python VM (with which I mean the normal eval loop) would only be
involved when you import or call normal Python code.


> I am looking at using Cython because I need to use Python libs as
> well, but started wondering, if I did not need any Python libs at
> all.. Could I just use Cython for its pythonic syntax and semantics?

Well, Cython is not designed for writing stand-alone C programs. You always
have the overhead of requiring CPython as a dependency. That's great if you
can make use of Python libraries or at least Python data types or
exceptions, but if you use none of those, you just end up with a large,
unused dependency, so that's not the optimal use case.

Stefan

From lodatom at gmail.com  Sun Apr 19 15:16:46 2009
From: lodatom at gmail.com (Mark Lodato)
Date: Sun, 19 Apr 2009 09:16:46 -0400
Subject: [Cython] Cython just to avoid coding C?
In-Reply-To: <c4ab18df0904190356w6298837boec858b2e18034567@mail.gmail.com>
References: <c4ab18df0904190356w6298837boec858b2e18034567@mail.gmail.com>
Message-ID: <ca433830904190616v9cf56f9r26ca2ed803a4423e@mail.gmail.com>

On Sun, Apr 19, 2009 at 6:56 AM, Mohamed Lrhazi <lrhazi at gmail.com> wrote:
> is there a way to avoid that by compiling the foo.pyx
> directly into native executable?

There's no way to get rid of the Python interpreter, but you could
embed the interpreter and statically link your module.  This has two
main advantages: (1) you only have a single file to deal with, rather
than the script and the .so, and (2) you can profile using `gprof',
which does not work with shared objects.

Here's an example:

--------------------------------- mylib.pyx --------------------------
import sys

cdef extern from "math.h":
    double lgamma(double)
    double exp(double)


cdef inline double gamma(double n):
    return exp( lgamma(n) )


def main(argv = None):
    if argv is None:
        argv = sys.argv
    if len(argv) != 2:
        name = argv[0]  if argv  else '<prog>'
        sys.stderr.write("USAGE: %s n\nPrints gamma(n).\n" % name)
        sys.exit(1)

    print gamma(float(argv[1]))
----------------------------------------------------------------------

----------------------------------- main.c ---------------------------
#include <Python.h>

// For each Cython module you want to embed, you must declare an
// init<module> function, like so:
PyMODINIT_FUNC initmylib(void);

int
main(int argc, char *argv[])
{
    // The first step is to set up the Python interpreter:
    Py_Initialize();
    PySys_SetArgv(argc, argv);

    // Next, we need to tell Python that our module exists.  Call each
    // of the functions you declared above.
    initmylib();

    // Now do some Python stuff.  The easiest thing to do is to give
    // the interpreter a string of Python code that imports your
    // module and calls it.
    PyRun_SimpleString("from mylib import main\n"
                       "main()\n");

    // When we're done, tell Python to clean up.
    Py_Finalize();
    return 0;
}
----------------------------------------------------------------------

On my machine, I compile with:

cython  mylib.pyx
gcc -fPIC -g -O2 -I /usr/include/python2.5  -c -o main.o main.c
gcc -fPIC -g -O2 -I /usr/include/python2.5  -c -o mylib.o mylib.c
gcc -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions \
    main.o mylib.o /usr/lib/python2.5/config/libpython2.5.a \
    -lm -ldl -pthread -lutil \
    -o main

(See http://docs.python.org/extending/embedding.html for details on
how to properly compile.)

Now I can run my program as a normal executable:

$ ./main
USAGE: ./main n
Prints gamma(n).
$ ./main 6
120.0
$ ./main 100
9.33262154439e+155


--
Mark

From lrhazi at gmail.com  Sun Apr 19 16:06:09 2009
From: lrhazi at gmail.com (Mohamed Lrhazi)
Date: Sun, 19 Apr 2009 10:06:09 -0400
Subject: [Cython] Cython just to avoid coding C?
In-Reply-To: <ca433830904190616v9cf56f9r26ca2ed803a4423e@mail.gmail.com>
References: <c4ab18df0904190356w6298837boec858b2e18034567@mail.gmail.com>
	<ca433830904190616v9cf56f9r26ca2ed803a4423e@mail.gmail.com>
Message-ID: <c4ab18df0904190706p50d00031vaf9be5e40cbb5906@mail.gmail.com>

Thank you so much eveybody. you answered all my questions and gave me
more. Now, when you're nice to people, they want more... so let me ask
for a bit more advice:

What I want to write is indeed, like Stefan guessed, a plugin for a
network OS. So my Cython code would be calling functions to register
for events of interest, and then define function callbacks, then call
a poll() function and wait for events to fire and callbacks to be
called.

Because the Cython code would never return, the "import foo" would be
the last thing to run, essentially. So, How can I define the callbacks
in my normal Python land, if I wanted to?

The poll function for the system actually comes in flavors, one of
which is nonblocking, but am still trying to understand all options.

I am also wondering if there was an obvious way to "connect" this
event loop, in this NOS, with Twisted Python reactors? but that's even
more vague a question, aint it?

Thank you very much all, and thanks for Cython. BTW, how do you
pronounce it? with an "s" sound or "k" sound?

Mohamed.


On Sun, Apr 19, 2009 at 9:16 AM, Mark Lodato <lodatom at gmail.com> wrote:
> On Sun, Apr 19, 2009 at 6:56 AM, Mohamed Lrhazi <lrhazi at gmail.com> wrote:
>> is there a way to avoid that by compiling the foo.pyx
>> directly into native executable?
>
> There's no way to get rid of the Python interpreter, but you could
> embed the interpreter and statically link your module. ?This has two
> main advantages: (1) you only have a single file to deal with, rather
> than the script and the .so, and (2) you can profile using `gprof',
> which does not work with shared objects.
>
> Here's an example:
>
> --------------------------------- mylib.pyx --------------------------
> import sys
>
> cdef extern from "math.h":
> ? ?double lgamma(double)
> ? ?double exp(double)
>
>
> cdef inline double gamma(double n):
> ? ?return exp( lgamma(n) )
>
>
> def main(argv = None):
> ? ?if argv is None:
> ? ? ? ?argv = sys.argv
> ? ?if len(argv) != 2:
> ? ? ? ?name = argv[0] ?if argv ?else '<prog>'
> ? ? ? ?sys.stderr.write("USAGE: %s n\nPrints gamma(n).\n" % name)
> ? ? ? ?sys.exit(1)
>
> ? ?print gamma(float(argv[1]))
> ----------------------------------------------------------------------
>
> ----------------------------------- main.c ---------------------------
> #include <Python.h>
>
> // For each Cython module you want to embed, you must declare an
> // init<module> function, like so:
> PyMODINIT_FUNC initmylib(void);
>
> int
> main(int argc, char *argv[])
> {
> ? ?// The first step is to set up the Python interpreter:
> ? ?Py_Initialize();
> ? ?PySys_SetArgv(argc, argv);
>
> ? ?// Next, we need to tell Python that our module exists. ?Call each
> ? ?// of the functions you declared above.
> ? ?initmylib();
>
> ? ?// Now do some Python stuff. ?The easiest thing to do is to give
> ? ?// the interpreter a string of Python code that imports your
> ? ?// module and calls it.
> ? ?PyRun_SimpleString("from mylib import main\n"
> ? ? ? ? ? ? ? ? ? ? ? "main()\n");
>
> ? ?// When we're done, tell Python to clean up.
> ? ?Py_Finalize();
> ? ?return 0;
> }
> ----------------------------------------------------------------------
>
> On my machine, I compile with:
>
> cython ?mylib.pyx
> gcc -fPIC -g -O2 -I /usr/include/python2.5 ?-c -o main.o main.c
> gcc -fPIC -g -O2 -I /usr/include/python2.5 ?-c -o mylib.o mylib.c
> gcc -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions \
> ? ?main.o mylib.o /usr/lib/python2.5/config/libpython2.5.a \
> ? ?-lm -ldl -pthread -lutil \
> ? ?-o main
>
> (See http://docs.python.org/extending/embedding.html for details on
> how to properly compile.)
>
> Now I can run my program as a normal executable:
>
> $ ./main
> USAGE: ./main n
> Prints gamma(n).
> $ ./main 6
> 120.0
> $ ./main 100
> 9.33262154439e+155
>
>
> --
> Mark
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
" Logic merely sanctions the conquests of the intuition."
Jacques Hadamard

From stefan_ml at behnel.de  Sun Apr 19 16:46:27 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 19 Apr 2009 16:46:27 +0200
Subject: [Cython] Cython just to avoid coding C?
In-Reply-To: <ca433830904190616v9cf56f9r26ca2ed803a4423e@mail.gmail.com>
References: <c4ab18df0904190356w6298837boec858b2e18034567@mail.gmail.com>
	<ca433830904190616v9cf56f9r26ca2ed803a4423e@mail.gmail.com>
Message-ID: <49EB3943.9090409@behnel.de>


Mark Lodato wrote:
> you could embed the interpreter and statically link your module.
> [...]
> ----------------------------------- main.c ---------------------------
> #include <Python.h>
> 
> // For each Cython module you want to embed, you must declare an
> // init<module> function, like so:
> PyMODINIT_FUNC initmylib(void);
> 
> int
> main(int argc, char *argv[])
> {
>     // The first step is to set up the Python interpreter:
>     Py_Initialize();
>     PySys_SetArgv(argc, argv);
> 
>     // Next, we need to tell Python that our module exists.  Call each
>     // of the functions you declared above.
>     initmylib();
> 
>     // Now do some Python stuff.  The easiest thing to do is to give
>     // the interpreter a string of Python code that imports your
>     // module and calls it.
>     PyRun_SimpleString("from mylib import main\n"
>                        "main()\n");
> 
>     // When we're done, tell Python to clean up.
>     Py_Finalize();
>     return 0;
> }
> ----------------------------------------------------------------------

I wonder if it makes sense to add support for this to Cython. You could
have a command line option that adds a suitable main() function that runs
the module code in an embedded CPython interpreter (and that does the setup
correctly for Py2 and Py3).

Or maybe an additional tool like Python's "freeze" (what about naming it
"cool" or "maincyfy"? Or maybe "(em)bedcy"?) could take a list of Cython
modules and generate a main .c file like the one above, which would simply
run the module initialisation in order. That would make it trivial to write
a 'main module' (passed as last argument) that simply contains the main
program code in its body.

We might even be able to check module interdependencies here, so that the
tool can bail out if it knows that the sequential module initialisations
cannot work in an embedded setup.

Tons of cool stuff to do here ...

Stefan

From dagss at student.matnat.uio.no  Sun Apr 19 22:50:02 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Sun, 19 Apr 2009 22:50:02 +0200
Subject: [Cython] Proposal: Packed struct support?
Message-ID: <49EB8E7A.5060106@student.matnat.uio.no>

Could we have some kind of keyword to define packed structs in Cython?
This is not part of the C standard, but is supported in most C compilers
through extensions. (Obviously one can do it now through writing a
header file, but that is inconvenient.)

I.e. something like:

cdef packed struct MyStruct:
     char a
     double b

would output conditional code to do either __attribute__((__packed__))
or #pragma pack(push,1), #pragma pack(pop) (gcc/icc and Visual C, 
respectively).

PEP-3118 buffers (like NumPy arrays) can contain arbitrarily packed 
data, and this would help me write readable testing code for that (as 
well as anyone actually using such types, however of course one should 
always align ones data properly...)

BTW, any volunteers for Visual C testing on this? Also, does anyone know 
wheter it is possible to have a #pragma in a #define, like this:

#define SOMEMACRO #pragma pack(push, 1)

?

-- 
Dag Sverre


From greg.ewing at canterbury.ac.nz  Mon Apr 20 02:52:30 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 20 Apr 2009 12:52:30 +1200
Subject: [Cython] Cython just to avoid coding C?
In-Reply-To: <f512b545782c02ba410bd78bcfcddf8d.squirrel@webmail.uio.no>
References: <c4ab18df0904190356w6298837boec858b2e18034567@mail.gmail.com>
	<f512b545782c02ba410bd78bcfcddf8d.squirrel@webmail.uio.no>
Message-ID: <49EBC74E.8090608@canterbury.ac.nz>

Dag Sverre Seljebotn wrote:

> Bottom line: Unless the memory usage or startup time is a problem, don't
> worry about this, just do it :-)

If your code won't be making any use of Python objects
or Python libraries, there probably won't be much
advantage to using Cython -- you'll still be writing
C code, just with a slightly different syntax.

On the other hand, if you want to take advantage of
Python features in your code, this is a perfectly
reasonable thing to do.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Mon Apr 20 02:56:13 2009
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 20 Apr 2009 12:56:13 +1200
Subject: [Cython] Cython just to avoid coding C?
In-Reply-To: <c4ab18df0904190356w6298837boec858b2e18034567@mail.gmail.com>
References: <c4ab18df0904190356w6298837boec858b2e18034567@mail.gmail.com>
Message-ID: <49EBC82D.9030200@canterbury.ac.nz>

Mohamed Lrhazi wrote:

> is there a way to avoid that by compiling the foo.pyx
> directly into native executable?

It's probably not feasible to create a Cython module
that will work with no Python interpreter at all.
Even a minimal Cython source file probably generates
references to various Python runtime routines.

-- 
Greg

From sturla at molden.no  Mon Apr 20 13:56:48 2009
From: sturla at molden.no (Sturla Molden)
Date: Mon, 20 Apr 2009 13:56:48 +0200
Subject: [Cython] Proposal: Packed struct support?
In-Reply-To: <49EB8E7A.5060106@student.matnat.uio.no>
References: <49EB8E7A.5060106@student.matnat.uio.no>
Message-ID: <49EC6300.1050907@molden.no>

On 4/19/2009 10:50 PM, Dag Sverre Seljebotn wrote:

> Could we have some kind of keyword to define packed structs in Cython?
> This is not part of the C standard, but is supported in most C compilers
> through extensions. 

Since it's not in the standard, there is no standard syntax for this.

But you can fake it using a char array and some constants storing offsets.

    /*
    cdef packed struct MyStruct:
       char a
       double b
    */
    const int mystruct_size = sizeof(char)+sizeof(double);
    const int mystruct_offset_a = 0;
    const int mystruct_offset_b = sizeof(char);

    /*
    cdef MyStruct mystruct
    */
    char mystruct[mystruct_size]; /* sizeof(char) is 1 by defintion */


    /*
    cdef char a
    cdef double b
    a = mystruct.a
    b = mystruct.b
    */
    char a;
    double b;
    a = *(char *)((char *)mystruct + mystruct_offset_a);
    b = *(double *)((char *)mystruct + mystruct_offset_b);



Sturla Molden


From dagss at student.matnat.uio.no  Mon Apr 20 16:17:32 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Mon, 20 Apr 2009 16:17:32 +0200 (CEST)
Subject: [Cython] Proposal: Packed struct support?
In-Reply-To: <49EC6300.1050907@molden.no>
References: <49EB8E7A.5060106@student.matnat.uio.no>
	<49EC6300.1050907@molden.no>
Message-ID: <999877f81f0134f2d1acd578effc2769.squirrel@webmail.uio.no>

Sturla Molden wrote:
> On 4/19/2009 10:50 PM, Dag Sverre Seljebotn wrote:
>
>> Could we have some kind of keyword to define packed structs in Cython?
>> This is not part of the C standard, but is supported in most C compilers
>> through extensions.
>
> Since it's not in the standard, there is no standard syntax for this.
>
> But you can fake it using a char array and some constants storing offsets.

Thanks, good idea. Indeed, since Cython would know the struct definitions,
we could just directly output the necesarry pointer arithmetic :-)

However using the various compiler extensions would take less time to
implement in Cython and make for more readable code. I'm not that worried
about compatability -- we're only considering the subset of C compilers on
which CPython compiles anyway, and it's a fairly universal compiler
extension (though with differing syntax).

So (assuming a positive vote on this) I suppose I'll go that route first,
and then take your approach if there's a need.

Dag Sverre


From jf.moulin at gmail.com  Mon Apr 20 17:17:29 2009
From: jf.moulin at gmail.com (Jean-Francois Moulin)
Date: Mon, 20 Apr 2009 17:17:29 +0200
Subject: [Cython] help with function pointer
Message-ID: <af6d35810904200817p62e92366ld616a491bf9b63da@mail.gmail.com>

Hi

I am struggling with a function pointer problem:

cdef class LstFileReader_FAST:
    cdef tuple (*do_read_ev)(LstFileReader_FAST)

     def __init__(self,fname):

        self.fname = fname
        self.goto_start()
        self.do_read_ev = self._next_ev
.....

results in a compilation error:

Cannot assign type 'tuple (LstFileReader_FAST, int
__pyx_skip_dispatch)' to 'tuple (*)(LstFileReader_FAST)'

I found no clear explanation for this __pyx_skip_dispatch... and
somewhere above in my code I have an almostt similar statement which
works:

cdef int (*locate)(DelayLine, np.ndarray)
...
    def __init__(self):
               ....
                self.locate = self.locate_single_terminal


Any clue?
Thanks in advance for helping a beginner!
JF

From dalcinl at gmail.com  Mon Apr 20 17:29:18 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 20 Apr 2009 12:29:18 -0300
Subject: [Cython] help with function pointer
In-Reply-To: <af6d35810904200817p62e92366ld616a491bf9b63da@mail.gmail.com>
References: <af6d35810904200817p62e92366ld616a491bf9b63da@mail.gmail.com>
Message-ID: <e7ba66e40904200829q19871895kf3c6f48214caf5ff@mail.gmail.com>

In the past, Cython accepted such code. Unfortunately, at some point
this functionality was lost. However, you can add a forward
declaration and a ctypedef like this:

cdef class LstFileReader_FAST # forward declaration

ctypedef tuple read_fn_t(LstFileReader_FAST) # ctypedef for function type

cdef class LstFileReader_FAST:
   cdef read_fn_t *do_read_ev # note do_read_ev is a pointer to read_fn_t !!!



On Mon, Apr 20, 2009 at 12:17 PM, Jean-Francois Moulin
<jf.moulin at gmail.com> wrote:
> Hi
>
> I am struggling with a function pointer problem:
>
> cdef class LstFileReader_FAST:
> ? ?cdef tuple (*do_read_ev)(LstFileReader_FAST)
>
> ? ? def __init__(self,fname):
>
> ? ? ? ?self.fname = fname
> ? ? ? ?self.goto_start()
> ? ? ? ?self.do_read_ev = self._next_ev
> .....
>
> results in a compilation error:
>
> Cannot assign type 'tuple (LstFileReader_FAST, int
> __pyx_skip_dispatch)' to 'tuple (*)(LstFileReader_FAST)'
>
> I found no clear explanation for this __pyx_skip_dispatch... and
> somewhere above in my code I have an almostt similar statement which
> works:
>
> cdef int (*locate)(DelayLine, np.ndarray)
> ...
> ? ?def __init__(self):
> ? ? ? ? ? ? ? ....
> ? ? ? ? ? ? ? ?self.locate = self.locate_single_terminal
>
>
> Any clue?
> Thanks in advance for helping a beginner!
> JF
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dalcinl at gmail.com  Mon Apr 20 17:34:52 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 20 Apr 2009 12:34:52 -0300
Subject: [Cython] Proposal: Packed struct support?
In-Reply-To: <49EB8E7A.5060106@student.matnat.uio.no>
References: <49EB8E7A.5060106@student.matnat.uio.no>
Message-ID: <e7ba66e40904200834w3fbc3114gee600d5832110bd9@mail.gmail.com>

On Sun, Apr 19, 2009 at 5:50 PM, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:
>
> #define SOMEMACRO #pragma pack(push, 1)
>

No way, the preprocessor will complain. You'll need the C99
"_Pragma()" operator.



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From robertwb at math.washington.edu  Mon Apr 20 20:38:26 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 20 Apr 2009 11:38:26 -0700
Subject: [Cython] Proposal: Packed struct support?
In-Reply-To: <999877f81f0134f2d1acd578effc2769.squirrel@webmail.uio.no>
References: <49EB8E7A.5060106@student.matnat.uio.no>
	<49EC6300.1050907@molden.no>
	<999877f81f0134f2d1acd578effc2769.squirrel@webmail.uio.no>
Message-ID: <1ADDF964-8FE1-4BA9-8B10-B31B06B1F8B5@math.washington.edu>

On Apr 20, 2009, at 7:17 AM, Dag Sverre Seljebotn wrote:

> Sturla Molden wrote:
>> On 4/19/2009 10:50 PM, Dag Sverre Seljebotn wrote:
>>
>>> Could we have some kind of keyword to define packed structs in  
>>> Cython?
>>> This is not part of the C standard, but is supported in most C  
>>> compilers
>>> through extensions.
>>
>> Since it's not in the standard, there is no standard syntax for this.
>>
>> But you can fake it using a char array and some constants storing  
>> offsets.
>
> Thanks, good idea. Indeed, since Cython would know the struct  
> definitions,
> we could just directly output the necesarry pointer arithmetic :-)

I'd rather avoid such messiness in the C code if we can. +1 to being  
able to specify a packed keyword though.

- Robert


From robertwb at math.washington.edu  Mon Apr 20 20:42:03 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 20 Apr 2009 11:42:03 -0700
Subject: [Cython] help with function pointer
In-Reply-To: <af6d35810904200817p62e92366ld616a491bf9b63da@mail.gmail.com>
References: <af6d35810904200817p62e92366ld616a491bf9b63da@mail.gmail.com>
Message-ID: <6F08D037-7371-4D43-B7FE-8185FD8B25DB@math.washington.edu>

On Apr 20, 2009, at 8:17 AM, Jean-Francois Moulin wrote:

> Hi
>
> I am struggling with a function pointer problem:
>
> cdef class LstFileReader_FAST:
>     cdef tuple (*do_read_ev)(LstFileReader_FAST)
>
>      def __init__(self,fname):
>
>         self.fname = fname
>         self.goto_start()
>         self.do_read_ev = self._next_ev
> .....
>
> results in a compilation error:
>
> Cannot assign type 'tuple (LstFileReader_FAST, int
> __pyx_skip_dispatch)' to 'tuple (*)(LstFileReader_FAST)'
>
> I found no clear explanation for this __pyx_skip_dispatch...

I bet you've declared self._next_ev to be a cpdef (not cdef)  
function. cpdef functions have an extra implicit argument  
"__pyx_skip_dispatch" which lets them know whether or not they should  
skip a possible lookup in the objects dictionary.

This should probably be a clearer error though...

- Robert

From robertwb at math.washington.edu  Mon Apr 20 20:48:28 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 20 Apr 2009 11:48:28 -0700
Subject: [Cython] Cython just to avoid coding C?
In-Reply-To: <49EB3943.9090409@behnel.de>
References: <c4ab18df0904190356w6298837boec858b2e18034567@mail.gmail.com>
	<ca433830904190616v9cf56f9r26ca2ed803a4423e@mail.gmail.com>
	<49EB3943.9090409@behnel.de>
Message-ID: <4192B3F1-8CF6-41CA-80EE-42B2D32623B0@math.washington.edu>

On Apr 19, 2009, at 7:46 AM, Stefan Behnel wrote:

> Mark Lodato wrote:
>> you could embed the interpreter and statically link your module.
>> [...]
>> ----------------------------------- main.c  
>> ---------------------------
>> #include <Python.h>
>>
>> // For each Cython module you want to embed, you must declare an
>> // init<module> function, like so:
>> PyMODINIT_FUNC initmylib(void);
>>
>> int
>> main(int argc, char *argv[])
>> {
>>     // The first step is to set up the Python interpreter:
>>     Py_Initialize();
>>     PySys_SetArgv(argc, argv);
>>
>>     // Next, we need to tell Python that our module exists.  Call  
>> each
>>     // of the functions you declared above.
>>     initmylib();
>>
>>     // Now do some Python stuff.  The easiest thing to do is to give
>>     // the interpreter a string of Python code that imports your
>>     // module and calls it.
>>     PyRun_SimpleString("from mylib import main\n"
>>                        "main()\n");
>>
>>     // When we're done, tell Python to clean up.
>>     Py_Finalize();
>>     return 0;
>> }
>> --------------------------------------------------------------------- 
>> -
>
> I wonder if it makes sense to add support for this to Cython. You  
> could
> have a command line option that adds a suitable main() function  
> that runs
> the module code in an embedded CPython interpreter (and that does  
> the setup
> correctly for Py2 and Py3).

+1, I think that's an excellent idea. http://trac.cython.org/ 
cython_trac/ticket/289

> Or maybe an additional tool like Python's "freeze" (what about  
> naming it
> "cool" or "maincyfy"? Or maybe "(em)bedcy"?) could take a list of  
> Cython
> modules and generate a main .c file like the one above, which would  
> simply
> run the module initialisation in order. That would make it trivial  
> to write
> a 'main module' (passed as last argument) that simply contains the  
> main
> program code in its body.
>
> We might even be able to check module interdependencies here, so  
> that the
> tool can bail out if it knows that the sequential module  
> initialisations
> cannot work in an embedded setup.
>
> Tons of cool stuff to do here ...

Yep, that'd be cool. With enough work, we could totally do a  
"freeze." (Actually, if we use freeze it might not be that much work  
at all, but I haven't looked into it yet.)

- Robert



From stefan_ml at behnel.de  Tue Apr 21 07:43:04 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 21 Apr 2009 07:43:04 +0200
Subject: [Cython] Path towards Python completeness
Message-ID: <49ED5CE8.6090708@behnel.de>

Hi,

I just read the release announcement of PyPy's 1.1 beta:

http://morepypy.blogspot.com/2009/04/beta-for-110-released.html

According to their benchmarks, they are now less than 2x slower than
CPython in most cases and in some cases even 20% faster. Cython is
currently about 30% faster than CPython for plain Python code, but that
only concerns the "Python code that Cython supports", which is not at all
comparable to what PyPy supports.

My feeling is that the goal to make CPython support "most of Python" is
getting more important simply because PyPy exists. There is not that much
that is missing, but the sad thing is that the main part that *is* missing
is a very important part of the language: closures.

I feel that closure support has become a real blocker on the path towards
Python completeness. Closure support will actually open up a can of worms,
but a friendly one, as it will lay out the path towards lambda support and
nested functions/classes. Especially support for generators seems to be an
extremely worthy goal, but in the current state of Cython, it just cannot
be done.

Robert has started work on this at the first Cython workshop, but it turned
out to be more work than what was doable at the time. So there are parts of
this implementation in Cython's code base already, but it still needs some
fair investment to finish it up.

I would like to focus some (wo)manpower on this topic. For that, I need to
understand better what is there and what remains to be done, so that we can
break down the task into doable steps and get going.

Anyone else interested?

Stefan

From stefan_ml at behnel.de  Tue Apr 21 08:30:06 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 21 Apr 2009 08:30:06 +0200
Subject: [Cython] Path towards Python completeness
In-Reply-To: <49ED5CE8.6090708@behnel.de>
References: <49ED5CE8.6090708@behnel.de>
Message-ID: <49ED67EE.5090801@behnel.de>


Stefan Behnel wrote:
> My feeling is that the goal to make CPython support "most of Python"

... that may not sound like an interesting goal, unless you replace
"CPython" with "Cython". ;)

Stefan


From jf.moulin at gmail.com  Tue Apr 21 09:45:20 2009
From: jf.moulin at gmail.com (Jean-Francois Moulin)
Date: Tue, 21 Apr 2009 09:45:20 +0200
Subject: [Cython] help with function pointer
In-Reply-To: <6F08D037-7371-4D43-B7FE-8185FD8B25DB@math.washington.edu>
References: <af6d35810904200817p62e92366ld616a491bf9b63da@mail.gmail.com>
	<6F08D037-7371-4D43-B7FE-8185FD8B25DB@math.washington.edu>
Message-ID: <af6d35810904210045k105c4178t29f68b621d1354c5@mail.gmail.com>

Hi,

first of all, thanks for the replies.

As for the first answer by Lisandro, I do not understand then why my
first code fragment at all works...
The only difference is that the function called has two params, one
instance of one of my classes and a numpy array, while the faulty
piece of code should be called on an instance
of one of my classes only...

As for the second, from Robert, now, very well... how do I specify a
value for this implicit parameter then!? It is a bint I guess... I
tried to set it to 1 and got new error messages:
delay_line.pyx:961:30: Cannot assign type 'tuple (LstFileReader_FAST,
int __pyx_skip_dispatch, struct
__pyx_opt_args_12delay_line_C_18LstFileReader_FAST_next_ev
*__pyx_optional_args)' to 'tuple (*)(LstFileReader_FAST)'

I also tried to use cdef in all cases, I get no compile time error but
at runtime my object LstFileReader_FAST appears no longer to be
defined!? I do not understand because all the calls are made from
within Cython code... (so that I expect the cpdef is not really
needed)


Thanks a lot again for your help!
JF

On Mon, Apr 20, 2009 at 8:42 PM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Apr 20, 2009, at 8:17 AM, Jean-Francois Moulin wrote:
>
>> Hi
>>
>> I am struggling with a function pointer problem:
>>
>> cdef class LstFileReader_FAST:
>>     cdef tuple (*do_read_ev)(LstFileReader_FAST)
>>
>>      def __init__(self,fname):
>>
>>         self.fname = fname
>>         self.goto_start()
>>         self.do_read_ev = self._next_ev
>> .....
>>
>> results in a compilation error:
>>
>> Cannot assign type 'tuple (LstFileReader_FAST, int
>> __pyx_skip_dispatch)' to 'tuple (*)(LstFileReader_FAST)'
>>
>> I found no clear explanation for this __pyx_skip_dispatch...
>
> I bet you've declared self._next_ev to be a cpdef (not cdef)
> function. cpdef functions have an extra implicit argument
> "__pyx_skip_dispatch" which lets them know whether or not they should
> skip a possible lookup in the objects dictionary.
>
> This should probably be a clearer error though...
>
> - Robert
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>

From stefan_ml at behnel.de  Tue Apr 21 10:02:05 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 21 Apr 2009 10:02:05 +0200 (CEST)
Subject: [Cython] help with function pointer
In-Reply-To: <af6d35810904210045k105c4178t29f68b621d1354c5@mail.gmail.com>
References: <af6d35810904200817p62e92366ld616a491bf9b63da@mail.gmail.com>
	<6F08D037-7371-4D43-B7FE-8185FD8B25DB@math.washington.edu>
	<af6d35810904210045k105c4178t29f68b621d1354c5@mail.gmail.com>
Message-ID: <fe44d821ed0a3a47a662ce0f9dff0957.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Jean-Francois Moulin wrote:
> As for the second, from Robert, now, very well... how do I specify a
> value for this implicit parameter then!?

You don't. "cpdef" functions have a special signature, so you can't use
them as callbacks.


> I also tried to use cdef in all cases, I get no compile time error

:)


> but at runtime my object LstFileReader_FAST appears no longer to be
> defined!?

Then you changed other things in your code that made this break. Unless
you show us the code you try to compile, we cannot help you.

Stefan


From robertwb at math.washington.edu  Tue Apr 21 10:23:06 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 21 Apr 2009 01:23:06 -0700
Subject: [Cython] help with function pointer
In-Reply-To: <af6d35810904210045k105c4178t29f68b621d1354c5@mail.gmail.com>
References: <af6d35810904200817p62e92366ld616a491bf9b63da@mail.gmail.com>
	<6F08D037-7371-4D43-B7FE-8185FD8B25DB@math.washington.edu>
	<af6d35810904210045k105c4178t29f68b621d1354c5@mail.gmail.com>
Message-ID: <CAD789E2-DAD2-4605-9878-318E7A2ECD33@math.washington.edu>

On Apr 21, 2009, at 12:45 AM, Jean-Francois Moulin wrote:

> Hi,
>
> first of all, thanks for the replies.
>
> As for the first answer by Lisandro, I do not understand then why my
> first code fragment at all works...
> The only difference is that the function called has two params, one
> instance of one of my classes and a numpy array, while the faulty
> piece of code should be called on an instance
> of one of my classes only...
>
> As for the second, from Robert, now, very well... how do I specify a
> value for this implicit parameter then!? It is a bint I guess... I
> tried to set it to 1 and got new error messages:
> delay_line.pyx:961:30: Cannot assign type 'tuple (LstFileReader_FAST,
> int __pyx_skip_dispatch, struct
> __pyx_opt_args_12delay_line_C_18LstFileReader_FAST_next_ev
> *__pyx_optional_args)' to 'tuple (*)(LstFileReader_FAST)'

You can't. I agree the errors are more cryptic than they should be,  
but C functions have no concept of runtime dispatching or default  
arguments, so it doesn't work to cast such a function to a C function  
(well, maybe one could hack it, but it'd be ugly and fragile).

> I also tried to use cdef in all cases, I get no compile time error but
> at runtime my object LstFileReader_FAST appears no longer to be
> defined!? I do not understand because all the calls are made from
> within Cython code... (so that I expect the cpdef is not really
> needed)

I am guessing you're doing something like

a. LstFileReader_FAST(...)

and it's complaining that "a has not attribute 'LstFileReader_FAST'".  
Usually this indicates that a is not typed correctly (e.g. it thinks  
a is an ordinary object). You can run cython with the -a option to  
easily see this kind of thing.

- Robert


From robertwb at math.washington.edu  Tue Apr 21 10:34:51 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 21 Apr 2009 01:34:51 -0700
Subject: [Cython] Path towards Python completeness
In-Reply-To: <49ED5CE8.6090708@behnel.de>
References: <49ED5CE8.6090708@behnel.de>
Message-ID: <80500050-599D-4F3A-9FA1-0291E43B3B99@math.washington.edu>

On Apr 20, 2009, at 10:43 PM, Stefan Behnel wrote:

> Hi,
>
> I just read the release announcement of PyPy's 1.1 beta:
>
> http://morepypy.blogspot.com/2009/04/beta-for-110-released.html
>
> According to their benchmarks, they are now less than 2x slower than
> CPython in most cases and in some cases even 20% faster. Cython is
> currently about 30% faster than CPython for plain Python code, but  
> that
> only concerns the "Python code that Cython supports", which is not  
> at all
> comparable to what PyPy supports.
>
> My feeling is that the goal to make CPython support "most of  
> Python" is
> getting more important simply because PyPy exists. There is not  
> that much
> that is missing, but the sad thing is that the main part that *is*  
> missing
> is a very important part of the language: closures.
>
> I feel that closure support has become a real blocker on the path  
> towards
> Python completeness. Closure support will actually open up a can of  
> worms,
> but a friendly one, as it will lay out the path towards lambda  
> support and
> nested functions/classes. Especially support for generators seems  
> to be an
> extremely worthy goal, but in the current state of Cython, it just  
> cannot
> be done.

All of the above are syntactic sugar for special closures.

> Robert has started work on this at the first Cython workshop, but  
> it turned
> out to be more work than what was doable at the time. So there are  
> parts of
> this implementation in Cython's code base already, but it still  
> needs some
> fair investment to finish it up.
>
> I would like to focus some (wo)manpower on this topic. For that, I  
> need to
> understand better what is there and what remains to be done, so  
> that we can
> break down the task into doable steps and get going.
>
> Anyone else interested?

This has been bugging me for a *long* time. I'll pull out and go  
though the random scattered stuff I've done over the past year  
(something else more urgent has always come up everytime I've set out  
to do this) and put it up within the week. Should I make a closure  
branch on the site?

- Robert



From robertwb at math.washington.edu  Tue Apr 21 11:16:05 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 21 Apr 2009 02:16:05 -0700
Subject: [Cython] Cython just to avoid coding C?
In-Reply-To: <4192B3F1-8CF6-41CA-80EE-42B2D32623B0@math.washington.edu>
References: <c4ab18df0904190356w6298837boec858b2e18034567@mail.gmail.com>
	<ca433830904190616v9cf56f9r26ca2ed803a4423e@mail.gmail.com>
	<49EB3943.9090409@behnel.de>
	<4192B3F1-8CF6-41CA-80EE-42B2D32623B0@math.washington.edu>
Message-ID: <4905F089-AC21-4C61-A54E-F6BAF813F828@math.washington.edu>

On Apr 20, 2009, at 11:48 AM, Robert Bradshaw wrote:

> On Apr 19, 2009, at 7:46 AM, Stefan Behnel wrote:
>
>> Mark Lodato wrote:
>>> you could embed the interpreter and statically link your module.
>>> [...]
>>> ----------------------------------- main.c
>>> ---------------------------
>>> #include <Python.h>
>>>
>>> // For each Cython module you want to embed, you must declare an
>>> // init<module> function, like so:
>>> PyMODINIT_FUNC initmylib(void);
>>>
>>> int
>>> main(int argc, char *argv[])
>>> {
>>>     // The first step is to set up the Python interpreter:
>>>     Py_Initialize();
>>>     PySys_SetArgv(argc, argv);
>>>
>>>     // Next, we need to tell Python that our module exists.  Call
>>> each
>>>     // of the functions you declared above.
>>>     initmylib();
>>>
>>>     // Now do some Python stuff.  The easiest thing to do is to give
>>>     // the interpreter a string of Python code that imports your
>>>     // module and calls it.
>>>     PyRun_SimpleString("from mylib import main\n"
>>>                        "main()\n");
>>>
>>>     // When we're done, tell Python to clean up.
>>>     Py_Finalize();
>>>     return 0;
>>> }
>>> -------------------------------------------------------------------- 
>>> -
>>> -
>>
>> I wonder if it makes sense to add support for this to Cython. You
>> could
>> have a command line option that adds a suitable main() function
>> that runs
>> the module code in an embedded CPython interpreter (and that does
>> the setup
>> correctly for Py2 and Py3).
>
> +1, I think that's an excellent idea. http://trac.cython.org/
> cython_trac/ticket/289

There's an --embed command line option now, which generates an  
appropriate main() method. One then compiles the resulting C file  
into an executable (which still needs to be linked against the Python  
library).

- Robert

From jf.moulin at gmail.com  Tue Apr 21 11:19:06 2009
From: jf.moulin at gmail.com (Jean-Francois Moulin)
Date: Tue, 21 Apr 2009 11:19:06 +0200
Subject: [Cython] help with function pointer
In-Reply-To: <CAD789E2-DAD2-4605-9878-318E7A2ECD33@math.washington.edu>
References: <af6d35810904200817p62e92366ld616a491bf9b63da@mail.gmail.com>
	<6F08D037-7371-4D43-B7FE-8185FD8B25DB@math.washington.edu>
	<af6d35810904210045k105c4178t29f68b621d1354c5@mail.gmail.com>
	<CAD789E2-DAD2-4605-9878-318E7A2ECD33@math.washington.edu>
Message-ID: <af6d35810904210219y45a12adbr644cbb219a3d6e1d@mail.gmail.com>

Hi!
thanks for the answers!

Here is the runtime error message:

python read_sweep_C.py
LstFileReader_FAST('/home/jfmoulin/Lab/Experiments/refsans/07_11_PMB/BPSi-JFM-B1-087.lst')
Traceback (most recent call last):
  File "read_sweep_C.py", line 278, in <module>
    ui = exple2()
  File "read_sweep_C.py", line 225, in exple2
    tof_resol=.05
  File "delay_line.pyx", line 103, in delay_line_C.UI.__init__
(delay_line.c:1212)
    self.det_reader = DetectorReader(fname=self.fname,reader=reader,
  File "delay_line.pyx", line 328, in
delay_line_C.DetectorReader.__init__ (delay_line.c:3445)
    self.list_mode_file = [ListModeFile(fname=fname,reader=reader)
  File "delay_line.pyx", line 719, in
delay_line_C.ListModeFile.__init__ (delay_line.c:7367)
    self.reader = eval('%s(\'%s\')'%(reader,self.fname))
  File "<string>", line 1, in <module>
NameError: name 'LstFileReader_FAST' is not defined

So it seems not to be a type error for an object which would have
a LstFileReader_FAST property or method, right ?

Okay, you asked for it..., here is the full code (yes it is still very
messy and full of FIXMEs but the python version does run...albeit
sooooo slow, hence Cython)

Cheers, JF

from __future__ import division
import numpy as np
cimport numpy as np
import scipy # FIXME??? cimport?
import exceptions
import types
import struct
import string
import sys
import subprocess
import cPickle
import time
import exceptions
from os.path import getsize
import struct



DTYPE_int = np.int
DTYPE_long = np.long
DTYPE_double = np.double
ctypedef np.int_t DTYPE_int_t
ctypedef np.long_t DTYPE_long_t
ctypedef np.double_t DTYPE_double_t

# Forward class declaration
cdef class UI
cdef class DetectorReader
cdef class TofSlabber
cdef class LstFileReader_FAST
cdef class DelayLine

cdef class UI:

    """
    A class dealing with the setup of a list mode file reader :

       fname: a filename or a list of filenames (lst files)
       reader: the name of the low-level reader class to use for this lst file
       delay_line_pars: is a dictonary or list of dicts describing the
                        delay line, keys are:
            'name': 'X' or 'Y' is the name of the delay line
            'anode': [devicenum,channelnum] is a list of 2 values telling where
                    to look for the anode signal (the files are numbered in
                    the same order as in fname)
            'cathodes': [[devnum1,channum1],[devnum2,channum2]] is a list of
                       one or two signal adresses in the format as above.
                       Each of these one/two addresses corresponds to one
                       end of the cathode
            'length': 342.7 is a value in ns for the length of the delay line
            'tol': 20 a tolerance in ns for the discrimination of events
            'npix': 256 the number of pixels for the full delay line length

      tof_resol : .1 the tof resolution used for histogramming
      tof_lims : the tof limits used for the histogram
                 (None goes to a 0-Tmax default)
      timing_accuracy : maximum possible mesuered missmatch in ns between two
                        events which are to be considered simultaneous
                        (mismatch could be due to cabling or timing of the
                        different cards, 3 ns should be safe)

    """

    cdef list fname # FIXME
    cdef list delay_line_pars
    cdef int timing_accuracy
    cdef DetectorReader det_reader
    cdef TofSlabber slabber
    cdef dict strobe_seq
    cdef str tof_histo

    cdef public np.ndarray slab, histo
    cdef public nsweep

    def __init__(self,\
                 fname,\
                 reader='LstFileReader_FAST',\
                 delay_line_pars=[{'name':'X',
                                   'anode':[0,0],
                                   'cathodes':[[0,1],[0,2]],
                                   'length':342.7,
                                   'tol':20,
                                   'npix':256
                                   },
                                  {'name':'Y',
                                   'anode':[0,0],
                                   'cathodes':[0,3],
                                   'length':342.7,
                                   'tol':20,
                                   'npix':256
                                  }
                                ],\
                 tof_resol = .1,\
                 tof_lims = None,\
                 timing_accuracy = 3


                ):

        self.fname = fname
        self.delay_line_pars = delay_line_pars
        self.timing_accuracy = timing_accuracy

        self.det_reader = DetectorReader(fname=self.fname,reader=reader,
                                         delay_line_pars=self.delay_line_pars,
                                         timing_accuracy = self.timing_accuracy

                                        )
        self.slabber = TofSlabber(resol=tof_resol,tof_lims=tof_lims,
                                  npix=(delay_line_pars[0]['npix'],
                                        delay_line_pars[1]['npix']
                                       )
                                 )
        #self.reporter = Lst_reporter(self.slabber)
        self.strobe_seq = {}
        tof_histo = 'tof_histo'


    def old_read(self,nsweep=1):
        """reads nsweep of data and does the tof analysis."""
        for i in range(nsweep):
            sweep = self.det_reader.next_sweep()

            cleaned = self.det_reader.clean_sweep(sweep)
            self.slabber._incr(cleaned)

    cdef bint _is_ok_sweep(self, tuple sweep):
        """
        Tests if a sweep is valid (full time between two starts) and increments
        the slabber.
        Returns False if sweep not valid, True if valid
        """
        cdef bint sweep_empty
        cdef int count, swc
        cdef str fi
        cdef tuple sw
        cdef tuple cleaned
        sweep_empty = False

        # For each subfile we test if we got a complete sweep (i.e not None)
        # and if it contains events or not
        for count,fi in enumerate(self.fname):
#           print 'uuuu',sweep[count]
            for swc,sw in enumerate(sweep):
                if sw is None:
                    print 'reached EOF for %s'%count
                    return False
                # A valid sweep maybe empty...
                if sw == ():
                    print 'No counts in last read() for line %s'\
                            %swc
                    sweep_empty = True
            # A valid non empty sweep needs cleaning and in either case
            # we should increment the slabber with the adequate values
            if not sweep_empty:
                cleaned = self.det_reader.clean_sweep(sweep)
                self.slabber._incr(cleaned)
                # let us make the slabs and histo directly
                # accessible from the UI (...in cython seems to be too deep)
                self.slab = self.slabber.slab.view()
                self.histo = self.slabber.tof_histo.view()
                self.nsweep = self.slabber.nsweep
            else:
                self.slabber._incr(())
                # let us make the slabs and histo directly
                # accessible from the UI (...in cython seems to be too
deep) FIXME property
                self.slab = self.slabber.slab.view()
                self.histo = self.slabber.tof_histo.view()
                self.nsweep = self.slabber.nsweep
            return True

    cpdef int read(self,int nsweep=0):
        """
        read nsweep sweeps.
        nsweep > 0 reads from present position in file.
        nsweep == 0 or None reads all remaining sweeps till EOF.
        nsweep <0 reads last nsweep sweeps in file.
        Returns the number of successfuly read sweeps
        """
        cdef int nsweeps_count
        cdef int answ
        cdef tuple sweep
        nsweeps_count = 0
        if nsweep == 0:
            answ = 1
            while  answ :#is not None:
                answ = self.read(1)# FAILS HERE!
            return nsweeps_count
        else:
            if nsweep > 0:
                for i in range(nsweep):
                    sweep = self.det_reader.next_sweep()
                    nsweeps_count += 1
                    if not self._is_ok_sweep(sweep):
                        return nsweeps_count - 1
            else:
                # To read the last sweeps we first need to go to EOF
                self.det_reader.goto_end()
                for i in range(-nsweep):
                    sweep = self.det_reader.prev_sweep()
                    nsweeps_count += 1
                    if not self._is_ok_sweep(sweep):
                        return nsweeps_count - 1
        return nsweeps_count

    def strobe(self,nrepeat=1,nsweep=100):
        """
        Stroboscopic measurement over nrepeat blocks of nsweep sweeps
        Updates self.strobe_seq into {'slab','tof_histo','intensity'}
        where each key leads to a list of corresponding values
        """
        li_intensity = []
        li_slab = []
        li_histo = []
        n_slabs = len(self.slabber.li_tof)+1
        npix = self.slabber.npix
        for repeat in range(nrepeat):
            self.slabber.slab = scipy.zeros((n_slabs,npix[0],npix[1]))
            self.slabber.tof_histo = scipy.zeros((n_slabs))
            for i in range(nsweep):
                sweep = self.det_reader.next_sweep()
                if sweep is not None:
                    cleaned = self.det_reader.clean_sweep(sweep)
                    self.slabber._incr(cleaned)
            try:
                li_slab.append(self.slabber.slab)
                li_histo.append(self.slabber.tof_histo)
                li_intensity.append(self.slabber.slab[0].sum())
            except:
                pass
        self.strobe_seq = {'slab':li_slab,
                           'tof_histo':li_histo,
                           'intensity':li_intensity
                          }

    def follow(self, intervall=5, begin_at_EOF=False):
        """
        reads up to EOF every intervall seconds.
        if begin_at_EOF is True waits for one intervall and then reads
        the data acquired since follow() was called (not since startMeas())
        """
        if begin_at_EOF:
            self.det_reader.goto_end()
            time.sleep(intervall)

        while True:
            print 'I start reading again!'
            self.read()
            print 'Done! I\'ll have a nap for %s seconds.'%intervall
            time.sleep(intervall)

    def analyze(self, nsweep=0, out='full'):
        """
        A convenience function which reads nsweep and returns data only
        (unlike read() which just updates the UI properties)
        out == 'full' returns {'intensity','histo','nsweep'}
        out == 'intensity' returns a Numpy array of intensities
        out == 'histo' returns a list of histogramms
        """
        # input sanity check
        #FIXME

        self.read(nsweep=nsweep)
        if out == 'full':
            out = {'intensity':self.slabber.slab,
                   'histo':self.slabber.tof_histo,
                   'nsweep':self.slabber.nsweep
                  }

        elif out == 'intensity':
            out = self.slabber.slab
        elif out == 'histo':
            out = self.slabber.tof_histo,
        else:
            raise exceptions.InputError

        return out

    def save_data(self,filename,gzip=True):
        """
        save the slab array to a file in [gziped] cPickle format
        """
        fiout = open(filename,'w')
        cPickle.dump(self.slabber.slab,fiout)
        fiout.close()
        if gzip:
            subprocess.call(['gzip',filename])
#       p.save(filename+'.gz',self.slabber.slab[0])

    #def display(self,slicenum=0):
    #    """
    #    Minimalistic display function
    #    """
    #    p.imshow(p.log10(self.slabber.slab[slicenum]+1),interpolation='nearest')
    #    p.colorbar()
    #    p.figure()
    #    p.semilogy(self.slabber.tof_histo + 1)


cdef class DetectorReader:
    """
    Reads a list mode file and returns the 2D slices using
    delay line parameters
    """

    cdef list fname # FIXME enforce it
    cdef list list_mode_file
    cdef int n_devices
    cdef list delay_line_pars # FIXME
    cdef list delay_line
    cdef int timing_accuracy

    def __init__(self,fname=None,reader='LstFileReader_FAST',\
                 delay_line_pars={'name':'X',\
                                  'anode':[0,0],\
                                  'cathodes':[[0,1],[0,2]] ,\
                                  'length':300,\
                                  'tol':4.00,\
                                  'npix':256\
                                 },\
                 timing_accuracy =3\
                ):


        self.fname = list(fname)


        # create a list of ListModeFile readers (one per input file)
        self.list_mode_file = [ListModeFile(fname=fname,reader=reader)
                               for fname in self.fname
                              ]
        self.n_devices = len(self.list_mode_file)

        if type(delay_line_pars) != types.ListType:
            delay_line_pars = [delay_line_pars]
        self.delay_line_pars = delay_line_pars

        # create a list of DelayLine instances
        # (one per dictionary of parameters given as input to __init__)
        self.delay_line = [DelayLine(dlp) for dlp in self.delay_line_pars]
        self.timing_accuracy = timing_accuracy

    def goto_end(self):
        """
        Goes to the last full sweep in lst
        """
        for fi in self.list_mode_file:
            fi.reader.goto_end()

    cdef tuple next_sweep(self):
        """
        Returns a raw sweep in the form [[signals dev0][signal dev1]...].
        Useful for sweep analysis (counter behaviour)
        """
        cdef tuple sweep
        cdef ListModeFile fi

        sweep = tuple([fi.next_sweep() for fi in self.list_mode_file])
        return sweep

    def prev_sweep(self):
        """
        Returns a raw sweep in the form [[signals dev0][signal dev1]...].
        Useful for sweep analysis (counter behaviour)
        """
        sweep = tuple([fi.prev_sweep() for fi in self.list_mode_file])
        return sweep


    cdef dict format_sweep(self,tuple raw_sweep,tuple
li_signals=('anode','cathodes')):
        """
        Returns a dictionary of signals according to the definitions
        given in self.delay_line_pars.
        This output is more human-readable
        """

        cdef dict out_sweep
        cdef dict dlp
        cdef str dl_name
        cdef str signal

        out_sweep = {}
        for dlp in self.delay_line_pars:
            dl_name = dlp['name']
            out_sweep[dl_name] = {}
            for signal in li_signals:

                #print 'tttttt',type(dlp[signal][0])
                #print 'dlp[sig]',type(dlp[signal][0])
                if type(dlp[signal][0]) != types.IntType:
                    try:
                        out_sweep[dl_name][signal] =
tuple([raw_sweep[dlp[signal]
                                                               [terminal]
                                                               [0]
                                                           ]
                                                           [dlp[signal]
                                                               [terminal]
                                                               [1]
                                                           ]
                                                  for terminal in [0,1]
                                                ])
                    except:
                        print 'Signal not available'
                        return None # FIXME dangerous for a tuple???
                #except:
                else:
                    try:
                        out_sweep[dl_name][signal] = raw_sweep[dlp[signal][0]]\
                                                          [dlp[signal][1]]
                    except:
                        print 'Signal not available'
                        return None

                #except: #exceptions.TypeError:

                    ## we got an empty raw_sweep!
                    #print 'we got an empty raw sweep',raw_sweep
                    #return None


        return out_sweep

    cpdef tuple clean_sweep(self, tuple sweep):
        """
        Performs the discri on the sweep and tries to reconstruct missing
        events as much as possible
        """
        cdef dict formated_sweep
        cdef np.ndarray discriminatedX, discriminatedY,evx # FIXME USE
buffer nottation !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!11
        cdef list ev#FIXME!!!!!!!!!!!!evx to tuple??
        cdef np.ndarray index_evy
        formated_sweep = self.format_sweep(sweep)

        if formated_sweep is None:
            return ()

        discriminatedX = self.delay_line[0].do_discri(formated_sweep)
        discriminatedY = self.delay_line[1].do_discri(formated_sweep)
        #print 'discriX',discriminatedX
        #print 'discriY',discriminatedY
        # !!!! FIXME should avoid the search by being cleverer
        ev = []
        for evx in discriminatedX:
            try:
                index_evy = scipy.nonzero(abs(discriminatedY[:,0]-evx[0]) <
                                          self.timing_accuracy
                                         )[0]
                if len(index_evy)!= 0:
                    ev.append(tuple([evx[0],self.delay_line[0]._locate(evx),
                                self.delay_line[1]._locate(
                                                discriminatedY[index_evy][0]
                                                    )
                            ])
                            )# FIXME !!!be sure that there coulld be
only one evy
            except:
                pass
            #    print 'fixme with user defined error npixrange 11'
            #except:
                #print 'ahaaaaa!'
                #pass
        return tuple(ev)



cdef class DelayLine:
    """
    A class describing a delay line (addresses and length) as well
    as providing the discrimination functions
    """
    cdef public dict delay_line_pars # FIXME is this optimal
    cdef bint has_anode #bool fails withdelay_line.c:443: warning:
implicit declaration of function ?PyBool_CheckExact? at compile and
<type 'exceptions.ImportError'>:
/home/jfmoulin/My_Progs/refsans/refsans_cython/read_sweep/C/delay_line_C.so:
undefined symbol: PyBool_CheckExact at runtime
    cdef int has_cathodes
    cdef double length
    cdef double tol
    cdef public tuple window
    cdef list li_sig #FIXME
    cdef int npix
    cdef double npix_2, npix_length, npix_length_2
    cdef int (*locate)(DelayLine, np.ndarray) #FIXME
http://www.mail-archive.com/cython-dev at codespeak.net/msg02700.html)
    cdef bint (*discri)(DelayLine, tuple)
        #------------------------

    def __init__(self,dict delay_line_pars):
        self.delay_line_pars = delay_line_pars
        self.has_anode = self.delay_line_pars.has_key('anode')
        self.has_cathodes = 0
        self.length = self.delay_line_pars['length']
        self.tol = self.delay_line_pars['tol']
        self.window = (self.length - self.tol , self.length + self.tol)
        if self.delay_line_pars.has_key('cathodes'):
            try:
                self.delay_line_pars['cathodes'][1][0]
                self.has_cathodes = 2
            except:
                self.has_cathodes = 1
                self.locate = self.locate_single_terminal
        if self.has_anode and self.has_cathodes == 2:
            self.li_sig = ['anode','cathodes']
            self.locate = self.locate_two_terminals
        self.npix = self.delay_line_pars['npix']
        self.npix_2 = self.npix * .5
        self.npix_length = self.npix  / self.length
        self.npix_length_2 = self.npix_2  / self.length

    cpdef int _locate(self, np.ndarray signals):
        return self.locate(self, signals) # FIXME cython strangeness cdef fails
#even if call is from cython code!

    cdef bint discri_complete(self, tuple signals):
        """
        The easy case where we have exactly one anode and two cathodes signals
        """
        cdef int calc_length
        cdef bint discri_sum

        calc_length = signals[1] + signals[2] - 2 * signals[0]
        discri_sum = calc_length >= self.window[0] and \
                     calc_length <= self.window[1]
        return discri_sum

    cdef bint discri_single_cath(self, tuple signals):
        #FIXME is tuplethe best here???
        cdef int calc_pos
        cdef bint discri_pos

        calc_pos = signals[1] - signals[0]
        discri_pos = calc_pos <= self.window[1]
        return discri_pos

    cpdef np.ndarray do_discri(self,dict sweep):
        cdef list li_discriminated
        cdef int count_anode
        cdef int anode
        cdef tuple li_anode,sig # FIXME NAME??
        cdef bint only_one_ev
        cdef tuple li_cathodes

        self.discri = self.discri_complete
        # extract what is relevant to this delay line only
        sweep = sweep[self.delay_line_pars['name']]
        li_discriminated = []
        for count_anode , anode in enumerate(sweep['anode']):

            li_anode = self.within_delay(sweep['anode'],anode)
            only_one_ev = len(li_anode) == 1
            if self.has_cathodes == 2:
                li_cathodes =
tuple([self.within_delay(sweep['cathodes'][cathcount],
                                                 anode
                                                )
                               for cathcount in [0,1]
                              ])
                only_one_ev = only_one_ev and \
                              len(li_cathodes[0]) == 1 and \
                              len(li_cathodes[1]) == 1
            else:
                li_cathodes = self.within_delay(sweep['cathodes'],anode)
                only_one_ev = only_one_ev and len(li_cathodes) == 1
                self.discri = self.discri_single_cath

            if only_one_ev:
                if self.has_cathodes==2:
                    sig = (li_anode[0],li_cathodes[0][0],li_cathodes[1][0])
                else:
                    sig = (li_anode[0],li_cathodes[0])
                if self.discri(self,sig):# FIXME why must I put self
in the args!?
                    li_discriminated.append(sig)
                #else:
                #    print 'rej',sig

        discriminated = scipy.array(li_discriminated)
        return discriminated

    cdef tuple within_delay(self,tuple li_signals,int anode):
        cdef int tmax
        cdef int ev
        tmax = self.window[1] + anode
        return tuple([ev for ev in li_signals if ev>=anode and ev<=tmax])

    cdef int locate_single_terminal(self,np.ndarray signals):
        """
        Position the event on a self.pix scale (if self.npix is None,
        the return is continuous over [0,1]
        Input should be an array [anode, c1]
        """
        cdef int pos
        pos = int(round((signals[1] - signals[0]) * self.npix_length))
        return pos

    cdef int locate_two_terminals(self,np.ndarray signals):
        """
        Position the event on a self.pix scale (if self.npix  is None,
        the return is continuous over [0,1]
        Input should be an array [anode, c1, c2]
        """
        cdef int pos

        try:
            pos =  int(round(self.npix_2 + \
                       (signals[:,2] - signals[:,1]) * self.npix_length_2)
                      )
        except:
            pos =  int(round(self.npix_2 + \
                       (signals[2] - signals[1]) * self.npix_length_2)
                      )
        return pos

cdef class TofSlabber:
    """A class implementing the TOF histogramming """

    # NOTE:  for performance reasons defined directly only for 2D images
    # do it for 1D counters...

    # FIXME add a check to avoid incrementing severall times for
    # the same sweep?????


    cdef tuple tof_lims
    cdef double resol
    cdef tuple npix
    cdef public int nsweep
    cdef np.ndarray li_tof, li_tof_low, li_tof_high
    cdef public np.ndarray slab, tof_histo



    def __init__(self,tof_lims=None,resol=.1,npix=(256,256)):
        self.tof_lims = tof_lims
        self.resol = resol
        self.npix = npix
        self.nsweep = 0

        if self.tof_lims  is not None:
            self.init_tof_array()

    def init_tof_array(self):
        """
        Initializes the tof histogram
        """
        n_slabs =
round(scipy.log(self.tof_lims[1]/self.tof_lims[0])/scipy.log(1 +
self.resol)) + 1
        #print 'tof_lims/resol (init_tof_array)',self.tof_lims,self.resol
        #print 'n_slabs (init_tof_array)',n_slabs
        self.li_tof = scipy.append(scipy.array([0]),
                                   self.tof_lims[0] *
                                   scipy.logspace(0,
                                                  n_slabs,
                                                  num=n_slabs,
                                                  base=1 + self.resol
                                                 )
                                  )
        #print 'li_tof (init_tof_array)',self.li_tof

        self.li_tof_low = (1 - .5 * self.resol) * self.li_tof
        self.li_tof_high = (1 + .5 * self.resol) * self.li_tof

        self.slab = scipy.zeros((n_slabs,self.npix[0],self.npix[1]))

        self.tof_histo = scipy.zeros((n_slabs))

    def _incr(self,sweep):
        self.incr(sweep)

    cdef void incr(self,tuple sweep) except *:
        """
        Increments the tof histogramm and the different channels with the
        events found in a sweep.
        sweep should be a tuple.
        """


        cdef tuple ev
        cdef int slab_num

        #print 'sweep (input to incr)',sweep
        self.nsweep += 1
        if sweep == () :

            return
        if self.tof_lims  is None:
#            print (s.array(sweep[0]).min() *
.9,s.array(sweep[:0]).max() * 1.15)
            self.tof_lims = (int(scipy.array(sweep)[:,0].min() * .9),\
                             1 + int(scipy.array(sweep)[:,0].max() * 1.5)
                            )
            self.init_tof_array()

        for ev in sweep:
            slab_num = scipy.nonzero(ev[0]>=self.li_tof_low)[0][-1]
            # try to increment the correct tof slice
        #    try:
            #print 'slab_num (incr)', slab_num
            if slab_num > len(self.li_tof)-2:
                print 'Event rejected, tof>tmax'
            else:
                try:
                    self.slab[slab_num][ev[1],ev[2]] += 1
                    # increment the integrated signal
                    self.slab[0][ev[1],ev[2]] += 1
                    # increment the tof histogram
                    self.tof_histo[slab_num] += 1
                except exceptions.IndexError:
                    print 'fixme with user error about range'#,sweep
              #  pass


cdef class ListModeFile:
    """
    Implementation of all operations on list mode files.
    This class provides a level of abstraction to separate from the
    details of the file structure.
    fname: name of the file to be read
    reader: the name of a class defining a reader for this file.
            See LstFileReader_FAST for details of necessary methods.
    """

    cdef str fname
    cdef  LstFileReader_FAST reader# FIXME!!!!

    def __init__(self,fname,reader='LstFileReader_FAST'):
        self.fname = fname
        print '%s(\'%s\')'%(reader,self.fname)
        self.reader = eval('%s(\'%s\')'%(reader,self.fname))

#        self.reader = eval('low_level_reader.%s(\'%s\')'\
#                           %(reader,self.fname)
#                          )#FIXME into something less harmful

    cpdef tuple next_sweep(self):
        """
        Returns the next available full sweep starting from the present
        position in the file
        """
        return self.reader.next_sweep()

    def prev_sweep(self):
        """
        Reads the previous full sweep starting from present pos in file
        """
        return self.reader.prev_sweep()

    def goto_start(self):
        """
        Goes to first sweep in file.
        """
        self.reader.goto_start()

    def goto_end(self):
        """
        Goes to last full sweep in file.
        """
        self.reader.goto_end()

    def nevents(self):
        """
        Returns the number of events in the file (incl start signals)
        """
        return self.reader.nevents()


cdef class LstFileReader_FAST:

    """
    Implement operations on a FAST  list mode file (.lst).
    This is the low level reader
    """
    cdef str fname
    cdef tuple (*do_read_ev)(LstFileReader_FAST)
    cdef file lstfile # FIXME
    cdef list header


    def __init__(self,fname):

        self.fname = fname
        self.goto_start()
        # we define a function pointer to the reader.next in order to be
        # able to change the reading direction easily
        self.do_read_ev = self._next_ev

    def __iter__(self):

        return self

    def goto_start(self):

        """
        Read the header. Goto first event in file.
        """

        try:
            self.lstfile.close()
        except exceptions.AttributeError:
            pass

        self.lstfile = open(self.fname,'rb')

        self.header = []
        while True:
            line = self.lstfile.readline()
            #print line
            if '[DATA]' in line:
                break
            else:
                self.header.append(line)


    def goto_end(self):

        """
        Goto the position of the last fully written event in file.
        """

        # Count the events in file, there are 4 bytes per event...
        nev = self.nevents()
        end_pos = nev * 4
        last_ev_pos = (nev - 1) * 4
        self.lstfile.seek(last_ev_pos,1)

    def nevents(self):
        """
        return the number of events in file (incl starts)
        """
        self.goto_start()
        start_offset = self.lstfile.tell()
        nev = round((getsize(self.fname) - start_offset)/4)
        return nev

    cdef tuple prev(self):
        return self._prev()

    cdef tuple _prev(self):

        """
        read previous event in file, jump back before its position and
        return [channel,value].
        Use to read a file backwards.
        """

        cdef tuple ev

        try:
            self.lstfile.seek(-4,1)
        except exceptions.IOError:
            return None
        ev = self.next_ev()
        self.lstfile.seek(-4,1)
        return ev


    cdef tuple next_ev(self):
        return self._next_ev()

    cdef tuple _next_ev(self):

        """
        Read the next event in file and decode it. Return [chan_number, value]
        """
        cdef int input_chan
        cdef int value
        cdef str st_data
        cdef int data
        cdef int bit30
        cdef int bit31

        input_chan = -1#None
        value =  -1#None
        # cython trick for strings from python (see manual p15))!
#        s = self.lstfile.read(4)
        #print self.lstfile.tell()
        #print 's',type(s),s
        st_data = self.lstfile.read(4)
        #print 'st_data',st_data
        #print data
        try:
            data = struct.unpack('i',st_data)[0]
         #   print 'data', data

        except struct.error:
          #  print 'struct_error!return None'
            return (None,)
        #Reading bits 30 and 31 for input channel:
        #struct.unpack('>i','\x40\x00\x00\x00')[0] != 0)
        bit30 = int(data & 1073741824!=0)
        #struct.unpack('>i','\x80\x00\x00\x00')[0] !=0)
        bit31 = int(data & -2147483648!=0)
        #print 'bits', bit30, bit31
        input_chan = 1 * bit30 + 2 * bit31
#        key = str(bit30) + str(bit31)
        #print 'key',key
        #print bit30, bit31, 'event @',dict_input[key]

        #value of int(bits0_29): turn the bits 30 and 31
        # off and convert the value
        #struct.unpack('>i','\x3f\xff\xff\xff')[0]
        value = 32 + data & 1073741823
        #print value
        #vals[dict_input[key]-1].append(value)
        #input_chan = LstFileReader_FAST.dict_input[key]
        #print 'chan,val',input_chan, value
        return (input_chan, value)

    cdef tuple next_sweep(self):
        return self._next_sweep()

    cdef tuple _next_sweep(self):



        """
        Returns the next available full sweep starting from the present
        position in the file (implement premature eof!!!) FIXME
        """
        cdef list channels#, last_ev
        cdef tuple last_ev
        cdef list chan


        channels = []
        last_ev = (None, None)
      #  last_ev = [[],[]]
        # if necessary look for start event
        try:
            while last_ev[1] != 32:
                last_ev = self.do_read_ev(self)# FIXME
         #       print 'last_ev',last_ev
        except exceptions.TypeError:
            return None
        last_ev = (None,None)
        # then accumulate data up to next start
        while last_ev[1] != 32:
            last_ev = self.do_read_ev(self) # FIXME
            # in case of EOF before START signal,
            # we return None (uncomplete sweep)
            if last_ev[0] is None : return None
            # if we got a start signal we are done
            if last_ev[1] == 32:
                return tuple([tuple(chan) for chan in channels])
            else:
                # auto adjusting the number of channels
                try:
                    channels[last_ev[0]].append(last_ev[1])
                except:
                    channels.extend([[] for count in
                                     range(last_ev[0]-len(channels)+1)
                                    ]
                                   )
                    channels[last_ev[0]].append(last_ev[1])

        return tuple([tuple(chan) for chan in channels])

    cdef tuple prev_sweep(self):
        """
        Reads previous full sweep and goes back to its starting pos.
        Use to read backwards.
        """
        cdef tuple channels
        self.do_read_ev = self.prev
        channels = self.next_sweep()
        if channels is not None:
            # we read backwards... so we have to reorder the events
            # in increasing time
            channels = [chan[::-1] for chan in channels]
            # restore the normal reading direction of the file
        self.do_read_ev = self.next_ev

        return channels


    def progress(self):
        """
        progress() returns (Mbytes_read,file_progress(%))
        """
        return(self.lstfile.tell()/(1024*1024))#,fprogress(self.lstfile))






On Tue, Apr 21, 2009 at 10:23 AM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Apr 21, 2009, at 12:45 AM, Jean-Francois Moulin wrote:
>
>> Hi,
>>
>> first of all, thanks for the replies.
>>
>> As for the first answer by Lisandro, I do not understand then why my
>> first code fragment at all works...
>> The only difference is that the function called has two params, one
>> instance of one of my classes and a numpy array, while the faulty
>> piece of code should be called on an instance
>> of one of my classes only...
>>
>> As for the second, from Robert, now, very well... how do I specify a
>> value for this implicit parameter then!? It is a bint I guess... I
>> tried to set it to 1 and got new error messages:
>> delay_line.pyx:961:30: Cannot assign type 'tuple (LstFileReader_FAST,
>> int __pyx_skip_dispatch, struct
>> __pyx_opt_args_12delay_line_C_18LstFileReader_FAST_next_ev
>> *__pyx_optional_args)' to 'tuple (*)(LstFileReader_FAST)'
>
> You can't. I agree the errors are more cryptic than they should be,
> but C functions have no concept of runtime dispatching or default
> arguments, so it doesn't work to cast such a function to a C function
> (well, maybe one could hack it, but it'd be ugly and fragile).
>
>> I also tried to use cdef in all cases, I get no compile time error but
>> at runtime my object LstFileReader_FAST appears no longer to be
>> defined!? I do not understand because all the calls are made from
>> within Cython code... (so that I expect the cpdef is not really
>> needed)
>
> I am guessing you're doing something like
>
> a. LstFileReader_FAST(...)
>
> and it's complaining that "a has not attribute 'LstFileReader_FAST'".
> Usually this indicates that a is not typed correctly (e.g. it thinks
> a is an ordinary object). You can run cython with the -a option to
> easily see this kind of thing.
>
> - Robert
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>

From stefan_ml at behnel.de  Tue Apr 21 11:23:45 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 21 Apr 2009 11:23:45 +0200 (CEST)
Subject: [Cython] Path towards Python completeness
In-Reply-To: <80500050-599D-4F3A-9FA1-0291E43B3B99@math.washington.edu>
References: <49ED5CE8.6090708@behnel.de>
	<80500050-599D-4F3A-9FA1-0291E43B3B99@math.washington.edu>
Message-ID: <380c38b6623e9a1a6d12a649dee43ec4.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Robert Bradshaw wrote:
> On Apr 20, 2009, at 10:43 PM, Stefan Behnel wrote:
>> Closure support will actually open up a can of worms,
>> but a friendly one, as it will lay out the path towards lambda
>> support and nested functions/classes. Especially support for
>> generators seems to be an extremely worthy goal, but in the current
>> state of Cython, it just cannot be done.
>
> All of the above are syntactic sugar for special closures.

Sure, but I would say that closure support is more or less finished when
nested functions work. The rest are additional steps to add more syntax
features on top of that.

For example, generator support needs code that supports continuing
function execution. Lambda functions need a transform into nested
functions. That's additional work, but nothing that's worth caring about
right now.

The only way to get this working is to set achievable partial goals that
we (or others) can build on. Even nested classes are out of scope for the
first milestone, IMHO.


> This has been bugging me for a *long* time. I'll pull out and go
> though the random scattered stuff I've done over the past year
> (something else more urgent has always come up everytime I've set out
> to do this) and put it up within the week.

Great!


> Should I make a closure branch on the site?

You'd create it from cython-unstable, I assume. Given that the current
development works reasonably well with two branches (pull-direction
unstable<-devel), I doubt that a third "closures" branch (pull-direction
closures<-unstable) would add any major additional overhead, while still
allowing us to finish up and release 0.12 from cython-unstable when it's
time.

I think that's a good idea. Should we make (basic?) closure support an
official 0.13 goal?

Stefan


From stefan_ml at behnel.de  Tue Apr 21 11:39:54 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 21 Apr 2009 11:39:54 +0200 (CEST)
Subject: [Cython] help with function pointer
In-Reply-To: <af6d35810904210219y45a12adbr644cbb219a3d6e1d@mail.gmail.com>
References: <af6d35810904200817p62e92366ld616a491bf9b63da@mail.gmail.com>
	<6F08D037-7371-4D43-B7FE-8185FD8B25DB@math.washington.edu>
	<af6d35810904210045k105c4178t29f68b621d1354c5@mail.gmail.com>
	<CAD789E2-DAD2-4605-9878-318E7A2ECD33@math.washington.edu>
	<af6d35810904210219y45a12adbr644cbb219a3d6e1d@mail.gmail.com>
Message-ID: <7e35120d0d391d2840926cf8e034e41b.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Jean-Francois Moulin wrote:
> Okay, you asked for it..., here is the full code

I didn't really ask for the "full" code. It's rather uncommon to post code
of that size to a mailing list, especially as (fragile) part of the
message body. Next time, please consider pasting it into a web site
(pastebin or whatever), or at least *attach* it to an e-mail, so that
people can safely safe it to a file.

In this specific case, the traceback would have been enough, or at least a
hint that you are using eval():

   delay_line_C.ListModeFile.__init__ (delay_line.c:7367)
       self.reader = eval('%s(\'%s\')'%(reader,self.fname))
     File "<string>", line 1, in <module>
   NameError: name 'LstFileReader_FAST' is not defined

Cython does not currently support globals(), which is used by eval()
internally. Any reason why you pass the class name as parameter instead of
the class itself?

Stefan


From robertwb at math.washington.edu  Tue Apr 21 11:39:54 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 21 Apr 2009 02:39:54 -0700
Subject: [Cython] help with function pointer
In-Reply-To: <af6d35810904210219y45a12adbr644cbb219a3d6e1d@mail.gmail.com>
References: <af6d35810904200817p62e92366ld616a491bf9b63da@mail.gmail.com>
	<6F08D037-7371-4D43-B7FE-8185FD8B25DB@math.washington.edu>
	<af6d35810904210045k105c4178t29f68b621d1354c5@mail.gmail.com>
	<CAD789E2-DAD2-4605-9878-318E7A2ECD33@math.washington.edu>
	<af6d35810904210219y45a12adbr644cbb219a3d6e1d@mail.gmail.com>
Message-ID: <90EEE860-DAED-474D-A2E3-F1857ED62699@math.washington.edu>

On Apr 21, 2009, at 2:19 AM, Jean-Francois Moulin wrote:

> Hi!
> thanks for the answers!
>
> Here is the runtime error message:
>
> python read_sweep_C.py
> LstFileReader_FAST('/home/jfmoulin/Lab/Experiments/refsans/ 
> 07_11_PMB/BPSi-JFM-B1-087.lst')
> Traceback (most recent call last):
>   File "read_sweep_C.py", line 278, in <module>
>     ui = exple2()
>   File "read_sweep_C.py", line 225, in exple2
>     tof_resol=.05
>   File "delay_line.pyx", line 103, in delay_line_C.UI.__init__
> (delay_line.c:1212)
>     self.det_reader = DetectorReader(fname=self.fname,reader=reader,
>   File "delay_line.pyx", line 328, in
> delay_line_C.DetectorReader.__init__ (delay_line.c:3445)
>     self.list_mode_file = [ListModeFile(fname=fname,reader=reader)
>   File "delay_line.pyx", line 719, in
> delay_line_C.ListModeFile.__init__ (delay_line.c:7367)
>     self.reader = eval('%s(\'%s\')'%(reader,self.fname))
>   File "<string>", line 1, in <module>
> NameError: name 'LstFileReader_FAST' is not defined
>
> So it seems not to be a type error for an object which would have
> a LstFileReader_FAST property or method, right ?

Well, it was a blind guess not having the code :).

> Okay, you asked for it..., here is the full code (yes it is still very
> messy and full of FIXMEs but the python version does run...albeit
> sooooo slow, hence Cython)

Not sure exactly what the bug is here, but it does look more like a  
bug. Maybe you could boil this down to a isolated couple of lines  
that produces the error?

- Robert


From robertwb at math.washington.edu  Tue Apr 21 11:40:58 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 21 Apr 2009 02:40:58 -0700
Subject: [Cython] Path towards Python completeness
In-Reply-To: <380c38b6623e9a1a6d12a649dee43ec4.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
References: <49ED5CE8.6090708@behnel.de>
	<80500050-599D-4F3A-9FA1-0291E43B3B99@math.washington.edu>
	<380c38b6623e9a1a6d12a649dee43ec4.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <85093F3A-89F0-4324-9156-9DADE52C8B95@math.washington.edu>

On Apr 21, 2009, at 2:23 AM, Stefan Behnel wrote:

> Robert Bradshaw wrote:
>> On Apr 20, 2009, at 10:43 PM, Stefan Behnel wrote:
>>> Closure support will actually open up a can of worms,
>>> but a friendly one, as it will lay out the path towards lambda
>>> support and nested functions/classes. Especially support for
>>> generators seems to be an extremely worthy goal, but in the current
>>> state of Cython, it just cannot be done.
>>
>> All of the above are syntactic sugar for special closures.
>
> Sure, but I would say that closure support is more or less finished  
> when
> nested functions work. The rest are additional steps to add more  
> syntax
> features on top of that.
>
> For example, generator support needs code that supports continuing
> function execution. Lambda functions need a transform into nested
> functions. That's additional work, but nothing that's worth caring  
> about
> right now.
>
> The only way to get this working is to set achievable partial goals  
> that
> we (or others) can build on. Even nested classes are out of scope  
> for the
> first milestone, IMHO.

+1

>
>
>> This has been bugging me for a *long* time. I'll pull out and go
>> though the random scattered stuff I've done over the past year
>> (something else more urgent has always come up everytime I've set out
>> to do this) and put it up within the week.
>
> Great!
>
>
>> Should I make a closure branch on the site?
>
> You'd create it from cython-unstable, I assume. Given that the current
> development works reasonably well with two branches (pull-direction
> unstable<-devel), I doubt that a third "closures" branch (pull- 
> direction
> closures<-unstable) would add any major additional overhead, while  
> still
> allowing us to finish up and release 0.12 from cython-unstable when  
> it's
> time.
>
> I think that's a good idea. Should we make (basic?) closure support an
> official 0.13 goal?

Yep, sounds like a good plan.

- Robert


From jf.moulin at gmail.com  Tue Apr 21 12:06:18 2009
From: jf.moulin at gmail.com (Jean-Francois Moulin)
Date: Tue, 21 Apr 2009 12:06:18 +0200
Subject: [Cython] help with function pointer
In-Reply-To: <90EEE860-DAED-474D-A2E3-F1857ED62699@math.washington.edu>
References: <af6d35810904200817p62e92366ld616a491bf9b63da@mail.gmail.com>
	<6F08D037-7371-4D43-B7FE-8185FD8B25DB@math.washington.edu>
	<af6d35810904210045k105c4178t29f68b621d1354c5@mail.gmail.com>
	<CAD789E2-DAD2-4605-9878-318E7A2ECD33@math.washington.edu>
	<af6d35810904210219y45a12adbr644cbb219a3d6e1d@mail.gmail.com>
	<90EEE860-DAED-474D-A2E3-F1857ED62699@math.washington.edu>
Message-ID: <af6d35810904210306p7ab1c80bk6f5433074f8eec2a@mail.gmail.com>

Very sorry for this netiquette mistake... was too impatient I guess :0(

I will try to produce something shorter as requested!

As for the use of eval and the name of the class, it comes straight of
my python version were I used to import a module which contained the
class definition. So the code was self adapting in each
new case...
The idea was that this part of the code, which is a low-level reader
for some binary file could be changed to accomodate different vendor
versions or types of input files.
Maybe that is a stupid way of doing...
So stupid question now...: can I use a pointer to a class?

From stefan_ml at behnel.de  Tue Apr 21 13:16:31 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 21 Apr 2009 13:16:31 +0200 (CEST)
Subject: [Cython] help with function pointer
In-Reply-To: <af6d35810904210306p7ab1c80bk6f5433074f8eec2a@mail.gmail.com>
References: <af6d35810904200817p62e92366ld616a491bf9b63da@mail.gmail.com>
	<6F08D037-7371-4D43-B7FE-8185FD8B25DB@math.washington.edu>
	<af6d35810904210045k105c4178t29f68b621d1354c5@mail.gmail.com>
	<CAD789E2-DAD2-4605-9878-318E7A2ECD33@math.washington.edu>
	<af6d35810904210219y45a12adbr644cbb219a3d6e1d@mail.gmail.com>
	<90EEE860-DAED-474D-A2E3-F1857ED62699@math.washington.edu>
	<af6d35810904210306p7ab1c80bk6f5433074f8eec2a@mail.gmail.com>
Message-ID: <b2476612f3966aba2541532431243305.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Jean-Francois Moulin wrote:
> As for the use of eval and the name of the class, it comes straight of
> my python version were I used to import a module which contained the
> class definition. So the code was self adapting in each
> new case...

Well, eval() is not a good idea even in Python, as it requires you to
convert the parameters to a string first and then parse them again, which
may break things if they cannot be property escaped (see "SQL injection" &
friends).


> The idea was that this part of the code, which is a low-level reader
> for some binary file could be changed to accomodate different vendor
> versions or types of input files.
> Maybe that is a stupid way of doing...
> So stupid question now...: can I use a pointer to a class?

Sure, a class is a Python object. Just do what you'd do in Python: pass
the object and call it.

Stefan


From stefan_ml at behnel.de  Tue Apr 21 13:16:31 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 21 Apr 2009 13:16:31 +0200 (CEST)
Subject: [Cython] help with function pointer
In-Reply-To: <af6d35810904210306p7ab1c80bk6f5433074f8eec2a@mail.gmail.com>
References: <af6d35810904200817p62e92366ld616a491bf9b63da@mail.gmail.com>
	<6F08D037-7371-4D43-B7FE-8185FD8B25DB@math.washington.edu>
	<af6d35810904210045k105c4178t29f68b621d1354c5@mail.gmail.com>
	<CAD789E2-DAD2-4605-9878-318E7A2ECD33@math.washington.edu>
	<af6d35810904210219y45a12adbr644cbb219a3d6e1d@mail.gmail.com>
	<90EEE860-DAED-474D-A2E3-F1857ED62699@math.washington.edu>
	<af6d35810904210306p7ab1c80bk6f5433074f8eec2a@mail.gmail.com>
Message-ID: <b2476612f3966aba2541532431243305.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Jean-Francois Moulin wrote:
> As for the use of eval and the name of the class, it comes straight of
> my python version were I used to import a module which contained the
> class definition. So the code was self adapting in each
> new case...

Well, eval() is not a good idea even in Python, as it requires you to
convert the parameters to a string first and then parse them again, which
may break things if they cannot be property escaped (see "SQL injection" &
friends).


> The idea was that this part of the code, which is a low-level reader
> for some binary file could be changed to accomodate different vendor
> versions or types of input files.
> Maybe that is a stupid way of doing...
> So stupid question now...: can I use a pointer to a class?

Sure, a class is a Python object. Just do what you'd do in Python: pass
the object and call it.

Stefan


From dagss at student.matnat.uio.no  Tue Apr 21 14:04:56 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 21 Apr 2009 14:04:56 +0200
Subject: [Cython] Fortran GSoC accepted
Message-ID: <49EDB668.7030502@student.matnat.uio.no>

I'm very happy to report that Kurt Smith has been accepted for a Google 
Summer of Code project for working on Cython/Fortran integration!

http://wiki.cython.org/kwsmith/soc09

I'm his mentor for the project, and I start this thread to discuss how 
the GSoC should be arranged. (I suppose part of this could apply to the 
other accepted project as well:

http://socghop.appspot.com/student_project/show/google/gsoc2009/python/t124024629578

).

The project is roughly going to be divided in two parts:
  (A) Use the f2py parser and output code for a Cython binding
  (B) Improve Cython in some areas (especially buffers) so that the 
bindings generated in a) can be much less messy and more convenient to use.

Some things:

1) Where should the mentoring take place? Opinions on how much should 
happen on this list and how much should be done in private mail? 
Obviously language decisions etc. must end up here in the end, but what 
about day-to-day mentoring etc.?

One possibility is setting up a seperate Google group just for this 
project, if it turns out that it would be too high traffic for the 
mailing list but other people would like to follow the project as well.

2) Code and patch workflow.

On the (A) side, the status with f2py is that the author started a full 
rewrite (f2py 3G) which hasn't seen work for half a year 
(http://f2py.org/). So anything can happen here and we have to get back 
to how it's going to be done and where it should live. A relevant 
question though is whether we would like to ship the tool in (A) as part 
of Cython or have it as an independent download. (I don't expect it to 
add anything significant to download size, but Python core inclusion 
would be something to consider for or against here.)

When it comes to (B), the obvious thing to do is for Kurt to pass the 
code to me for review and then merge it as often as we can with either 
-devel or -unstable (not sure which yet, possibly both).

I'm wondering though whether perhaps Kurt could just get commit rights 
for the repos, under the strict understanding that he doesn't push 
anything which is not reviewed (for now at least). That could take some 
day-to-day merges off my back and make Kurt responsible for doing stable 
merges and think about the code integration schedule right from the start.

We could then use e.g. codereview.appspot.com for doing the code 
reviews; I would then simply give thumbs up and Kurt would do the push.

3) Feedback

There should be some kind of feedback mechanism; I expect to hear from 
Kurt at least weekly over the summer unless otherwise is arranged. I 
expect the ticket system will serve as the status update, and hope Kurt 
will make his progress visible through the ticket system (we can use the 
"fortran" tag on all GSoC related tickets for instance).

I guess (A) would perhaps use a different tracker than (B), we need to 
get back to that. We could set something up at e.g. Google Code if the 
rest of the Cython team wouldn't like (A)-related tickets cluttering up 
the trac. We could use a seperate milestone though...

Some mentors require students to keep a blog about what they are doing. 
Unless the Python Software Foundation makes this a requirement I'm going 
to leave this one up to Kurt; if the status is dutifully reported via 
tickets I'll have what I need personally, though a blog would be 
interesting as well.

Dag Sverre

From dagss at student.matnat.uio.no  Tue Apr 21 14:18:07 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 21 Apr 2009 14:18:07 +0200
Subject: [Cython] Path towards Python completeness
In-Reply-To: <49ED5CE8.6090708@behnel.de>
References: <49ED5CE8.6090708@behnel.de>
Message-ID: <49EDB97F.4030004@student.matnat.uio.no>

Stefan Behnel wrote:
> Hi,
> 
> I just read the release announcement of PyPy's 1.1 beta:
> 
> http://morepypy.blogspot.com/2009/04/beta-for-110-released.html
> 
> According to their benchmarks, they are now less than 2x slower than
> CPython in most cases and in some cases even 20% faster. Cython is
> currently about 30% faster than CPython for plain Python code, but that
> only concerns the "Python code that Cython supports", which is not at all
> comparable to what PyPy supports.
> 
> My feeling is that the goal to make CPython support "most of Python" is
> getting more important simply because PyPy exists. There is not that much
> that is missing, but the sad thing is that the main part that *is* missing
> is a very important part of the language: closures. 

I'm very happy that work on closures will be resumed and this is simply 
a digression:

I don't think we have to think of ourselves as in direct competition 
with PyPy (and/or Unladen Swallow). Personally I'm more than happy to 
use pure Python rather than Cython for the kind of things that can be 
sped up by those projects anyway (makes for a shorter compile/run cycle).

What I think sets Cython apart is that we add a superset to the Python 
language for adding types; which are simply needed in some situations. I 
think that is where we "compete".

When PyPy runs as fast as CPython, perhaps PyPy can start supporting the 
Cython type annotations (at least the pure Python mode ones); Jython 
could support the same annotations in time, and so on.

I.e., I see Cython as the language and philosophy at least as important 
as the speed we can run pure Python code at vs. PyPy :-)

-- 
Dag Sverre

From robin at reportlab.com  Tue Apr 21 14:54:04 2009
From: robin at reportlab.com (Robin Becker)
Date: Tue, 21 Apr 2009 13:54:04 +0100
Subject: [Cython] download for 0.11.1
Message-ID: <49EDC1EC.7030703@chamonix.reportlab.co.uk>

Is it intentional that the zip file contains the .hg folder. I see 5.365Mb in 
total of which 3.193Mb appeasr to be in the .hg folder?
-- 
Robin Becker

From ilmarw at simula.no  Tue Apr 21 14:57:20 2009
From: ilmarw at simula.no (Ilmar Wilbers)
Date: Tue, 21 Apr 2009 14:57:20 +0200
Subject: [Cython] download for 0.11.1
In-Reply-To: <49EDC1EC.7030703@chamonix.reportlab.co.uk>
References: <49EDC1EC.7030703@chamonix.reportlab.co.uk>
Message-ID: <49EDC2B0.3060700@simula.no>

Discussed recently:
http://codespeak.net/pipermail/cython-dev/2009-April/004883.html

ilmar

Robin Becker wrote:
> Is it intentional that the zip file contains the .hg folder. I see 5.365Mb in 
> total of which 3.193Mb appeasr to be in the .hg folder?
>   

From vic.kelson at gmail.com  Tue Apr 21 15:07:10 2009
From: vic.kelson at gmail.com (Vic Kelson)
Date: Tue, 21 Apr 2009 09:07:10 -0400
Subject: [Cython] Fortran GSoC accepted
In-Reply-To: <49EDB668.7030502@student.matnat.uio.no>
References: <49EDB668.7030502@student.matnat.uio.no>
Message-ID: <d5e3120f0904210607tb699866q5202a115f691926b@mail.gmail.com>

Congratulations!

This will be very useful in my research work. I'll be happy to assist with
testing, if that would be helpful.

Best of luck,
Vic

On Tue, Apr 21, 2009 at 8:04 AM, Dag Sverre Seljebotn <
dagss at student.matnat.uio.no> wrote:

> I'm very happy to report that Kurt Smith has been accepted for a Google
> Summer of Code project for working on Cython/Fortran integration!
>
> http://wiki.cython.org/kwsmith/soc09
>
> I'm his mentor for the project, and I start this thread to discuss how
> the GSoC should be arranged. (I suppose part of this could apply to the
> other accepted project as well:
>
>
> http://socghop.appspot.com/student_project/show/google/gsoc2009/python/t124024629578
>
> ).
>
> The project is roughly going to be divided in two parts:
>  (A) Use the f2py parser and output code for a Cython binding
>  (B) Improve Cython in some areas (especially buffers) so that the
> bindings generated in a) can be much less messy and more convenient to use.
>
> Some things:
>
> 1) Where should the mentoring take place? Opinions on how much should
> happen on this list and how much should be done in private mail?
> Obviously language decisions etc. must end up here in the end, but what
> about day-to-day mentoring etc.?
>
> One possibility is setting up a seperate Google group just for this
> project, if it turns out that it would be too high traffic for the
> mailing list but other people would like to follow the project as well.
>
> 2) Code and patch workflow.
>
> On the (A) side, the status with f2py is that the author started a full
> rewrite (f2py 3G) which hasn't seen work for half a year
> (http://f2py.org/). So anything can happen here and we have to get back
> to how it's going to be done and where it should live. A relevant
> question though is whether we would like to ship the tool in (A) as part
> of Cython or have it as an independent download. (I don't expect it to
> add anything significant to download size, but Python core inclusion
> would be something to consider for or against here.)
>
> When it comes to (B), the obvious thing to do is for Kurt to pass the
> code to me for review and then merge it as often as we can with either
> -devel or -unstable (not sure which yet, possibly both).
>
> I'm wondering though whether perhaps Kurt could just get commit rights
> for the repos, under the strict understanding that he doesn't push
> anything which is not reviewed (for now at least). That could take some
> day-to-day merges off my back and make Kurt responsible for doing stable
> merges and think about the code integration schedule right from the start.
>
> We could then use e.g. codereview.appspot.com for doing the code
> reviews; I would then simply give thumbs up and Kurt would do the push.
>
> 3) Feedback
>
> There should be some kind of feedback mechanism; I expect to hear from
> Kurt at least weekly over the summer unless otherwise is arranged. I
> expect the ticket system will serve as the status update, and hope Kurt
> will make his progress visible through the ticket system (we can use the
> "fortran" tag on all GSoC related tickets for instance).
>
> I guess (A) would perhaps use a different tracker than (B), we need to
> get back to that. We could set something up at e.g. Google Code if the
> rest of the Cython team wouldn't like (A)-related tickets cluttering up
> the trac. We could use a seperate milestone though...
>
> Some mentors require students to keep a blog about what they are doing.
> Unless the Python Software Foundation makes this a requirement I'm going
> to leave this one up to Kurt; if the status is dutifully reported via
> tickets I'll have what I need personally, though a blog would be
> interesting as well.
>
> Dag Sverre
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Vic Kelson
vic.kelson at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20090421/f7783b82/attachment.htm 

From robin at reportlab.com  Tue Apr 21 15:11:53 2009
From: robin at reportlab.com (Robin Becker)
Date: Tue, 21 Apr 2009 14:11:53 +0100
Subject: [Cython] download for 0.11.1
In-Reply-To: <49EDC2B0.3060700@simula.no>
References: <49EDC1EC.7030703@chamonix.reportlab.co.uk>
	<49EDC2B0.3060700@simula.no>
Message-ID: <49EDC619.5000602@chamonix.reportlab.co.uk>

Ilmar Wilbers wrote:
> Discussed recently:
> http://codespeak.net/pipermail/cython-dev/2009-April/004883.html
> 
> ilmar
> 
> Robin Becker wrote:
>> Is it intentional that the zip file contains the .hg folder. I see 5.365Mb in 
>> total of which 3.193Mb appeasr to be in the .hg folder?
>>   
>......

so just laziness then :)
-- 
Robin Becker

From dalcinl at gmail.com  Tue Apr 21 15:17:36 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Tue, 21 Apr 2009 10:17:36 -0300
Subject: [Cython] Cython just to avoid coding C?
In-Reply-To: <4905F089-AC21-4C61-A54E-F6BAF813F828@math.washington.edu>
References: <c4ab18df0904190356w6298837boec858b2e18034567@mail.gmail.com>
	<ca433830904190616v9cf56f9r26ca2ed803a4423e@mail.gmail.com>
	<49EB3943.9090409@behnel.de>
	<4192B3F1-8CF6-41CA-80EE-42B2D32623B0@math.washington.edu>
	<4905F089-AC21-4C61-A54E-F6BAF813F828@math.washington.edu>
Message-ID: <e7ba66e40904210617t230c5142v2bae1671504ebc0@mail.gmail.com>

Robert, your main() implementation needs more work IMHO.

1) For Py2, you need to if(PyErr_Occurred())
PyErr_WriteUnraisable(PyErr_Occurred()).

2) In Py3, you have to save the return of PyInit_<modname>, if NULL,
report error (WriteUnraisable again?), if not, decref it.

3) In case of errors, process return value should match what core
CPython returns en case of unhandled execptions.

3) However, a SystemExit exceptions should need some special
consideration, as you should extract the process return value from it,
as it could be 0 (zero).







On Tue, Apr 21, 2009 at 6:16 AM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Apr 20, 2009, at 11:48 AM, Robert Bradshaw wrote:
>
>> On Apr 19, 2009, at 7:46 AM, Stefan Behnel wrote:
>>
>>> Mark Lodato wrote:
>>>> you could embed the interpreter and statically link your module.
>>>> [...]
>>>> ----------------------------------- main.c
>>>> ---------------------------
>>>> #include <Python.h>
>>>>
>>>> // For each Cython module you want to embed, you must declare an
>>>> // init<module> function, like so:
>>>> PyMODINIT_FUNC initmylib(void);
>>>>
>>>> int
>>>> main(int argc, char *argv[])
>>>> {
>>>> ? ? // The first step is to set up the Python interpreter:
>>>> ? ? Py_Initialize();
>>>> ? ? PySys_SetArgv(argc, argv);
>>>>
>>>> ? ? // Next, we need to tell Python that our module exists. ?Call
>>>> each
>>>> ? ? // of the functions you declared above.
>>>> ? ? initmylib();
>>>>
>>>> ? ? // Now do some Python stuff. ?The easiest thing to do is to give
>>>> ? ? // the interpreter a string of Python code that imports your
>>>> ? ? // module and calls it.
>>>> ? ? PyRun_SimpleString("from mylib import main\n"
>>>> ? ? ? ? ? ? ? ? ? ? ? ?"main()\n");
>>>>
>>>> ? ? // When we're done, tell Python to clean up.
>>>> ? ? Py_Finalize();
>>>> ? ? return 0;
>>>> }
>>>> --------------------------------------------------------------------
>>>> -
>>>> -
>>>
>>> I wonder if it makes sense to add support for this to Cython. You
>>> could
>>> have a command line option that adds a suitable main() function
>>> that runs
>>> the module code in an embedded CPython interpreter (and that does
>>> the setup
>>> correctly for Py2 and Py3).
>>
>> +1, I think that's an excellent idea. http://trac.cython.org/
>> cython_trac/ticket/289
>
> There's an --embed command line option now, which generates an
> appropriate main() method. One then compiles the resulting C file
> into an executable (which still needs to be linked against the Python
> library).
>
> - Robert
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dagss at student.matnat.uio.no  Tue Apr 21 15:35:06 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 21 Apr 2009 15:35:06 +0200
Subject: [Cython] download for 0.11.1
In-Reply-To: <49EDC619.5000602@chamonix.reportlab.co.uk>
References: <49EDC1EC.7030703@chamonix.reportlab.co.uk>	<49EDC2B0.3060700@simula.no>
	<49EDC619.5000602@chamonix.reportlab.co.uk>
Message-ID: <49EDCB8A.6040705@student.matnat.uio.no>

Robin Becker wrote:
> Ilmar Wilbers wrote:
>> Discussed recently:
>> http://codespeak.net/pipermail/cython-dev/2009-April/004883.html
>>
>> ilmar
>>
>> Robin Becker wrote:
>>> Is it intentional that the zip file contains the .hg folder. I see 5.365Mb in 
>>> total of which 3.193Mb appeasr to be in the .hg folder?
>>>   
>> ......
> 
> so just laziness then :)

No, quote:

 > An argument in favor is that end-users can very easily contribute a
 > bugfix. If users suddenly encounter a bug they can fix then the fix
 > they
 > do is very likely to happen in the downloaded release directory, and
 > having done that then it's a lot less effort to submit a patch if
 > it's a
 > working repository.
 >
 > Especially with Cython, whose end-users are all developers, this is a
 > nice idea.

This is exactly why we do it. Sage ships this way and I think it's
been a significant factor in attracting developers.


-- 
Dag Sverre

From jf.moulin at gmail.com  Tue Apr 21 15:33:41 2009
From: jf.moulin at gmail.com (Jean-Francois Moulin)
Date: Tue, 21 Apr 2009 15:33:41 +0200
Subject: [Cython] help with function pointer
In-Reply-To: <b2476612f3966aba2541532431243305.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
References: <af6d35810904200817p62e92366ld616a491bf9b63da@mail.gmail.com>
	<6F08D037-7371-4D43-B7FE-8185FD8B25DB@math.washington.edu>
	<af6d35810904210045k105c4178t29f68b621d1354c5@mail.gmail.com>
	<CAD789E2-DAD2-4605-9878-318E7A2ECD33@math.washington.edu>
	<af6d35810904210219y45a12adbr644cbb219a3d6e1d@mail.gmail.com>
	<90EEE860-DAED-474D-A2E3-F1857ED62699@math.washington.edu>
	<af6d35810904210306p7ab1c80bk6f5433074f8eec2a@mail.gmail.com>
	<b2476612f3966aba2541532431243305.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <af6d35810904210633i39bb1b4fs113379a5094c17b7@mail.gmail.com>

Great! Just tried it and it works fine!
I gain something like a tenfold factor in speed with respect to my
initial python code.
>From what I read I expect I should still gain quite a lot by
optimizing the calls to the Numpy functions.
Namely using the buffer notation defining the type of array I am using.
A question arises, should I also apply this notation in the definition
of my pointers? and if yes, how?

so, I presently have statements like:
cdef int (*locate)(DelayLine, np.ndarray)

and would write something like:
 cdef int (*locate)(DelayLine, np.ndarray[DTYPE_t,ndim=2])

but compilation fails saying that a closing square bracket is missing
(])... in the description of the notation in the docs, one sees that
the name of the numpy array directly follows the
description... How to proceed?

From robin at reportlab.com  Tue Apr 21 15:48:00 2009
From: robin at reportlab.com (Robin Becker)
Date: Tue, 21 Apr 2009 14:48:00 +0100
Subject: [Cython] inherit from list
Message-ID: <49EDCE90.2050604@chamonix.reportlab.co.uk>

I have some older code that used to run with pyrex without complaint, but now 
the following

cdef extern from "listobject.h":
	ctypedef class __builtin__.list [object PyListObject]:
		pass


causes a warning
> warning: C:\code\users\robin\_tex_wrap.pyx:144:1: list already a builtin Cython type


however, I cannot remove the above code as that causes an error at my class 
declaration



> cdef class ObjectList(list):


what's the correct cython way to inherit from a list or other builtin? I have 
looked in the extension_types section, but am no wiser.
--
Robin Becker

From dagss at student.matnat.uio.no  Tue Apr 21 16:23:18 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 21 Apr 2009 16:23:18 +0200 (CEST)
Subject: [Cython] help with function pointer
In-Reply-To: <af6d35810904210633i39bb1b4fs113379a5094c17b7@mail.gmail.com>
References: <af6d35810904200817p62e92366ld616a491bf9b63da@mail.gmail.com>
	<6F08D037-7371-4D43-B7FE-8185FD8B25DB@math.washington.edu>
	<af6d35810904210045k105c4178t29f68b621d1354c5@mail.gmail.com>
	<CAD789E2-DAD2-4605-9878-318E7A2ECD33@math.washington.edu>
	<af6d35810904210219y45a12adbr644cbb219a3d6e1d@mail.gmail.com>
	<90EEE860-DAED-474D-A2E3-F1857ED62699@math.washington.edu>
	<af6d35810904210306p7ab1c80bk6f5433074f8eec2a@mail.gmail.com>
	<b2476612f3966aba2541532431243305.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<af6d35810904210633i39bb1b4fs113379a5094c17b7@mail.gmail.com>
Message-ID: <d416dd90cdcf283d0aae347b41e40d28.squirrel@webmail.uio.no>

Jean-Francois Moulin wrote:
> Great! Just tried it and it works fine!
> I gain something like a tenfold factor in speed with respect to my
> initial python code.
>>From what I read I expect I should still gain quite a lot by
> optimizing the calls to the Numpy functions.
> Namely using the buffer notation defining the type of array I am using.
> A question arises, should I also apply this notation in the definition
> of my pointers? and if yes, how?
>
> so, I presently have statements like:
> cdef int (*locate)(DelayLine, np.ndarray)
>
> and would write something like:
>  cdef int (*locate)(DelayLine, np.ndarray[DTYPE_t,ndim=2])

This is simply not supported with cdef functions. You need to do

cdef int (*locate)(DelayLine, np.ndarray)

and then inside the implementation of the function you assign to it, you do

cdef int (*locate)(DelayLine a, np.ndarray b)
    cdef np.ndarray[DTYPE_t, ndim=2] b_buf = b
    .... access b through b_buf ...

Dag Sverre


From stefan_ml at behnel.de  Tue Apr 21 18:26:17 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 21 Apr 2009 18:26:17 +0200 (CEST)
Subject: [Cython] Path towards Python completeness
In-Reply-To: <49EDB97F.4030004@student.matnat.uio.no>
References: <49ED5CE8.6090708@behnel.de>
	<49EDB97F.4030004@student.matnat.uio.no>
Message-ID: <ab1e0fdf3d34a12989a138890c192bfd.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Dag Sverre Seljebotn wrote:
> I don't think we have to think of ourselves as in direct competition
> with PyPy (and/or Unladen Swallow). Personally I'm more than happy to
> use pure Python rather than Cython for the kind of things that can be
> sped up by those projects anyway (makes for a shorter compile/run cycle).

I do actually see PyPy as a direct competitor of Cython (much more than
Jython and IronPython, BTW), as both projects aim to compile Python code
to fast native code. Currently, we are faster and they are more feature
complete. But if we can have the cake and eat it, too...


> What I think sets Cython apart is that we add a superset to the Python
> language for adding types; which are simply needed in some situations. I
> think that is where we "compete".

That's rather where we are beyond competition.


> When PyPy runs as fast as CPython, perhaps PyPy can start supporting the
> Cython type annotations (at least the pure Python mode ones); Jython
> could support the same annotations in time, and so on.

Let's wait and see, but I doubt that there will be a major interest in
doing that. I actually find it more important for Cython to gain type
inference mechanisms that lift the requirement for defining types in the
first place, at least for standard use cases.

Stefan


From stefan_ml at behnel.de  Tue Apr 21 18:32:30 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 21 Apr 2009 18:32:30 +0200 (CEST)
Subject: [Cython] inherit from list
In-Reply-To: <49EDCE90.2050604@chamonix.reportlab.co.uk>
References: <49EDCE90.2050604@chamonix.reportlab.co.uk>
Message-ID: <36cc114ef2dc49b452300d16e4a7660d.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Robin Becker wrote:
> I have some older code that used to run with pyrex without complaint, but
> now the following
>
> cdef extern from "listobject.h":
> 	ctypedef class __builtin__.list [object PyListObject]:
> 		pass
>
> causes a warning
>> warning: C:\code\users\robin\_tex_wrap.pyx:144:1: list already a builtin
>> Cython type

Yes, that warning is currently a bit misleading in this specific case, as
there is not yet a way to achieve inheritance of builtin types without the
above declaration. It's meant more for code that uses the declared type in
other places than supertypes, where Cython now handles these types better
on its own.

This is what ticket #258 is supposed to mean.

http://trac.cython.org/cython_trac/ticket/258

See also ticket #170:

http://trac.cython.org/cython_trac/ticket/170

If you use the "list" type only for subtypes, just ignore the warning. It
will become an error once Cython can deal with this case directly, so you
will notice when it's time to remove the declaration.

Stefan


From robin at reportlab.com  Tue Apr 21 18:40:16 2009
From: robin at reportlab.com (Robin Becker)
Date: Tue, 21 Apr 2009 17:40:16 +0100
Subject: [Cython] inherit from list
In-Reply-To: <36cc114ef2dc49b452300d16e4a7660d.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
References: <49EDCE90.2050604@chamonix.reportlab.co.uk>
	<36cc114ef2dc49b452300d16e4a7660d.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <49EDF6F0.504@chamonix.reportlab.co.uk>

.........
> 
> Yes, that warning is currently a bit misleading in this specific case, as
> there is not yet a way to achieve inheritance of builtin types without the
> above declaration. It's meant more for code that uses the declared type in
> other places than supertypes, where Cython now handles these types better
> on its own.
> 
> This is what ticket #258 is supposed to mean.
> 
> http://trac.cython.org/cython_trac/ticket/258
> 
> See also ticket #170:
> 
> http://trac.cython.org/cython_trac/ticket/170
> 
> If you use the "list" type only for subtypes, just ignore the warning. It
> will become an error once Cython can deal with this case directly, so you
> will notice when it's time to remove the declaration.
> 
> Stefan
......

OK thanks. I noticed that all seems to work. Is the intent to allow inheriting 
from builtin types to be handled automatically as it is in python where list is 
both a constructor and a type?
-- 
Robin Becker

From stefan_ml at behnel.de  Tue Apr 21 19:01:27 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 21 Apr 2009 19:01:27 +0200 (CEST)
Subject: [Cython] Fortran GSoC accepted
In-Reply-To: <49EDB668.7030502@student.matnat.uio.no>
References: <49EDB668.7030502@student.matnat.uio.no>
Message-ID: <d86a3133f5a7b216f98fb14045e6f6f6.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Dag Sverre Seljebotn wrote:
> I'm very happy to report that Kurt Smith has been accepted for a Google
> Summer of Code project for working on Cython/Fortran integration!

Yep, congrats!


> I'm his mentor for the project, and I start this thread to discuss how
> the GSoC should be arranged. (I suppose part of this could apply to the
> other accepted project as well:
>
> http://socghop.appspot.com/student_project/show/google/gsoc2009/python/t124024629578

Congrats on that one, too!

Good luck to our GSoCers.


> 1) Where should the mentoring take place? Opinions on how much should
> happen on this list and how much should be done in private mail?
> Obviously language decisions etc. must end up here in the end, but what
> about day-to-day mentoring etc.?

Depends. Trivial/basic things may go into private mail, but I would prefer
having most of the discussions on the list, where others can
help/find/learn, too. This is an open source project, so things should
happen in the open.


> One possibility is setting up a seperate Google group just for this
> project, if it turns out that it would be too high traffic for the
> mailing list but other people would like to follow the project as well.

Let's see about that when we notice that it gets too much traffic.

I would lean more towards a separate cython-users list than a separate
sub-dev list. (I have ignored threads before, so I know I'll cope ;)


> 2) Code and patch workflow.
>
> On the (A) side, the status with f2py is that the author started a full
> rewrite (f2py 3G) which hasn't seen work for half a year
> (http://f2py.org/). So anything can happen here and we have to get back
> to how it's going to be done and where it should live.

Sounds like the right thing to ask the author first.


> A relevant
> question though is whether we would like to ship the tool in (A) as part
> of Cython or have it as an independent download. (I don't expect it to
> add anything significant to download size, but Python core inclusion
> would be something to consider for or against here.)

Let's decide that when we can take a look at it.


> When it comes to (B), the obvious thing to do is for Kurt to pass the
> code to me for review and then merge it as often as we can with either
> -devel or -unstable (not sure which yet, possibly both).

Definitely not the current cython-devel, but without further detail (that
may become available once the project has left the initial planning
phase), I can't tell if it makes more sense to add another branch. Given
the time-frame of the GSoC, maybe even cython-unstable is too close to
release to add major new, unfinished features. Parts may go in earlier,
obviously, so a branch from which we can import stuff may make sense.

IMHO, too early to decide.


> I'm wondering though whether perhaps Kurt could just get commit rights
> for the repos, under the strict understanding that he doesn't push
> anything which is not reviewed (for now at least). That could take some
> day-to-day merges off my back and make Kurt responsible for doing stable
> merges and think about the code integration schedule right from the start.

I wouldn't mind giving him write access, though I don't see an immediate
need for that. Patches will require reviews anyway, even major design
reviews.


> We could then use e.g. codereview.appspot.com for doing the code
> reviews; I would then simply give thumbs up and Kurt would do the push.

Looks like they didn't fix the login issue yet. It still requires a Google
account to comment.


> 3) Feedback
>
> There should be some kind of feedback mechanism; I expect to hear from
> Kurt at least weekly over the summer unless otherwise is arranged. I
> expect the ticket system will serve as the status update, and hope Kurt
> will make his progress visible through the ticket system (we can use the
> "fortran" tag on all GSoC related tickets for instance).

Sure. Please take care to cut down the subject of tickets to make sure it
has a provable result. Otherwise, it's not clear when the ticket is
solved.

It's fine to have a couple of tickets that depend on each other.


> I guess (A) would perhaps use a different tracker than (B), we need to
> get back to that.

Why? As long as we consider integrating it with Cython, I'm fine with
tracking the required work here.


> Some mentors require students to keep a blog about what they are doing.
> Unless the Python Software Foundation makes this a requirement I'm going
> to leave this one up to Kurt; if the status is dutifully reported via
> tickets I'll have what I need personally, though a blog would be
> interesting as well.

It would be nice, yes.

Stefan


From stefan_ml at behnel.de  Tue Apr 21 19:04:12 2009
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 21 Apr 2009 19:04:12 +0200 (CEST)
Subject: [Cython] inherit from list
In-Reply-To: <49EDF6F0.504@chamonix.reportlab.co.uk>
References: <49EDCE90.2050604@chamonix.reportlab.co.uk>
	<36cc114ef2dc49b452300d16e4a7660d.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<49EDF6F0.504@chamonix.reportlab.co.uk>
Message-ID: <5868d3e61cc56e30541512544f9c00e8.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Robin Becker wrote:
> Is the intent to allow inheriting
> from builtin types to be handled automatically as it is in python where
> list is both a constructor and a type?

Yes, that's what ticket #258 is supposed to track (although the
description is a bit short to really get the idea).

Stefan


From dagss at student.matnat.uio.no  Tue Apr 21 19:39:28 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 21 Apr 2009 19:39:28 +0200 (CEST)
Subject: [Cython] Fortran GSoC accepted
In-Reply-To: <d86a3133f5a7b216f98fb14045e6f6f6.squirrel@groupware.dvs.informatik.tu
	-darmstadt.de>
References: <49EDB668.7030502@student.matnat.uio.no>
	<d86a3133f5a7b216f98fb14045e6f6f6.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <4becd63dd8da72233794f517ffa03fea.squirrel@webmail.uio.no>

Stefan Behnel wrote:
> Dag Sverre Seljebotn wrote:
>> We could then use e.g. codereview.appspot.com for doing the code
>> reviews; I would then simply give thumbs up and Kurt would do the push.
>
> Looks like they didn't fix the login issue yet. It still requires a Google
> account to comment.

Have I missed a post somewhere on the dangers of getting a Google account?

Me and Kurt are obviously required to have those anyway, in order to
participate in the program.

Dag Sverre


From kwmsmith at gmail.com  Tue Apr 21 19:42:27 2009
From: kwmsmith at gmail.com (Kurt Smith)
Date: Tue, 21 Apr 2009 12:42:27 -0500
Subject: [Cython] Fortran GSoC accepted
In-Reply-To: <d86a3133f5a7b216f98fb14045e6f6f6.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
References: <49EDB668.7030502@student.matnat.uio.no>
	<d86a3133f5a7b216f98fb14045e6f6f6.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <c266a7930904211042o373f4356r546ccabbfe392d3d@mail.gmail.com>

On Tue, Apr 21, 2009 at 12:01 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Dag Sverre Seljebotn wrote:
>> I'm very happy to report that Kurt Smith has been accepted for a Google
>> Summer of Code project for working on Cython/Fortran integration!
>
> Yep, congrats!

Thanks!  And thanks to you, Robert, Dag and everyone else for
accepting the project.  I'm pleased to see that Cython has 2 GSoC
projects this year, too (one more than last year?).


>> 1) Where should the mentoring take place? Opinions on how much should
>> happen on this list and how much should be done in private mail?
>> Obviously language decisions etc. must end up here in the end, but what
>> about day-to-day mentoring etc.?
>
> Depends. Trivial/basic things may go into private mail, but I would prefer
> having most of the discussions on the list, where others can
> help/find/learn, too. This is an open source project, so things should
> happen in the open.

I'm with you there -- we'll get a feel as time goes on, but by default
the substantial threads would be on the list.

>
>
>> One possibility is setting up a seperate Google group just for this
>> project, if it turns out that it would be too high traffic for the
>> mailing list but other people would like to follow the project as well.
>
> Let's see about that when we notice that it gets too much traffic.
>
> I would lean more towards a separate cython-users list than a separate
> sub-dev list. (I have ignored threads before, so I know I'll cope ;)
>
>
>> 2) Code and patch workflow.
>>
>> On the (A) side, the status with f2py is that the author started a full
>> rewrite (f2py 3G) which hasn't seen work for half a year
>> (http://f2py.org/). So anything can happen here and we have to get back
>> to how it's going to be done and where it should live.
>
> Sounds like the right thing to ask the author first.

Dag just sent off an email to Pearu this morning outlining our
approach and possible options.  We'll let you know the results & cc
the list if need be.

>
>
>> A relevant
>> question though is whether we would like to ship the tool in (A) as part
>> of Cython or have it as an independent download. (I don't expect it to
>> add anything significant to download size, but Python core inclusion
>> would be something to consider for or against here.)
>
> Let's decide that when we can take a look at it.
>
>
>> When it comes to (B), the obvious thing to do is for Kurt to pass the
>> code to me for review and then merge it as often as we can with either
>> -devel or -unstable (not sure which yet, possibly both).
>
> Definitely not the current cython-devel, but without further detail (that
> may become available once the project has left the initial planning
> phase), I can't tell if it makes more sense to add another branch. Given
> the time-frame of the GSoC, maybe even cython-unstable is too close to
> release to add major new, unfinished features. Parts may go in earlier,
> obviously, so a branch from which we can import stuff may make sense.
>
> IMHO, too early to decide.

I/we should have a better grasp on this in a couple of weeks at most.

>
>
>> I'm wondering though whether perhaps Kurt could just get commit rights
>> for the repos, under the strict understanding that he doesn't push
>> anything which is not reviewed (for now at least). That could take some
>> day-to-day merges off my back and make Kurt responsible for doing stable
>> merges and think about the code integration schedule right from the start.
>
> I wouldn't mind giving him write access, though I don't see an immediate
> need for that. Patches will require reviews anyway, even major design
> reviews.

I have no great eagerness for write access, and I'll need to prove
trustworthiness to get there.  I hope that at a future date after
patches are reviewed and committed by yourselves it'll make more sense
to grant me write access, with the understanding that I'll put up the
patches for review first.

Honestly, my mercurial-fu isn't marvelous at the moment (I can do the
basic things and am establishing my workflow, but if I were to mess
something up I'd need help fixing it) and I'd like some more
experience in the day-to-day work before being given access to
something that matters.

>
>
>> We could then use e.g. codereview.appspot.com for doing the code
>> reviews; I would then simply give thumbs up and Kurt would do the push.
>
> Looks like they didn't fix the login issue yet. It still requires a Google
> account to comment.
>
>
>> 3) Feedback
>>
>> There should be some kind of feedback mechanism; I expect to hear from
>> Kurt at least weekly over the summer unless otherwise is arranged. I
>> expect the ticket system will serve as the status update, and hope Kurt
>> will make his progress visible through the ticket system (we can use the
>> "fortran" tag on all GSoC related tickets for instance).
>
> Sure. Please take care to cut down the subject of tickets to make sure it
> has a provable result. Otherwise, it's not clear when the ticket is
> solved.
>
> It's fine to have a couple of tickets that depend on each other.

Sounds fine to me.

>> I guess (A) would perhaps use a different tracker than (B), we need to
>> get back to that.
>
> Why? As long as we consider integrating it with Cython, I'm fine with
> tracking the required work here.
>
>
>> Some mentors require students to keep a blog about what they are doing.
>> Unless the Python Software Foundation makes this a requirement I'm going
>> to leave this one up to Kurt; if the status is dutifully reported via
>> tickets I'll have what I need personally, though a blog would be
>> interesting as well.
>
> It would be nice, yes.

I'll get one set up.  Any canonical place for GSoC blogs that anyone knows of?

Kurt

From dagss at student.matnat.uio.no  Tue Apr 21 19:48:25 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 21 Apr 2009 19:48:25 +0200 (CEST)
Subject: [Cython] Fortran GSoC accepted
In-Reply-To: <c266a7930904211042o373f4356r546ccabbfe392d3d@mail.gmail.com>
References: <49EDB668.7030502@student.matnat.uio.no>
	<d86a3133f5a7b216f98fb14045e6f6f6.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<c266a7930904211042o373f4356r546ccabbfe392d3d@mail.gmail.com>
Message-ID: <f6fe0d10df80e021bd5c74a315121eb2.squirrel@webmail.uio.no>

Kurt Smith wrote:
> On Tue, Apr 21, 2009 at 12:01 PM, Stefan Behnel <stefan_ml at behnel.de>
> wrote:
>> Dag Sverre Seljebotn wrote:
>>> Some mentors require students to keep a blog about what they are doing.
>>> Unless the Python Software Foundation makes this a requirement I'm
>>> going
>>> to leave this one up to Kurt; if the status is dutifully reported via
>>> tickets I'll have what I need personally, though a blog would be
>>> interesting as well.
>>
>> It would be nice, yes.
>
> I'll get one set up.  Any canonical place for GSoC blogs that anyone knows
> of?

I don't think so, what happened last year though is that PSF aggregated
the GSoC blogs here: http://soc.python.org/ . So as long as you have RSS
support you're good.

Dag Sverre


From fperez.net at gmail.com  Tue Apr 21 20:29:35 2009
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 21 Apr 2009 11:29:35 -0700
Subject: [Cython] Fortran GSoC accepted
In-Reply-To: <49EDB668.7030502@student.matnat.uio.no>
References: <49EDB668.7030502@student.matnat.uio.no>
Message-ID: <db6b5ecc0904211129g4d48d632y95a8dbba5bafff8f@mail.gmail.com>

On Tue, Apr 21, 2009 at 5:04 AM, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:
> I'm very happy to report that Kurt Smith has been accepted for a Google
> Summer of Code project for working on Cython/Fortran integration!
>
> http://wiki.cython.org/kwsmith/soc09
>
> I'm his mentor for the project, and I start this thread to discuss how

I just wanted to congratulate you and thank you for getting this
going!  Fortran support is an oft-overlooked yet *critical* piece of
making Python a serious scientific computing tool, and f2py has been
getting a bit long in the tooth.  For a long time many of us have
thought that the cython-f2py marriage would be an ideal evolution,
seeing how conceptually they can share so much infrastructure, and
cython is a much more lively project than f2py today.

So a huge thanks from me, and best wishes for the success of this
project over the summer!

Cheers,

f

From robertwb at math.washington.edu  Tue Apr 21 20:33:43 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 21 Apr 2009 11:33:43 -0700
Subject: [Cython] Cython just to avoid coding C?
In-Reply-To: <e7ba66e40904210617t230c5142v2bae1671504ebc0@mail.gmail.com>
References: <c4ab18df0904190356w6298837boec858b2e18034567@mail.gmail.com>
	<ca433830904190616v9cf56f9r26ca2ed803a4423e@mail.gmail.com>
	<49EB3943.9090409@behnel.de>
	<4192B3F1-8CF6-41CA-80EE-42B2D32623B0@math.washington.edu>
	<4905F089-AC21-4C61-A54E-F6BAF813F828@math.washington.edu>
	<e7ba66e40904210617t230c5142v2bae1671504ebc0@mail.gmail.com>
Message-ID: <E7DA7325-D6F2-416C-B104-072D2CF34281@math.washington.edu>

On Apr 21, 2009, at 6:17 AM, Lisandro Dalcin wrote:

> Robert, your main() implementation needs more work IMHO.

Thanks for the feedback. I just did the basic embedding, as on David  
McNab's site. It would be good to have it behave more like python -m  
http://trac.cython.org/cython_trac/ticket/291


> 1) For Py2, you need to if(PyErr_Occurred())
> PyErr_WriteUnraisable(PyErr_Occurred()).

Actually, it might be nice to dump the whole traceback here.

> 2) In Py3, you have to save the return of PyInit_<modname>, if NULL,
> report error (WriteUnraisable again?), if not, decref it.

Is PyErr_Occurred not a good enough check here? Yes, I should  
probably get and decref the module (though it won't matter much as I  
call Py_Finalize next.

> 3) In case of errors, process return value should match what core
> CPython returns en case of unhandled execptions.

Do you know of a good list? (Otherwise, I'll just try some.)

> 3) However, a SystemExit exceptions should need some special
> consideration, as you should extract the process return value from it,
> as it could be 0 (zero).

Good point.

- Robert

From robertwb at math.washington.edu  Tue Apr 21 20:36:49 2009
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 21 Apr 2009 11:36:49 -0700
Subject: [Cython] help with function pointer
In-Reply-To: <d416dd90cdcf283d0aae347b41e40d28.squirrel@webmail.uio.no>
References: <af6d35810904200817p62e92366ld616a491bf9b63da@mail.gmail.com>
	<6F08D037-7371-4D43-B7FE-8185FD8B25DB@math.washington.edu>
	<af6d35810904210045k105c4178t29f68b621d1354c5@mail.gmail.com>
	<CAD789E2-DAD2-4605-9878-318E7A2ECD33@math.washington.edu>
	<af6d35810904210219y45a12adbr644cbb219a3d6e1d@mail.gmail.com>
	<90EEE860-DAED-474D-A2E3-F1857ED62699@math.washington.edu>
	<af6d35810904210306p7ab1c80bk6f5433074f8eec2a@