From ondrej at certik.cz  Thu Apr  1 02:40:41 2010
From: ondrej at certik.cz (Ondrej Certik)
Date: Wed, 31 Mar 2010 17:40:41 -0700
Subject: [Cython] LiE in Cython
In-Reply-To: <17A344E1-C792-4D8B-9CA3-BEE580FB28EF@gmail.com>
References: <20091017115713.GC4492@zephyr>
	<EC4210E5-8AA4-420C-9858-4E6E644C7090@math.washington.edu>
	<17A344E1-C792-4D8B-9CA3-BEE580FB28EF@gmail.com>
Message-ID: <r2i85b5c3131003311740i976afccfs654c724d760fb30e@mail.gmail.com>

On Tue, Mar 30, 2010 at 9:20 PM, David Simmons-Duffin
<dsimmons.com at gmail.com> wrote:
> Dear All,
>
> I've finally gotten around to posting the code for my cython wrapper for the computer algebra package LiE:
>
> http://github.com/davidsd/lie
>
> Basically, the project contains a copy of the LiE source, some extra C files, and a cython file (lie.pyx) that references functions from LiE and defines extension classes and some more abstract python objects.
>
> It currently works well enough for what I made it for (http://arxiv.org/abs/0910.4585). ?However, there are many ways it could be improved. ?Since I'm no longer striving towards a specific goal, and I have other projects to work on, it's hard for me to prioritize one improvement over another. ?So I'd be grateful if those who are interested could take a look, let me know what improvements/features they think are important, and perhaps help me work towards them.

I don't have time at the moment to figure out how to compile it on
linux, but looking at the readme and examples in there, it looks very
interesting. And your article too.

Ondrej

From baihaoyu at gmail.com  Fri Apr  2 07:53:54 2010
From: baihaoyu at gmail.com (Haoyu Bai)
Date: Fri, 2 Apr 2010 13:53:54 +0800
Subject: [Cython] GSoC project for improving Python compatibility
In-Reply-To: <4A9A1D0E-EB65-4838-A59B-9E1737390DDC@math.washington.edu>
References: <3eb99a591003210431s12042237ubd1be5c73b24bfec@mail.gmail.com> 
	<4BA72159.6010007@student.matnat.uio.no>
	<3eb99a591003220939r3c5a94c3ibdd30826f885bc7e@mail.gmail.com> 
	<a8c4ff4c1003221207w74d19898uf2c8bff011b3c67b@mail.gmail.com> 
	<3eb99a591003270813x38a6d554u54419e693a345efc@mail.gmail.com> 
	<4A9A1D0E-EB65-4838-A59B-9E1737390DDC@math.washington.edu>
Message-ID: <h2i3eb99a591004012253k83f8539u3efda34280151c0c@mail.gmail.com>

On Wed, Mar 31, 2010 at 1:36 AM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
>
> There's also Py3 scope binding rules for list comprehensions,
> exception blocks, etc.
>

Thanks. I added these to my list.

>> 2. Pure Python mode with py3 features
>> I agree that we need to support tuple as annotation for a convention
>> to be compatible with others. Besides function annotation, I'm also
>> thinking to make use of decorators. As decorators now supported for
>> classes, we could have @cdef @cpdef as decorator. This is a possible
>> solution for get rid of .pxd files (ticket #112). A demo:
>>
>> import cython
>> @cython.cdef
>> class Foo:
>> ? ?@cython.cdef
>> ? ?def bar(self, x: cython.int) -> cython.p_char:
>> ? ? ? ?...
>>
>> Also, one could always use "from cython import *" to make the code
>> more concise.
>
> That's an interesting idea, but how would one resolve
>
> ? ? from cython import *
> ? ? from X import *
>
> ?
>
> Also cython.int != __builtin__.int is important. Perhaps this would
> just be disallowed...
>

Yes this should be discouraged, as in normal Python coding standard.
But it is allowed by Python, so we should allow it too. A user who did
"from cython import *" should always aware of the potential name
conflict.

At least, if the code running in Python, then cython.int ==
__builtin__.int, right? (That's what I read from Shadow.py) I think it
is good to keep cython.int==__builtin__.int, and
cython.double==cython.float==__builtin__.float, maybe also
cython.p_char==__builtin__.bytearray, etc.  Then we can always drop in
a type checker when the code is running in Python, instead  of report
type error after running the compiled code.

Thanks!

-- 
Haoyu BAI
School of Computing,
National University of Singapore.

From maximem at melchiorscientificprogramming.com  Fri Apr  2 16:05:35 2010
From: maximem at melchiorscientificprogramming.com (maximem at melchiorscientificprogramming.com)
Date: Fri, 2 Apr 2010 16:05:35 +0200 (CEST)
Subject: [Cython] Combination of decorator and cpdef with cython 0.13
Message-ID: <ec742a77f21d331f73dd3714cf02857a.squirrel@webmail.ovh.net>

Dear developer of cython,

First of all I thank you for all your work. Thanks to Cython and the
documentation and tutorial that you displayed on internet, I have
successfully managed to create a cython code approximately 250 times
quicker than with python and even 1.4times quicker than C.

In fact I created two cython codes corresponding to two python codes of
the same algorithm (one with a single function and one with several
classes and function).

I hesitated several minutes before posting in this list or in the user
list but since this is not simple cython problem but a problem with the
new version of cython, i have finally chosen to post here. I hope I made
the right choice.

I downloaded the version 0.13 of cython yesterday (I was previously using
the version 0.12.1) and two problems appeared when I used cython.

The first one is that it seems that we can no more combine cpdef and
decorator :

    @cython.boundscheck(False)
    @cython.wraparound(False)
    cpdef run(self,adjacencyMatrix_,nodesPosition_,int dimension,nbLink_):
   ^
------------------------------------------------------------

/homelocal/max/Memoire/Sources/CAttractionRepulsionAlgorithm.pyx:48:4:
Decorators can only be followed by functions

I have the same problem with cdef and I have not the problem with def.

The second problem is that it seems that we can nore more used
@cython.wraparound(False).
I have this error

    @cython.wraparound(False)
   ^
------------------------------------------------------------

/homelocal/max/Memoire/Sources/CAttractionRepulsionAlgorithm.pyx:47:4:
'wraparound' not a valid cython attribute or is being used incorrectly

and this error for the same line :

    @cython.wraparound(False)
   ^
------------------------------------------------------------

/homelocal/max/Memoire/Sources/CAttractionRepulsionAlgorithm.pyx:47:4:
'wraparound' not a valid cython language construct

The use of this decorator with cpdef is important for me since it speed up
my cython code by nearly a factor 2.

Max.



From dagss at student.matnat.uio.no  Fri Apr  2 17:30:25 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Fri, 02 Apr 2010 17:30:25 +0200
Subject: [Cython] Combination of decorator and cpdef with cython 0.13
In-Reply-To: <ec742a77f21d331f73dd3714cf02857a.squirrel@webmail.ovh.net>
References: <ec742a77f21d331f73dd3714cf02857a.squirrel@webmail.ovh.net>
Message-ID: <4BB60D91.2000405@student.matnat.uio.no>

maximem at melchiorscientificprogramming.com wrote:
> Dear developer of cython,
>
> First of all I thank you for all your work. Thanks to Cython and the
> documentation and tutorial that you displayed on internet, I have
> successfully managed to create a cython code approximately 250 times
> quicker than with python and even 1.4times quicker than C.
>
> In fact I created two cython codes corresponding to two python codes of
> the same algorithm (one with a single function and one with several
> classes and function).
>
> I hesitated several minutes before posting in this list or in the user
> list but since this is not simple cython problem but a problem with the
> new version of cython, i have finally chosen to post here. I hope I made
> the right choice.
>
> I downloaded the version 0.13 of cython yesterday (I was previously using
> the version 0.12.1) and two problems appeared when I used cython.
>
> The first one is that it seems that we can no more combine cpdef and
> decorator :
>
>     @cython.boundscheck(False)
>     @cython.wraparound(False)
>     cpdef run(self,adjacencyMatrix_,nodesPosition_,int dimension,nbLink_):
>    ^
> ------------------------------------------------------------
>
> /homelocal/max/Memoire/Sources/CAttractionRepulsionAlgorithm.pyx:48:4:
> Decorators can only be followed by functions
>
> I have the same problem with cdef and I have not the problem with def.
>   
This is a bug. Thanks for reporting, filed at 
http://trac.cython.org/cython_trac/ticket/522.

In the meantime, you can use declarations such as np.ndarray[int, 
ndim=2, negative_indices=False] for the same effect.

> The second problem is that it seems that we can nore more used
> @cython.wraparound(False).
> I have this error
>
>     @cython.wraparound(False)
>    ^
> ------------------------------------------------------------
>
> /homelocal/max/Memoire/Sources/CAttractionRepulsionAlgorithm.pyx:47:4:
> 'wraparound' not a valid cython attribute or is being used incorrectly
>
> and this error for the same line :
>
>     @cython.wraparound(False)
>    ^
> ------------------------------------------------------------
>
> /homelocal/max/Memoire/Sources/CAttractionRepulsionAlgorithm.pyx:47:4:
> 'wraparound' not a valid cython language construct
>   

I do not understand you. Can you provide some more context (some more 
lines around where this happened, or an entire file which is as small as 
possible but reproduce the error). Are you sure you remembered to 
"cimport cython"?

Dag Sverre

From maximem at melchiorscientificprogramming.com  Fri Apr  2 18:36:26 2010
From: maximem at melchiorscientificprogramming.com (maximem at melchiorscientificprogramming.com)
Date: Fri, 2 Apr 2010 18:36:26 +0200 (CEST)
Subject: [Cython] Combination of decorator and cpdef with cython 0.13
In-Reply-To: <4BB60D91.2000405@student.matnat.uio.no>
References: <ec742a77f21d331f73dd3714cf02857a.squirrel@webmail.ovh.net>
	<4BB60D91.2000405@student.matnat.uio.no>
Message-ID: <55dd8ccb65c530aa98ee9847558c23c6.squirrel@webmail.ovh.net>

> maximem at melchiorscientificprogramming.com wrote:
>> Dear developer of cython,
>>
>> First of all I thank you for all your work. Thanks to Cython and the
>> documentation and tutorial that you displayed on internet, I have
>> successfully managed to create a cython code approximately 250 times
>> quicker than with python and even 1.4times quicker than C.
>>
>> In fact I created two cython codes corresponding to two python codes of
>> the same algorithm (one with a single function and one with several
>> classes and function).
>>
>> I hesitated several minutes before posting in this list or in the user
>> list but since this is not simple cython problem but a problem with the
>> new version of cython, i have finally chosen to post here. I hope I made
>> the right choice.
>>
>> I downloaded the version 0.13 of cython yesterday (I was previously
>> using
>> the version 0.12.1) and two problems appeared when I used cython.
>>
>> The first one is that it seems that we can no more combine cpdef and
>> decorator :
>>
>>     @cython.boundscheck(False)
>>     @cython.wraparound(False)
>>     cpdef run(self,adjacencyMatrix_,nodesPosition_,int
>> dimension,nbLink_):
>>    ^
>> ------------------------------------------------------------
>>
>> /homelocal/max/Memoire/Sources/CAttractionRepulsionAlgorithm.pyx:48:4:
>> Decorators can only be followed by functions
>>
>> I have the same problem with cdef and I have not the problem with def.
>>
> This is a bug. Thanks for reporting, filed at
> http://trac.cython.org/cython_trac/ticket/522.
>
> In the meantime, you can use declarations such as np.ndarray[int,
> ndim=2, negative_indices=False] for the same effect.
>

This seems to reduce the time computation but not as
@cython.boundscheck(False)
@cython.wraparound(False)

Speedup between decorators and  nothing = 1.5 (and not 2 as mentionned
previously)
Speedup between negative_indices=False and nothing = 1.25

>> The second problem is that it seems that we can nore more used
>> @cython.wraparound(False).
>> I have this error
>>
>>     @cython.wraparound(False)
>>    ^
>> ------------------------------------------------------------
>>
>> /homelocal/max/Memoire/Sources/CAttractionRepulsionAlgorithm.pyx:47:4:
>> 'wraparound' not a valid cython attribute or is being used incorrectly
>>
>> and this error for the same line :
>>
>>     @cython.wraparound(False)
>>    ^
>> ------------------------------------------------------------
>>
>> /homelocal/max/Memoire/Sources/CAttractionRepulsionAlgorithm.pyx:47:4:
>> 'wraparound' not a valid cython language construct
>>
>
> I do not understand you. Can you provide some more context (some more
> lines around where this happened, or an entire file which is as small as
> possible but reproduce the error). Are you sure you remembered to
> "cimport cython"?

Sorry, after sending the email, I was thinking that I had given too few
informations.
Here is the first part of my code :

import CWiredSpace
cimport CWiredSpace
import numpy as np
cimport numpy as np
import time as t
cimport cython

DTYPE = np.int
DTYPEF = np.float
ctypedef np.int_t DTYPE_t
ctypedef np.float_t DTYPEF_t

cdef extern from "math.h":
    double sqrt(double)
    double min(double,double)
    double max(double,double)

cdef inline double double_min(double a, double b): return a if a <= b else b
cdef inline double double_max(double a, double b): return a if a >= b else b

cdef class AttractionRepulsionAlgorithm:
    '''
    classdocs
    '''

    cdef int _nbIteration
    cdef int _startAttraction
    cdef double _Temperature
    cdef CWiredSpace.WiredSpace _WiredSpace

    def __init__(self,int nbIteration, int startAttraction, double
temperature,WiredSpace):
        '''
        Constructor
        '''
        self._nbIteration = nbIteration
        self._startAttraction = startAttraction
        self._Temperature = temperature
        self._WiredSpace = WiredSpace

    @cython.boundscheck(False)
    @cython.wraparound(False)
    def run(self,adjacencyMatrix_,nodesPosition_,int dimension,nbLink_):

Maxime Melchior.


From jonas at lophus.org  Sat Apr  3 01:26:16 2010
From: jonas at lophus.org (Jonas H.)
Date: Sat, 03 Apr 2010 01:26:16 +0200
Subject: [Cython] __dealloc__ with gil
Message-ID: <4BB67D18.9070206@lophus.org>

I need to acquire the GIL for a `__dealloc__` method, but the Cython 
grammar does not allow this:

     cdef  __dealloc__ with gil:: "special methods must be `def`"
     def   __dealloc__ with gil:: Python syntax error (of course)
     cpdef __dealloc__ with gil:: see `cdef`

What is the right way to do this?

     Jonas

From stefan_ml at behnel.de  Sat Apr  3 08:40:38 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 03 Apr 2010 08:40:38 +0200
Subject: [Cython] __dealloc__ with gil
In-Reply-To: <4BB67D18.9070206@lophus.org>
References: <4BB67D18.9070206@lophus.org>
Message-ID: <4BB6E2E6.4000303@behnel.de>

Hi,

note that this is a question for the cython-users mailing list.

Jonas H., 03.04.2010 01:26:
> I need to acquire the GIL for a `__dealloc__` method, but the Cython
> grammar does not allow this:
>
>       cdef  __dealloc__ with gil:: "special methods must be `def`"
>       def   __dealloc__ with gil:: Python syntax error (of course)
>       cpdef __dealloc__ with gil:: see `cdef`
>
> What is the right way to do this?

No need to do that. The __dealloc__ method is only called when the GIL is held.

Stefan

From baihaoyu at gmail.com  Sat Apr  3 09:18:24 2010
From: baihaoyu at gmail.com (Haoyu Bai)
Date: Sat, 3 Apr 2010 15:18:24 +0800
Subject: [Cython] Compile-time decorator
Message-ID: <q2m3eb99a591004030018haa683ad6id1395d1e6bb593b6@mail.gmail.com>

Hi,

Thinking about how to implement Cython decorators (@cclass, @cfunc
etc.) for pure Python mode, I come up with an idea of "compile-time
decorator". Basically, we want these decorators to have effect during
compile time, rather than normal Python decorator that is called in
runtime. So, to implement @cfunc, we can actually implement a function
cfunc(node), which inputs a DefNode and transform it to a
CFuncDefNode, then put the function into the compile-time scope.

Cython directives could also be dealt in this way. This may even
extend to a kind of plugin framework, which allows user to write their
compile-time decorator.

A problem is in which phase these decorators should be called. I guess
it should be somewhere between DecoratorTransform and
AnalyseDeclarationsTransform, but would that be  too late for compiler
directives?

Any comment about this? Thanks!

-- 
Haoyu BAI
School of Computing,
National University of Singapore.

From stefan_ml at behnel.de  Sat Apr  3 12:50:20 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 03 Apr 2010 12:50:20 +0200
Subject: [Cython] Compile-time decorator
In-Reply-To: <q2m3eb99a591004030018haa683ad6id1395d1e6bb593b6@mail.gmail.com>
References: <q2m3eb99a591004030018haa683ad6id1395d1e6bb593b6@mail.gmail.com>
Message-ID: <4BB71D6C.3070901@behnel.de>

Haoyu Bai, 03.04.2010 09:18:
> Thinking about how to implement Cython decorators (@cclass, @cfunc
> etc.) for pure Python mode, I come up with an idea of "compile-time
> decorator". Basically, we want these decorators to have effect during
> compile time, rather than normal Python decorator that is called in
> runtime. So, to implement @cfunc, we can actually implement a function
> cfunc(node), which inputs a DefNode and transform it to a
> CFuncDefNode, then put the function into the compile-time scope.

-1, we shouldn't expose Cython compiler internals to the language level.

It's better to use a transform that intercepts on certain decorators.

Stefan

From baihaoyu at gmail.com  Sat Apr  3 14:15:09 2010
From: baihaoyu at gmail.com (Haoyu Bai)
Date: Sat, 3 Apr 2010 20:15:09 +0800
Subject: [Cython] Compile-time decorator
In-Reply-To: <4BB71D6C.3070901@behnel.de>
References: <q2m3eb99a591004030018haa683ad6id1395d1e6bb593b6@mail.gmail.com> 
	<4BB71D6C.3070901@behnel.de>
Message-ID: <y2u3eb99a591004030515te0fc91fbvd8b527940833a58e@mail.gmail.com>

On Sat, Apr 3, 2010 at 6:50 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Haoyu Bai, 03.04.2010 09:18:
>> Thinking about how to implement Cython decorators (@cclass, @cfunc
>> etc.) for pure Python mode, I come up with an idea of "compile-time
>> decorator". Basically, we want these decorators to have effect during
>> compile time, rather than normal Python decorator that is called in
>> runtime. So, to implement @cfunc, we can actually implement a function
>> cfunc(node), which inputs a DefNode and transform it to a
>> CFuncDefNode, then put the function into the compile-time scope.
>
> -1, we shouldn't expose Cython compiler internals to the language level.
>
> It's better to use a transform that intercepts on certain decorators.
>
> Stefan

By implement @cfunc as a function, I'm not mean to implement it *in
Cython*, it is still a part of the Cython compiler, as the transforms.
And the main motivation is not to expose Cython internal to the users,
but to have a flexible mechanism to support the various decorators -
Cython directives, @cclass @cfunc, and even some Python builtin
decorator (@classmethod, @staticmethod and @property).

These are decorators need to be executed in compile time. Currently,
the code to deal with these decorators are scattered in many places,
some in transforms, some in nodes. It would be better to have a
unified mechanism to deal with all of these decorators. That's what I
proposed.

Indeed, the node classes do not provide an API so we should not expose
them to end-user. But there is nothing to prevent us to write
compile-time function to process them inside Cython (and decorator is
just functions).

-- 
Haoyu BAI
School of Computing,
National University of Singapore.

From stefan_ml at behnel.de  Sat Apr  3 14:24:51 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 03 Apr 2010 14:24:51 +0200
Subject: [Cython] Compile-time decorator
In-Reply-To: <y2u3eb99a591004030515te0fc91fbvd8b527940833a58e@mail.gmail.com>
References: <q2m3eb99a591004030018haa683ad6id1395d1e6bb593b6@mail.gmail.com>
	<4BB71D6C.3070901@behnel.de>
	<y2u3eb99a591004030515te0fc91fbvd8b527940833a58e@mail.gmail.com>
Message-ID: <4BB73393.1030903@behnel.de>

Haoyu Bai, 03.04.2010 14:15:
> On Sat, Apr 3, 2010 at 6:50 PM, Stefan Behnel<stefan_ml at behnel.de>  wrote:
>> Haoyu Bai, 03.04.2010 09:18:
>>> Thinking about how to implement Cython decorators (@cclass, @cfunc
>>> etc.) for pure Python mode, I come up with an idea of "compile-time
>>> decorator". Basically, we want these decorators to have effect during
>>> compile time, rather than normal Python decorator that is called in
>>> runtime. So, to implement @cfunc, we can actually implement a function
>>> cfunc(node), which inputs a DefNode and transform it to a
>>> CFuncDefNode, then put the function into the compile-time scope.
>>
>> -1, we shouldn't expose Cython compiler internals to the language level.
>>
>> It's better to use a transform that intercepts on certain decorators.
>
> By implement @cfunc as a function, I'm not mean to implement it *in
> Cython*, it is still a part of the Cython compiler, as the transforms.
> And the main motivation is not to expose Cython internal to the users,
> but to have a flexible mechanism to support the various decorators -
> Cython directives, @cclass @cfunc, and even some Python builtin
> decorator (@classmethod, @staticmethod and @property).
>
> These are decorators need to be executed in compile time. Currently,
> the code to deal with these decorators are scattered in many places,
> some in transforms, some in nodes. It would be better to have a
> unified mechanism to deal with all of these decorators. That's what I
> proposed.
>
> Indeed, the node classes do not provide an API so we should not expose
> them to end-user. But there is nothing to prevent us to write
> compile-time function to process them inside Cython (and decorator is
> just functions).

In that case, there is nothing to gain from making them "compile time 
evaluated functions", because the implementation itself would still be 
inside of the compiler. So the decorator would still just be a name for 
something that is triggered inside of the compiler, which may be a code 
block, a function, a method or a whole transformation class.

While I do not doubt that there is potential for a code cleanup, the main 
reason why the "decorators are scattered in many places" is that their 
implementation requires code to be executed during different compiler 
stages. This cannot be helped by stuffing everything into a single function.

Stefan

From robertwb at math.washington.edu  Sat Apr  3 21:59:17 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 3 Apr 2010 12:59:17 -0700
Subject: [Cython] Compile-time decorator
In-Reply-To: <4BB73393.1030903@behnel.de>
References: <q2m3eb99a591004030018haa683ad6id1395d1e6bb593b6@mail.gmail.com>
	<4BB71D6C.3070901@behnel.de>
	<y2u3eb99a591004030515te0fc91fbvd8b527940833a58e@mail.gmail.com>
	<4BB73393.1030903@behnel.de>
Message-ID: <F4A07E8F-A27E-4CDD-9342-E37D0DED3F1E@math.washington.edu>

On Apr 3, 2010, at 5:24 AM, Stefan Behnel wrote:

> Haoyu Bai, 03.04.2010 14:15:
>> On Sat, Apr 3, 2010 at 6:50 PM, Stefan Behnel<stefan_ml at behnel.de>   
>> wrote:
>>> Haoyu Bai, 03.04.2010 09:18:
>>>> Thinking about how to implement Cython decorators (@cclass, @cfunc
>>>> etc.) for pure Python mode, I come up with an idea of "compile-time
>>>> decorator". Basically, we want these decorators to have effect  
>>>> during
>>>> compile time, rather than normal Python decorator that is called in
>>>> runtime. So, to implement @cfunc, we can actually implement a  
>>>> function
>>>> cfunc(node), which inputs a DefNode and transform it to a
>>>> CFuncDefNode, then put the function into the compile-time scope.
>>>
>>> -1, we shouldn't expose Cython compiler internals to the language  
>>> level.
>>>
>>> It's better to use a transform that intercepts on certain  
>>> decorators.
>>
>> By implement @cfunc as a function, I'm not mean to implement it *in
>> Cython*, it is still a part of the Cython compiler, as the  
>> transforms.
>> And the main motivation is not to expose Cython internal to the  
>> users,
>> but to have a flexible mechanism to support the various decorators -
>> Cython directives, @cclass @cfunc, and even some Python builtin
>> decorator (@classmethod, @staticmethod and @property).
>>
>> These are decorators need to be executed in compile time. Currently,
>> the code to deal with these decorators are scattered in many places,
>> some in transforms, some in nodes. It would be better to have a
>> unified mechanism to deal with all of these decorators. That's what I
>> proposed.
>>
>> Indeed, the node classes do not provide an API so we should not  
>> expose
>> them to end-user. But there is nothing to prevent us to write
>> compile-time function to process them inside Cython (and decorator is
>> just functions).
>
> In that case, there is nothing to gain from making them "compile time
> evaluated functions", because the implementation itself would still be
> inside of the compiler. So the decorator would still just be a name  
> for
> something that is triggered inside of the compiler, which may be a  
> code
> block, a function, a method or a whole transformation class.
>
> While I do not doubt that there is potential for a code cleanup, the  
> main
> reason why the "decorators are scattered in many places" is that their
> implementation requires code to be executed during different compiler
> stages. This cannot be helped by stuffing everything into a single  
> function.

Also, many (most?) directives (e.g. boundscheck, cdivision,  
infer_types) aren't best expressed as node transformations, rather  
they simply affect the behavior of the contained code. I would  
certainly imagine that @cfunc, @cclass, (and others) will be  
implemented a visitor that in turn calls such transforming functions.

- Robert
  

From jonas at lophus.org  Sat Apr  3 22:06:54 2010
From: jonas at lophus.org (Jonas H.)
Date: Sat, 03 Apr 2010 22:06:54 +0200
Subject: [Cython] __dealloc__ with gil
In-Reply-To: <4BB6E2E6.4000303@behnel.de>
References: <4BB67D18.9070206@lophus.org> <4BB6E2E6.4000303@behnel.de>
Message-ID: <4BB79FDE.4000003@lophus.org>

On 04/03/2010 08:40 AM, Stefan Behnel wrote:
> No need to do that. The __dealloc__ method is only called when the GIL
> is held.

It seems not. I'm not completely sure, but if I debug the generated C 
code and look at the return value of an inserted `PyThreadState_GET()`, 
it is 0x0 -- which is only possible if the GIL is not held properly, 
isn't it?

However if I do something like

     def __dealloc__(self):
         self.__dealloc__with_gil__()
     def __dealloc__with_gil__(self) with gil:
         ...

my program does not crash/receive a Segmentation fault error.

... same with __len__ and so on, is there also "guaranteed" that the GIL 
is held?

     Jonas

From greg.ewing at canterbury.ac.nz  Sun Apr  4 11:58:20 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 04 Apr 2010 21:58:20 +1200
Subject: [Cython] __dealloc__ with gil
In-Reply-To: <4BB67D18.9070206@lophus.org>
References: <4BB67D18.9070206@lophus.org>
Message-ID: <4BB862BC.70908@canterbury.ac.nz>

Jonas H. wrote:
> I need to acquire the GIL for a `__dealloc__` method,

Are you sure? The only way __dealloc__ can get called is
when a Py_DECREF drops the object's reference count to
zero, and nobody should be calling Py_DECREF without
holding the GIL.

-- 
Greg

From baihaoyu at gmail.com  Mon Apr  5 18:31:12 2010
From: baihaoyu at gmail.com (Haoyu Bai)
Date: Tue, 6 Apr 2010 00:31:12 +0800
Subject: [Cython] Compile-time decorator
In-Reply-To: <F4A07E8F-A27E-4CDD-9342-E37D0DED3F1E@math.washington.edu>
References: <q2m3eb99a591004030018haa683ad6id1395d1e6bb593b6@mail.gmail.com> 
	<4BB71D6C.3070901@behnel.de>
	<y2u3eb99a591004030515te0fc91fbvd8b527940833a58e@mail.gmail.com>
	<4BB73393.1030903@behnel.de>
	<F4A07E8F-A27E-4CDD-9342-E37D0DED3F1E@math.washington.edu>
Message-ID: <g2x3eb99a591004050931w9af213f6oc676822842046422@mail.gmail.com>

On Sun, Apr 4, 2010 at 3:59 AM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Apr 3, 2010, at 5:24 AM, Stefan Behnel wrote:
>
>> Haoyu Bai, 03.04.2010 14:15:
>>> On Sat, Apr 3, 2010 at 6:50 PM, Stefan Behnel<stefan_ml at behnel.de>
>>> wrote:
>>>> Haoyu Bai, 03.04.2010 09:18:
>>>>> Thinking about how to implement Cython decorators (@cclass, @cfunc
>>>>> etc.) for pure Python mode, I come up with an idea of "compile-time
>>>>> decorator". Basically, we want these decorators to have effect
>>>>> during
>>>>> compile time, rather than normal Python decorator that is called in
>>>>> runtime. So, to implement @cfunc, we can actually implement a
>>>>> function
>>>>> cfunc(node), which inputs a DefNode and transform it to a
>>>>> CFuncDefNode, then put the function into the compile-time scope.
>>>>
>>>> -1, we shouldn't expose Cython compiler internals to the language
>>>> level.
>>>>
>>>> It's better to use a transform that intercepts on certain
>>>> decorators.
>>>
>>> By implement @cfunc as a function, I'm not mean to implement it *in
>>> Cython*, it is still a part of the Cython compiler, as the
>>> transforms.
>>> And the main motivation is not to expose Cython internal to the
>>> users,
>>> but to have a flexible mechanism to support the various decorators -
>>> Cython directives, @cclass @cfunc, and even some Python builtin
>>> decorator (@classmethod, @staticmethod and @property).
>>>
>>> These are decorators need to be executed in compile time. Currently,
>>> the code to deal with these decorators are scattered in many places,
>>> some in transforms, some in nodes. It would be better to have a
>>> unified mechanism to deal with all of these decorators. That's what I
>>> proposed.
>>>
>>> Indeed, the node classes do not provide an API so we should not
>>> expose
>>> them to end-user. But there is nothing to prevent us to write
>>> compile-time function to process them inside Cython (and decorator is
>>> just functions).
>>
>> In that case, there is nothing to gain from making them "compile time
>> evaluated functions", because the implementation itself would still be
>> inside of the compiler. So the decorator would still just be a name
>> for
>> something that is triggered inside of the compiler, which may be a
>> code
>> block, a function, a method or a whole transformation class.
>>
>> While I do not doubt that there is potential for a code cleanup, the
>> main
>> reason why the "decorators are scattered in many places" is that their
>> implementation requires code to be executed during different compiler
>> stages. This cannot be helped by stuffing everything into a single
>> function.
>
> Also, many (most?) directives (e.g. boundscheck, cdivision,
> infer_types) aren't best expressed as node transformations, rather
> they simply affect the behavior of the contained code. I would
> certainly imagine that @cfunc, @cclass, (and others) will be
> implemented a visitor that in turn calls such transforming functions.
>
> - Robert
>

In my opinion, decorators should only responsible for either setting
directives or calling a transforming method of the node to transform
the node to another. Maybe we can refactor the current
InterpretCompilerDirectives transform to add generic support for
decorators and directives. Additionally, I think it is better to have
something like a CythonModule.py as a counterpart of the Shadow.py,
then it is easy to assure the content of the 'cython' module is the
same between Cython and pure Python mode.

Another interesting problem is how to deal with name overriding. What
to do if you have:

import cython as C
...
C = 'foo'
...

or similar thing like:

import cython
...
foo = cython.cdef
...

and as mentioned by Robert:

from cython import *
from foo import *  # where foo contains something clash to cython

Note that these may also apply to some Python built-in such as
@property, which may need some special treatment so can not just seen
as normal Python built-in.

As InterpretCompilerDirectives happens before
AnalyseDeclarationsTransform, seems the only choice is to forbid these
things. This could work in this way:

1. All the cython names are imported to a protected global scope
2. The necessary directive setting and node transforms are all done in
the InterpretCompilerDirectives stage.
3. After InterpretCompilerDirectives, raise an error for any access to
the protected global scope. ('int' and 'float' may be exceptions)
4. After "from cython import *", any "from foo import *" is disallowed
(or warned?). (An alternative is to forbid "from cython import *")

This may be the best we can do. Of course, one can always do
exec('cython = "foo" '), well...

-- 
Haoyu BAI
School of Computing,
National University of Singapore.

From robertwb at math.washington.edu  Mon Apr  5 21:00:16 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 5 Apr 2010 12:00:16 -0700
Subject: [Cython] Compile-time decorator
In-Reply-To: <g2x3eb99a591004050931w9af213f6oc676822842046422@mail.gmail.com>
References: <q2m3eb99a591004030018haa683ad6id1395d1e6bb593b6@mail.gmail.com>
	<4BB71D6C.3070901@behnel.de>
	<y2u3eb99a591004030515te0fc91fbvd8b527940833a58e@mail.gmail.com>
	<4BB73393.1030903@behnel.de>
	<F4A07E8F-A27E-4CDD-9342-E37D0DED3F1E@math.washington.edu>
	<g2x3eb99a591004050931w9af213f6oc676822842046422@mail.gmail.com>
Message-ID: <51EB9181-F4C6-43CE-B747-22F9903AB2EA@math.washington.edu>

On Apr 5, 2010, at 9:31 AM, Haoyu Bai wrote:

> On Sun, Apr 4, 2010 at 3:59 AM, Robert Bradshaw
> <robertwb at math.washington.edu> wrote:
>> On Apr 3, 2010, at 5:24 AM, Stefan Behnel wrote:
>>
>>> Haoyu Bai, 03.04.2010 14:15:
>>>> On Sat, Apr 3, 2010 at 6:50 PM, Stefan Behnel<stefan_ml at behnel.de>
>>>> wrote:
>>>>> Haoyu Bai, 03.04.2010 09:18:
>>>>>> Thinking about how to implement Cython decorators (@cclass,  
>>>>>> @cfunc
>>>>>> etc.) for pure Python mode, I come up with an idea of "compile- 
>>>>>> time
>>>>>> decorator". Basically, we want these decorators to have effect
>>>>>> during
>>>>>> compile time, rather than normal Python decorator that is  
>>>>>> called in
>>>>>> runtime. So, to implement @cfunc, we can actually implement a
>>>>>> function
>>>>>> cfunc(node), which inputs a DefNode and transform it to a
>>>>>> CFuncDefNode, then put the function into the compile-time scope.
>>>>>
>>>>> -1, we shouldn't expose Cython compiler internals to the language
>>>>> level.
>>>>>
>>>>> It's better to use a transform that intercepts on certain
>>>>> decorators.
>>>>
>>>> By implement @cfunc as a function, I'm not mean to implement it *in
>>>> Cython*, it is still a part of the Cython compiler, as the
>>>> transforms.
>>>> And the main motivation is not to expose Cython internal to the
>>>> users,
>>>> but to have a flexible mechanism to support the various  
>>>> decorators -
>>>> Cython directives, @cclass @cfunc, and even some Python builtin
>>>> decorator (@classmethod, @staticmethod and @property).
>>>>
>>>> These are decorators need to be executed in compile time.  
>>>> Currently,
>>>> the code to deal with these decorators are scattered in many  
>>>> places,
>>>> some in transforms, some in nodes. It would be better to have a
>>>> unified mechanism to deal with all of these decorators. That's  
>>>> what I
>>>> proposed.
>>>>
>>>> Indeed, the node classes do not provide an API so we should not
>>>> expose
>>>> them to end-user. But there is nothing to prevent us to write
>>>> compile-time function to process them inside Cython (and  
>>>> decorator is
>>>> just functions).
>>>
>>> In that case, there is nothing to gain from making them "compile  
>>> time
>>> evaluated functions", because the implementation itself would  
>>> still be
>>> inside of the compiler. So the decorator would still just be a name
>>> for
>>> something that is triggered inside of the compiler, which may be a
>>> code
>>> block, a function, a method or a whole transformation class.
>>>
>>> While I do not doubt that there is potential for a code cleanup, the
>>> main
>>> reason why the "decorators are scattered in many places" is that  
>>> their
>>> implementation requires code to be executed during different  
>>> compiler
>>> stages. This cannot be helped by stuffing everything into a single
>>> function.
>>
>> Also, many (most?) directives (e.g. boundscheck, cdivision,
>> infer_types) aren't best expressed as node transformations, rather
>> they simply affect the behavior of the contained code. I would
>> certainly imagine that @cfunc, @cclass, (and others) will be
>> implemented a visitor that in turn calls such transforming functions.
>>
>> - Robert
>>
>
> In my opinion, decorators should only responsible for either setting
> directives or calling a transforming method of the node to transform
> the node to another.

This is currently the case.

> Maybe we can refactor the current
> InterpretCompilerDirectives transform to add generic support for
> decorators and directives.

As mentioned earlier, one difficulty is that different transformations  
need to be handled at different phases of the compilation pipeline.  
Doing something like this for plugins and/or macros could be an  
interesting avenue to pursue once we have full Python compatibility.

> Additionally, I think it is better to have
> something like a CythonModule.py as a counterpart of the Shadow.py,
> then it is easy to assure the content of the 'cython' module is the
> same between Cython and pure Python mode.

I'm not convinced that grouping internal Cython constructions based on  
whether they have a "pure" counterpart is the best way to do things-- 
in my ind the directives having to do with division are best handled  
in the division node, those dealing with buffers best handled in the  
buffer code, etc.

> Another interesting problem is how to deal with name overriding.

Yes, this is an issue that has not been adequately solved yet. I'm not  
sure it can be (see below).

> What to do if you have:
>
> import cython as C
> ...
> C = 'foo'
> ...
>
> or similar thing like:
>
> import cython
> ...
> foo = cython.cdef
> ...

from cython import cclass

if random.random() < 0.5:
     cclass = lambda x: x

%cclass:
class Foo:
     ...

>
> and as mentioned by Robert:
>
> from cython import *
> from foo import *  # where foo contains something clash to cython
>
> Note that these may also apply to some Python built-in such as
> @property, which may need some special treatment so can not just seen
> as normal Python built-in.
>
> As InterpretCompilerDirectives happens before
> AnalyseDeclarationsTransform, seems the only choice is to forbid these
> things. This could work in this way:
>
> 1. All the cython names are imported to a protected global scope
> 2. The necessary directive setting and node transforms are all done in
> the InterpretCompilerDirectives stage.
> 3. After InterpretCompilerDirectives, raise an error for any access to
> the protected global scope. ('int' and 'float' may be exceptions)

Hmm... Something like this (or at least tying whether or not something  
is a cython directive into the Entry objects) may improve semantics  
somewhat.

> 4. After "from cython import *", any "from foo import *" is disallowed
> (or warned?). (An alternative is to forbid "from cython import *")

I think it's much safer to simply forbid "from cython import *",  
especially as its contents may change with time.

> This may be the best we can do. Of course, one can always do
> exec('cython = "foo" '), well...

At least Cython wouldn't choke on that.

- Robert


From stefan_ml at behnel.de  Mon Apr  5 22:25:44 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 05 Apr 2010 22:25:44 +0200
Subject: [Cython] Compile-time decorator
In-Reply-To: <51EB9181-F4C6-43CE-B747-22F9903AB2EA@math.washington.edu>
References: <q2m3eb99a591004030018haa683ad6id1395d1e6bb593b6@mail.gmail.com>	<4BB71D6C.3070901@behnel.de>	<y2u3eb99a591004030515te0fc91fbvd8b527940833a58e@mail.gmail.com>	<4BB73393.1030903@behnel.de>	<F4A07E8F-A27E-4CDD-9342-E37D0DED3F1E@math.washington.edu>	<g2x3eb99a591004050931w9af213f6oc676822842046422@mail.gmail.com>
	<51EB9181-F4C6-43CE-B747-22F9903AB2EA@math.washington.edu>
Message-ID: <4BBA4748.6090201@behnel.de>

Robert Bradshaw, 05.04.2010 21:00:
> On Apr 5, 2010, at 9:31 AM, Haoyu Bai wrote:
>> Additionally, I think it is better to have
>> something like a CythonModule.py as a counterpart of the Shadow.py,
>> then it is easy to assure the content of the 'cython' module is the
>> same between Cython and pure Python mode.
>
> I'm not convinced that grouping internal Cython constructions based on
> whether they have a "pure" counterpart is the best way to do things--
> in my ind the directives having to do with division are best handled
> in the division node, those dealing with buffers best handled in the
> buffer code, etc.

+1

It's nice to have a simple language interface to all these features, but 
that doesn't mean they have to be any similar on the implementation side.


>> What to do if you have:
>>
>> import cython as C
>> ...
>> C = 'foo'
>> ...
>>
>> or similar thing like:
>>
>> import cython
>> ...
>> foo = cython.cdef
>> ...
>
> from cython import cclass
>
> if random.random()<  0.5:
>       cclass = lambda x: x
>
> %cclass:
> class Foo:
>       ...

I was about to say that these things would require at least a full featured 
flow control analysis mechanism, but it's true that there will always be 
cases where this can't be solved at all.


> I think it's much safer to simply forbid "from cython import *",
> especially as its contents may change with time.

+1

I'm even fine with forbidding reassignments to names imported from the 
cython namespace. The cython module is special in every way, so making it 
just a bit more special so that we can simplify the compiler sounds 
reasonable to me. I can't imagine any use case for any of the above 
examples. When we find them in code, it's almost certainly a bug that 
Cython should bark at.


>> This may be the best we can do. Of course, one can always do
>> exec('cython = "foo" '), well...
>
> At least Cython wouldn't choke on that.

... and that's not even because it can't currently work. ;)

Stefan

From baihaoyu at gmail.com  Tue Apr  6 05:17:59 2010
From: baihaoyu at gmail.com (Haoyu Bai)
Date: Tue, 6 Apr 2010 11:17:59 +0800
Subject: [Cython] Compile-time decorator
In-Reply-To: <4BBA4748.6090201@behnel.de>
References: <q2m3eb99a591004030018haa683ad6id1395d1e6bb593b6@mail.gmail.com> 
	<4BB71D6C.3070901@behnel.de>
	<y2u3eb99a591004030515te0fc91fbvd8b527940833a58e@mail.gmail.com>
	<4BB73393.1030903@behnel.de>
	<F4A07E8F-A27E-4CDD-9342-E37D0DED3F1E@math.washington.edu> 
	<g2x3eb99a591004050931w9af213f6oc676822842046422@mail.gmail.com> 
	<51EB9181-F4C6-43CE-B747-22F9903AB2EA@math.washington.edu> 
	<4BBA4748.6090201@behnel.de>
Message-ID: <p2m3eb99a591004052017i86a64f24t47d5bf0c92024f98@mail.gmail.com>

On Tue, Apr 6, 2010 at 4:25 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Robert Bradshaw, 05.04.2010 21:00:
>> On Apr 5, 2010, at 9:31 AM, Haoyu Bai wrote:
>>> Additionally, I think it is better to have
>>> something like a CythonModule.py as a counterpart of the Shadow.py,
>>> then it is easy to assure the content of the 'cython' module is the
>>> same between Cython and pure Python mode.
>>
>> I'm not convinced that grouping internal Cython constructions based on
>> whether they have a "pure" counterpart is the best way to do things--
>> in my ind the directives having to do with division are best handled
>> in the division node, those dealing with buffers best handled in the
>> buffer code, etc.
>
> +1
>
> It's nice to have a simple language interface to all these features, but
> that doesn't mean they have to be any similar on the implementation side.
>
>

I'm not mean to unify the implementation of these features. The
decorators would just set directive or flags to the node, then the
nodes only need to dealing with these directives or flags, rather than
check the decorators directly.

>> I think it's much safer to simply forbid "from cython import *",
>> especially as its contents may change with time.
>
> +1
>
> I'm even fine with forbidding reassignments to names imported from the
> cython namespace. The cython module is special in every way, so making it
> just a bit more special so that we can simplify the compiler sounds
> reasonable to me. I can't imagine any use case for any of the above
> examples. When we find them in code, it's almost certainly a bug that
> Cython should bark at.
>
>

Agree with forbidding reassignments. But if we forbid "from cython
import *", then do we allow this:

from cython import cfunc, int, float

otherwise, I just imagine, it could be pain to write a lot of
@cython.cfunc instead of the simply cdef.


-- 
Haoyu BAI
School of Computing,
National University of Singapore.

From robertwb at math.washington.edu  Tue Apr  6 07:55:28 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 5 Apr 2010 22:55:28 -0700
Subject: [Cython] Compile-time decorator
In-Reply-To: <p2m3eb99a591004052017i86a64f24t47d5bf0c92024f98@mail.gmail.com>
References: <q2m3eb99a591004030018haa683ad6id1395d1e6bb593b6@mail.gmail.com>
	<4BB71D6C.3070901@behnel.de>
	<y2u3eb99a591004030515te0fc91fbvd8b527940833a58e@mail.gmail.com>
	<4BB73393.1030903@behnel.de>
	<F4A07E8F-A27E-4CDD-9342-E37D0DED3F1E@math.washington.edu>
	<g2x3eb99a591004050931w9af213f6oc676822842046422@mail.gmail.com>
	<51EB9181-F4C6-43CE-B747-22F9903AB2EA@math.washington.edu>
	<4BBA4748.6090201@behnel.de>
	<p2m3eb99a591004052017i86a64f24t47d5bf0c92024f98@mail.gmail.com>
Message-ID: <0FB80306-01E6-442F-8995-E00573D5777D@math.washington.edu>

On Apr 5, 2010, at 8:17 PM, Haoyu Bai wrote:

> On Tue, Apr 6, 2010 at 4:25 AM, Stefan Behnel <stefan_ml at behnel.de>  
> wrote:
>> Robert Bradshaw, 05.04.2010 21:00:
>>> On Apr 5, 2010, at 9:31 AM, Haoyu Bai wrote:
>>>> Additionally, I think it is better to have
>>>> something like a CythonModule.py as a counterpart of the Shadow.py,
>>>> then it is easy to assure the content of the 'cython' module is the
>>>> same between Cython and pure Python mode.
>>>
>>> I'm not convinced that grouping internal Cython constructions  
>>> based on
>>> whether they have a "pure" counterpart is the best way to do  
>>> things--
>>> in my ind the directives having to do with division are best handled
>>> in the division node, those dealing with buffers best handled in the
>>> buffer code, etc.
>>
>> +1
>>
>> It's nice to have a simple language interface to all these  
>> features, but
>> that doesn't mean they have to be any similar on the implementation  
>> side.
>
> I'm not mean to unify the implementation of these features. The
> decorators would just set directive or flags to the node, then the
> nodes only need to dealing with these directives or flags, rather than
> check the decorators directly.

This sounds very much like what the InterpretCompilerDirectives  
visitor does. it also unifies the various ways of setting directives  
(decorators are only one option).

>>> I think it's much safer to simply forbid "from cython import *",
>>> especially as its contents may change with time.
>>
>> +1
>>
>> I'm even fine with forbidding reassignments to names imported from  
>> the
>> cython namespace. The cython module is special in every way, so  
>> making it
>> just a bit more special so that we can simplify the compiler sounds
>> reasonable to me.

Sounds reasonable to me.

>> I can't imagine any use case for any of the above
>> examples. When we find them in code, it's almost certainly a bug that
>> Cython should bark at.
>
> Agree with forbidding reassignments. But if we forbid "from cython
> import *", then do we allow this:
>
> from cython import cfunc, int, float
>
> otherwise, I just imagine, it could be pain to write a lot of
> @cython.cfunc instead of the simply cdef.

Yes, for sure we will (and currently do) allow named imports, even  
"from cython import long as my_long."

- Robert



From robertwb at math.washington.edu  Tue Apr  6 07:56:36 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 5 Apr 2010 22:56:36 -0700
Subject: [Cython] GSoC project for improving Python compatibility
In-Reply-To: <h2i3eb99a591004012253k83f8539u3efda34280151c0c@mail.gmail.com>
References: <3eb99a591003210431s12042237ubd1be5c73b24bfec@mail.gmail.com>
	<4BA72159.6010007@student.matnat.uio.no>
	<3eb99a591003220939r3c5a94c3ibdd30826f885bc7e@mail.gmail.com>
	<a8c4ff4c1003221207w74d19898uf2c8bff011b3c67b@mail.gmail.com>
	<3eb99a591003270813x38a6d554u54419e693a345efc@mail.gmail.com>
	<4A9A1D0E-EB65-4838-A59B-9E1737390DDC@math.washington.edu>
	<h2i3eb99a591004012253k83f8539u3efda34280151c0c@mail.gmail.com>
Message-ID: <5171D334-8DBC-4911-9ED1-3C49438909C1@math.washington.edu>

On Apr 1, 2010, at 10:53 PM, Haoyu Bai wrote:

> On Wed, Mar 31, 2010 at 1:36 AM, Robert Bradshaw
> <robertwb at math.washington.edu> wrote:
>>
>> There's also Py3 scope binding rules for list comprehensions,
>> exception blocks, etc.
>>
>
> Thanks. I added these to my list.

Do you have a proposal written up yet? It may be useful to put it up  
on the Cython wiki before it's due for feedback (though, of course, I  
can't promise anything).

- Robert



From stefan_ml at behnel.de  Tue Apr  6 08:00:26 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 06 Apr 2010 08:00:26 +0200
Subject: [Cython] Compile-time decorator
In-Reply-To: <p2m3eb99a591004052017i86a64f24t47d5bf0c92024f98@mail.gmail.com>
References: <q2m3eb99a591004030018haa683ad6id1395d1e6bb593b6@mail.gmail.com>
	<4BB71D6C.3070901@behnel.de>	<y2u3eb99a591004030515te0fc91fbvd8b527940833a58e@mail.gmail.com>	<4BB73393.1030903@behnel.de>	<F4A07E8F-A27E-4CDD-9342-E37D0DED3F1E@math.washington.edu>
	<g2x3eb99a591004050931w9af213f6oc676822842046422@mail.gmail.com>
	<51EB9181-F4C6-43CE-B747-22F9903AB2EA@math.washington.edu>
	<4BBA4748.6090201@behnel.de>
	<p2m3eb99a591004052017i86a64f24t47d5bf0c92024f98@mail.gmail.com>
Message-ID: <4BBACDFA.70709@behnel.de>

Haoyu Bai, 06.04.2010 05:17:
> On Tue, Apr 6, 2010 at 4:25 AM, Stefan Behnel wrote:
>> Robert Bradshaw, 05.04.2010 21:00:
>>> I'm not convinced that grouping internal Cython constructions based on
>>> whether they have a "pure" counterpart is the best way to do things--
>>> in my ind the directives having to do with division are best handled
>>> in the division node, those dealing with buffers best handled in the
>>> buffer code, etc.
>>
>> It's nice to have a simple language interface to all these features, but
>> that doesn't mean they have to be any similar on the implementation side.
>
> I'm not mean to unify the implementation of these features. The
> decorators would just set directive or flags to the node

Well, again: it's not just nodes.

> then the
> nodes only need to dealing with these directives or flags, rather than
> check the decorators directly.

The decorators work on subtrees, usually functions, so they'd have to apply 
their "directive or flags" to the whole subtree, and they'd have to know 
what nodes to intercept on. The easiest way to do these things is - a 
transform visitor, just what we currently have.


> if we forbid "from cython import *", then do we allow this:
>
> from cython import cfunc, int, float

Sure.


> otherwise, I just imagine, it could be pain to write a lot of
> @cython.cfunc instead of the simply cdef.

If you have to write that a lot, just use a selective import or

     cimport cython as cy

I get the impression that you are chasing non-problems here.

Stefan

From baihaoyu at gmail.com  Tue Apr  6 08:58:23 2010
From: baihaoyu at gmail.com (Haoyu Bai)
Date: Tue, 6 Apr 2010 14:58:23 +0800
Subject: [Cython] GSoC project for improving Python compatibility
In-Reply-To: <5171D334-8DBC-4911-9ED1-3C49438909C1@math.washington.edu>
References: <3eb99a591003210431s12042237ubd1be5c73b24bfec@mail.gmail.com> 
	<4BA72159.6010007@student.matnat.uio.no>
	<3eb99a591003220939r3c5a94c3ibdd30826f885bc7e@mail.gmail.com> 
	<a8c4ff4c1003221207w74d19898uf2c8bff011b3c67b@mail.gmail.com> 
	<3eb99a591003270813x38a6d554u54419e693a345efc@mail.gmail.com> 
	<4A9A1D0E-EB65-4838-A59B-9E1737390DDC@math.washington.edu> 
	<h2i3eb99a591004012253k83f8539u3efda34280151c0c@mail.gmail.com> 
	<5171D334-8DBC-4911-9ED1-3C49438909C1@math.washington.edu>
Message-ID: <h2v3eb99a591004052358pe8f9b871p819b8b21ad813813@mail.gmail.com>

On Tue, Apr 6, 2010 at 1:56 PM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Apr 1, 2010, at 10:53 PM, Haoyu Bai wrote:
>
>> On Wed, Mar 31, 2010 at 1:36 AM, Robert Bradshaw
>> <robertwb at math.washington.edu> wrote:
>>>
>>> There's also Py3 scope binding rules for list comprehensions,
>>> exception blocks, etc.
>>>
>>
>> Thanks. I added these to my list.
>
> Do you have a proposal written up yet? It may be useful to put it up
> on the Cython wiki before it's due for feedback (though, of course, I
> can't promise anything).
>
> - Robert
>
>

Actually I have posted it to GSoC website and sent a copy to Craig,
I'll put it onto the wiki sooner.


-- 
Haoyu BAI
School of Computing,
National University of Singapore.

From baihaoyu at gmail.com  Tue Apr  6 08:52:01 2010
From: baihaoyu at gmail.com (Haoyu Bai)
Date: Tue, 6 Apr 2010 14:52:01 +0800
Subject: [Cython] Compile-time decorator
In-Reply-To: <4BBACDFA.70709@behnel.de>
References: <q2m3eb99a591004030018haa683ad6id1395d1e6bb593b6@mail.gmail.com> 
	<4BB71D6C.3070901@behnel.de>
	<y2u3eb99a591004030515te0fc91fbvd8b527940833a58e@mail.gmail.com>
	<4BB73393.1030903@behnel.de>
	<F4A07E8F-A27E-4CDD-9342-E37D0DED3F1E@math.washington.edu> 
	<g2x3eb99a591004050931w9af213f6oc676822842046422@mail.gmail.com> 
	<51EB9181-F4C6-43CE-B747-22F9903AB2EA@math.washington.edu> 
	<4BBA4748.6090201@behnel.de>
	<p2m3eb99a591004052017i86a64f24t47d5bf0c92024f98@mail.gmail.com>
	<4BBACDFA.70709@behnel.de>
Message-ID: <y2n3eb99a591004052352yc478c691j5382f6d385de321c@mail.gmail.com>

On Tue, Apr 6, 2010 at 2:00 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Haoyu Bai, 06.04.2010 05:17:
>> On Tue, Apr 6, 2010 at 4:25 AM, Stefan Behnel wrote:
>>> Robert Bradshaw, 05.04.2010 21:00:
>>>> I'm not convinced that grouping internal Cython constructions based on
>>>> whether they have a "pure" counterpart is the best way to do things--
>>>> in my ind the directives having to do with division are best handled
>>>> in the division node, those dealing with buffers best handled in the
>>>> buffer code, etc.
>>>
>>> It's nice to have a simple language interface to all these features, but
>>> that doesn't mean they have to be any similar on the implementation side.
>>
>> I'm not mean to unify the implementation of these features. The
>> decorators would just set directive or flags to the node
>
> Well, again: it's not just nodes.
>
>> then the
>> nodes only need to dealing with these directives or flags, rather than
>> check the decorators directly.
>
> The decorators work on subtrees, usually functions, so they'd have to apply
> their "directive or flags" to the whole subtree, and they'd have to know
> what nodes to intercept on. The easiest way to do these things is - a
> transform visitor, just what we currently have.
>
>
>> if we forbid "from cython import *", then do we allow this:
>>
>> from cython import cfunc, int, float
>
> Sure.
>
>
>> otherwise, I just imagine, it could be pain to write a lot of
>> @cython.cfunc instead of the simply cdef.
>
> If you have to write that a lot, just use a selective import or
>
> ? ? cimport cython as cy
>
> I get the impression that you are chasing non-problems here.
>
> Stefan

I just thinking what could I help for improving Cython. Maybe I am
thinking the wrong way since I'm not such familiar with the Cython
design concept yet. However, no one intend to chase non-problems.


-- 
Haoyu BAI
School of Computing,
National University of Singapore.

From robertwb at math.washington.edu  Tue Apr  6 09:11:14 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 6 Apr 2010 00:11:14 -0700
Subject: [Cython] Compile-time decorator
In-Reply-To: <y2n3eb99a591004052352yc478c691j5382f6d385de321c@mail.gmail.com>
References: <q2m3eb99a591004030018haa683ad6id1395d1e6bb593b6@mail.gmail.com>
	<4BB71D6C.3070901@behnel.de>
	<y2u3eb99a591004030515te0fc91fbvd8b527940833a58e@mail.gmail.com>
	<4BB73393.1030903@behnel.de>
	<F4A07E8F-A27E-4CDD-9342-E37D0DED3F1E@math.washington.edu>
	<g2x3eb99a591004050931w9af213f6oc676822842046422@mail.gmail.com>
	<51EB9181-F4C6-43CE-B747-22F9903AB2EA@math.washington.edu>
	<4BBA4748.6090201@behnel.de>
	<p2m3eb99a591004052017i86a64f24t47d5bf0c92024f98@mail.gmail.com>
	<4BBACDFA.70709@behnel.de>
	<y2n3eb99a591004052352yc478c691j5382f6d385de321c@mail.gmail.com>
Message-ID: <63651FA5-F391-4384-93BF-18B3032C1F21@math.washington.edu>

On Apr 5, 2010, at 11:52 PM, Haoyu Bai wrote:

> On Tue, Apr 6, 2010 at 2:00 PM, Stefan Behnel <stefan_ml at behnel.de>  
> wrote:
>> Haoyu Bai, 06.04.2010 05:17:
>>> On Tue, Apr 6, 2010 at 4:25 AM, Stefan Behnel wrote:
>>>> Robert Bradshaw, 05.04.2010 21:00:
>>>>> I'm not convinced that grouping internal Cython constructions  
>>>>> based on
>>>>> whether they have a "pure" counterpart is the best way to do  
>>>>> things--
>>>>> in my ind the directives having to do with division are best  
>>>>> handled
>>>>> in the division node, those dealing with buffers best handled in  
>>>>> the
>>>>> buffer code, etc.
>>>>
>>>> It's nice to have a simple language interface to all these  
>>>> features, but
>>>> that doesn't mean they have to be any similar on the  
>>>> implementation side.
>>>
>>> I'm not mean to unify the implementation of these features. The
>>> decorators would just set directive or flags to the node
>>
>> Well, again: it's not just nodes.
>>
>>> then the
>>> nodes only need to dealing with these directives or flags, rather  
>>> than
>>> check the decorators directly.
>>
>> The decorators work on subtrees, usually functions, so they'd have  
>> to apply
>> their "directive or flags" to the whole subtree, and they'd have to  
>> know
>> what nodes to intercept on. The easiest way to do these things is - a
>> transform visitor, just what we currently have.
>>
>>
>>> if we forbid "from cython import *", then do we allow this:
>>>
>>> from cython import cfunc, int, float
>>
>> Sure.
>>
>>
>>> otherwise, I just imagine, it could be pain to write a lot of
>>> @cython.cfunc instead of the simply cdef.
>>
>> If you have to write that a lot, just use a selective import or
>>
>>     cimport cython as cy
>>
>> I get the impression that you are chasing non-problems here.
>>
>> Stefan
>
> I just thinking what could I help for improving Cython. Maybe I am
> thinking the wrong way since I'm not such familiar with the Cython
> design concept yet. However, no one intend to chase non-problems.

And thank you for your input. It takes a bit to get familiar with the  
Cython design and codebase, but I'm confident you will. New ideas and  
perspectives are very valuable, and I think the things you're bringing  
up shows you're thinking about the right kind of issues.

- Robert


From stefan_ml at behnel.de  Tue Apr  6 09:50:34 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 06 Apr 2010 09:50:34 +0200
Subject: [Cython] Compile-time decorator
In-Reply-To: <y2n3eb99a591004052352yc478c691j5382f6d385de321c@mail.gmail.com>
References: <q2m3eb99a591004030018haa683ad6id1395d1e6bb593b6@mail.gmail.com>
	<4BB71D6C.3070901@behnel.de>	<y2u3eb99a591004030515te0fc91fbvd8b527940833a58e@mail.gmail.com>	<4BB73393.1030903@behnel.de>	<F4A07E8F-A27E-4CDD-9342-E37D0DED3F1E@math.washington.edu>
	<g2x3eb99a591004050931w9af213f6oc676822842046422@mail.gmail.com>
	<51EB9181-F4C6-43CE-B747-22F9903AB2EA@math.washington.edu>
	<4BBA4748.6090201@behnel.de>	<p2m3eb99a591004052017i86a64f24t47d5bf0c92024f98@mail.gmail.com>	<4BBACDFA.70709@behnel.de>
	<y2n3eb99a591004052352yc478c691j5382f6d385de321c@mail.gmail.com>
Message-ID: <4BBAE7CA.2040902@behnel.de>

Haoyu Bai, 06.04.2010 08:52:
> I just thinking what could I help for improving Cython. Maybe I am
> thinking the wrong way since I'm not such familiar with the Cython
> design concept yet. However, no one intend to chase non-problems.

I know, sorry if I sounded a bit harsh. What you described certainly 
belongs to the issues that we keep dealing with while designing both the 
compiler and the language. It's clear that you can't be as familiar with 
the design as those who have been working on it for years - no-one expects 
you to be, and as Robert said, fresh ideas are always appreciated, and they 
often arise from taking an innocent look at existing structures.

There is a certain grey area between the adherence to Python language 
principles and what we allow or enforce in the Cython language. The issues 
you mentioned can be dealt with by restricting the Cython language, which 
simplifies them considerably. It's always a good design principle to keep 
things simple, and the solution I proposed ("imports from the 'cython' 
module are fixed") keeps things simple to understand for users and simple 
in the compiler. They also do not restrict any obvious use cases - why 
should you assign something else to a special name that you imported?

Stefan

From baihaoyu at gmail.com  Tue Apr  6 10:58:57 2010
From: baihaoyu at gmail.com (Haoyu Bai)
Date: Tue, 6 Apr 2010 16:58:57 +0800
Subject: [Cython] GSoC project for improving Python compatibility
In-Reply-To: <h2v3eb99a591004052358pe8f9b871p819b8b21ad813813@mail.gmail.com>
References: <3eb99a591003210431s12042237ubd1be5c73b24bfec@mail.gmail.com> 
	<4BA72159.6010007@student.matnat.uio.no>
	<3eb99a591003220939r3c5a94c3ibdd30826f885bc7e@mail.gmail.com> 
	<a8c4ff4c1003221207w74d19898uf2c8bff011b3c67b@mail.gmail.com> 
	<3eb99a591003270813x38a6d554u54419e693a345efc@mail.gmail.com> 
	<4A9A1D0E-EB65-4838-A59B-9E1737390DDC@math.washington.edu> 
	<h2i3eb99a591004012253k83f8539u3efda34280151c0c@mail.gmail.com> 
	<5171D334-8DBC-4911-9ED1-3C49438909C1@math.washington.edu> 
	<h2v3eb99a591004052358pe8f9b871p819b8b21ad813813@mail.gmail.com>
Message-ID: <t2i3eb99a591004060158zfac59fabl5477e56a256fca65@mail.gmail.com>

On Tue, Apr 6, 2010 at 2:58 PM, Haoyu Bai <baihaoyu at gmail.com> wrote:
> On Tue, Apr 6, 2010 at 1:56 PM, Robert Bradshaw
> <robertwb at math.washington.edu> wrote:
>> On Apr 1, 2010, at 10:53 PM, Haoyu Bai wrote:
>>
>>> On Wed, Mar 31, 2010 at 1:36 AM, Robert Bradshaw
>>> <robertwb at math.washington.edu> wrote:
>>>>
>>>> There's also Py3 scope binding rules for list comprehensions,
>>>> exception blocks, etc.
>>>>
>>>
>>> Thanks. I added these to my list.
>>
>> Do you have a proposal written up yet? It may be useful to put it up
>> on the Cython wiki before it's due for feedback (though, of course, I
>> can't promise anything).
>>
>> - Robert
>>
>>
>
> Actually I have posted it to GSoC website and sent a copy to Craig,
> I'll put it onto the wiki sooner.
>
>

Hi,

Now my proposal is on the Cython wiki for comment:

http://wiki.cython.org/HaoyuBai/GSoC2010

Again, thanks for all the worthy suggestions and patience for my ignorance. :)

-- 
Haoyu BAI
School of Computing,
National University of Singapore.

From robertwb at math.washington.edu  Tue Apr  6 11:15:40 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 6 Apr 2010 02:15:40 -0700
Subject: [Cython] GSoC project for improving Python compatibility
In-Reply-To: <t2i3eb99a591004060158zfac59fabl5477e56a256fca65@mail.gmail.com>
References: <3eb99a591003210431s12042237ubd1be5c73b24bfec@mail.gmail.com>
	<4BA72159.6010007@student.matnat.uio.no>
	<3eb99a591003220939r3c5a94c3ibdd30826f885bc7e@mail.gmail.com>
	<a8c4ff4c1003221207w74d19898uf2c8bff011b3c67b@mail.gmail.com>
	<3eb99a591003270813x38a6d554u54419e693a345efc@mail.gmail.com>
	<4A9A1D0E-EB65-4838-A59B-9E1737390DDC@math.washington.edu>
	<h2i3eb99a591004012253k83f8539u3efda34280151c0c@mail.gmail.com>
	<5171D334-8DBC-4911-9ED1-3C49438909C1@math.washington.edu>
	<h2v3eb99a591004052358pe8f9b871p819b8b21ad813813@mail.gmail.com>
	<t2i3eb99a591004060158zfac59fabl5477e56a256fca65@mail.gmail.com>
Message-ID: <4B8661A3-50E1-4E9B-B4EB-3A51CC0B8561@math.washington.edu>

> Hi,
>
> Now my proposal is on the Cython wiki for comment:
>
> http://wiki.cython.org/HaoyuBai/GSoC2010

Looking very good. Two things jumped out at me as I read this:

1) It may be worth swapping the annotation and decorator support. I am  
suggesting this because if any surprising and/or unforeseen  
difficulties arise it will probably be in the latter, and it'd be  
better to start attacking that sooner rather than right at the end.

2) I like how concrete your proposal is, but it would be good to have  
a specific milestone by the midterm review set ahead of time. This  
should include getting stuff merged (and your first third looks like  
it has lots of useful stuff that could get merged, as well as  
producing a lot of useful stuff while getting to know your way around  
the codebase).

- Robert


From stefan_ml at behnel.de  Tue Apr  6 11:53:52 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 06 Apr 2010 11:53:52 +0200
Subject: [Cython] GSoC project for improving Python compatibility
In-Reply-To: <t2i3eb99a591004060158zfac59fabl5477e56a256fca65@mail.gmail.com>
References: <3eb99a591003210431s12042237ubd1be5c73b24bfec@mail.gmail.com>
	<4BA72159.6010007@student.matnat.uio.no>	<3eb99a591003220939r3c5a94c3ibdd30826f885bc7e@mail.gmail.com>
	<a8c4ff4c1003221207w74d19898uf2c8bff011b3c67b@mail.gmail.com>
	<3eb99a591003270813x38a6d554u54419e693a345efc@mail.gmail.com>
	<4A9A1D0E-EB65-4838-A59B-9E1737390DDC@math.washington.edu>
	<h2i3eb99a591004012253k83f8539u3efda34280151c0c@mail.gmail.com>
	<5171D334-8DBC-4911-9ED1-3C49438909C1@math.washington.edu>
	<h2v3eb99a591004052358pe8f9b871p819b8b21ad813813@mail.gmail.com>
	<t2i3eb99a591004060158zfac59fabl5477e56a256fca65@mail.gmail.com>
Message-ID: <4BBB04B0.1070807@behnel.de>

Haoyu Bai, 06.04.2010 10:58:
> Now my proposal is on the Cython wiki for comment:
>
> http://wiki.cython.org/HaoyuBai/GSoC2010

Thanks, that looks really good!

My comments:

In general, the tickets you propose to fix (mostly in part 1 of your 
schedule) touch most of the Cython compiler, so there is a lot you will 
(have to) learn and look into. Don't make your schedule too tight. I really 
like the patches that you have written so far, so I'm confident that you 
can achieve what you have proposed.

#465 may be a little more work than you expect, especially when you are new 
to the Cython code base. It may require a new scope class for 
comprehensions, and the comprehension must then get analysed within that 
scope. This kind of stuff can be pretty tricky to get right. The exception 
scoping may end up having a similar solution.

Support for __annotations__ is certainly not worth a week. All you have to 
do is build a DictNode that maps parameter names to their annotation value, 
let it generate its code during module setup, and pass the result to 
PyFunction_SetAnnotations(). Given how much you will have learned from the 
other tickets before that, this will likely not take you more than a day to 
implement. I'd expect that the complete annotation support, including the 
extraction of type annotations (and writing tests for them), can be done in 
less than one week. Remember that Cython /has/ static typing, so this is 
just adding a new syntax for it. Almost everything you need is there 
already, you just have to stick it together. The only bit that isn't there 
already is return types for Python (def) functions - we will need to 
discuss what impact this should have on the generated code (function 
internal/external type checks? different calling convention like auto-cpdef?).

So, basically, part 2 of your proposed schedule will give you time to 
finish the first part, which I think will take longer. Part 3 may also take 
less time (again, the semantics is implemented, so it's mostly syntactic 
changes).

I certainly support your proposal.

Stefan

From craigcitro at gmail.com  Tue Apr  6 23:29:53 2010
From: craigcitro at gmail.com (Craig Citro)
Date: Tue, 6 Apr 2010 14:29:53 -0700
Subject: [Cython] GSoC project for improving Python compatibility
In-Reply-To: <t2i3eb99a591004060158zfac59fabl5477e56a256fca65@mail.gmail.com>
References: <3eb99a591003210431s12042237ubd1be5c73b24bfec@mail.gmail.com> 
	<4BA72159.6010007@student.matnat.uio.no>
	<3eb99a591003220939r3c5a94c3ibdd30826f885bc7e@mail.gmail.com> 
	<a8c4ff4c1003221207w74d19898uf2c8bff011b3c67b@mail.gmail.com> 
	<3eb99a591003270813x38a6d554u54419e693a345efc@mail.gmail.com> 
	<4A9A1D0E-EB65-4838-A59B-9E1737390DDC@math.washington.edu> 
	<h2i3eb99a591004012253k83f8539u3efda34280151c0c@mail.gmail.com> 
	<5171D334-8DBC-4911-9ED1-3C49438909C1@math.washington.edu> 
	<h2v3eb99a591004052358pe8f9b871p819b8b21ad813813@mail.gmail.com> 
	<t2i3eb99a591004060158zfac59fabl5477e56a256fca65@mail.gmail.com>
Message-ID: <h2wa8c4ff4c1004061429v2a2ca64dvdc0391d5b4decac2@mail.gmail.com>

Hi Haoyu,

> Now my proposal is on the Cython wiki for comment:
>
> http://wiki.cython.org/HaoyuBai/GSoC2010
>

Sorry about the delay -- I'm finally getting some cycles to sit down
and work on this.

First, minor edits -- I went ahead and edited the page on the wiki and
made minor grammatical/spelling/formatting changes. I also left a few
notes where I thought you could use more explanation and/or a link --
just search for "-cc" on the page. (We could also renumber the
references, but that's not really a big deal.)

I had some thoughts along the same lines as Robert and Stefan for the
first part of the project -- I think that there will probably be a
little extra time needed at the beginning acclimating to the codebase.
(Though maybe the fact that you've already got two other patches up
and waiting for review suggests otherwise? ...) Also taking into
account Robert and Stefan's advice on ordering of events and how much
time things will take, I'd recommend slightly restructuring things, as
follows:

 * Weeks 1-3: bug fixes as-is, making sure to get each passing tests
and posted on trac before moving on to the next one
 * Week 4: Work on pure Python mode documentation, start pinging
people for code reviews on patches remaining from weeks 1-3
 * Week 5: respond to any comments on patches, and make getting as
many patches as possible merged the goal for midterm review.
 * Week 6-7: decorator support (note that some of the documentation
work was moved up)
 * Week 8-9: annotation support (I'm taking into account Stefan's
comment that this is not as deep as you might expect at first)
 * Week 10: cleanup, documentation, tests, etc.

Basically, I want to really focus on getting a bunch of patches and
documentation knocked out in the first half, for two reasons: first,
as your mentor, I can be reviewing your patches and responding as a
way of monitoring your progress, and second, it means that you'll get
more familiarity with the current code and documentation before
jumping into the slightly-larger-scale decorator and annotation
projects. It also means that we have a little extra padding at the
beginning for things that take extra time (like Stefan predicted for
#465).

Does that seems sensible? Feel free to tell me if you think I'm crazy,
or if you'd rather do it a different way.

I'm going to go work on finishing up whatever else I have to do for
mentor registration, but I should be able to be fairly responsive by
email for the rest of the week ... and I'm getting excited about this
project!

-cc

From robertwb at math.washington.edu  Wed Apr  7 19:46:47 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Wed, 7 Apr 2010 10:46:47 -0700
Subject: [Cython] GSoC project for improving Python compatibility
In-Reply-To: <t2i3eb99a591004060158zfac59fabl5477e56a256fca65@mail.gmail.com>
References: <3eb99a591003210431s12042237ubd1be5c73b24bfec@mail.gmail.com>
	<4BA72159.6010007@student.matnat.uio.no>
	<3eb99a591003220939r3c5a94c3ibdd30826f885bc7e@mail.gmail.com>
	<a8c4ff4c1003221207w74d19898uf2c8bff011b3c67b@mail.gmail.com>
	<3eb99a591003270813x38a6d554u54419e693a345efc@mail.gmail.com>
	<4A9A1D0E-EB65-4838-A59B-9E1737390DDC@math.washington.edu>
	<h2i3eb99a591004012253k83f8539u3efda34280151c0c@mail.gmail.com>
	<5171D334-8DBC-4911-9ED1-3C49438909C1@math.washington.edu>
	<h2v3eb99a591004052358pe8f9b871p819b8b21ad813813@mail.gmail.com>
	<t2i3eb99a591004060158zfac59fabl5477e56a256fca65@mail.gmail.com>
Message-ID: <7B210ECE-3AC8-487A-9B1A-5B3E5E57A177@math.washington.edu>

On Apr 6, 2010, at 1:58 AM, Haoyu Bai wrote:

> On Tue, Apr 6, 2010 at 2:58 PM, Haoyu Bai <baihaoyu at gmail.com> wrote:
>> On Tue, Apr 6, 2010 at 1:56 PM, Robert Bradshaw
>> <robertwb at math.washington.edu> wrote:
>>> On Apr 1, 2010, at 10:53 PM, Haoyu Bai wrote:
>>>
>>>> On Wed, Mar 31, 2010 at 1:36 AM, Robert Bradshaw
>>>> <robertwb at math.washington.edu> wrote:
>>>>>
>>>>> There's also Py3 scope binding rules for list comprehensions,
>>>>> exception blocks, etc.
>>>>>
>>>>
>>>> Thanks. I added these to my list.
>>>
>>> Do you have a proposal written up yet? It may be useful to put it up
>>> on the Cython wiki before it's due for feedback (though, of  
>>> course, I
>>> can't promise anything).
>>>
>>> - Robert
>>>
>>>
>>
>> Actually I have posted it to GSoC website and sent a copy to Craig,
>> I'll put it onto the wiki sooner.
>>
>>
>
> Hi,
>
> Now my proposal is on the Cython wiki for comment:
>
> http://wiki.cython.org/HaoyuBai/GSoC2010
>
> Again, thanks for all the worthy suggestions and patience for my  
> ignorance. :)

Another comment, in terms of documenting the new decorators, it would  
be great if a quality presentation of pure mode finally made it to http://docs.cython.org/src/tutorial/pure.html 
  now that it'll be fully functional.

- Robert



From baihaoyu at gmail.com  Thu Apr  8 08:01:54 2010
From: baihaoyu at gmail.com (Haoyu Bai)
Date: Thu, 8 Apr 2010 14:01:54 +0800
Subject: [Cython] GSoC project for improving Python compatibility
In-Reply-To: <h2wa8c4ff4c1004061429v2a2ca64dvdc0391d5b4decac2@mail.gmail.com>
References: <3eb99a591003210431s12042237ubd1be5c73b24bfec@mail.gmail.com> 
	<3eb99a591003220939r3c5a94c3ibdd30826f885bc7e@mail.gmail.com> 
	<a8c4ff4c1003221207w74d19898uf2c8bff011b3c67b@mail.gmail.com> 
	<3eb99a591003270813x38a6d554u54419e693a345efc@mail.gmail.com> 
	<4A9A1D0E-EB65-4838-A59B-9E1737390DDC@math.washington.edu> 
	<h2i3eb99a591004012253k83f8539u3efda34280151c0c@mail.gmail.com> 
	<5171D334-8DBC-4911-9ED1-3C49438909C1@math.washington.edu> 
	<h2v3eb99a591004052358pe8f9b871p819b8b21ad813813@mail.gmail.com> 
	<t2i3eb99a591004060158zfac59fabl5477e56a256fca65@mail.gmail.com> 
	<h2wa8c4ff4c1004061429v2a2ca64dvdc0391d5b4decac2@mail.gmail.com>
Message-ID: <l2v3eb99a591004072301kb02172fem539146a2f30a6fd1@mail.gmail.com>

On Wed, Apr 7, 2010 at 5:29 AM, Craig Citro <craigcitro at gmail.com> wrote:
> Hi Haoyu,
>
>> Now my proposal is on the Cython wiki for comment:
>>
>> http://wiki.cython.org/HaoyuBai/GSoC2010
>>
>
> Sorry about the delay -- I'm finally getting some cycles to sit down
> and work on this.
>
> First, minor edits -- I went ahead and edited the page on the wiki and
> made minor grammatical/spelling/formatting changes. I also left a few
> notes where I thought you could use more explanation and/or a link --
> just search for "-cc" on the page. (We could also renumber the
> references, but that's not really a big deal.)
>
> I had some thoughts along the same lines as Robert and Stefan for the
> first part of the project -- I think that there will probably be a
> little extra time needed at the beginning acclimating to the codebase.
> (Though maybe the fact that you've already got two other patches up
> and waiting for review suggests otherwise? ...) Also taking into
> account Robert and Stefan's advice on ordering of events and how much
> time things will take, I'd recommend slightly restructuring things, as
> follows:
>
> ?* Weeks 1-3: bug fixes as-is, making sure to get each passing tests
> and posted on trac before moving on to the next one
> ?* Week 4: Work on pure Python mode documentation, start pinging
> people for code reviews on patches remaining from weeks 1-3
> ?* Week 5: respond to any comments on patches, and make getting as
> many patches as possible merged the goal for midterm review.
> ?* Week 6-7: decorator support (note that some of the documentation
> work was moved up)
> ?* Week 8-9: annotation support (I'm taking into account Stefan's
> comment that this is not as deep as you might expect at first)
> ?* Week 10: cleanup, documentation, tests, etc.
>
> Basically, I want to really focus on getting a bunch of patches and
> documentation knocked out in the first half, for two reasons: first,
> as your mentor, I can be reviewing your patches and responding as a
> way of monitoring your progress, and second, it means that you'll get
> more familiarity with the current code and documentation before
> jumping into the slightly-larger-scale decorator and annotation
> projects. It also means that we have a little extra padding at the
> beginning for things that take extra time (like Stefan predicted for
> #465).
>
> Does that seems sensible? Feel free to tell me if you think I'm crazy,
> or if you'd rather do it a different way.
>
> I'm going to go work on finishing up whatever else I have to do for
> mentor registration, but I should be able to be fairly responsive by
> email for the rest of the week ... and I'm getting excited about this
> project!
>
> -cc

Hi,

Thanks for all the comments! They are all very reasonable and I've
updated the proposal on the wiki according to these. I have adjusted
some unclear paragraphs and rearranged the schedule to follow the
concern of workload and milestone for mid-term review.

Hopefully now the proposal is clearer and better shaped. Thanks for
your help! :)

-- 
Haoyu BAI
School of Computing,
National University of Singapore.

From baihaoyu at gmail.com  Thu Apr  8 08:08:13 2010
From: baihaoyu at gmail.com (Haoyu Bai)
Date: Thu, 8 Apr 2010 14:08:13 +0800
Subject: [Cython] GSoC project for improving Python compatibility
In-Reply-To: <7B210ECE-3AC8-487A-9B1A-5B3E5E57A177@math.washington.edu>
References: <3eb99a591003210431s12042237ubd1be5c73b24bfec@mail.gmail.com> 
	<3eb99a591003220939r3c5a94c3ibdd30826f885bc7e@mail.gmail.com> 
	<a8c4ff4c1003221207w74d19898uf2c8bff011b3c67b@mail.gmail.com> 
	<3eb99a591003270813x38a6d554u54419e693a345efc@mail.gmail.com> 
	<4A9A1D0E-EB65-4838-A59B-9E1737390DDC@math.washington.edu> 
	<h2i3eb99a591004012253k83f8539u3efda34280151c0c@mail.gmail.com> 
	<5171D334-8DBC-4911-9ED1-3C49438909C1@math.washington.edu> 
	<h2v3eb99a591004052358pe8f9b871p819b8b21ad813813@mail.gmail.com> 
	<t2i3eb99a591004060158zfac59fabl5477e56a256fca65@mail.gmail.com> 
	<7B210ECE-3AC8-487A-9B1A-5B3E5E57A177@math.washington.edu>
Message-ID: <n2t3eb99a591004072308i842879a1q8095d064ae37419b@mail.gmail.com>

On Thu, Apr 8, 2010 at 1:46 AM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Apr 6, 2010, at 1:58 AM, Haoyu Bai wrote:
>
>> On Tue, Apr 6, 2010 at 2:58 PM, Haoyu Bai <baihaoyu at gmail.com> wrote:
>>> On Tue, Apr 6, 2010 at 1:56 PM, Robert Bradshaw
>>> <robertwb at math.washington.edu> wrote:
>>>> On Apr 1, 2010, at 10:53 PM, Haoyu Bai wrote:
>>>>
>>>>> On Wed, Mar 31, 2010 at 1:36 AM, Robert Bradshaw
>>>>> <robertwb at math.washington.edu> wrote:
>>>>>>
>>>>>> There's also Py3 scope binding rules for list comprehensions,
>>>>>> exception blocks, etc.
>>>>>>
>>>>>
>>>>> Thanks. I added these to my list.
>>>>
>>>> Do you have a proposal written up yet? It may be useful to put it up
>>>> on the Cython wiki before it's due for feedback (though, of
>>>> course, I
>>>> can't promise anything).
>>>>
>>>> - Robert
>>>>
>>>>
>>>
>>> Actually I have posted it to GSoC website and sent a copy to Craig,
>>> I'll put it onto the wiki sooner.
>>>
>>>
>>
>> Hi,
>>
>> Now my proposal is on the Cython wiki for comment:
>>
>> http://wiki.cython.org/HaoyuBai/GSoC2010
>>
>> Again, thanks for all the worthy suggestions and patience for my
>> ignorance. :)
>
> Another comment, in terms of documenting the new decorators, it would
> be great if a quality presentation of pure mode finally made it to http://docs.cython.org/src/tutorial/pure.html
> ?now that it'll be fully functional.
>
> - Robert
>
>

Yes this is what I intend to do. Besides this tutorial, do we need to
have a separate chapter in the reference guide, or just mention the
pure Py syntax as an alternative in the corresponding chapter of the
reference guide is OK?


-- 
Haoyu BAI
School of Computing,
National University of Singapore.

From robertwb at math.washington.edu  Thu Apr  8 09:01:28 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 8 Apr 2010 00:01:28 -0700
Subject: [Cython] GSoC project for improving Python compatibility
In-Reply-To: <n2t3eb99a591004072308i842879a1q8095d064ae37419b@mail.gmail.com>
References: <3eb99a591003210431s12042237ubd1be5c73b24bfec@mail.gmail.com>
	<3eb99a591003220939r3c5a94c3ibdd30826f885bc7e@mail.gmail.com>
	<a8c4ff4c1003221207w74d19898uf2c8bff011b3c67b@mail.gmail.com>
	<3eb99a591003270813x38a6d554u54419e693a345efc@mail.gmail.com>
	<4A9A1D0E-EB65-4838-A59B-9E1737390DDC@math.washington.edu>
	<h2i3eb99a591004012253k83f8539u3efda34280151c0c@mail.gmail.com>
	<5171D334-8DBC-4911-9ED1-3C49438909C1@math.washington.edu>
	<h2v3eb99a591004052358pe8f9b871p819b8b21ad813813@mail.gmail.com>
	<t2i3eb99a591004060158zfac59fabl5477e56a256fca65@mail.gmail.com>
	<7B210ECE-3AC8-487A-9B1A-5B3E5E57A177@math.washington.edu>
	<n2t3eb99a591004072308i842879a1q8095d064ae37419b@mail.gmail.com>
Message-ID: <E620CE38-980D-4CFD-A178-1307709B175D@math.washington.edu>

On Apr 7, 2010, at 11:08 PM, Haoyu Bai wrote:

> On Thu, Apr 8, 2010 at 1:46 AM, Robert Bradshaw
> <robertwb at math.washington.edu> wrote:
>> On Apr 6, 2010, at 1:58 AM, Haoyu Bai wrote:
>>
>>> On Tue, Apr 6, 2010 at 2:58 PM, Haoyu Bai <baihaoyu at gmail.com>  
>>> wrote:
>>>> On Tue, Apr 6, 2010 at 1:56 PM, Robert Bradshaw
>>>> <robertwb at math.washington.edu> wrote:
>>>>> On Apr 1, 2010, at 10:53 PM, Haoyu Bai wrote:
>>>>>
>>>>>> On Wed, Mar 31, 2010 at 1:36 AM, Robert Bradshaw
>>>>>> <robertwb at math.washington.edu> wrote:
>>>>>>>
>>>>>>> There's also Py3 scope binding rules for list comprehensions,
>>>>>>> exception blocks, etc.
>>>>>>>
>>>>>>
>>>>>> Thanks. I added these to my list.
>>>>>
>>>>> Do you have a proposal written up yet? It may be useful to put  
>>>>> it up
>>>>> on the Cython wiki before it's due for feedback (though, of
>>>>> course, I
>>>>> can't promise anything).
>>>>>
>>>>> - Robert
>>>>>
>>>>>
>>>>
>>>> Actually I have posted it to GSoC website and sent a copy to Craig,
>>>> I'll put it onto the wiki sooner.
>>>>
>>>>
>>>
>>> Hi,
>>>
>>> Now my proposal is on the Cython wiki for comment:
>>>
>>> http://wiki.cython.org/HaoyuBai/GSoC2010
>>>
>>> Again, thanks for all the worthy suggestions and patience for my
>>> ignorance. :)
>>
>> Another comment, in terms of documenting the new decorators, it would
>> be great if a quality presentation of pure mode finally made it to http://docs.cython.org/src/tutorial/pure.html
>>  now that it'll be fully functional.
>>
>> - Robert
>>
>>
>
> Yes this is what I intend to do. Besides this tutorial, do we need to
> have a separate chapter in the reference guide, or just mention the
> pure Py syntax as an alternative in the corresponding chapter of the
> reference guide is OK?

I think having the reference guide point to the tutorial is just fine.

- Robert



From greg.ewing at canterbury.ac.nz  Thu Apr  8 13:37:30 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 08 Apr 2010 23:37:30 +1200
Subject: [Cython] clasmethod still broken
In-Reply-To: <4B1A7E5A.20602@behnel.de>
References: <e7ba66e40912042003j301aaa3du3fd8080f5f2eac46@mail.gmail.com>
	<4B1A0A41.3010103@behnel.de> <4B1A4F0A.7020804@behnel.de>
	<4B1A7E5A.20602@behnel.de>
Message-ID: <4BBDBFFA.90203@canterbury.ac.nz>

Stefan Behnel wrote:

>>    cdef class cclass:
>>         def test(*args):
>>             pass
 >
> Pyrex suffers from the same problem:
> 
> $ python ./bin/pyrexc cdef_methods_T462.pyx
> [...]
> .../cdef_methods_T462.pyx:26:4: Method test_args has wrong number of
> arguments (0 declared, 1 or more expected)

I've just been looking into this. The problem is that at the C
level, "self" is passed into methods as a separate argument, so
that you need to declare the method as

          def test(self, *args):
             ...

I'm inclined to not bother trying to fix this in Pyrex, because
it would take extra effort to merge "self" into the *args, and
it seems like a rather rare thing to want to do.

-- 
Greg


From nepenthesdev at gmail.com  Thu Apr  8 15:51:05 2010
From: nepenthesdev at gmail.com (Nepenthes Development Team)
Date: Thu, 8 Apr 2010 15:51:05 +0200
Subject: [Cython] cython 0.12.1 unexpected behaviour
Message-ID: <s2v60d7a5221004080651z8ed44acch3a96375a1cc7dd9d@mail.gmail.com>

Hi,

while I use cython in a very specific way, creating bindings for
software which embeds python(3), I'm glad it works that great.
I've been using cython 0.11.(4?) and 0.12 before, both worked fine,
but 0.12.1 breaks, cython aborts.

The error message provided is below.

If you want to reproduce, simply try to compile "dionaea", some low
interaction honeypot software, the python bindings is meant to use
cython (http://dionaea.carnivore.it/).

There is some thing in the binding which is not standard,
I need to be able to log exceptions myself, as logging exception
within cython does not work, I export the functions to the c-part of
the binding, and call them over there, so I can log the exceptions
properly.
But this worked fine, and I do not expect cython to fail here, as the
function exporting it is a totally valid operation.

The error:

cython -v binding.pyx
Compiling ~/git.carnivore.it/dionaea/modules/python/binding.pyx
Traceback (most recent call last):
  File "/usr/local/bin/cython", line 8, in <module>
    main(command_line = 1)
  File "/usr/local/lib/python2.6/dist-packages/Cython/Compiler/Main.py",
line 743, in main
    result = compile(sources, options)
  File "/usr/local/lib/python2.6/dist-packages/Cython/Compiler/Main.py",
line 718, in compile
    return compile_multiple(source, options)
  File "/usr/local/lib/python2.6/dist-packages/Cython/Compiler/Main.py",
line 690, in compile_multiple
    result = run_pipeline(source, options)
  File "/usr/local/lib/python2.6/dist-packages/Cython/Compiler/Main.py",
line 561, in run_pipeline
    err, enddata = context.run_pipeline(pipeline, source)
  File "/usr/local/lib/python2.6/dist-packages/Cython/Compiler/Main.py",
line 221, in run_pipeline
    data = phase(data)
  File "Visitor.py", line 326, in
Cython.Compiler.Visitor.EnvTransform.__call__
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:5908)
  File "Visitor.py", line 275, in
Cython.Compiler.Visitor.CythonTransform.__call__
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4932)
  File "Visitor.py", line 258, in
Cython.Compiler.Visitor.VisitorTransform.__call__
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4692)
  File "Visitor.py", line 28, in
Cython.Compiler.Visitor.BasicVisitor.visit
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:1150)
  File "Visitor.py", line 253, in
Cython.Compiler.Visitor.VisitorTransform.recurse_to_children
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4620)
  File "Visitor.py", line 254, in
Cython.Compiler.Visitor.VisitorTransform.recurse_to_children
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4576)
  File "Visitor.py", line 237, in
Cython.Compiler.Visitor.VisitorTransform.visitchildren
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4255)
  File "Visitor.py", line 210, in
Cython.Compiler.Visitor.TreeVisitor._visitchildren
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4019)
  File "Visitor.py", line 183, in
Cython.Compiler.Visitor.TreeVisitor.visitchild
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3500)
  File "Visitor.py", line 178, in
Cython.Compiler.Visitor.TreeVisitor.visitchild
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3399)
  File "Visitor.py", line 28, in
Cython.Compiler.Visitor.BasicVisitor.visit
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:1150)
  File "Visitor.py", line 253, in
Cython.Compiler.Visitor.VisitorTransform.recurse_to_children
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4620)
  File "Visitor.py", line 254, in
Cython.Compiler.Visitor.VisitorTransform.recurse_to_children
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4576)
  File "Visitor.py", line 237, in
Cython.Compiler.Visitor.VisitorTransform.visitchildren
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4255)
  File "Visitor.py", line 208, in
Cython.Compiler.Visitor.TreeVisitor._visitchildren
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3997)
  File "Visitor.py", line 183, in
Cython.Compiler.Visitor.TreeVisitor.visitchild
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3500)
  File "Visitor.py", line 178, in
Cython.Compiler.Visitor.TreeVisitor.visitchild
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3399)
  File "Visitor.py", line 28, in
Cython.Compiler.Visitor.BasicVisitor.visit
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:1150)
  File "Visitor.py", line 253, in
Cython.Compiler.Visitor.VisitorTransform.recurse_to_children
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4620)
  File "Visitor.py", line 254, in
Cython.Compiler.Visitor.VisitorTransform.recurse_to_children
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4576)
  File "Visitor.py", line 237, in
Cython.Compiler.Visitor.VisitorTransform.visitchildren
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4255)
  File "Visitor.py", line 208, in
Cython.Compiler.Visitor.TreeVisitor._visitchildren
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3997)
  File "Visitor.py", line 183, in
Cython.Compiler.Visitor.TreeVisitor.visitchild
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3500)
  File "Visitor.py", line 178, in
Cython.Compiler.Visitor.TreeVisitor.visitchild
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3399)
  File "Visitor.py", line 28, in
Cython.Compiler.Visitor.BasicVisitor.visit
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:1150)
  File "Visitor.py", line 253, in
Cython.Compiler.Visitor.VisitorTransform.recurse_to_children
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4620)
  File "Visitor.py", line 254, in
Cython.Compiler.Visitor.VisitorTransform.recurse_to_children
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4576)
  File "Visitor.py", line 237, in
Cython.Compiler.Visitor.VisitorTransform.visitchildren
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4255)
  File "Visitor.py", line 210, in
Cython.Compiler.Visitor.TreeVisitor._visitchildren
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4019)
  File "Visitor.py", line 183, in
Cython.Compiler.Visitor.TreeVisitor.visitchild
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3500)
  File "Visitor.py", line 178, in
Cython.Compiler.Visitor.TreeVisitor.visitchild
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3399)
  File "Visitor.py", line 28, in
Cython.Compiler.Visitor.BasicVisitor.visit
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:1150)
  File "Visitor.py", line 253, in
Cython.Compiler.Visitor.VisitorTransform.recurse_to_children
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4620)
  File "Visitor.py", line 254, in
Cython.Compiler.Visitor.VisitorTransform.recurse_to_children
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4576)
  File "Visitor.py", line 237, in
Cython.Compiler.Visitor.VisitorTransform.visitchildren
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4255)
  File "Visitor.py", line 208, in
Cython.Compiler.Visitor.TreeVisitor._visitchildren
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3997)
  File "Visitor.py", line 183, in
Cython.Compiler.Visitor.TreeVisitor.visitchild
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3500)
  File "Visitor.py", line 178, in
Cython.Compiler.Visitor.TreeVisitor.visitchild
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3399)
  File "Visitor.py", line 28, in
Cython.Compiler.Visitor.BasicVisitor.visit
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:1150)
  File "Visitor.py", line 253, in
Cython.Compiler.Visitor.VisitorTransform.recurse_to_children
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4620)
  File "Visitor.py", line 254, in
Cython.Compiler.Visitor.VisitorTransform.recurse_to_children
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4576)
  File "Visitor.py", line 237, in
Cython.Compiler.Visitor.VisitorTransform.visitchildren
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4255)
  File "Visitor.py", line 210, in
Cython.Compiler.Visitor.TreeVisitor._visitchildren
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4019)
  File "Visitor.py", line 183, in
Cython.Compiler.Visitor.TreeVisitor.visitchild
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3500)
  File "Visitor.py", line 178, in
Cython.Compiler.Visitor.TreeVisitor.visitchild
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3399)
  File "Visitor.py", line 28, in
Cython.Compiler.Visitor.BasicVisitor.visit
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:1150)
  File "Visitor.py", line 253, in
Cython.Compiler.Visitor.VisitorTransform.recurse_to_children
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4620)
  File "Visitor.py", line 254, in
Cython.Compiler.Visitor.VisitorTransform.recurse_to_children
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4576)
  File "Visitor.py", line 237, in
Cython.Compiler.Visitor.VisitorTransform.visitchildren
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4255)
  File "Visitor.py", line 208, in
Cython.Compiler.Visitor.TreeVisitor._visitchildren
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3997)
  File "Visitor.py", line 183, in
Cython.Compiler.Visitor.TreeVisitor.visitchild
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3500)
  File "Visitor.py", line 178, in
Cython.Compiler.Visitor.TreeVisitor.visitchild
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3399)
  File "Visitor.py", line 28, in
Cython.Compiler.Visitor.BasicVisitor.visit
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:1150)
  File "Visitor.py", line 330, in
Cython.Compiler.Visitor.EnvTransform.visit_FuncDefNode
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:6021)
  File "Visitor.py", line 236, in
Cython.Compiler.Visitor.VisitorTransform.visitchildren
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4512)
  File "Visitor.py", line 237, in
Cython.Compiler.Visitor.VisitorTransform.visitchildren
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4255)
  File "Visitor.py", line 210, in
Cython.Compiler.Visitor.TreeVisitor._visitchildren
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4019)
  File "Visitor.py", line 183, in
Cython.Compiler.Visitor.TreeVisitor.visitchild
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3500)
  File "Visitor.py", line 178, in
Cython.Compiler.Visitor.TreeVisitor.visitchild
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3399)
  File "Visitor.py", line 28, in
Cython.Compiler.Visitor.BasicVisitor.visit
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:1150)
  File "Visitor.py", line 253, in
Cython.Compiler.Visitor.VisitorTransform.recurse_to_children
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4620)
  File "Visitor.py", line 254, in
Cython.Compiler.Visitor.VisitorTransform.recurse_to_children
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4576)
  File "Visitor.py", line 237, in
Cython.Compiler.Visitor.VisitorTransform.visitchildren
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4255)
  File "Visitor.py", line 208, in
Cython.Compiler.Visitor.TreeVisitor._visitchildren
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3997)
  File "Visitor.py", line 183, in
Cython.Compiler.Visitor.TreeVisitor.visitchild
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3500)
  File "Visitor.py", line 178, in
Cython.Compiler.Visitor.TreeVisitor.visitchild
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3399)
  File "Visitor.py", line 28, in
Cython.Compiler.Visitor.BasicVisitor.visit
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:1150)
  File "Visitor.py", line 253, in
Cython.Compiler.Visitor.VisitorTransform.recurse_to_children
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4620)
  File "Visitor.py", line 254, in
Cython.Compiler.Visitor.VisitorTransform.recurse_to_children
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4576)
  File "Visitor.py", line 237, in
Cython.Compiler.Visitor.VisitorTransform.visitchildren
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4255)
  File "Visitor.py", line 210, in
Cython.Compiler.Visitor.TreeVisitor._visitchildren
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:4019)
  File "Visitor.py", line 183, in
Cython.Compiler.Visitor.TreeVisitor.visitchild
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3500)
  File "Visitor.py", line 178, in
Cython.Compiler.Visitor.TreeVisitor.visitchild
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:3399)
  File "Visitor.py", line 28, in
Cython.Compiler.Visitor.BasicVisitor.visit
(/tmp/Cython-0.12.1/Cython/Compiler/Visitor.c:1150)
  File "/usr/local/lib/python2.6/dist-packages/Cython/Compiler/Optimize.py",
line 910, in visit_SimpleCallNode
    node, function, args)
  File "/usr/local/lib/python2.6/dist-packages/Cython/Compiler/Optimize.py",
line 1033, in _dispatch_to_handler
    return method_handler(node, arg_list, is_unbound_method)
  File "/usr/local/lib/python2.6/dist-packages/Cython/Compiler/Optimize.py",
line 1481, in _handle_simple_method_bytes_decode
    parameters = self._unpack_encoding_and_error_mode(node.pos, args)
  File "/usr/local/lib/python2.6/dist-packages/Cython/Compiler/Optimize.py",
line 1524, in _unpack_encoding_and_error_mode
    encoding_node = args[1]
IndexError: list index out of range


MfG
Markus

From stefan_ml at behnel.de  Thu Apr  8 18:15:44 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 08 Apr 2010 18:15:44 +0200
Subject: [Cython] cython 0.12.1 unexpected behaviour
In-Reply-To: <s2v60d7a5221004080651z8ed44acch3a96375a1cc7dd9d@mail.gmail.com>
References: <s2v60d7a5221004080651z8ed44acch3a96375a1cc7dd9d@mail.gmail.com>
Message-ID: <4BBE0130.60608@behnel.de>

Nepenthes Development Team, 08.04.2010 15:51:
>    File "/usr/local/lib/python2.6/dist-packages/Cython/Compiler/Optimize.py",
> line 910, in visit_SimpleCallNode
>      node, function, args)
>    File "/usr/local/lib/python2.6/dist-packages/Cython/Compiler/Optimize.py",
> line 1033, in _dispatch_to_handler
>      return method_handler(node, arg_list, is_unbound_method)
>    File "/usr/local/lib/python2.6/dist-packages/Cython/Compiler/Optimize.py",
> line 1481, in _handle_simple_method_bytes_decode
>      parameters = self._unpack_encoding_and_error_mode(node.pos, args)
>    File "/usr/local/lib/python2.6/dist-packages/Cython/Compiler/Optimize.py",
> line 1524, in _unpack_encoding_and_error_mode
>      encoding_node = args[1]
> IndexError: list index out of range

Hmm, my bad, I guess. I assume you do "some_byte_string.decode()" 
somewhere, without arguments? The behaviour of that is platform dependant, 
you should use an explicit encoding instead

That being said, the crash is a bug in Cython. Could you try the attached 
patch?

Stefan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bytes_decode_fix.patch
Type: text/x-diff
Size: 1897 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20100408/35f163c1/attachment.bin 

From nepenthesdev at gmail.com  Thu Apr  8 18:32:57 2010
From: nepenthesdev at gmail.com (Nepenthes Development Team)
Date: Thu, 8 Apr 2010 18:32:57 +0200
Subject: [Cython] cython 0.12.1 unexpected behaviour
In-Reply-To: <4BBE0130.60608@behnel.de>
References: <s2v60d7a5221004080651z8ed44acch3a96375a1cc7dd9d@mail.gmail.com>
	<4BBE0130.60608@behnel.de>
Message-ID: <p2k60d7a5221004080932nd166735ai7349b9fc535c2bb1@mail.gmail.com>

Hi,

On Thu, Apr 8, 2010 at 6:15 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Hmm, my bad, I guess. I assume you do "some_byte_string.decode()" somewhere,
> without arguments? The behaviour of that is platform dependant, you should
> use an explicit encoding instead

Yes, I do, but I took your advice.

> That being said, the crash is a bug in Cython. Could you try the attached
> patch?

cythoning works with the patch, compiling the c code failed for const
correctness.
Taking the faq into account, it works now.

Thanks.

Markus

From dalcinl at gmail.com  Sat Apr 10 00:11:00 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Fri, 9 Apr 2010 19:11:00 -0300
Subject: [Cython] [PATCH] basic C types in pure mode.
Message-ID: <h2ze7ba66e41004091511rdd16c3f4o1abdf5d5dd5e5408@mail.gmail.com>

Is this patch fine? I mean, look at the new testcase in pure.pyx...
Should we support all basic C types in cython.declare()?

BTW, I would really like to remove that "modifiers_and_name_to_type" dictionary


-- 
Lisandro Dalcin
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pure.diff
Type: text/x-patch
Size: 6692 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20100409/b4a45ac2/attachment.bin 

From stefan_ml at behnel.de  Sun Apr 11 19:00:42 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 11 Apr 2010 19:00:42 +0200
Subject: [Cython] [PATCH] basic C types in pure mode.
In-Reply-To: <h2ze7ba66e41004091511rdd16c3f4o1abdf5d5dd5e5408@mail.gmail.com>
References: <h2ze7ba66e41004091511rdd16c3f4o1abdf5d5dd5e5408@mail.gmail.com>
Message-ID: <4BC2003A.9020906@behnel.de>

Lisandro Dalcin, 10.04.2010 00:11:
> Is this patch fine?

I think the reason why no-one replied so far is that this kind of patch is 
basically impossible to validate on eye-sight. The changes in 
"modifiers_and_name_to_type" seem to be mostly whitespace noise - it would 
be easier to read the patch without that.


 > I mean, look at the new testcase in pure.pyx...
> Should we support all basic C types in cython.declare()?

Certainly all C types that can be mapped to a Python type in some way (i.e. 
those that make sense in pure more).


> BTW, I would really like to remove that "modifiers_and_name_to_type" dictionary

How would you do that?

Stefan

From dalcinl at gmail.com  Mon Apr 12 01:50:33 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Sun, 11 Apr 2010 20:50:33 -0300
Subject: [Cython] int complex failure in C++ with MSVC
Message-ID: <q2oe7ba66e41004111650w35b057c7x26018d385d244207@mail.gmail.com>

See this...

----------------------------------------------------------------------
File "C:\Users\dalcinl\DEVEL\cython-devel\BUILD\run\cpp\complex_int_T446.pyd",
line ?, in complex_int_T446.__test__.test_arith (line 3)
Failed example:
    test_arith(29+11j, 5+7j)
Expected:
    ((-29-11j), (34+18j), (24+4j), (68+258j), (3-2j))
Got:
    ((-29-11j), (34+18j), (24+4j), (68+258j), (1-4j))

It seems that std::complex<int> in MSVC does not work well for
integrals, operator/() was written with floating types in mind.

What should we do?

-- 
Lisandro Dalcin
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dalcinl at gmail.com  Mon Apr 12 01:57:17 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Sun, 11 Apr 2010 20:57:17 -0300
Subject: [Cython] [PATCH] basic C types in pure mode.
In-Reply-To: <4BC2003A.9020906@behnel.de>
References: <h2ze7ba66e41004091511rdd16c3f4o1abdf5d5dd5e5408@mail.gmail.com>
	<4BC2003A.9020906@behnel.de>
Message-ID: <p2ve7ba66e41004111657s8b3ca8b0i435ace18eaab55e@mail.gmail.com>

On 11 April 2010 14:00, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Lisandro Dalcin, 10.04.2010 00:11:
>> Is this patch fine?
>
> I think the reason why no-one replied so far is that this kind of patch is
> basically impossible to validate on eye-sight. The changes in
> "modifiers_and_name_to_type" seem to be mostly whitespace noise - it would
> be easier to read the patch without that.
>

Well, these changes are really uninteresting.

>
> ?> I mean, look at the new testcase in pure.pyx...
>> Should we support all basic C types in cython.declare()?
>
> Certainly all C types that can be mapped to a Python type in some way (i.e.
> those that make sense in pure more).
>

Sorry, could you clarify? Are you fine what the code I've added in
tests/run/pure.pyx ?


>
>> BTW, I would really like to remove that "modifiers_and_name_to_type" dictionary
>
> How would you do that?
>

Still not sure, but we have to large dicts in PyrexTypes, that smells
to unnecessary duplication.


-- 
Lisandro Dalcin
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From greg.ewing at canterbury.ac.nz  Mon Apr 12 08:17:49 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 12 Apr 2010 18:17:49 +1200
Subject: [Cython] [PATCH] basic C types in pure mode.
In-Reply-To: <4BC2003A.9020906@behnel.de>
References: <h2ze7ba66e41004091511rdd16c3f4o1abdf5d5dd5e5408@mail.gmail.com>
	<4BC2003A.9020906@behnel.de>
Message-ID: <4BC2BB0D.40109@canterbury.ac.nz>

Stefan Behnel wrote:
 >Lisandro Dalcin, 10.04.2010 00:11:
>>BTW, I would really like to remove that "modifiers_and_name_to_type" dictionary
> 
> How would you do that?

FYI, this dictionary is used by the parser when it gets a type
name such as 'int' and a set of modifiers such as 'long', 'unsigned'
etc. and wants to get a PyrexType corresponding to it. The mapping
is haphazard enough that a lookup table seemed like the best way
to do it when I was writing Pyrex.

I've been reworking the int type stuff in Pyrex recently -- you
might want to have a look at how it's done in the 0.9.9 release
I just made. I replaced all the magic numbers in that table with
named constants, and that's made it considerably more readable.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Mon Apr 12 08:24:08 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 12 Apr 2010 18:24:08 +1200
Subject: [Cython] [PATCH] basic C types in pure mode.
In-Reply-To: <p2ve7ba66e41004111657s8b3ca8b0i435ace18eaab55e@mail.gmail.com>
References: <h2ze7ba66e41004091511rdd16c3f4o1abdf5d5dd5e5408@mail.gmail.com>
	<4BC2003A.9020906@behnel.de>
	<p2ve7ba66e41004111657s8b3ca8b0i435ace18eaab55e@mail.gmail.com>
Message-ID: <4BC2BC88.4040809@canterbury.ac.nz>

Lisandro Dalcin wrote:

> Still not sure, but we have to large dicts in PyrexTypes, that smells
> to unnecessary duplication.

And I discovered that the other one wasn't being used any
more in Pyrex and eliminated it -- you might want to
check whether the same is true in Cython.

-- 
Greg

From stefan_ml at behnel.de  Mon Apr 12 09:10:42 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 12 Apr 2010 09:10:42 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC269ED.1070703@canterbury.ac.nz>
References: <4BC269ED.1070703@canterbury.ac.nz>
Message-ID: <4BC2C772.8040000@behnel.de>

Gregory Ewing, 12.04.2010 02:31:
> Pyrex 0.9.9 is now available:
>
> http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/

 > A more detailed discussion of these changes can be found
 > in the release notes here:
 >
 > 
http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/version/Doc/Release_Notes_0.9.9.html


> * Some facilities for interfacing with C++ code have been
> added.

Looks like Pyrex and Cython are really starting to diverge syntaxwise for 
similar features.

http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/version/Doc/Manual/using_with_c++.html

http://wiki.cython.org/WrappingCPlusPlus


> * Changes have been made to the semantics of exception
> catching to improve efficiency.

Since this violates Python semantics, I guess this will stay a Pyrex-only 
feature. A similar kind of behaviour could only appear in Cython if the 
except clause could be determined to be free of code that required those 
Python semantics, which I guess is hard enough to prove to make it mostly 
worthless. (Note that the except-pass case is already optimised in Cython.)


> * Preparation is being made for making 'not None' the
> default for extension type parameters to Python functions.

I agree that this has been a source of bugs in the past, so I'm not 
generally opposed to this. I really dislike breaking existing code, but I 
would expect that this is only a minor source of trouble for users. I'm 
+0.3 for such a change in Cython.


 > * Non-GC Extension Types

+1 for the feature, -1 for the syntax. Python 3 has keyword arguments in 
the baseclass tuple, so this would be much better here:

     cdef class Spam(gc=False):
         cdef object sausage


 > * Operations between two int types of the same rank now return an
 > unsigned result if either of the operands is unsigned; if the ranks
 > differ, the result has the same type as the wider-ranked operand. I
 > think this is the best approximation of the ANSI C rules that is
 > possible without knowing the exact sizes of the types.

I'm not sure how Cython handles this - can anyone comment?


 > * PyString_InternFromString is now exposed under the name cintern rather
 > than intern, because it is not a complete replacement for the Python
 > intern function (it can't handle strings containing null bytes).

The way Cython currently handles intern() is that it only works at the 
Python level, requiring byte string input in Py2 and unicode string input 
in Py3. Given this behaviour, I would expect that "cintern" returns the 
native 'str' type for each platform. BTW, note that intern() is no longer a 
builtin in Py3.

In Cython, neither of the two C-API functions that work on char* are 
currently exposed at the language level, so interning from a char* only 
works in Py2 (where it does exactly the same as a call to 
PyString_InternAsString due to the normal string coercion semantics).

It would become a little funny if "intern(char*)" worked in Py3 but 
"intern(bytes)" didn't, so I'm reluctant to enabling support for char* in 
Py3. That would be the only reason to use a new builtin, I guess, so I'm 
+0.5 for supporting it.


 > * The size check that was previously generated when importing an
 > extension type has been disabled for the time being until I can think of
 > something better. It was generating too many false positives, for
 > example from different versions of numpy.

IIRC, Cython has also stepped back from this, but still has some kind of 
check enabled. Don't remember the exact details.


 > * The __fastcall calling convention option is now supported. Also, Pyrex
 > no longer assumes that __cdecl is the default calling convention. To be
 > considered compatible, two function types must either be declared with
 > the same calling convention, or both must leave it unspecified.

Can't comment on this, but worth considering for Cython as well, I guess.


 > * As I have been threatening for some time, using __new__ as the name of
 > the initialisation method of an extension type has become an error
 > rather than just a warning. In some future release, __new__ will
 > re-emerge with more Python-like semantics.

I have wanted __new__ support for quite a while now, so I think this is the 
right thing to do. Cython already has a ticket about __new__, and we 
discussed this feature before I opened it, so no problem here, I guess.

http://trac.cython.org/cython_trac/ticket/238

Stefan

From dalcinl at gmail.com  Mon Apr 12 15:05:09 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 12 Apr 2010 10:05:09 -0300
Subject: [Cython] [PATCH] basic C types in pure mode.
In-Reply-To: <4BC2BC88.4040809@canterbury.ac.nz>
References: <h2ze7ba66e41004091511rdd16c3f4o1abdf5d5dd5e5408@mail.gmail.com>
	<4BC2003A.9020906@behnel.de>
	<p2ve7ba66e41004111657s8b3ca8b0i435ace18eaab55e@mail.gmail.com>
	<4BC2BC88.4040809@canterbury.ac.nz>
Message-ID: <s2je7ba66e41004120605ude84ea05lcbbd9789cbd1664e@mail.gmail.com>

On 12 April 2010 03:24, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Lisandro Dalcin wrote:
>
>> Still not sure, but we have to large dicts in PyrexTypes, that smells
>> to unnecessary duplication.
>
> And I discovered that the other one wasn't being used any
> more in Pyrex and eliminated it -- you might want to
> check whether the same is true in Cython.
>

Wow! Yes, I'm looking at your changes right now... I think I'll
back-port them..

Just a little comment on your patch: you defined this:

# Signedness values
UNSIGNED = 0
NOSIGN = 1
SIGNED = 2

# Longness values
SHORT = -1
NOLEN = 0
LONG = 1
LONGLONG = 2

That very nice, so... Why not take advantage of them to define all the
c_xxx_type instances?

-- 
Lisandro Dalcin
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dalcinl at gmail.com  Mon Apr 12 15:35:12 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 12 Apr 2010 10:35:12 -0300
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC2C772.8040000@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC2C772.8040000@behnel.de>
Message-ID: <y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>

On 12 April 2010 04:10, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Gregory Ewing, 12.04.2010 02:31:
>
>> * Preparation is being made for making 'not None' the
>> default for extension type parameters to Python functions.
>
> I agree that this has been a source of bugs in the past, so I'm not
> generally opposed to this. I really dislike breaking existing code, but I
> would expect that this is only a minor source of trouble for users. I'm
> +0.3 for such a change in Cython.
>

Big +1 from my side. About code breakages: Cython could use a compiler
directive for this in order to get the old behavior.


>
> ?> * Non-GC Extension Types
>
> +1 for the feature, -1 for the syntax. Python 3 has keyword arguments in
> the baseclass tuple, so this would be much better here:
>
> ? ? cdef class Spam(gc=False):
> ? ? ? ? cdef object sausage
>

I agree with Stefan here... Moreover, Cython currently uses bracket
for the 'type' and 'object' specifiers of extension types, but perhaps
it should use Foo(type=FooType, object=FooObject) ??

>
> ?> * Operations between two int types of the same rank now return an
> ?> unsigned result if either of the operands is unsigned; if the ranks
> ?> differ, the result has the same type as the wider-ranked operand. I
> ?> think this is the best approximation of the ANSI C rules that is
> ?> possible without knowing the exact sizes of the types.
>
> I'm not sure how Cython handles this - can anyone comment?
>

I'm working right now on this to try to backport the change  in order
to simplify the code in PyrexTypes.py

IIUC, the Cython and Pyrex behave the same, though Pyrex's code is
simpler, so worth to backport .


>
> ?> * The __fastcall calling convention option is now supported. Also, Pyrex
> ?> no longer assumes that __cdecl is the default calling convention. To be
> ?> considered compatible, two function types must either be declared with
> ?> the same calling convention, or both must leave it unspecified.
>
> Can't comment on this, but worth considering for Cython as well, I guess.
>

I've addedd __fastcall support long ago. About the other change, I
think that not assuming __cdecl as the default is the right thing to
do, so we should check for this in Cython. I could try to review this,
unless a interested Windows user/contributor beats me ;-)

>
> ?> * As I have been threatening for some time, using __new__ as the name of
> ?> the initialisation method of an extension type has become an error
> ?> rather than just a warning. In some future release, __new__ will
> ?> re-emerge with more Python-like semantics.
>
> I have wanted __new__ support for quite a while now, so I think this is the
> right thing to do. Cython already has a ticket about __new__, and we
> discussed this feature before I opened it, so no problem here, I guess.
>
> http://trac.cython.org/cython_trac/ticket/238
>

+1


-- 
Lisandro Dalcin
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From stefan_ml at behnel.de  Mon Apr 12 16:25:46 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 12 Apr 2010 16:25:46 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC2C772.8040000@behnel.de>
	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
Message-ID: <4BC32D6A.1060604@behnel.de>

Lisandro Dalcin, 12.04.2010 15:35:
> On 12 April 2010 04:10, Stefan Behnel wrote:
>> Gregory Ewing, 12.04.2010 02:31:
>>
>>> * Preparation is being made for making 'not None' the
>>> default for extension type parameters to Python functions.
>>
>> I agree that this has been a source of bugs in the past, so I'm not
>> generally opposed to this. I really dislike breaking existing code, but I
>> would expect that this is only a minor source of trouble for users. I'm
>> +0.3 for such a change in Cython.
>
> Big +1 from my side. About code breakages: Cython could use a compiler
> directive for this in order to get the old behavior.

... which would mean that users must still adapt their code. If they do 
that, I think it's better to fix it while you're at it, rather than going 
with the quick fix of putting in a directive. So I'm -0 on such a 
directive, given that we can expect the average user code to contain at 
least a couple of bugs due to a missing None test. If we emit a warning for 
a missing "or None", it's easy enough to spot where work needs to be done.


>>   >  * Non-GC Extension Types
>>
>> +1 for the feature, -1 for the syntax. Python 3 has keyword arguments in
>> the baseclass tuple, so this would be much better here:
>>
>>      cdef class Spam(gc=False):
>>          cdef object sausage
>
> I agree with Stefan here... Moreover, Cython currently uses bracket
> for the 'type' and 'object' specifiers of extension types, but perhaps
> it should use Foo(type=FooType, object=FooObject) ??

Rather "FooType" as a string and maybe something like "ctypename" instead 
of just "type", but, yes, I like that.


>>   >  * Operations between two int types of the same rank now return an
>>   >  unsigned result if either of the operands is unsigned; if the ranks
>>   >  differ, the result has the same type as the wider-ranked operand. I
>>   >  think this is the best approximation of the ANSI C rules that is
>>   >  possible without knowing the exact sizes of the types.
>>
>> I'm not sure how Cython handles this - can anyone comment?
>
> I'm working right now on this to try to backport the change  in order
> to simplify the code in PyrexTypes.py

Thanks!


> IIUC, the Cython and Pyrex behave the same, though Pyrex's code is
> simpler, so worth to backport .

Except that "back"-port isn't quite the right word here. ;)


>>   >  * The __fastcall calling convention option is now supported. Also, Pyrex
>>   >  no longer assumes that __cdecl is the default calling convention. To be
>>   >  considered compatible, two function types must either be declared with
>>   >  the same calling convention, or both must leave it unspecified.
>>
>> Can't comment on this, but worth considering for Cython as well, I guess.
>
> I've addedd __fastcall support long ago. About the other change, I
> think that not assuming __cdecl as the default is the right thing to
> do, so we should check for this in Cython. I could try to review this,
> unless a interested Windows user/contributor beats me ;-)

Let's not wait for that. ;)

Stefan

From dagss at student.matnat.uio.no  Mon Apr 12 16:34:49 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Mon, 12 Apr 2010 16:34:49 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC2C772.8040000@behnel.de>
	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
Message-ID: <4BC32F89.40808@student.matnat.uio.no>

Lisandro Dalcin wrote:
> On 12 April 2010 04:10, Stefan Behnel <stefan_ml at behnel.de> wrote:
>> Gregory Ewing, 12.04.2010 02:31:
>>
>>> * Preparation is being made for making 'not None' the
>>> default for extension type parameters to Python functions.
>> I agree that this has been a source of bugs in the past, so I'm not
>> generally opposed to this. I really dislike breaking existing code, but I
>> would expect that this is only a minor source of trouble for users. I'm
>> +0.3 for such a change in Cython.
>>
> 
> Big +1 from my side. About code breakages: Cython could use a compiler
> directive for this in order to get the old behavior.

I guess I should repeat my big -1 for this then.

I think Java-like behaviour is much more appropriate (i.e. raise proper 
exceptions in the code using the variable, but allow None as a value).

Let's not break the language forever just because nobody has time to 
implement control flow analysis now.

-- 
Dag Sverre

From dalcinl at gmail.com  Mon Apr 12 18:37:01 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 12 Apr 2010 13:37:01 -0300
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC32F89.40808@student.matnat.uio.no>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC2C772.8040000@behnel.de>
	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
	<4BC32F89.40808@student.matnat.uio.no>
Message-ID: <l2ye7ba66e41004120937j9dde305eye31f4e6a47517a66@mail.gmail.com>

On 12 April 2010 11:34, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:
> Lisandro Dalcin wrote:
>> On 12 April 2010 04:10, Stefan Behnel <stefan_ml at behnel.de> wrote:
>>> Gregory Ewing, 12.04.2010 02:31:
>>>
>>>> * Preparation is being made for making 'not None' the
>>>> default for extension type parameters to Python functions.
>>> I agree that this has been a source of bugs in the past, so I'm not
>>> generally opposed to this. I really dislike breaking existing code, but I
>>> would expect that this is only a minor source of trouble for users. I'm
>>> +0.3 for such a change in Cython.
>>>
>>
>> Big +1 from my side. About code breakages: Cython could use a compiler
>> directive for this in order to get the old behavior.
>
> I guess I should repeat my big -1 for this then.
>
> I think Java-like behaviour is much more appropriate (i.e. raise proper
> exceptions in the code using the variable, but allow None as a value).
>
> Let's not break the language forever just because nobody has time to
> implement control flow analysis now.
>

Would you accept a directive to ENABLE automatic "not None"?



-- 
Lisandro Dalcin
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dagss at student.matnat.uio.no  Mon Apr 12 18:50:08 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Mon, 12 Apr 2010 18:50:08 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <l2ye7ba66e41004120937j9dde305eye31f4e6a47517a66@mail.gmail.com>
References: <4BC269ED.1070703@canterbury.ac.nz>
	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>
	<l2ye7ba66e41004120937j9dde305eye31f4e6a47517a66@mail.gmail.com>
Message-ID: <4BC34F40.5010101@student.matnat.uio.no>

Lisandro Dalcin wrote:
> On 12 April 2010 11:34, Dag Sverre Seljebotn
> <dagss at student.matnat.uio.no> wrote:
>   
>> Lisandro Dalcin wrote:
>>     
>>> On 12 April 2010 04:10, Stefan Behnel <stefan_ml at behnel.de> wrote:
>>>       
>>>> Gregory Ewing, 12.04.2010 02:31:
>>>>
>>>>         
>>>>> * Preparation is being made for making 'not None' the
>>>>> default for extension type parameters to Python functions.
>>>>>           
>>>> I agree that this has been a source of bugs in the past, so I'm not
>>>> generally opposed to this. I really dislike breaking existing code, but I
>>>> would expect that this is only a minor source of trouble for users. I'm
>>>> +0.3 for such a change in Cython.
>>>>
>>>>         
>>> Big +1 from my side. About code breakages: Cython could use a compiler
>>> directive for this in order to get the old behavior.
>>>       
>> I guess I should repeat my big -1 for this then.
>>
>> I think Java-like behaviour is much more appropriate (i.e. raise proper
>> exceptions in the code using the variable, but allow None as a value).
>>
>> Let's not break the language forever just because nobody has time to
>> implement control flow analysis now.
>>
>>     
>
> Would you accept a directive to ENABLE automatic "not None"?
>   
I don't feel very happy about it, no. (But it's not for me to accept or 
reject.)

We already have cdivision which is comparable (changes semantics in 
non-exceptional case in a hard-to-spot fashion), but it was introduced 
because of backwards compatibility and the 20% performance penalty.

This one seems to have the same cons, but without backwards 
compatibility and performance as pro-arguments (leaving only 
convenience). And "not None" is already in the syntax. "Explicit is 
better than implicit" and all that.

Dag Sverre

From robertwb at math.washington.edu  Mon Apr 12 19:14:56 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 12 Apr 2010 10:14:56 -0700
Subject: [Cython] int complex failure in C++ with MSVC
In-Reply-To: <q2oe7ba66e41004111650w35b057c7x26018d385d244207@mail.gmail.com>
References: <q2oe7ba66e41004111650w35b057c7x26018d385d244207@mail.gmail.com>
Message-ID: <2EA9CDD9-34BC-47D4-A3C1-0B5018ADB0E7@math.washington.edu>

On Apr 11, 2010, at 4:50 PM, Lisandro Dalcin wrote:

> See this...
>
> ----------------------------------------------------------------------
> File "C:\Users\dalcinl\DEVEL\cython-devel\BUILD\run\cpp 
> \complex_int_T446.pyd",
> line ?, in complex_int_T446.__test__.test_arith (line 3)
> Failed example:
>    test_arith(29+11j, 5+7j)
> Expected:
>    ((-29-11j), (34+18j), (24+4j), (68+258j), (3-2j))
> Got:
>    ((-29-11j), (34+18j), (24+4j), (68+258j), (1-4j))
>
> It seems that std::complex<int> in MSVC does not work well for
> integrals, operator/() was written with floating types in mind.
>
> What should we do?

Hmm... I would be OK with complex_int_T446 not testing division (with  
a note). I'd assume this has to do with the order in which one does  
the intermal operations (and subsequent truncation).

- Robert


From robertwb at math.washington.edu  Mon Apr 12 19:23:26 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 12 Apr 2010 10:23:26 -0700
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC2C772.8040000@behnel.de>
	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
Message-ID: <56FF72E0-7D63-4273-A478-49D0100F6D09@math.washington.edu>

On Apr 12, 2010, at 6:35 AM, Lisandro Dalcin wrote:

> On 12 April 2010 04:10, Stefan Behnel <stefan_ml at behnel.de> wrote:
>>
>>  > * Non-GC Extension Types
>>
>> +1 for the feature, -1 for the syntax. Python 3 has keyword  
>> arguments in
>> the baseclass tuple, so this would be much better here:
>>
>>     cdef class Spam(gc=False):
>>         cdef object sausage
>
> I agree with Stefan here...

I think a class decorator would be even more natural here.

> Moreover, Cython currently uses bracket
> for the 'type' and 'object' specifiers of extension types, but perhaps
> it should use Foo(type=FooType, object=FooObject) ??

I like this idea.

>>
>>  > * Operations between two int types of the same rank now return an
>>  > unsigned result if either of the operands is unsigned; if the  
>> ranks
>>  > differ, the result has the same type as the wider-ranked  
>> operand. I
>>  > think this is the best approximation of the ANSI C rules that is
>>  > possible without knowing the exact sizes of the types.
>>
>> I'm not sure how Cython handles this - can anyone comment?
>>
>
> I'm working right now on this to try to backport the change  in order
> to simplify the code in PyrexTypes.py
>
> IIUC, the Cython and Pyrex behave the same, though Pyrex's code is
> simpler, so worth to backport .

Yep, we changed to unsigned + signed = unsigned unless one is wider a  
long time ago.

>>  > * The __fastcall calling convention option is now supported.  
>> Also, Pyrex
>>  > no longer assumes that __cdecl is the default calling  
>> convention. To be
>>  > considered compatible, two function types must either be  
>> declared with
>>  > the same calling convention, or both must leave it unspecified.
>>
>> Can't comment on this, but worth considering for Cython as well, I  
>> guess.
>>
>
> I've addedd __fastcall support long ago. About the other change, I
> think that not assuming __cdecl as the default is the right thing to
> do, so we should check for this in Cython. I could try to review this,
> unless a interested Windows user/contributor beats me ;-)

I have no idea what these even mean, so I'll defer to you with a bias  
towards backwards compatibility.

>>  > * As I have been threatening for some time, using __new__ as the  
>> name of
>>  > the initialisation method of an extension type has become an error
>>  > rather than just a warning. In some future release, __new__ will
>>  > re-emerge with more Python-like semantics.
>>
>> I have wanted __new__ support for quite a while now, so I think  
>> this is the
>> right thing to do. Cython already has a ticket about __new__, and we
>> discussed this feature before I opened it, so no problem here, I  
>> guess.
>>
>> http://trac.cython.org/cython_trac/ticket/238
>>
>
> +1

Lets defer this change until I can make sure the Sage codebase is  
clean. We should make it a visible warning at least first.

- Robert


From Chris.Barker at noaa.gov  Mon Apr 12 20:06:53 2010
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Mon, 12 Apr 2010 11:06:53 -0700
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC34F40.5010101@student.matnat.uio.no>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC2C772.8040000@behnel.de>
	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
	<4BC32F89.40808@student.matnat.uio.no>
	<l2ye7ba66e41004120937j9dde305eye31f4e6a47517a66@mail.gmail.com>
	<4BC34F40.5010101@student.matnat.uio.no>
Message-ID: <4BC3613D.7040806@noaa.gov>

>>>>>> * Preparation is being made for making 'not None' the
>>>>>> default for extension type parameters to Python functions.

>>> I guess I should repeat my big -1 for this then.
>>>
>>> I think Java-like behaviour is much more appropriate (i.e. raise proper
>>> exceptions in the code using the variable, but allow None as a value).

I'm a bit confused here -- how could you have the code checking for None 
everywhere without a performance hit?

I understand the utility of being able to pass None, but it is really 
dangerous unless it is handled properly, which means the user code needs 
to check for it up front.

>> Would you accept a directive to ENABLE automatic "not None"?

That seems reasonable to me -- by the way, is there a deprecation policy 
for Cython?

> And "not None" is already in the syntax. "Explicit is 
> better than implicit" and all that.

Sure, but leaving it out is not explicit, and it is the more dangerous 
option. An "allow None" would be the right way to be explicit.

-just a couple thoughts from a user.

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

From robertwb at math.washington.edu  Mon Apr 12 20:11:59 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 12 Apr 2010 11:11:59 -0700
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC34F40.5010101@student.matnat.uio.no>
References: <4BC269ED.1070703@canterbury.ac.nz>
	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>
	<l2ye7ba66e41004120937j9dde305eye31f4e6a47517a66@mail.gmail.com>
	<4BC34F40.5010101@student.matnat.uio.no>
Message-ID: <BB7E49F6-EC28-4283-A6FD-F3FEAF882794@math.washington.edu>

On Apr 12, 2010, at 9:50 AM, Dag Sverre Seljebotn wrote:

> Lisandro Dalcin wrote:
>> On 12 April 2010 11:34, Dag Sverre Seljebotn
>> <dagss at student.matnat.uio.no> wrote:
>>
>>> Lisandro Dalcin wrote:
>>>
>>>> On 12 April 2010 04:10, Stefan Behnel <stefan_ml at behnel.de> wrote:
>>>>
>>>>> Gregory Ewing, 12.04.2010 02:31:
>>>>>
>>>>>
>>>>>> * Preparation is being made for making 'not None' the
>>>>>> default for extension type parameters to Python functions.
>>>>>>
>>>>> I agree that this has been a source of bugs in the past, so I'm  
>>>>> not
>>>>> generally opposed to this. I really dislike breaking existing  
>>>>> code, but I
>>>>> would expect that this is only a minor source of trouble for  
>>>>> users. I'm
>>>>> +0.3 for such a change in Cython.
>>>>>
>>>>>
>>>> Big +1 from my side. About code breakages: Cython could use a  
>>>> compiler
>>>> directive for this in order to get the old behavior.
>>>>
>>> I guess I should repeat my big -1 for this then.
>>>
>>> I think Java-like behaviour is much more appropriate (i.e. raise  
>>> proper
>>> exceptions in the code using the variable, but allow None as a  
>>> value).
>>>
>>> Let's not break the language forever just because nobody has time to
>>> implement control flow analysis now.

I'm -1 to changing the default as well--control flow analysis is the  
way to correctly handle this. We also have the noncheck directive if  
people are worried about this.

>> Would you accept a directive to ENABLE automatic "not None"?
>>
> I don't feel very happy about it, no. (But it's not for me to accept  
> or
> reject.)
>
> We already have cdivision which is comparable (changes semantics in
> non-exceptional case in a hard-to-spot fashion), but it was introduced
> because of backwards compatibility and the 20% performance penalty.
>
> This one seems to have the same cons, but without backwards
> compatibility and performance as pro-arguments (leaving only
> convenience). And "not None" is already in the syntax. "Explicit is
> better than implicit" and all that.

Pyrex is about writing extension modules, and (conceptually) has a  
much sharper line between "python-level" and "c-level" code than we  
want to have in Cython. (In fact, the cdivision questions really  
boiled down to trying to erase this line. There was also the issue of  
giving subtly different answers rather than crashing...) As I've  
proposed before, our goal should be taking any valid Python code, pass  
it through the Cython compiler, and have it behave the same (except  
for speed, introspection, and possibly a finite list of other  
necessary exceptions).

- Robert


From robertwb at math.washington.edu  Mon Apr 12 20:53:12 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 12 Apr 2010 11:53:12 -0700
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC3613D.7040806@noaa.gov>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC2C772.8040000@behnel.de>
	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
	<4BC32F89.40808@student.matnat.uio.no>
	<l2ye7ba66e41004120937j9dde305eye31f4e6a47517a66@mail.gmail.com>
	<4BC34F40.5010101@student.matnat.uio.no>
	<4BC3613D.7040806@noaa.gov>
Message-ID: <A490290F-26BC-45DA-86DB-27E98C54BB97@math.washington.edu>

On Apr 12, 2010, at 11:06 AM, Christopher Barker wrote:

>>>>>>> * Preparation is being made for making 'not None' the
>>>>>>> default for extension type parameters to Python functions.
>
>>>> I guess I should repeat my big -1 for this then.
>>>>
>>>> I think Java-like behaviour is much more appropriate (i.e. raise  
>>>> proper
>>>> exceptions in the code using the variable, but allow None as a  
>>>> value).
>
> I'm a bit confused here -- how could you have the code checking for  
> None
> everywhere without a performance hit?

Control flow analysis. This will also handle much more than just  
Python function arguments.

> I understand the utility of being able to pass None, but it is really
> dangerous unless it is handled properly, which means the user code  
> needs
> to check for it up front.
>
>>> Would you accept a directive to ENABLE automatic "not None"?
>
> That seems reasonable to me

I would not be opposed to it either given that users would really like  
this behavior, but I would hope that it becomes unnecessary before too  
long.

> -- by the way, is there a deprecation policy for Cython?

No, other than giving ample warnings ahead of time, and we try to  
maintain backwards compatibility.

>> And "not None" is already in the syntax. "Explicit is
>> better than implicit" and all that.
>
> Sure, but leaving it out is not explicit,

it's just like Python

> and it is the more dangerous  option.

Yes, unfortunately the resulting crash is harder...

> An "allow None" would be the right way to be explicit.

If we have a directive, we should also support this (or "or None"  
following Pyrex's choice of syntax).

> -just a couple thoughts from a user.


And thank you for your sharing. Would you be happy with raising same  
exception you'd see in Python if there was no (or very little)  
performance impact?

- Robert


From dalcinl at gmail.com  Mon Apr 12 22:42:39 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 12 Apr 2010 17:42:39 -0300
Subject: [Cython] int complex failure in C++ with MSVC
In-Reply-To: <2EA9CDD9-34BC-47D4-A3C1-0B5018ADB0E7@math.washington.edu>
References: <q2oe7ba66e41004111650w35b057c7x26018d385d244207@mail.gmail.com>
	<2EA9CDD9-34BC-47D4-A3C1-0B5018ADB0E7@math.washington.edu>
Message-ID: <r2me7ba66e41004121342h8e59de80n4ea99d865f60fbe@mail.gmail.com>

On 12 April 2010 14:14, Robert Bradshaw <robertwb at math.washington.edu> wrote:
> On Apr 11, 2010, at 4:50 PM, Lisandro Dalcin wrote:
>
>> See this...
>>
>> ----------------------------------------------------------------------
>> File "C:\Users\dalcinl\DEVEL\cython-devel\BUILD\run\cpp
>> \complex_int_T446.pyd",
>> line ?, in complex_int_T446.__test__.test_arith (line 3)
>> Failed example:
>> ? ?test_arith(29+11j, 5+7j)
>> Expected:
>> ? ?((-29-11j), (34+18j), (24+4j), (68+258j), (3-2j))
>> Got:
>> ? ?((-29-11j), (34+18j), (24+4j), (68+258j), (1-4j))
>>
>> It seems that std::complex<int> in MSVC does not work well for
>> integrals, operator/() was written with floating types in mind.
>>
>> What should we do?
>
> Hmm... I would be OK with complex_int_T446 not testing division (with
> a note).

Do you mean a little comment saying this is broken with MSVC?

> I'd assume this has to do with the order in which one does
> the intermal operations (and subsequent truncation).
>

Yes.


-- 
Lisandro Dalcin
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From robertwb at math.washington.edu  Mon Apr 12 23:04:30 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 12 Apr 2010 14:04:30 -0700
Subject: [Cython] int complex failure in C++ with MSVC
In-Reply-To: <r2me7ba66e41004121342h8e59de80n4ea99d865f60fbe@mail.gmail.com>
References: <q2oe7ba66e41004111650w35b057c7x26018d385d244207@mail.gmail.com>
	<2EA9CDD9-34BC-47D4-A3C1-0B5018ADB0E7@math.washington.edu>
	<r2me7ba66e41004121342h8e59de80n4ea99d865f60fbe@mail.gmail.com>
Message-ID: <520D0D09-16AA-4B1E-8401-36C104A3B836@math.washington.edu>

On Apr 12, 2010, at 1:42 PM, Lisandro Dalcin wrote:

> On 12 April 2010 14:14, Robert Bradshaw  
> <robertwb at math.washington.edu> wrote:
>> On Apr 11, 2010, at 4:50 PM, Lisandro Dalcin wrote:
>>
>>> See this...
>>>
>>> ----------------------------------------------------------------------
>>> File "C:\Users\dalcinl\DEVEL\cython-devel\BUILD\run\cpp
>>> \complex_int_T446.pyd",
>>> line ?, in complex_int_T446.__test__.test_arith (line 3)
>>> Failed example:
>>>    test_arith(29+11j, 5+7j)
>>> Expected:
>>>    ((-29-11j), (34+18j), (24+4j), (68+258j), (3-2j))
>>> Got:
>>>    ((-29-11j), (34+18j), (24+4j), (68+258j), (1-4j))
>>>
>>> It seems that std::complex<int> in MSVC does not work well for
>>> integrals, operator/() was written with floating types in mind.
>>>
>>> What should we do?
>>
>> Hmm... I would be OK with complex_int_T446 not testing division (with
>> a note).
>
> Do you mean a little comment saying this is broken with MSVC?


Yep. For non-exact division I could see calling the result undefined,  
but that's not the case here.

- Robert


From dalcinl at gmail.com  Mon Apr 12 23:16:19 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 12 Apr 2010 18:16:19 -0300
Subject: [Cython] int complex failure in C++ with MSVC
In-Reply-To: <520D0D09-16AA-4B1E-8401-36C104A3B836@math.washington.edu>
References: <q2oe7ba66e41004111650w35b057c7x26018d385d244207@mail.gmail.com>
	<2EA9CDD9-34BC-47D4-A3C1-0B5018ADB0E7@math.washington.edu>
	<r2me7ba66e41004121342h8e59de80n4ea99d865f60fbe@mail.gmail.com>
	<520D0D09-16AA-4B1E-8401-36C104A3B836@math.washington.edu>
Message-ID: <v2qe7ba66e41004121416p7292fb48p7423ca9e999d9a43@mail.gmail.com>

On 12 April 2010 18:04, Robert Bradshaw <robertwb at math.washington.edu> wrote:
> On Apr 12, 2010, at 1:42 PM, Lisandro Dalcin wrote:
>
>> On 12 April 2010 14:14, Robert Bradshaw
>> <robertwb at math.washington.edu> wrote:
>>> On Apr 11, 2010, at 4:50 PM, Lisandro Dalcin wrote:
>>>
>>>> See this...
>>>>
>>>> ----------------------------------------------------------------------
>>>> File "C:\Users\dalcinl\DEVEL\cython-devel\BUILD\run\cpp
>>>> \complex_int_T446.pyd",
>>>> line ?, in complex_int_T446.__test__.test_arith (line 3)
>>>> Failed example:
>>>> ? ?test_arith(29+11j, 5+7j)
>>>> Expected:
>>>> ? ?((-29-11j), (34+18j), (24+4j), (68+258j), (3-2j))
>>>> Got:
>>>> ? ?((-29-11j), (34+18j), (24+4j), (68+258j), (1-4j))
>>>>
>>>> It seems that std::complex<int> in MSVC does not work well for
>>>> integrals, operator/() was written with floating types in mind.
>>>>
>>>> What should we do?
>>>
>>> Hmm... I would be OK with complex_int_T446 not testing division (with
>>> a note).
>>
>> Do you mean a little comment saying this is broken with MSVC?
>
>
> Yep. For non-exact division I could see calling the result undefined,
> but that's not the case here.
>

Mmm.. Wait a minute... Now this makes me think a bit different ... In
general, we cannot trust std::complex implementations [except GCC, of
course :-)] ... So what about switching to struct-based complexes if
the real-type is integral? Or perhaps better, use an ad-hoc quot()
implementation?



-- 
Lisandro Dalcin
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dalcinl at gmail.com  Mon Apr 12 23:55:07 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 12 Apr 2010 18:55:07 -0300
Subject: [Cython] issue while "back-porting" Pyrex change (Greg,
	please comment).
Message-ID: <z2je7ba66e41004121455hdc824583l36d515f044c0234d@mail.gmail.com>

Pyrex:  <unsigned int> + <long> --> <long>
Cython: <unsigned int> + <long> --> <unsigned long>


This is the Pyrex algorithm:

    if type1.rank < type2.rank:
        widest_type = type2
    elif type1.rank > type2.rank:
        widest_type = type1
    elif type1.signed < type2.signed:
        widest_type = type1
    else:
        widest_type = type2

Below I pasted what the C standard says... IIUC, Pyrex does not follow
it, right? Mmm... Now I'm not sure about getting rid of Cython's
"sign_and_rank_to_type" mapping.


Otherwise, the integer promotions are performed on both operands. Then the
following rules are applied to the promoted operands:

If both operands have the same type, then no further conversion is needed.

Otherwise, if both operands have signed integer types or both have unsigned
integer types, the operand with the type of lesser integer conversion rank is
converted to the type of the operand with greater rank.

Otherwise, if the operand that has unsigned integer type has rank greater or
equal to the rank of the type of the other operand, then the operand with
signed integer type is converted to the type of the operand with unsigned
integer type.

Otherwise, if the type of the operand with signed integer type can represent
all of the values of the type of the operand with unsigned integer type, then
the operand with unsigned integer type is converted to the type of the
operand with signed integer type.

Otherwise, both operands are converted to the unsigned integer type
corresponding to the type of the operand with signed integer type.



-- 
Lisandro Dalcin
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From stefan_ml at behnel.de  Tue Apr 13 06:25:55 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 13 Apr 2010 06:25:55 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <56FF72E0-7D63-4273-A478-49D0100F6D09@math.washington.edu>
References: <4BC269ED.1070703@canterbury.ac.nz>
	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
	<56FF72E0-7D63-4273-A478-49D0100F6D09@math.washington.edu>
Message-ID: <4BC3F253.30002@behnel.de>

Robert Bradshaw, 12.04.2010 19:23:
> On Apr 12, 2010, at 6:35 AM, Lisandro Dalcin wrote:
>
>> On 12 April 2010 04:10, Stefan Behnel wrote:
>>>
>>>   >  * Non-GC Extension Types
>>>
>>> +1 for the feature, -1 for the syntax. Python 3 has keyword
>>> arguments in
>>> the baseclass tuple, so this would be much better here:
>>>
>>>      cdef class Spam(gc=False):
>>>          cdef object sausage
>>
>> I agree with Stefan here...
>
> I think a class decorator would be even more natural here.

Right, and it would work more easily in pure Python (2.x) code as well.


>>>   >  * As I have been threatening for some time, using __new__ as the
>>> name of
>>>   >  the initialisation method of an extension type has become an error
>>>   >  rather than just a warning. In some future release, __new__ will
>>>   >  re-emerge with more Python-like semantics.
>>>
>>> I have wanted __new__ support for quite a while now, so I think
>>> this is the
>>> right thing to do. Cython already has a ticket about __new__, and we
>>> discussed this feature before I opened it, so no problem here, I
>>> guess.
>>>
>>> http://trac.cython.org/cython_trac/ticket/238
>>
>> +1
>
> Lets defer this change until I can make sure the Sage codebase is
> clean. We should make it a visible warning at least first.

The compile time error that you will get if your __new__ function does not 
define its return type or returns without a return value is very clear and 
will catch 100% of the then incorrect cases, so I don't see a major use in 
making this deprecated first. All a user has to do to fix this is rename 
__new__ to __cinit__.

Stefan

From stefan_ml at behnel.de  Tue Apr 13 06:42:33 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 13 Apr 2010 06:42:33 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC32F89.40808@student.matnat.uio.no>
References: <4BC269ED.1070703@canterbury.ac.nz>
	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
	<4BC32F89.40808@student.matnat.uio.no>
Message-ID: <4BC3F639.4070200@behnel.de>

Dag Sverre Seljebotn, 12.04.2010 16:34:
> Lisandro Dalcin wrote:
>> On 12 April 2010 04:10, Stefan Behnel wrote:
>>> Gregory Ewing, 12.04.2010 02:31:
>>>
>>>> * Preparation is being made for making 'not None' the
>>>> default for extension type parameters to Python functions.
>>> I agree that this has been a source of bugs in the past, so I'm not
>>> generally opposed to this. I really dislike breaking existing code, but I
>>> would expect that this is only a minor source of trouble for users. I'm
>>> +0.3 for such a change in Cython.
>>
>> Big +1 from my side. About code breakages: Cython could use a compiler
>> directive for this in order to get the old behavior.
>
> I guess I should repeat my big -1 for this then.
>
> I think Java-like behaviour is much more appropriate (i.e. raise proper
> exceptions in the code using the variable, but allow None as a value).
>
> Let's not break the language forever just because nobody has time to
> implement control flow analysis now.

Control flow analysis can't prevent the case that a user passes None into a 
Python function. It can only print a warning (not even an error) when it 
finds out that the code accesses C level members of an explicitly typed 
extension type argument where the value *may* be None. In that case, the 
user will have to recognise the warning as important and add a "not None" 
to the function signature (in an expected 95% of the cases) or handle this 
in explicit code (in at most 5%). Given that the simple "reject None" case 
is so extremely common at the Python API level, it sounds like the right 
thing to do by default.

Note that we are only dealing with Python functions here, and only with the 
case that the input argument is explicitly typed, i.e like this:

     def func(MyExtType value):

So there already is a rather large performance impact in calling this 
function, and there already is an explicit annotation saying that the value 
is an extension type. It won't even hurt cpdef functions, where the type 
check would happen inside of the Python wrapper.

I accept the argument that function parameters are not the only dangerous 
place here and that values received from a function call or global values 
are best checked as well, so control flow analysis would still be able to 
print helpful warnings in other places. But the case of function arguments 
is so important that it's worth giving it distinct semantics.

Stefan

From robertwb at math.washington.edu  Tue Apr 13 07:10:48 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 12 Apr 2010 22:10:48 -0700
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC3F639.4070200@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz>
	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
	<4BC32F89.40808@student.matnat.uio.no> <4BC3F639.4070200@behnel.de>
Message-ID: <903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>

On Apr 12, 2010, at 9:42 PM, Stefan Behnel wrote:

> Dag Sverre Seljebotn, 12.04.2010 16:34:
>> Lisandro Dalcin wrote:
>>> On 12 April 2010 04:10, Stefan Behnel wrote:
>>>> Gregory Ewing, 12.04.2010 02:31:
>>>>
>>>>> * Preparation is being made for making 'not None' the
>>>>> default for extension type parameters to Python functions.
>>>> I agree that this has been a source of bugs in the past, so I'm not
>>>> generally opposed to this. I really dislike breaking existing  
>>>> code, but I
>>>> would expect that this is only a minor source of trouble for  
>>>> users. I'm
>>>> +0.3 for such a change in Cython.
>>>
>>> Big +1 from my side. About code breakages: Cython could use a  
>>> compiler
>>> directive for this in order to get the old behavior.
>>
>> I guess I should repeat my big -1 for this then.
>>
>> I think Java-like behaviour is much more appropriate (i.e. raise  
>> proper
>> exceptions in the code using the variable, but allow None as a  
>> value).
>>
>> Let's not break the language forever just because nobody has time to
>> implement control flow analysis now.
>
> Control flow analysis can't prevent the case that a user passes None  
> into a
> Python function. It can only print a warning (not even an error)  
> when it
> finds out that the code accesses C level members of an explicitly  
> typed
> extension type argument where the value *may* be None. In that case,  
> the
> user will have to recognise the warning as important and add a "not  
> None"
> to the function signature (in an expected 95% of the cases) or  
> handle this
> in explicit code (in at most 5%). Given that the simple "reject  
> None" case
> is so extremely common at the Python API level, it sounds like the  
> right
> thing to do by default.
>
> Note that we are only dealing with Python functions here, and only  
> with the
> case that the input argument is explicitly typed, i.e like this:
>
>     def func(MyExtType value):
>
> So there already is a rather large performance impact in calling this
> function, and there already is an explicit annotation saying that  
> the value
> is an extension type. It won't even hurt cpdef functions, where the  
> type
> check would happen inside of the Python wrapper.

So you'd have cases where

     (<object>x).func(None) # runtime error
     (<MyClass>x).func(None) # segfault

> I accept the argument that function parameters are not the only  
> dangerous
> place here and that values received from a function call or global  
> values
> are best checked as well, so control flow analysis would still be  
> able to
> print helpful warnings in other places. But the case of function  
> arguments
> is so important that it's worth giving it distinct semantics.

You are missing the point of control flow analysis here--it's not to  
warn the user when code might be None, but to raise (runtime)  
exceptions when it is. We can turn that on now with the nonecheck  
directive, but that comes at a cost. With control flow analysis, one  
can make nonecheck=True be the default, and the check is only  
performed the first time a cdef attribute is accessed (in a given  
path), which will have the advantage of behaving like Python and  
handling more than arguments without the performance degradation, as  
well as not segfaulting on bad input.

I think this is the ideal long term solution, and we shouldn't be  
changing the default just to change it back later.

- Robert


From dagss at student.matnat.uio.no  Tue Apr 13 08:19:02 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 13 Apr 2010 08:19:02 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>
	<4BC3F639.4070200@behnel.de>
	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>
Message-ID: <4BC40CD6.6080206@student.matnat.uio.no>

Robert Bradshaw wrote:
> On Apr 12, 2010, at 9:42 PM, Stefan Behnel wrote:
>
>   
>> Dag Sverre Seljebotn, 12.04.2010 16:34:
>>     
>>> Lisandro Dalcin wrote:
>>>       
>>>> On 12 April 2010 04:10, Stefan Behnel wrote:
>>>>         
>>>>> Gregory Ewing, 12.04.2010 02:31:
>>>>>
>>>>>           
>>>>>> * Preparation is being made for making 'not None' the
>>>>>> default for extension type parameters to Python functions.
>>>>>>             
>>>>> I agree that this has been a source of bugs in the past, so I'm not
>>>>> generally opposed to this. I really dislike breaking existing  
>>>>> code, but I
>>>>> would expect that this is only a minor source of trouble for  
>>>>> users. I'm
>>>>> +0.3 for such a change in Cython.
>>>>>           
>>>> Big +1 from my side. About code breakages: Cython could use a  
>>>> compiler
>>>> directive for this in order to get the old behavior.
>>>>         
>>> I guess I should repeat my big -1 for this then.
>>>
>>> I think Java-like behaviour is much more appropriate (i.e. raise  
>>> proper
>>> exceptions in the code using the variable, but allow None as a  
>>> value).
>>>
>>> Let's not break the language forever just because nobody has time to
>>> implement control flow analysis now.
>>>       
>> Control flow analysis can't prevent the case that a user passes None  
>> into a
>> Python function. It can only print a warning (not even an error)  
>> when it
>> finds out that the code accesses C level members of an explicitly  
>> typed
>> extension type argument where the value *may* be None. In that case,  
>> the
>> user will have to recognise the warning as important and add a "not  
>> None"
>> to the function signature (in an expected 95% of the cases) or  
>> handle this
>> in explicit code (in at most 5%). Given that the simple "reject  
>> None" case
>> is so extremely common at the Python API level, it sounds like the  
>> right
>> thing to do by default.
>>
>> Note that we are only dealing with Python functions here, and only  
>> with the
>> case that the input argument is explicitly typed, i.e like this:
>>
>>     def func(MyExtType value):
>>
>> So there already is a rather large performance impact in calling this
>> function, and there already is an explicit annotation saying that  
>> the value
>> is an extension type. It won't even hurt cpdef functions, where the  
>> type
>> check would happen inside of the Python wrapper.
>>     
>
> So you'd have cases where
>
>      (<object>x).func(None) # runtime error
>      (<MyClass>x).func(None) # segfault
>
>   
>> I accept the argument that function parameters are not the only  
>> dangerous
>> place here and that values received from a function call or global  
>> values
>> are best checked as well, so control flow analysis would still be  
>> able to
>> print helpful warnings in other places. But the case of function  
>> arguments
>> is so important that it's worth giving it distinct semantics.
>>     
>
> You are missing the point of control flow analysis here--it's not to  
> warn the user when code might be None, but to raise (runtime)  
> exceptions when it is. We can turn that on now with the nonecheck  
> directive, but that comes at a cost. With control flow analysis, one  
> can make nonecheck=True be the default, and the check is only  
> performed the first time a cdef attribute is accessed (in a given  
> path), which will have the advantage of behaving like Python and  
> handling more than arguments without the performance degradation, as  
> well as not segfaulting on bad input.
>   
Also there's the big advantage of allowing code like

def f(MyClass arg=None):
    if arg is None:
        arg = new_default_instance()
    print arg.x

without any "allow None".

Dag Sverre
> I think this is the ideal long term solution, and we shouldn't be  
> changing the default just to change it back later.
>
> - Robert
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>   


From stefan_ml at behnel.de  Tue Apr 13 08:28:32 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 13 Apr 2010 08:28:32 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>
	<4BC3F639.4070200@behnel.de>
	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>
Message-ID: <4BC40F10.9060300@behnel.de>

Robert Bradshaw, 13.04.2010 07:10:
> On Apr 12, 2010, at 9:42 PM, Stefan Behnel wrote:
>> Note that we are only dealing with Python functions here, and only
>> with the
>> case that the input argument is explicitly typed, i.e like this:
>>
>>      def func(MyExtType value):
>>
>> So there already is a rather large performance impact in calling this
>> function, and there already is an explicit annotation saying that
>> the value
>> is an extension type. It won't even hurt cpdef functions, where the
>> type check would happen inside of the Python wrapper.
>
> So you'd have cases where
>
>       (<object>x).func(None) # runtime error
>       (<MyClass>x).func(None) # segfault

The cdef calling convention dictates that type checks and coercions happen 
outside of the function body. So, when you have this:

     cpdef func(MyType x not None):

it's the *calling* code that has to do the None check in addition to its 
type check, not the cdef function implementation. If that doesn't happen 
(didn't check, but I assume it doesn't), it's a bug in the current 
implementation. This is certainly something where control flow analysis can 
help, but as long as that doesn't work reliably, a None check before the 
call is required.


>> I accept the argument that function parameters are not the only
>> dangerous
>> place here and that values received from a function call or global
>> values
>> are best checked as well, so control flow analysis would still be
>> able to
>> print helpful warnings in other places. But the case of function
>> arguments
>> is so important that it's worth giving it distinct semantics.
>
> You are missing the point of control flow analysis here--it's not to
> warn the user when code might be None, but to raise (runtime)
> exceptions when it is.

Isn't that orthogonal to the default behaviour of typed arguments? Or would 
you want to drop the "not None" feature entirely once we can trace variable 
values? I think it's still a feature then, but I agree with Greg that it 
works the wrong way around.

The way it currently works is that you get a type error for *every* 
explicitly typed input value that cannot be converted to the target type, 
*except* for the single case of passing None into an explicitly typed 
extension type argument. So this is the only case that is not safe, and a 
very common case where things can segfault because it's easy to forget 
about. It just doesn't behave the way it looks in the code. The proposed 
change is about fixing exactly this pitfall.

Stefan

From stefan_ml at behnel.de  Tue Apr 13 08:30:55 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 13 Apr 2010 08:30:55 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC40CD6.6080206@student.matnat.uio.no>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>
	<4BC40CD6.6080206@student.matnat.uio.no>
Message-ID: <4BC40F9F.6080204@behnel.de>

Dag Sverre Seljebotn, 13.04.2010 08:19:
> Also there's the big advantage of allowing code like
>
> def f(MyClass arg=None):
>      if arg is None:
>          arg = new_default_instance()
>      print arg.x
>
> without any "allow None".

Sorry, but that case is so trivial to handle implicitly that I can't see 
how this is supposed to be an argument against changing the default.

Stefan

From greg.ewing at canterbury.ac.nz  Tue Apr 13 08:30:50 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 13 Apr 2010 18:30:50 +1200
Subject: [Cython] [PATCH] basic C types in pure mode.
In-Reply-To: <s2je7ba66e41004120605ude84ea05lcbbd9789cbd1664e@mail.gmail.com>
References: <h2ze7ba66e41004091511rdd16c3f4o1abdf5d5dd5e5408@mail.gmail.com>
	<4BC2003A.9020906@behnel.de>
	<p2ve7ba66e41004111657s8b3ca8b0i435ace18eaab55e@mail.gmail.com>
	<4BC2BC88.4040809@canterbury.ac.nz>
	<s2je7ba66e41004120605ude84ea05lcbbd9789cbd1664e@mail.gmail.com>
Message-ID: <4BC40F9A.70808@canterbury.ac.nz>

Lisandro Dalcin wrote:

> # Signedness values
> UNSIGNED = 0
> ...
> 
> # Longness values
> SHORT = -1
> ...
> 
> That very nice, so... Why not take advantage of them to define all the
> c_xxx_type instances?

That's a good idea. I'll do that next time I'm looking at that
piece of code.

-- 
Greg



From greg.ewing at canterbury.ac.nz  Tue Apr 13 08:45:17 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 13 Apr 2010 18:45:17 +1200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC32F89.40808@student.matnat.uio.no>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC2C772.8040000@behnel.de>
	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
	<4BC32F89.40808@student.matnat.uio.no>
Message-ID: <4BC412FD.20708@canterbury.ac.nz>

Dag Sverre Seljebotn wrote:

> I guess I should repeat my big -1 for this then.
> 
> I think Java-like behaviour is much more appropriate (i.e. raise proper 
> exceptions in the code using the variable, but allow None as a value).

This may indeed be a better way for Cython to go, given
its goal of matching Python semantics. I don't think it's
right for Pyrex at the moment, though, because making it
efficient would require rather more analysis than I'm
intending to do in the foreseeable future.

> Let's not break the language forever

I don't think it needs to be broken forever. If I ever
decided to do None-checks at time of use, I could just
stop taking any notice of 'or None' and 'not None'
declarations. They would be allowed so that old code
would still compile, but they would be ignored.

-- 
Greg


From greg.ewing at canterbury.ac.nz  Tue Apr 13 08:57:17 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 13 Apr 2010 18:57:17 +1200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <56FF72E0-7D63-4273-A478-49D0100F6D09@math.washington.edu>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC2C772.8040000@behnel.de>
	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
	<56FF72E0-7D63-4273-A478-49D0100F6D09@math.washington.edu>
Message-ID: <4BC415CD.4080402@canterbury.ac.nz>

Robert Bradshaw wrote:
> On Apr 12, 2010, at 6:35 AM, Lisandro Dalcin wrote:
> 
>>I've addedd __fastcall support long ago. About the other change, I
>>think that not assuming __cdecl as the default is the right thing to
>>do,
> 
> I have no idea what these even mean, so I'll defer to you with a bias  
> towards backwards compatibility.

I don't actually know exactly what the various calling conventions mean
either. The important thing is that the caller and callee have to agree
about them.

The point about unspecified calling conventions is that the default
can be changed with an option to the C compiler, so Pyrex has to be
conservative and assume that if it's not specified, then it's not
compatible with one that is specified.

-- 
Greg



From greg.ewing at canterbury.ac.nz  Tue Apr 13 09:03:00 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 13 Apr 2010 19:03:00 +1200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <BB7E49F6-EC28-4283-A6FD-F3FEAF882794@math.washington.edu>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC2C772.8040000@behnel.de>
	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
	<4BC32F89.40808@student.matnat.uio.no>
	<l2ye7ba66e41004120937j9dde305eye31f4e6a47517a66@mail.gmail.com>
	<4BC34F40.5010101@student.matnat.uio.no>
	<BB7E49F6-EC28-4283-A6FD-F3FEAF882794@math.washington.edu>
Message-ID: <4BC41724.8030809@canterbury.ac.nz>

Robert Bradshaw wrote:

> I'm -1 to changing the default as well--control flow analysis is the  
> way to correctly handle this.

Have you thought through how much analysis is going to be
required? Keep in mind that potential None values don't
just come in via arguments to functions. They can be
stored in data structures, returned by other functions,
etc. etc. Seems to me you would need whole-program
analysis to do this comprehensively.

-- 
Greg


From dagss at student.matnat.uio.no  Tue Apr 13 09:11:38 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 13 Apr 2010 09:11:38 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC40F9F.6080204@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>
	<4BC40F9F.6080204@behnel.de>
Message-ID: <4BC4192A.6040106@student.matnat.uio.no>

Stefan Behnel wrote:
> Dag Sverre Seljebotn, 13.04.2010 08:19:
>> Also there's the big advantage of allowing code like
>>
>> def f(MyClass arg=None):
>>      if arg is None:
>>          arg = new_default_instance()
>>      print arg.x
>>
>> without any "allow None".
> 
> Sorry, but that case is so trivial to handle implicitly that I can't see 
> how this is supposed to be an argument against changing the default.
> 

The argument against changing the default is simply that there is 
another option which is much closer to Python semantics (and IMHO more 
user friendly as the user don't have to care about declaring "not None" 
manually).

Java has had decent "None" behaviour for ages using control flow 
analysis, so it can definitely be done. As Robert says, it is just about 
moving the check (that should happen at some point anyway) to the point 
of first attribute access in a code path and then raise the exception 
there, just like Python would.

-- 
Dag Sverre

From robertwb at math.washington.edu  Tue Apr 13 09:09:32 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 13 Apr 2010 00:09:32 -0700
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC40F10.9060300@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>
	<4BC3F639.4070200@behnel.de>
	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>
	<4BC40F10.9060300@behnel.de>
Message-ID: <4428764B-DEDE-4EA7-9FB4-118722CA87C0@math.washington.edu>

On Apr 12, 2010, at 11:28 PM, Stefan Behnel wrote:

> Robert Bradshaw, 13.04.2010 07:10:
>> On Apr 12, 2010, at 9:42 PM, Stefan Behnel wrote:
>>> Note that we are only dealing with Python functions here, and only
>>> with the
>>> case that the input argument is explicitly typed, i.e like this:
>>>
>>>     def func(MyExtType value):
>>>
>>> So there already is a rather large performance impact in calling  
>>> this
>>> function, and there already is an explicit annotation saying that
>>> the value
>>> is an extension type. It won't even hurt cpdef functions, where the
>>> type check would happen inside of the Python wrapper.
>>
>> So you'd have cases where
>>
>>      (<object>x).func(None) # runtime error
>>      (<MyClass>x).func(None) # segfault
>
> The cdef calling convention dictates that type checks and coercions  
> happen
> outside of the function body. So, when you have this:
>
>     cpdef func(MyType x not None):
>
> it's the *calling* code that has to do the None check in addition to  
> its
> type check, not the cdef function implementation. If that doesn't  
> happen
> (didn't check, but I assume it doesn't), it's a bug in the current
> implementation. This is certainly something where control flow  
> analysis can
> help, but as long as that doesn't work reliably, a None check before  
> the
> call is required.

OK, fair enough.

>>> I accept the argument that function parameters are not the only
>>> dangerous
>>> place here and that values received from a function call or global
>>> values
>>> are best checked as well, so control flow analysis would still be
>>> able to
>>> print helpful warnings in other places. But the case of function
>>> arguments
>>> is so important that it's worth giving it distinct semantics.
>>
>> You are missing the point of control flow analysis here--it's not to
>> warn the user when code might be None, but to raise (runtime)
>> exceptions when it is.
>
> Isn't that orthogonal to the default behaviour of typed arguments?

No, I see it as very related--the whole reason "not none" is being  
promoted is to avoid segfaults. Do you still want None to be rejected  
by default when we have control-flow enabled nonecheck around?

> Or would you want to drop the "not None" feature entirely once we  
> can trace variable values?

We can still keep it around.

> I think it's still a feature then, but I agree with Greg that it
> works the wrong way around.
>
> The way it currently works is that you get a type error for *every*
> explicitly typed input value that cannot be converted to the target  
> type,
> *except* for the single case of passing None into an explicitly typed
> extension type argument. So this is the only case that is not safe,  
> and a
> very common case where things can segfault because it's easy to forget
> about. It just doesn't behave the way it looks in the code. The  
> proposed
> change is about fixing exactly this pitfall.

But this is the case of *every* typed extension type variable, and  
you're proposing special casing python function arguments. I see

def func(MyType x):
     ...

as shorthand for

def func(o):
     cdef MyType x = o
     ...

or at least feel uneasy with them behaving differently.

- Robert



From robertwb at math.washington.edu  Tue Apr 13 09:11:37 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 13 Apr 2010 00:11:37 -0700
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC412FD.20708@canterbury.ac.nz>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC2C772.8040000@behnel.de>
	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
	<4BC32F89.40808@student.matnat.uio.no>
	<4BC412FD.20708@canterbury.ac.nz>
Message-ID: <7D102801-8411-4B9E-B9CC-67800599D79A@math.washington.edu>

On Apr 12, 2010, at 11:45 PM, Greg Ewing wrote:

> Dag Sverre Seljebotn wrote:
>
>> I guess I should repeat my big -1 for this then.
>>
>> I think Java-like behaviour is much more appropriate (i.e. raise  
>> proper
>> exceptions in the code using the variable, but allow None as a  
>> value).
>
> This may indeed be a better way for Cython to go, given
> its goal of matching Python semantics. I don't think it's
> right for Pyrex at the moment, though, because making it
> efficient would require rather more analysis than I'm
> intending to do in the foreseeable future.
>
>> Let's not break the language forever
>
> I don't think it needs to be broken forever. If I ever
> decided to do None-checks at time of use, I could just
> stop taking any notice of 'or None' and 'not None'
> declarations. They would be allowed so that old code
> would still compile, but they would be ignored.

But people would start depending on this, and then ignoring those  
checks would break people's code in other ways.

- Robert


From greg.ewing at canterbury.ac.nz  Tue Apr 13 09:50:22 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 13 Apr 2010 19:50:22 +1200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC2C772.8040000@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC2C772.8040000@behnel.de>
Message-ID: <4BC4223E.8070105@canterbury.ac.nz>

Stefan Behnel wrote:
> +1 for the feature, -1 for the syntax. Python 3 has keyword arguments in 
> the baseclass tuple, so this would be much better here:
> 
>      cdef class Spam(gc=False):
>          cdef object sausage

I'm in two minds about this. In Python, those keyword arguments are
actual expressions that are evaluated at run time and passed to the
type's __init__ method. Options to Pyrex extension types only have
meaning at compile time, and they're all special in different ways.
To me they have quite a different flavour.

> It would become a little funny if "intern(char*)" worked in Py3 but 
> "intern(bytes)" didn't, so I'm reluctant to enabling support for char* in 
> Py3.

If Cython is able to generate different code for an intern() call
depending on what type of argument it's passed, then that's fine.
Pyrex only needed the name change because its builtin function
handling isn't that smart, and I'd rather not spend a lot of effort
on changing it just to handle rare cases like this.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Tue Apr 13 10:17:25 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 13 Apr 2010 20:17:25 +1200
Subject: [Cython] issue while "back-porting" Pyrex change (Greg,
 please comment).
In-Reply-To: <z2je7ba66e41004121455hdc824583l36d515f044c0234d@mail.gmail.com>
References: <z2je7ba66e41004121455hdc824583l36d515f044c0234d@mail.gmail.com>
Message-ID: <4BC42895.9030402@canterbury.ac.nz>

Lisandro Dalcin wrote:
> 
> This is the Pyrex algorithm:
> 
>     if type1.rank < type2.rank:
>         widest_type = type2
>     elif type1.rank > type2.rank:
>         widest_type = type1
>     elif type1.signed < type2.signed:
>         widest_type = type1
>     else:
>         widest_type = type2
> 
> Below I pasted what the C standard says... IIUC, Pyrex does not follow
> it, right?

I'm trying to do the best I can.

> Otherwise, if the type of the operand with signed integer type can represent
> all of the values of the type of the operand with unsigned integer type, then
> the operand with unsigned integer type is converted to the type of the
> operand with signed integer type.

This is the part that neither Pyrex nor Cython can follow exactly,
because it requires knowning the actual sizes of the types. E.g. if
int and long are the same size, then unsigned int + long should be
unsigned long, because signed long can't represent all the values
of unsigned int. But if long is wider than int, then it can, so
the result should be signed long.

 > Pyrex:  <unsigned int> + <long> --> <long>
 > Cython: <unsigned int> + <long> --> <unsigned long>

In other words, Pyrex is guessing that long is wider than int,
which will sometimes be wrong.

But Cython seems to be guessing that anything unsigned is at
least as wide as anything signed, which seems less likely to
be right to me.

The only guaranteed correct thing to do would be to treat the
result as being of unknown signedness in these cases, and to
refuse to do anything with it that would require knowing the
signedness (e.g. choosing which Python conversion function to
call). I might consider doing that in a future version.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Tue Apr 13 11:47:59 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 13 Apr 2010 21:47:59 +1200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC40CD6.6080206@student.matnat.uio.no>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC2C772.8040000@behnel.de>
	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
	<4BC32F89.40808@student.matnat.uio.no> <4BC3F639.4070200@behnel.de>
	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>
	<4BC40CD6.6080206@student.matnat.uio.no>
Message-ID: <4BC43DCF.1060603@canterbury.ac.nz>

Dag Sverre Seljebotn wrote:

> Also there's the big advantage of allowing code like
> 
> def f(MyClass arg=None):
>     if arg is None:
>         arg = new_default_instance()
>     print arg.x

Hmmm, that's a point. Seems like it should be easy enough to
recognise a default value of None and take it as implying
'or None'. I'll add it to my list...

-- 
Greg

From greg.ewing at canterbury.ac.nz  Tue Apr 13 12:02:45 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 13 Apr 2010 22:02:45 +1200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <7D102801-8411-4B9E-B9CC-67800599D79A@math.washington.edu>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC2C772.8040000@behnel.de>
	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
	<4BC32F89.40808@student.matnat.uio.no>
	<4BC412FD.20708@canterbury.ac.nz>
	<7D102801-8411-4B9E-B9CC-67800599D79A@math.washington.edu>
Message-ID: <4BC44145.9000009@canterbury.ac.nz>

Robert Bradshaw wrote:
>> I could just
>>stop taking any notice of 'or None' and 'not None'
>>declarations. They would be allowed so that old code
>>would still compile, but they would be ignored.
> 
> But people would start depending on this, and then ignoring those  
> checks would break people's code in other ways.

In what ways are you imagining that people would depend
on it?

The only way I can see of it breaking code is if someone
relied on a TypeError getting raised at the exact point
where a function was entered, instead of waiting until
a bit later. That seems like a rather unlikely scenario
to me.

-- 
Greg

From stefan_ml at behnel.de  Tue Apr 13 13:13:36 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 13 Apr 2010 13:13:36 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC4192A.6040106@student.matnat.uio.no>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>	<4BC40F9F.6080204@behnel.de>
	<4BC4192A.6040106@student.matnat.uio.no>
Message-ID: <4BC451E0.5090602@behnel.de>

Dag Sverre Seljebotn, 13.04.2010 09:11:
> Stefan Behnel wrote:
>> Dag Sverre Seljebotn, 13.04.2010 08:19:
>>> Also there's the big advantage of allowing code like
>>>
>>> def f(MyClass arg=None):
>>>       if arg is None:
>>>           arg = new_default_instance()
>>>       print arg.x
>>>
>>> without any "allow None".
>>
>> Sorry, but that case is so trivial to handle implicitly that I can't see
>> how this is supposed to be an argument against changing the default.
>
> The argument against changing the default is simply that there is
> another option which is much closer to Python semantics (and IMHO more
> user friendly as the user don't have to care about declaring "not None"
> manually).

Well, but that's the whole point of doing this change. To keep users from 
having to write "not None" manually. Requiring "or None" instead would a) 
immediately drop all dangerous cases where users forget to test for None, 
b) make it obvious in the source where None is a valid value, and c)  allow 
the future control flow analysis to know that the argument cannot be None 
unless specified otherwise, so that it can drop all implicit None tests 
from the source. We could then even emit a warning for explicit "is None" 
tests in the source to make users aware that the value cannot be None at 
this point.

So I really don't see why future control flow analysis capabilities should 
be negatively impacted by this change, or why they would render this 
feature redundant. All this change really does is relieve users from having 
to write "not None" in most of their Python level functions.

Stefan

From stefan_ml at behnel.de  Tue Apr 13 13:32:10 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 13 Apr 2010 13:32:10 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC4192A.6040106@student.matnat.uio.no>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>	<4BC40F9F.6080204@behnel.de>
	<4BC4192A.6040106@student.matnat.uio.no>
Message-ID: <4BC4563A.8020007@behnel.de>

Dag Sverre Seljebotn, 13.04.2010 09:11:
> The argument against changing the default is simply that there is
> another option which is much closer to Python semantics

I think the "another option" bit in this sentence is at the core of our 
disagreement. My point is that incorrect input should be caught early, and 
None is incorrect (and dangerous) input in almost all cases where a typed 
value is expected. So rejecting this dangerous input should be the default.

Your position is that the error should be caught automatically but at a 
later point where it is required to occur in order to prevent a crash. I'm 
perfectly fine with catching this error automatically, but I think this is 
an orthogonal feature that has nothing to do with input value checking and 
coercion.

My point is that for typed arguments, we generate errors in all cases but 
this one, and this particular inconsistency is a very dangerous one that 
leads to very common bugs that are rarely caught by test suites.

Stefan

From dalcinl at gmail.com  Tue Apr 13 16:25:56 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Tue, 13 Apr 2010 11:25:56 -0300
Subject: [Cython] issue while "back-porting" Pyrex change (Greg,
	please 	comment).
In-Reply-To: <4BC42895.9030402@canterbury.ac.nz>
References: <z2je7ba66e41004121455hdc824583l36d515f044c0234d@mail.gmail.com>
	<4BC42895.9030402@canterbury.ac.nz>
Message-ID: <j2we7ba66e41004130725m5f53ce62h32336de8512e115a@mail.gmail.com>

On 13 April 2010 05:17, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Lisandro Dalcin wrote:
>>
>> This is the Pyrex algorithm:
>>
>> ? ? if type1.rank < type2.rank:
>> ? ? ? ? widest_type = type2
>> ? ? elif type1.rank > type2.rank:
>> ? ? ? ? widest_type = type1
>> ? ? elif type1.signed < type2.signed:
>> ? ? ? ? widest_type = type1
>> ? ? else:
>> ? ? ? ? widest_type = type2
>>
>> Below I pasted what the C standard says... IIUC, Pyrex does not follow
>> it, right?
>
> I'm trying to do the best I can.
>

Me too! I was just trying to understand your current promotion
algorithm and the wording in the standard.

>> Otherwise, if the type of the operand with signed integer type can represent
>> all of the values of the type of the operand with unsigned integer type, then
>> the operand with unsigned integer type is converted to the type of the
>> operand with signed integer type.
>
> This is the part that neither Pyrex nor Cython can follow exactly,
> because it requires knowning the actual sizes of the types. E.g. if
> int and long are the same size, then unsigned int + long should be
> unsigned long, because signed long can't represent all the values
> of unsigned int. But if long is wider than int, then it can, so
> the result should be signed long.
>

Yep.

> ?> Pyrex: ?<unsigned int> + <long> --> <long>
> ?> Cython: <unsigned int> + <long> --> <unsigned long>
>
> In other words, Pyrex is guessing that long is wider than int,
> which will sometimes be wrong.
>
> But Cython seems to be guessing that anything unsigned is at
> least as wide as anything signed, which seems less likely to
> be right to me.
>

OK! Now I see your point:

In [1]: from Cython.Compiler.PyrexTypes import *

In [2]: widest_numeric_type (c_uchar_type, c_longlong_type)
Out[2]: <CNumericType unsigned PY_LONG_LONG>

That's clearly wrong (on common platforms)

> The only guaranteed correct thing to do would be to treat the
> result as being of unknown signedness in these cases, and to
> refuse to do anything with it that would require knowing the
> signedness (e.g. choosing which Python conversion function to
> call). I might consider doing that in a future version.
>

Or defer C -> Py conversion to C land, where you can use sizeof()...
But this would be a bit complicate, we will need to manage the
generation of code for binops involving signed and unsigned types...


-- 
Lisandro Dalcin
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dagss at student.matnat.uio.no  Tue Apr 13 16:28:03 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 13 Apr 2010 16:28:03 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC4563A.8020007@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>	<4BC40F9F.6080204@behnel.de>	<4BC4192A.6040106@student.matnat.uio.no>
	<4BC4563A.8020007@behnel.de>
Message-ID: <4BC47F73.4090304@student.matnat.uio.no>

Stefan Behnel wrote:
> Dag Sverre Seljebotn, 13.04.2010 09:11:
>   
>> The argument against changing the default is simply that there is
>> another option which is much closer to Python semantics
>>     
>
> I think the "another option" bit in this sentence is at the core of our 
> disagreement. My point is that incorrect input should be caught early, and 
> None is incorrect (and dangerous) input in almost all cases where a typed 
> value is expected. So rejecting this dangerous input should be the default.
>   

I'll just repeat Robert: I think the core of the disagreement is that

def f(X x):
    ...

should correspond directly to

def f(obj):
    cdef X x = obj

I.e., no special casing of arguments vs. other typed variables.

This is simply because anything else is not going to be obvious to 
anybody learning the language. It's seems very complicated to hold a 
Cython tutorial where I have to explain that "MyType myvar" means 
something else for function arguments than in local variable or class 
field declarations.

If you want to propose that

cdef X x = None

is also disallowed, then that's something else entirely.

Anyway: It's 2 vs. 2 at this point and I don't see anybody changing 
sides, so I hope to resign from the discussion at this point.

Dag Sverre

From dagss at student.matnat.uio.no  Tue Apr 13 16:33:12 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 13 Apr 2010 16:33:12 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC47F73.4090304@student.matnat.uio.no>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>	<4BC40F9F.6080204@behnel.de>	<4BC4192A.6040106@student.matnat.uio.no>	<4BC4563A.8020007@behnel.de>
	<4BC47F73.4090304@student.matnat.uio.no>
Message-ID: <4BC480A8.5000604@student.matnat.uio.no>

Dag Sverre Seljebotn wrote:
> Stefan Behnel wrote:
>   
>> Dag Sverre Seljebotn, 13.04.2010 09:11:
>>   
>>     
>>> The argument against changing the default is simply that there is
>>> another option which is much closer to Python semantics
>>>     
>>>       
>> I think the "another option" bit in this sentence is at the core of our 
>> disagreement. My point is that incorrect input should be caught early, and 
>> None is incorrect (and dangerous) input in almost all cases where a typed 
>> value is expected. So rejecting this dangerous input should be the default.
>>   
>>     
>
> I'll just repeat Robert: I think the core of the disagreement is that
>
> def f(X x):
>     ...
>
> should correspond directly to
>
> def f(obj):
>     cdef X x = obj
>
> I.e., no special casing of arguments vs. other typed variables.
>
> This is simply because anything else is not going to be obvious to 
> anybody learning the language. It's seems very complicated to hold a 
> Cython tutorial where I have to explain that "MyType myvar" means 
> something else for function arguments than in local variable or class 
> field declarations.
>
> If you want to propose that
>
> cdef X x = None
>
> is also disallowed, then that's something else entirely.
>
> Anyway: It's 2 vs. 2 at this point and I don't see anybody changing 
> sides, so I hope to resign from the discussion at this point.
>   
Actually, I overlooked Chris Barker's post in that count, sorry. And 
also there's getting feedback from other users in an appropriately named 
thread etc.

Anyway, I think we can conclude already that this is controversial 
enough to demand a CEP? It's been up several times in the past with the 
same arguments reiterated IIRC.

Dag Sverre

From dalcinl at gmail.com  Tue Apr 13 17:09:14 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Tue, 13 Apr 2010 12:09:14 -0300
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC480A8.5000604@student.matnat.uio.no>
References: <4BC269ED.1070703@canterbury.ac.nz>
	<4BC32F89.40808@student.matnat.uio.no> <4BC3F639.4070200@behnel.de>
	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>
	<4BC40CD6.6080206@student.matnat.uio.no> <4BC40F9F.6080204@behnel.de>
	<4BC4192A.6040106@student.matnat.uio.no> <4BC4563A.8020007@behnel.de>
	<4BC47F73.4090304@student.matnat.uio.no>
	<4BC480A8.5000604@student.matnat.uio.no>
Message-ID: <o2ne7ba66e41004130809pf1d151e9jc04046ccb60fd3c9@mail.gmail.com>

On 13 April 2010 11:33, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:
>>
> Actually, I overlooked Chris Barker's post in that count, sorry. And
> also there's getting feedback from other users in an appropriately named
> thread etc.
>

Does my complains also count as from user-side?

> Anyway, I think we can conclude already that this is controversial
> enough to demand a CEP? It's been up several times in the past with the
> same arguments reiterated IIRC.
>

Given the scarce time Stefan and me have for this, asking for a CEP is
an obvious way to maintain the status-quo ;-) ...

Come on! I really do not understand... "not None" and "or None" are
certainly not part of Python' semantics, nor are type declaration (a
subject people use and abuse here for argumentation)... So, for the
very moment I use a typed argument, I abandoned strict Python
semantics, so I should be able to decide how to manage None in a
case-by-case basis (using "Type arg not None", "Type arg or None", or
"Type arg=None", etc.) and moreover I should be able to choose the
default handling, i.e. use a COMPILER DIRECTIVE!!!

I really cannot understand the opposition to the None handling being
determined by a user-selected compiler directive... Of course, one
this is available, we could have discussions, write CEP's, fight with
swords,  whatever.... about what should the default handling be if
the directive is not used. I would probably not make any strong
argument about this; I'm not a language lawyer; just a user how makes
some contributions to core from time to time, had the experience of
wrapping medium-sized C API's, and is upset of being forced to
manually write "not None" in almost every method! Because if I forget
to write "not None", my code is likely broken!!!

PS: Even if 100% correct control flow is ever implemented and
available, I would still like to use "Type arg not None", as that
exposes errors early-on.


-- 
Lisandro Dalcin
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From stefan_ml at behnel.de  Tue Apr 13 17:12:52 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 13 Apr 2010 17:12:52 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC47F73.4090304@student.matnat.uio.no>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>	<4BC40F9F.6080204@behnel.de>	<4BC4192A.6040106@student.matnat.uio.no>	<4BC4563A.8020007@behnel.de>
	<4BC47F73.4090304@student.matnat.uio.no>
Message-ID: <4BC489F4.2080609@behnel.de>

Dag Sverre Seljebotn, 13.04.2010 16:28:
> Stefan Behnel wrote:
>> Dag Sverre Seljebotn, 13.04.2010 09:11:
>>
>>> The argument against changing the default is simply that there is
>>> another option which is much closer to Python semantics
>>>
>>
>> I think the "another option" bit in this sentence is at the core of our
>> disagreement. My point is that incorrect input should be caught early, and
>> None is incorrect (and dangerous) input in almost all cases where a typed
>> value is expected. So rejecting this dangerous input should be the default.
>
> I'll just repeat Robert: I think the core of the disagreement is that
>
> def f(X x):
>      ...
>
> should correspond directly to
>
> def f(obj):
>      cdef X x = obj
>
> I.e., no special casing of arguments vs. other typed variables.

Yes, I disagree with that, simply because they already are different. You 
can't do

     cdef X not None x = obj

but you can do

     cdef X x = <X?>obj

to mimic the "not None" semantics. You can't do the same thing for 
arguments, though, where the requirement is much more common. That's why 
there is a dedicated syntax, just the wrong one.


> This is simply because anything else is not going to be obvious to
> anybody learning the language. It's seems very complicated to hold a
> Cython tutorial where I have to explain that "MyType myvar" means
> something else for function arguments than in local variable or class
> field declarations.
>
> If you want to propose that
>
> cdef X x = None
>
> is also disallowed, then that's something else entirely.

I'm not proposing that because it doesn't make sense without supporting 
'del' and control flow analysis correctly. Otherwise, it would be 
impossible to reset a typed reference or to initialise it with an 'empty' 
value. Input parameters are an entirely different beast.

I think we should stick to helpful examples, so I guess a CEP is really 
required.

Stefan

From dagss at student.matnat.uio.no  Tue Apr 13 17:17:23 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 13 Apr 2010 17:17:23 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <o2ne7ba66e41004130809pf1d151e9jc04046ccb60fd3c9@mail.gmail.com>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC32F89.40808@student.matnat.uio.no>
	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>
	<4BC40F9F.6080204@behnel.de>	<4BC4192A.6040106@student.matnat.uio.no>
	<4BC4563A.8020007@behnel.de>	<4BC47F73.4090304@student.matnat.uio.no>	<4BC480A8.5000604@student.matnat.uio.no>
	<o2ne7ba66e41004130809pf1d151e9jc04046ccb60fd3c9@mail.gmail.com>
Message-ID: <4BC48B03.2030105@student.matnat.uio.no>

Lisandro Dalcin wrote:
> On 13 April 2010 11:33, Dag Sverre Seljebotn
> <dagss at student.matnat.uio.no> wrote:
>   
>> Actually, I overlooked Chris Barker's post in that count, sorry. And
>> also there's getting feedback from other users in an appropriately named
>> thread etc.
>>
>>     
>
> Does my complains also count as from user-side?
>   
To get 2+2, I counted (Stefan, Lisandro) vs. (Robert, Dag).

Dag Sverre

From dalcinl at gmail.com  Tue Apr 13 17:29:28 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Tue, 13 Apr 2010 12:29:28 -0300
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC48B03.2030105@student.matnat.uio.no>
References: <4BC269ED.1070703@canterbury.ac.nz>
	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>
	<4BC40CD6.6080206@student.matnat.uio.no> <4BC40F9F.6080204@behnel.de>
	<4BC4192A.6040106@student.matnat.uio.no> <4BC4563A.8020007@behnel.de>
	<4BC47F73.4090304@student.matnat.uio.no>
	<4BC480A8.5000604@student.matnat.uio.no>
	<o2ne7ba66e41004130809pf1d151e9jc04046ccb60fd3c9@mail.gmail.com>
	<4BC48B03.2030105@student.matnat.uio.no>
Message-ID: <w2he7ba66e41004130829v84d4154cp72999432b5575bfc@mail.gmail.com>

On 13 April 2010 12:17, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:
> Lisandro Dalcin wrote:
>> On 13 April 2010 11:33, Dag Sverre Seljebotn
>> <dagss at student.matnat.uio.no> wrote:
>>
>>> Actually, I overlooked Chris Barker's post in that count, sorry. And
>>> also there's getting feedback from other users in an appropriately named
>>> thread etc.
>>>
>>>
>>
>> Does my complains also count as from user-side?
>>
> To get 2+2, I counted (Stefan, Lisandro) vs. (Robert, Dag).
>

OK, I got your comment wrong, sorry... I understood you were giving
Chris's comments special relevance because he is an end-user, as if
Stefan and me were just complaining in a more "philosophical" way...
and I think that Stefan and me do complain based in the experience
accumulated by wrapping medium-sized to large C/C++ API's to provide a
Pythonic interface to them.


-- 
Lisandro Dalcin
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From stefan_ml at behnel.de  Tue Apr 13 17:35:18 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 13 Apr 2010 17:35:18 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <w2he7ba66e41004130829v84d4154cp72999432b5575bfc@mail.gmail.com>
References: <4BC269ED.1070703@canterbury.ac.nz>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>
	<4BC40F9F.6080204@behnel.de>	<4BC4192A.6040106@student.matnat.uio.no>
	<4BC4563A.8020007@behnel.de>	<4BC47F73.4090304@student.matnat.uio.no>	<4BC480A8.5000604@student.matnat.uio.no>	<o2ne7ba66e41004130809pf1d151e9jc04046ccb60fd3c9@mail.gmail.com>	<4BC48B03.2030105@student.matnat.uio.no>
	<w2he7ba66e41004130829v84d4154cp72999432b5575bfc@mail.gmail.com>
Message-ID: <4BC48F36.5080202@behnel.de>

Lisandro Dalcin, 13.04.2010 17:29:
> I think that Stefan and me do complain based in the experience
> accumulated by wrapping medium-sized to large C/C++ API's to provide a
> Pythonic interface to them.

Absolutely.

Stefan

From stefan_ml at behnel.de  Tue Apr 13 17:37:02 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 13 Apr 2010 17:37:02 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <o2ne7ba66e41004130809pf1d151e9jc04046ccb60fd3c9@mail.gmail.com>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC32F89.40808@student.matnat.uio.no>
	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>
	<4BC40F9F.6080204@behnel.de>	<4BC4192A.6040106@student.matnat.uio.no>
	<4BC4563A.8020007@behnel.de>	<4BC47F73.4090304@student.matnat.uio.no>	<4BC480A8.5000604@student.matnat.uio.no>
	<o2ne7ba66e41004130809pf1d151e9jc04046ccb60fd3c9@mail.gmail.com>
Message-ID: <4BC48F9E.4080602@behnel.de>

Lisandro Dalcin, 13.04.2010 17:09:
> I really cannot understand the opposition to the None handling being
> determined by a user-selected compiler directive...

There is little use in a directive that everyone uses anyway. After finding 
out about it, that is...

Stefan

From dagss at student.matnat.uio.no  Tue Apr 13 17:51:05 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 13 Apr 2010 17:51:05 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <o2ne7ba66e41004130809pf1d151e9jc04046ccb60fd3c9@mail.gmail.com>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC32F89.40808@student.matnat.uio.no>
	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>
	<4BC40F9F.6080204@behnel.de>	<4BC4192A.6040106@student.matnat.uio.no>
	<4BC4563A.8020007@behnel.de>	<4BC47F73.4090304@student.matnat.uio.no>	<4BC480A8.5000604@student.matnat.uio.no>
	<o2ne7ba66e41004130809pf1d151e9jc04046ccb60fd3c9@mail.gmail.com>
Message-ID: <4BC492E9.5070803@student.matnat.uio.no>

Sorry, I somehow missed the rest of your post...

Lisandro Dalcin wrote:
>> Anyway, I think we can conclude already that this is controversial
>> enough to demand a CEP? It's been up several times in the past with the
>> same arguments reiterated IIRC.
>>
>>     
>
> Given the scarce time Stefan and me have for this, asking for a CEP is
> an obvious way to maintain the status-quo ;-) ...
>   
Heh, I guess...

I actually forgot somewhere in the heat that your request was for a 
compiler directive turned off by default. Sorry about that. Yes, I can 
see that for e.g. lxml and mpi4py such a directive would be more useful 
than it would be for me.

I'll yield my opposition towards a directive turned off by default, 
provided the name is clear enough ("check_arguments_for_none", 
"no_none_arguments"...?).

Dag Sverre

> Come on! I really do not understand... "not None" and "or None" are
> certainly not part of Python' semantics, nor are type declaration (a
> subject people use and abuse here for argumentation)... So, for the
> very moment I use a typed argument, I abandoned strict Python
> semantics, so I should be able to decide how to manage None in a
> case-by-case basis (using "Type arg not None", "Type arg or None", or
> "Type arg=None", etc.) and moreover I should be able to choose the
> default handling, i.e. use a COMPILER DIRECTIVE!!!
>
> I really cannot understand the opposition to the None handling being
> determined by a user-selected compiler directive... Of course, one
> this is available, we could have discussions, write CEP's, fight with
> swords,  whatever.... about what should the default handling be if
> the directive is not used. I would probably not make any strong
> argument about this; I'm not a language lawyer; just a user how makes
> some contributions to core from time to time, had the experience of
> wrapping medium-sized C API's, and is upset of being forced to
> manually write "not None" in almost every method! Because if I forget
> to write "not None", my code is likely broken!!!
>
> PS: Even if 100% correct control flow is ever implemented and
> available, I would still like to use "Type arg not None", as that
> exposes errors early-on.
>
>
>   


From dalcinl at gmail.com  Tue Apr 13 18:05:20 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Tue, 13 Apr 2010 13:05:20 -0300
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC48F9E.4080602@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz>
	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>
	<4BC40CD6.6080206@student.matnat.uio.no> <4BC40F9F.6080204@behnel.de>
	<4BC4192A.6040106@student.matnat.uio.no> <4BC4563A.8020007@behnel.de>
	<4BC47F73.4090304@student.matnat.uio.no>
	<4BC480A8.5000604@student.matnat.uio.no>
	<o2ne7ba66e41004130809pf1d151e9jc04046ccb60fd3c9@mail.gmail.com>
	<4BC48F9E.4080602@behnel.de>
Message-ID: <q2re7ba66e41004130905h62762043pf64c1ff231ea60e4@mail.gmail.com>

On 13 April 2010 12:37, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Lisandro Dalcin, 13.04.2010 17:09:
>> I really cannot understand the opposition to the None handling being
>> determined by a user-selected compiler directive...
>
> There is little use in a directive that everyone uses anyway. After finding
> out about it, that is...
>

I agree with your, but let's start by having something others accept
and we can use ;-) ...



-- 
Lisandro Dalcin
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From robertwb at math.washington.edu  Tue Apr 13 18:27:42 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 13 Apr 2010 09:27:42 -0700
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC489F4.2080609@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>	<4BC40F9F.6080204@behnel.de>	<4BC4192A.6040106@student.matnat.uio.no>	<4BC4563A.8020007@behnel.de>
	<4BC47F73.4090304@student.matnat.uio.no>
	<4BC489F4.2080609@behnel.de>
Message-ID: <E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>

On Apr 13, 2010, at 8:12 AM, Stefan Behnel wrote:

> Dag Sverre Seljebotn, 13.04.2010 16:28:
>> Stefan Behnel wrote:
>>> Dag Sverre Seljebotn, 13.04.2010 09:11:
>>>
>>>> The argument against changing the default is simply that there is
>>>> another option which is much closer to Python semantics
>>>>
>>>
>>> I think the "another option" bit in this sentence is at the core  
>>> of our
>>> disagreement. My point is that incorrect input should be caught  
>>> early, and
>>> None is incorrect (and dangerous) input in almost all cases where  
>>> a typed
>>> value is expected. So rejecting this dangerous input should be the  
>>> default.
>>
>> I'll just repeat Robert: I think the core of the disagreement is that
>>
>> def f(X x):
>>     ...
>>
>> should correspond directly to
>>
>> def f(obj):
>>     cdef X x = obj
>>
>> I.e., no special casing of arguments vs. other typed variables.
>
> Yes, I disagree with that, simply because they already are  
> different. You
> can't do
>
>     cdef X not None x = obj
>
> but you can do
>
>     cdef X x = <X?>obj
>
> to mimic the "not None" semantics.

No, <X?>obj allows None just fine. We could extend the syntax to <X?  
not None> but that's not there now.

- Robert

From Chris.Barker at noaa.gov  Tue Apr 13 18:34:06 2010
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Tue, 13 Apr 2010 09:34:06 -0700
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4428764B-DEDE-4EA7-9FB4-118722CA87C0@math.washington.edu>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC2C772.8040000@behnel.de>
	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
	<4BC32F89.40808@student.matnat.uio.no> <4BC3F639.4070200@behnel.de>
	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>
	<4BC40F10.9060300@behnel.de>
	<4428764B-DEDE-4EA7-9FB4-118722CA87C0@math.washington.edu>
Message-ID: <4BC49CFE.9050408@noaa.gov>

Dag Sverre Seljebotn wrote:

> The argument against changing the default is simply that there is 
> another option which is much closer to Python semantics (and IMHO more 
> user friendly as the user don't have to care about declaring "not None" 
> manually).
> 
> Java has had decent "None" behaviour for ages using control flow 
> analysis, so it can definitely be done.

Java is a far more static language (though I guess we're talking about 
static Cython anyway).

> As Robert says, it is just about 
> moving the check (that should happen at some point anyway) to the point 
> of first attribute access in a code path and then raise the exception 
> there, just like Python would.

This sounds simple enough, but would there really be a limited 
performance hit? I'm a bit over my head here, but in something like this:

cdef func(MyType x):
      ...

     for i in a_really_big_loop:
         if a_conditional:
             do_something_with( x )

wouldn't you need to check if x were None inside that loop? Or could you 
know that it _may_ be used, and therefor check before the loop?

If so, then great, but the current default really is broken.

Dag Sverre Seljebotn wrote:
> It's seems very complicated to hold a 
> Cython tutorial where I have to explain that "MyType myvar" means 
> something else for function arguments than in local variable or class 
> field declarations.

as a newbie, yes, you are right that is confusing, but...

> If you want to propose that
> 
> cdef X x = None
> 
> is also disallowed, then that's something else entirely.

You mean that is allowed? I wouldn't have expected that! I guess I've 
never tried it, nor been bitten by it, but yes, I'd think that would be 
illegal.

For just that the same reason -- I'm statically typing so that I can 
count on an object being a certain type -- if I have to write "is None" 
checks all over the place, what's the point? And anything that can cause 
a segfault is a bug, period.

so, can I now do:

cdef f( int x ):
    ....

def myfunc(x, y):

   cdef int z

   ...

   z = x

   ...

   f( z )

and pass in None for x, and have f( z ) get None? If so, might it crash? 
and if not, where is it caught?


I guess it comes down to this, from this newbie's perspective:

If I type a variable, the code should not crash if I pass in something 
else, anything else, it should coerce or fail with an exception, unless 
I _explicitly_ tell Cython that it could be something else, in which 
case, I'd better have written the code to handle that.

> Anyway: It's 2 vs. 2 at this point 

I really don't care if anyone is counting my "vote" or not, this sort of 
decision shouldn't be made by majority choice anyway.

And I think I am less of a "user" than all of you, but more of a newbie, 
which means I may very well not understand important subtleties.


-Chris






-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

From robertwb at math.washington.edu  Tue Apr 13 18:47:25 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 13 Apr 2010 09:47:25 -0700
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC451E0.5090602@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>	<4BC40F9F.6080204@behnel.de>
	<4BC4192A.6040106@student.matnat.uio.no>
	<4BC451E0.5090602@behnel.de>
Message-ID: <EEBA10ED-5431-4E08-8463-C27E5A38A99D@math.washington.edu>

On Apr 13, 2010, at 4:13 AM, Stefan Behnel wrote:

> Dag Sverre Seljebotn, 13.04.2010 09:11:
>> Stefan Behnel wrote:
>>> Dag Sverre Seljebotn, 13.04.2010 08:19:
>>>> Also there's the big advantage of allowing code like
>>>>
>>>> def f(MyClass arg=None):
>>>>      if arg is None:
>>>>          arg = new_default_instance()
>>>>      print arg.x
>>>>
>>>> without any "allow None".
>>>
>>> Sorry, but that case is so trivial to handle implicitly that I  
>>> can't see
>>> how this is supposed to be an argument against changing the default.
>>
>> The argument against changing the default is simply that there is
>> another option which is much closer to Python semantics (and IMHO  
>> more
>> user friendly as the user don't have to care about declaring "not  
>> None"
>> manually).
>
> Well, but that's the whole point of doing this change. To keep users  
> from
> having to write "not None" manually. Requiring "or None" instead  
> would a)
> immediately drop all dangerous cases where users forget to test for  
> None,
> b) make it obvious in the source where None is a valid value, and  
> c)  allow
> the future control flow analysis to know that the argument cannot be  
> None
> unless specified otherwise, so that it can drop all implicit None  
> tests
> from the source. We could then even emit a warning for explicit "is  
> None"
> tests in the source to make users aware that the value cannot be  
> None at
> this point.

Are you saying that

def foo(X x not None):
       x = None

would raise an error? (I.e. x is now some special type that can only  
be declared in arguments?) I do think such a type could be useful in  
general, but there's the issue of initial values...

> So I really don't see why future control flow analysis capabilities  
> should
> be negatively impacted by this change, or why they would render this
> feature redundant. All this change really does is relieve users from  
> having
> to write "not None" in most of their Python level functions.

Perhaps here is the difference in perspective--I don't need to write  
"not None" in most of my Python level functions--None is a perfectly  
valid value in a lot of the code I write. Control flow analysis  
handles the dangerous part, but your point of view is that even if it  
fails gracefully, allowing None is just an error waiting to happen.

- Robert



From robertwb at math.washington.edu  Tue Apr 13 18:46:18 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 13 Apr 2010 09:46:18 -0700
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC49CFE.9050408@noaa.gov>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC2C772.8040000@behnel.de>
	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
	<4BC32F89.40808@student.matnat.uio.no> <4BC3F639.4070200@behnel.de>
	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>
	<4BC40F10.9060300@behnel.de>
	<4428764B-DEDE-4EA7-9FB4-118722CA87C0@math.washington.edu>
	<4BC49CFE.9050408@noaa.gov>
Message-ID: <FEE8736E-EDD1-4B15-94B0-63DB1468FCBA@math.washington.edu>

On Apr 13, 2010, at 9:34 AM, Christopher Barker wrote:

> Dag Sverre Seljebotn wrote:
>
>> The argument against changing the default is simply that there is
>> another option which is much closer to Python semantics (and IMHO  
>> more
>> user friendly as the user don't have to care about declaring "not  
>> None"
>> manually).
>>
>> Java has had decent "None" behaviour for ages using control flow
>> analysis, so it can definitely be done.
>
> Java is a far more static language (though I guess we're talking about
> static Cython anyway).
>
>> As Robert says, it is just about
>> moving the check (that should happen at some point anyway) to the  
>> point
>> of first attribute access in a code path and then raise the exception
>> there, just like Python would.
>
> This sounds simple enough, but would there really be a limited
> performance hit? I'm a bit over my head here, but in something like  
> this:
>
> cdef func(MyType x):
>      ...
>
>     for i in a_really_big_loop:
>         if a_conditional:
>             do_something_with( x )
>
> wouldn't you need to check if x were None inside that loop? Or could  
> you
> know that it _may_ be used, and therefor check before the loop?

I think we could avoid the performance hit in most, but not all, cases.

> If so, then great, but the current default really is broken.
>
> Dag Sverre Seljebotn wrote:
>> It's seems very complicated to hold a
>> Cython tutorial where I have to explain that "MyType myvar" means
>> something else for function arguments than in local variable or class
>> field declarations.
>
> as a newbie, yes, you are right that is confusing, but...
>
>> If you want to propose that
>>
>> cdef X x = None
>>
>> is also disallowed, then that's something else entirely.
>
> You mean that is allowed? I wouldn't have expected that! I guess I've
> never tried it, nor been bitten by it, but yes, I'd think that would  
> be
> illegal.

The issue is that

cdef X x

and

cdef class Y:
     cdef X x

have to provide some kind of default value for x, and None is really  
the only option here. This means that None can get essentially  
everywhere.

> For just that the same reason -- I'm statically typing so that I can
> count on an object being a certain type -- if I have to write "is  
> None"
> checks all over the place, what's the point? And anything that can  
> cause
> a segfault is a bug, period.

Yes, but we haven't managed to fix that one without slowing everything  
down yet :)

- Robert


From dagss at student.matnat.uio.no  Tue Apr 13 18:53:30 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 13 Apr 2010 18:53:30 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC49CFE.9050408@noaa.gov>
References: <4BC269ED.1070703@canterbury.ac.nz>
	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>
	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40F10.9060300@behnel.de>	<4428764B-DEDE-4EA7-9FB4-118722CA87C0@math.washington.edu>
	<4BC49CFE.9050408@noaa.gov>
Message-ID: <4BC4A18A.4080708@student.matnat.uio.no>

Christopher Barker wrote:
> Dag Sverre Seljebotn wrote:
>
>   
>> The argument against changing the default is simply that there is 
>> another option which is much closer to Python semantics (and IMHO more 
>> user friendly as the user don't have to care about declaring "not None" 
>> manually).
>>
>> Java has had decent "None" behaviour for ages using control flow 
>> analysis, so it can definitely be done.
>>     
>
> Java is a far more static language (though I guess we're talking about 
> static Cython anyway).
>
>   
>> As Robert says, it is just about 
>> moving the check (that should happen at some point anyway) to the point 
>> of first attribute access in a code path and then raise the exception 
>> there, just like Python would.
>>     
>
> This sounds simple enough, but would there really be a limited 
> performance hit? I'm a bit over my head here, but in something like this:
>
> cdef func(MyType x):
>       ...
>
>      for i in a_really_big_loop:
>          if a_conditional:
>              do_something_with( x )
>
> wouldn't you need to check if x were None inside that loop? Or could you 
> know that it _may_ be used, and therefor check before the loop?
>   

Yes, it gets messy -- but one can e.g. unroll the first iteration of the 
loop. Or simpler, accept the hit and document that you should do "assert 
x is not None" prior to a loop...

All I know is that Java does it so it can't be too bad, but the compiler 
may have to be quite sophisticated in order to optimize everything that 
can be optimized.

>> If you want to propose that
>>
>> cdef X x = None
>>
>> is also disallowed, then that's something else entirely.
>>     
>
> You mean that is allowed? I wouldn't have expected that! I guess I've 
> never tried it, nor been bitten by it, but yes, I'd think that would be 
> illegal.
>   
Yes, and that's exactly why I want the above behaviour -- one can in 
principle get bitten by global variables as well as arguments (but 
Lisandro, Stefan and Greg would argue that the majority of the 
None-values come from user code in Python calling Cython code, so that 
they come in through arguments initially. I'm as concerned about just 
having a safeguard against my own bugs, and then global variables etc. 
are more important.).

I coded the nonecheck directive for this reason, but the performance is 
hurt.

The main problem in changing the behaviour is this: How to e.g. write 
the constructor of a cdef class if the typed attributes of self don't 
start off as None? (C++ gets around this with a custom functional-style 
syntax for just this purpose, requiring initialization of all fields 
prior to any procedural code).

Also None is often convenient just to flag "not available" or similar. 
But then it is typically natural to explicitly test for None everywhere 
when using such a variable.

Dag Sverre

From dagss at student.matnat.uio.no  Tue Apr 13 18:55:31 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 13 Apr 2010 18:55:31 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC48F9E.4080602@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC32F89.40808@student.matnat.uio.no>	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>	<4BC40F9F.6080204@behnel.de>	<4BC4192A.6040106@student.matnat.uio.no>	<4BC4563A.8020007@behnel.de>	<4BC47F73.4090304@student.matnat.uio.no>	<4BC480A8.5000604@student.matnat.uio.no>	<o2ne7ba66e41004130809pf1d151e9jc04046ccb60fd3c9@mail.gmail.com>
	<4BC48F9E.4080602@behnel.de>
Message-ID: <4BC4A203.8080507@student.matnat.uio.no>

Stefan Behnel wrote:
> Lisandro Dalcin, 13.04.2010 17:09:
>   
>> I really cannot understand the opposition to the None handling being
>> determined by a user-selected compiler directive...
>>     
>
> There is little use in a directive that everyone uses anyway. After finding 
> out about it, that is...
>   
Well, a lot of my typed arguments *are* None, and has None as a 
perfectly valid value, so I wouldn't enable it.

Dag Sverre

From dalcinl at gmail.com  Tue Apr 13 18:58:47 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Tue, 13 Apr 2010 13:58:47 -0300
Subject: [Cython] promotion of integral types
Message-ID: <t2ve7ba66e41004130958r369208c9y1f46ea5fee96e5c8@mail.gmail.com>

After discussion with Greg in previous thread and careful reading of
C99 std (TC3 from Nov. 2007) , I believe we should do this:

def widest_numeric_type(type1, type2):
    # Given two numeric types, return the narrowest type
    # encompassing both of them.
    if type1.is_complex or type2.is_complex:
        def real_type(ntype):
            if ntype.is_complex:
                return ntype.real_type
            return ntype
        widest_type = CComplexType(
            widest_numeric_type(
                real_type(type1),
                real_type(type2)))
    elif type1.is_enum and type2.is_enum:
        widest_type = c_int_type
    elif type1.rank < type2.rank:
        widest_type = type2
    elif type1.rank > type2.rank:
        widest_type = type1
    elif type1.signed < type2.signed:
        widest_type = type1
    else:
        widest_type = type2
    return widest_type


This algorithm will be wrong for binops mixing signed and unsigned
integrals, because we cannot know if if a signed type can or not
represent all the values of other unsigned type.

As part of my on-going work to cleanup PyrexTypes.py, I propose to use
Greg's algorithm, and adding a warning somewhere would be nice (can
someone can point me were should I look for adding such warning?).

Comments?


-- 
Lisandro Dalcin
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From stefan_ml at behnel.de  Tue Apr 13 19:08:04 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 13 Apr 2010 19:08:04 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC4A203.8080507@student.matnat.uio.no>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC32F89.40808@student.matnat.uio.no>	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>	<4BC40F9F.6080204@behnel.de>	<4BC4192A.6040106@student.matnat.uio.no>	<4BC4563A.8020007@behnel.de>	<4BC47F73.4090304@student.matnat.uio.no>	<4BC480A8.5000604@student.matnat.uio.no>	<o2ne7ba66e41004130809pf1d151e9jc04046ccb60fd3c9@mail.gmail.com>	<4BC48F9E.4080602@behnel.de>
	<4BC4A203.8080507@student.matnat.uio.no>
Message-ID: <4BC4A4F4.2080001@behnel.de>

Dag Sverre Seljebotn, 13.04.2010 18:55:
> Stefan Behnel wrote:
>> Lisandro Dalcin, 13.04.2010 17:09:
>>
>>> I really cannot understand the opposition to the None handling being
>>> determined by a user-selected compiler directive...
>>>
>>
>> There is little use in a directive that everyone uses anyway. After finding
>> out about it, that is...
>>
> Well, a lot of my typed arguments *are* None, and has None as a
> perfectly valid value, so I wouldn't enable it.

Ah, so now we're getting to the 'why' of this discussion. lxml contains 
one(!) function with a typed argument that allows None, and that has a None 
default argument. So we see the world from completely different angles. 
Could you give code examples where typed None arguments make sense for you?

Stefan

From stefan_ml at behnel.de  Tue Apr 13 19:14:20 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 13 Apr 2010 19:14:20 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC49CFE.9050408@noaa.gov>
References: <4BC269ED.1070703@canterbury.ac.nz>
	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>
	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40F10.9060300@behnel.de>	<4428764B-DEDE-4EA7-9FB4-118722CA87C0@math.washington.edu>
	<4BC49CFE.9050408@noaa.gov>
Message-ID: <4BC4A66C.2010209@behnel.de>

Christopher Barker, 13.04.2010 18:34:
> so, can I now do:
>
> cdef f( int x ):
>      ....
>
> def myfunc(x, y):
>
>     cdef int z
>     ...
>     z = x
>     ...
>     f( z )
>
> and pass in None for x, and have f( z ) get None? If so, might it crash?
> and if not, where is it caught?

'f' is a cdef function, so the *caller* sees the type of the argument and 
tries the conversion. Since this fails, the exception will be raised from 
within myfunc() and occur in the code line that calls f().


> I guess it comes down to this, from this newbie's perspective:
>
> If I type a variable, the code should not crash if I pass in something
> else, anything else, it should coerce or fail with an exception, unless
> I _explicitly_ tell Cython that it could be something else, in which
> case, I'd better have written the code to handle that.

That's the theory. However, None is special, and you actually *want* None 
to be allowed in most cases, except for function input values. You 
certainly want to be able to set a typed Python object variable to None to 
delete the reference it contains.


>> Anyway: It's 2 vs. 2 at this point
>
> I really don't care if anyone is counting my "vote" or not, this sort of
> decision shouldn't be made by majority choice anyway.

Agreed.

Stefan

From robertwb at math.washington.edu  Tue Apr 13 19:44:52 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 13 Apr 2010 10:44:52 -0700
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>	<4BC40F9F.6080204@behnel.de>	<4BC4192A.6040106@student.matnat.uio.no>	<4BC4563A.8020007@behnel.de>
	<4BC47F73.4090304@student.matnat.uio.no>
	<4BC489F4.2080609@behnel.de>
	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>
Message-ID: <93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>

OK, Here's what we're going to do:

1) We'll make a directive controlling the behavior of typed arguments.
2) We'll support the "or None" annotation, so the directive will be  
useful both directions.

(Both of these should be very easy to implement.) The pending release,  
however, will keep the default behavior. Even if my preference was to  
change, I think it's too late in the release cycle to change now, and  
this will give users the option of seeing if the proposed change would  
impact their code. Someone should write up a CEP at least summarizing  
the discussion here and in previous threads (if no one beats me to it,  
I will, but not until after I graduate) and then we can re-open the  
discussion of whether its best to change the default from both a  
language design and end user perspective at that point.

- Robert


From dagss at student.matnat.uio.no  Tue Apr 13 19:53:48 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 13 Apr 2010 19:53:48 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC4A4F4.2080001@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC32F89.40808@student.matnat.uio.no>	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>	<4BC40F9F.6080204@behnel.de>	<4BC4192A.6040106@student.matnat.uio.no>	<4BC4563A.8020007@behnel.de>	<4BC47F73.4090304@student.matnat.uio.no>	<4BC480A8.5000604@student.matnat.uio.no>	<o2ne7ba66e41004130809pf1d151e9jc04046ccb60fd3c9@mail.gmail.com>	<4BC48F9E.4080602@behnel.de>	<4BC4A203.8080507@student.matnat.uio.no>
	<4BC4A4F4.2080001@behnel.de>
Message-ID: <4BC4AFAC.6030706@student.matnat.uio.no>

Stefan Behnel wrote:
> Dag Sverre Seljebotn, 13.04.2010 18:55:
>   
>> Stefan Behnel wrote:
>>     
>>> Lisandro Dalcin, 13.04.2010 17:09:
>>>
>>>       
>>>> I really cannot understand the opposition to the None handling being
>>>> determined by a user-selected compiler directive...
>>>>
>>>>         
>>> There is little use in a directive that everyone uses anyway. After finding
>>> out about it, that is...
>>>
>>>       
>> Well, a lot of my typed arguments *are* None, and has None as a
>> perfectly valid value, so I wouldn't enable it.
>>     
>
> Ah, so now we're getting to the 'why' of this discussion. lxml contains 
> one(!) function with a typed argument that allows None, and that has a None 
> default argument. So we see the world from completely different angles. 
> Could you give code examples where typed None arguments make sense for you?
>   
I do this a lot:

def some_operation(np.ndarray[double] a, np.ndarray[double] b, 
np.ndarray[double] out=None):
    if out is None:
        # no output buffer allocated; allocate one
        out = np.zeros_like(a)
    # computation, store result in out
    return out

But there's also stuff like (making up the concrete example):

def some_operation(np.ndarray values, np.ndarray jacobian):
    for k in ...:
        ...
        if jacobian is not None: # include it in the computation
...

These could all be handled by special-casing "arg=None", but I'd really, 
really prefer choosing default values for arguments to be an orthogonal 
issue to this. Perhaps I want to force users into thinking about 
supplying a jacobian or not -- it shouldn't be up to the Cython language.

Dag Sverre

From stefan_ml at behnel.de  Tue Apr 13 20:22:59 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 13 Apr 2010 20:22:59 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC4AFAC.6030706@student.matnat.uio.no>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC32F89.40808@student.matnat.uio.no>	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>	<4BC40F9F.6080204@behnel.de>	<4BC4192A.6040106@student.matnat.uio.no>	<4BC4563A.8020007@behnel.de>	<4BC47F73.4090304@student.matnat.uio.no>	<4BC480A8.5000604@student.matnat.uio.no>	<o2ne7ba66e41004130809pf1d151e9jc04046ccb60fd3c9@mail.gmail.com>	<4BC48F9E.4080602@behnel.de>	<4BC4A203.8080507@student.matnat.uio.no>	<4BC4A4F4.2080001@behnel.de>
	<4BC4AFAC.6030706@student.matnat.uio.no>
Message-ID: <4BC4B683.2090304@behnel.de>

Dag Sverre Seljebotn, 13.04.2010 19:53:
> def some_operation(np.ndarray[double] a, np.ndarray[double] b,
>   np.ndarray[double] out=None):
>      if out is None:
>          # no output buffer allocated; allocate one
>          out = np.zeros_like(a)
>      # computation, store result in out
>      return out

Hmm, I expect that you still want a and b to be non-None, don't you? So 
assuming the proposed change, the above would have two bugs less and 
otherwise run out-of-the-box.


> But there's also stuff like (making up the concrete example):
>
> def some_operation(np.ndarray values, np.ndarray jacobian):
>      for k in ...:
>          ...
>          if jacobian is not None: # include it in the computation
> ...
>
> These could all be handled by special-casing "arg=None", but I'd really,
> really prefer choosing default values for arguments to be an orthogonal
> issue to this.

It's not completely orthogonal. If you use "np.ndarray jacobian=None", this 
would get expanded to "ns.ndarray jacobian or None = None" (ugly as it 
looks). However, in the above example, you'd have to be explicit and say 
"np.ndarray jacobian or None", so your code would need to change. I 
actually find that "or None" represents your intention very well, much 
better than a plain "np.ndarray jacobian", which suggests that the argument 
must be of type "np.ndarray" (especially when read in the autogenerated 
docstring signature).

Stefan

From greg.ewing at canterbury.ac.nz  Wed Apr 14 02:22:30 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 14 Apr 2010 12:22:30 +1200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC4A18A.4080708@student.matnat.uio.no>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC2C772.8040000@behnel.de>
	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
	<4BC32F89.40808@student.matnat.uio.no> <4BC3F639.4070200@behnel.de>
	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>
	<4BC40F10.9060300@behnel.de>
	<4428764B-DEDE-4EA7-9FB4-118722CA87C0@math.washington.edu>
	<4BC49CFE.9050408@noaa.gov> <4BC4A18A.4080708@student.matnat.uio.no>
Message-ID: <4BC50AC6.3000709@canterbury.ac.nz>

Dag Sverre Seljebotn wrote:
> one can in 
> principle get bitten by global variables as well as arguments (but 
> Lisandro, Stefan and Greg would argue that the majority of the 
> None-values come from user code in Python calling Cython code

Typed global variables are not exposed to Python, so Python
code can't stick None values directly into them.

It's not so much that the "majority" of None values are assumed
to come from Python code, but that the Pyrex programmer is
assumed to know what he's doing with his own code and refrain
from misusing None. Python code, on the other hand, lives out
there beyond the pale and can't be trusted.

-- 
Greg

From dagss at student.matnat.uio.no  Wed Apr 14 10:01:12 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Wed, 14 Apr 2010 10:01:12 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC50AC6.3000709@canterbury.ac.nz>
References: <4BC269ED.1070703@canterbury.ac.nz>
	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>
	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40F10.9060300@behnel.de>	<4428764B-DEDE-4EA7-9FB4-118722CA87C0@math.washington.edu>	<4BC49CFE.9050408@noaa.gov>
	<4BC4A18A.4080708@student.matnat.uio.no>
	<4BC50AC6.3000709@canterbury.ac.nz>
Message-ID: <4BC57648.9000301@student.matnat.uio.no>

Greg Ewing wrote:
> Dag Sverre Seljebotn wrote:
>   
>> one can in 
>> principle get bitten by global variables as well as arguments (but 
>> Lisandro, Stefan and Greg would argue that the majority of the 
>> None-values come from user code in Python calling Cython code
>>     
>
> Typed global variables are not exposed to Python, so Python
> code can't stick None values directly into them.
>
> It's not so much that the "majority" of None values are assumed
> to come from Python code, but that the Pyrex programmer is
> assumed to know what he's doing with his own code and refrain
> from misusing None. Python code, on the other hand, lives out
> there beyond the pale and can't be trusted.
>   
Yes, that was my point: I'm fallible, I make bugs, sometimes I don't 
know what I'm doing :-)

Dag Sverre

From stefan_ml at behnel.de  Wed Apr 14 10:37:05 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 14 Apr 2010 10:37:05 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC57648.9000301@student.matnat.uio.no>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40F10.9060300@behnel.de>	<4428764B-DEDE-4EA7-9FB4-118722CA87C0@math.washington.edu>	<4BC49CFE.9050408@noaa.gov>	<4BC4A18A.4080708@student.matnat.uio.no>	<4BC50AC6.3000709@canterbury.ac.nz>
	<4BC57648.9000301@student.matnat.uio.no>
Message-ID: <4BC57EB1.3040906@behnel.de>

Dag Sverre Seljebotn, 14.04.2010 10:01:
> Greg Ewing wrote:
>> Dag Sverre Seljebotn wrote:
>>
>>> one can in
>>> principle get bitten by global variables as well as arguments (but
>>> Lisandro, Stefan and Greg would argue that the majority of the
>>> None-values come from user code in Python calling Cython code
>>
>> Typed global variables are not exposed to Python, so Python
>> code can't stick None values directly into them.
>>
>> It's not so much that the "majority" of None values are assumed
>> to come from Python code, but that the Pyrex programmer is
>> assumed to know what he's doing with his own code and refrain
>> from misusing None. Python code, on the other hand, lives out
>> there beyond the pale and can't be trusted.
>>
> Yes, that was my point: I'm fallible, I make bugs, sometimes I don't
> know what I'm doing :-)

Sure, and the proposed change is targeting one common source of those bugs.

Stefan

From dagss at student.matnat.uio.no  Wed Apr 14 11:09:51 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Wed, 14 Apr 2010 11:09:51 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC4B683.2090304@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC32F89.40808@student.matnat.uio.no>	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>	<4BC40F9F.6080204@behnel.de>	<4BC4192A.6040106@student.matnat.uio.no>	<4BC4563A.8020007@behnel.de>	<4BC47F73.4090304@student.matnat.uio.no>	<4BC480A8.5000604@student.matnat.uio.no>	<o2ne7ba66e41004130809pf1d151e9jc04046ccb60fd3c9@mail.gmail.com>	<4BC48F9E.4080602@behnel.de>	<4BC4A203.8080507@student.matnat.uio.no>	<4BC4A4F4.2080001@behnel.de>	<4BC4AFAC.6030706@student.matnat.uio.no>
	<4BC4B683.2090304@behnel.de>
Message-ID: <4BC5865F.3010509@student.matnat.uio.no>

Stefan Behnel wrote:
> Dag Sverre Seljebotn, 13.04.2010 19:53:
>   
>> def some_operation(np.ndarray[double] a, np.ndarray[double] b,
>>   np.ndarray[double] out=None):
>>      if out is None:
>>          # no output buffer allocated; allocate one
>>          out = np.zeros_like(a)
>>      # computation, store result in out
>>      return out
>>     
>
> Hmm, I expect that you still want a and b to be non-None, don't you? So 
> assuming the proposed change, the above would have two bugs less and 
> otherwise run out-of-the-box.
>
>
>   
>> But there's also stuff like (making up the concrete example):
>>
>> def some_operation(np.ndarray values, np.ndarray jacobian):
>>      for k in ...:
>>          ...
>>          if jacobian is not None: # include it in the computation
>> ...
>>
>> These could all be handled by special-casing "arg=None", but I'd really,
>> really prefer choosing default values for arguments to be an orthogonal
>> issue to this.
>>     
>
> It's not completely orthogonal. If you use "np.ndarray jacobian=None", this 
> would get expanded to "ns.ndarray jacobian or None = None" (ugly as it 
> looks). However, in the above example, you'd have to be explicit and say 
> "np.ndarray jacobian or None", so your code would need to change. I 
> actually find that "or None" represents your intention very well, much 
> better than a plain "np.ndarray jacobian", which suggests that the argument 
> must be of type "np.ndarray" (especially when read in the autogenerated 
> docstring signature).
>   
Sure. You make good points.

But it can still be compared to Cython-with-control-flow-analysis, at 
which point I don't have to bother about "or None" or "not None" at all, 
meaning one less thing to think about (and learn!) with respect to pure 
Python. (I almost always write my code in Python first and only 
Cythonize if needed -- anything that can be kept the same between pure 
Python and Cython is good.)

So

1) I agree that in the absence of control flow analysis, requiring "or 
None" would be a much safer default behavior (no accidental segfaults). 
I certainly wouldn't have voted for today's semantics!

2) But, one presumably want control flow analysis eventually, and it 
seems bad to change behaviour twice, however minor the changes are 
(point at which exception is raised and its type).

Anyway, I think/hope we understand each other now. I'm all for Robert's 
suggestion of doing a compiler directive first and reviving this debate 
later.

Dag Sverre

From stefan_ml at behnel.de  Wed Apr 14 11:31:53 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 14 Apr 2010 11:31:53 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>	<4BC40F9F.6080204@behnel.de>	<4BC4192A.6040106@student.matnat.uio.no>	<4BC4563A.8020007@behnel.de>	<4BC47F73.4090304@student.matnat.uio.no>	<4BC489F4.2080609@behnel.de>
	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>
Message-ID: <4BC58B89.705@behnel.de>

Robert Bradshaw, 13.04.2010 18:27:
> <X?>obj allows None just fine.

Hmmm, that sounds pretty dangerous, too. It really doesn't look like it does.

Stefan

From stefan_ml at behnel.de  Wed Apr 14 11:54:56 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 14 Apr 2010 11:54:56 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>	<4BC40F9F.6080204@behnel.de>	<4BC4192A.6040106@student.matnat.uio.no>	<4BC4563A.8020007@behnel.de>	<4BC47F73.4090304@student.matnat.uio.no>	<4BC489F4.2080609@behnel.de>	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>
	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>
Message-ID: <4BC590F0.1090707@behnel.de>

Robert Bradshaw, 13.04.2010 19:44:
> OK, Here's what we're going to do:
>
> 1) We'll make a directive controlling the behavior of typed arguments.
> 2) We'll support the "or None" annotation, so the directive will be
> useful both directions.
>
> (Both of these should be very easy to implement.) The pending release,
> however, will keep the default behavior. Even if my preference was to
> change, I think it's too late in the release cycle to change now, and
> this will give users the option of seeing if the proposed change would
> impact their code. Someone should write up a CEP at least summarizing
> the discussion here and in previous threads (if no one beats me to it,
> I will, but not until after I graduate) and then we can re-open the
> discussion of whether its best to change the default from both a
> language design and end user perspective at that point.

I've written up a CEP for now:

http://wiki.cython.org/enhancements/argumentnonechecks

Stefan

From dagss at student.matnat.uio.no  Wed Apr 14 12:42:56 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Wed, 14 Apr 2010 12:42:56 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC590F0.1090707@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>	<4BC40F9F.6080204@behnel.de>	<4BC4192A.6040106@student.matnat.uio.no>	<4BC4563A.8020007@behnel.de>	<4BC47F73.4090304@student.matnat.uio.no>	<4BC489F4.2080609@behnel.de>	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>
	<4BC590F0.1090707@behnel.de>
Message-ID: <4BC59C30.1020208@student.matnat.uio.no>

Stefan Behnel wrote:
> Robert Bradshaw, 13.04.2010 19:44:
>   
>> OK, Here's what we're going to do:
>>
>> 1) We'll make a directive controlling the behavior of typed arguments.
>> 2) We'll support the "or None" annotation, so the directive will be
>> useful both directions.
>>
>> (Both of these should be very easy to implement.) The pending release,
>> however, will keep the default behavior. Even if my preference was to
>> change, I think it's too late in the release cycle to change now, and
>> this will give users the option of seeing if the proposed change would
>> impact their code. Someone should write up a CEP at least summarizing
>> the discussion here and in previous threads (if no one beats me to it,
>> I will, but not until after I graduate) and then we can re-open the
>> discussion of whether its best to change the default from both a
>> language design and end user perspective at that point.
>>     
>
> I've written up a CEP for now:
>
> http://wiki.cython.org/enhancements/argumentnonechecks
>   
The surge protection is hitting me for some reason... and I didn't think 
I'm even behind a NAT...

Dag Sverre

From stefan_ml at behnel.de  Wed Apr 14 16:56:53 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 14 Apr 2010 16:56:53 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>	<4BC40F9F.6080204@behnel.de>	<4BC4192A.6040106@student.matnat.uio.no>	<4BC4563A.8020007@behnel.de>	<4BC47F73.4090304@student.matnat.uio.no>	<4BC489F4.2080609@behnel.de>	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>
	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>
Message-ID: <4BC5D7B5.8090102@behnel.de>

Robert Bradshaw, 13.04.2010 19:44:
> OK, Here's what we're going to do:
>
> 1) We'll make a directive controlling the behavior of typed arguments.
> 2) We'll support the "or None" annotation, so the directive will be
> useful both directions.

http://hg.cython.org/cython-devel/rev/5dfcb27621fe

I kept both flags as attributes in the argument declarator class, that 
makes it easier to drop one of them in case we decide to do so.

Stefan

From dalcinl at gmail.com  Wed Apr 14 17:52:55 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 14 Apr 2010 12:52:55 -0300
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC5D7B5.8090102@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz>
	<4BC40CD6.6080206@student.matnat.uio.no> <4BC40F9F.6080204@behnel.de>
	<4BC4192A.6040106@student.matnat.uio.no> <4BC4563A.8020007@behnel.de>
	<4BC47F73.4090304@student.matnat.uio.no> <4BC489F4.2080609@behnel.de>
	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>
	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>
	<4BC5D7B5.8090102@behnel.de>
Message-ID: <m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>

On 14 April 2010 11:56, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Robert Bradshaw, 13.04.2010 19:44:
>> OK, Here's what we're going to do:
>>
>> 1) We'll make a directive controlling the behavior of typed arguments.
>> 2) We'll support the "or None" annotation, so the directive will be
>> useful both directions.
>
> http://hg.cython.org/cython-devel/rev/5dfcb27621fe
>

Tests?


-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From Chris.Barker at noaa.gov  Wed Apr 14 18:21:40 2010
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Wed, 14 Apr 2010 09:21:40 -0700
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC590F0.1090707@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC2C772.8040000@behnel.de>
	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
	<4BC32F89.40808@student.matnat.uio.no> <4BC3F639.4070200@behnel.de>
	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>
	<4BC40CD6.6080206@student.matnat.uio.no> <4BC40F9F.6080204@behnel.de>
	<4BC4192A.6040106@student.matnat.uio.no> <4BC4563A.8020007@behnel.de>
	<4BC47F73.4090304@student.matnat.uio.no> <4BC489F4.2080609@behnel.de>
	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>
	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>
	<4BC590F0.1090707@behnel.de>
Message-ID: <4BC5EB94.7060009@noaa.gov>

Stefan Behnel wrote:
> I've written up a CEP for now:
> 
> http://wiki.cython.org/enhancements/argumentnonechecks

great, thanks!

A couple comments:


"""
Typed argument values that must reject None are more common than those 
than can safely allow them (without being optional arguments). So the 
current syntax requires more overhead on the user side than the proposed 
syntax.
"""

This seems to be very much style dependent. If I followed the 
conversation right, Lisandro (and I) would use a lot of "not None", 
whereas Dag would use a lot of "or None". So I think the more compelling 
argument is that leaving out a "not None" could result in a segfault, 
whereas leaving out an "or None" would result in an exception, and be 
more likely to be caught in tests.


About "or None": None is a very common and useful case for an extra type 
one might want, but it's not the only one -- is there a way to tell 
Cython that you want to accept one of two (or a few) types?



"""Changing the behavior of cdef MyExtType x is not an option as the 
variable requires an initial value and users must be able to set it to a 
neutral value to destroy the reference they contain.
"""

This is probably a result of my limited understanding, but I'm still a 
bit confused about this -- there has got to be a less dangerous way to 
do that -- after all, people write a lot of code in C without needing it.
  - Can't you destroy the reference with a "del"?
  - It doesn't seem onerous to me that an initial value needs to be of 
the type specified -- that's the cost of static typing.


Dag Sverre Seljebotn wrote:
> But it can still be compared to Cython-with-control-flow-analysis, at 
> which point I don't have to bother about "or None" or "not None" at all, 
> meaning one less thing to think about (and learn!) with respect to pure 
> Python.

Yes, that will be wonderful -- but are we talking months or years before 
it's a reality?

-Chris








-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

From stefan_ml at behnel.de  Wed Apr 14 18:22:58 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 14 Apr 2010 18:22:58 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC40CD6.6080206@student.matnat.uio.no>
	<4BC40F9F.6080204@behnel.de>	<4BC4192A.6040106@student.matnat.uio.no>
	<4BC4563A.8020007@behnel.de>	<4BC47F73.4090304@student.matnat.uio.no>
	<4BC489F4.2080609@behnel.de>	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>	<4BC5D7B5.8090102@behnel.de>
	<m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>
Message-ID: <4BC5EBE2.5030005@behnel.de>

Lisandro Dalcin, 14.04.2010 17:52:
> On 14 April 2010 11:56, Stefan Behnel wrote:
>> Robert Bradshaw, 13.04.2010 19:44:
>>> OK, Here's what we're going to do:
>>>
>>> 1) We'll make a directive controlling the behavior of typed arguments.
>>> 2) We'll support the "or None" annotation, so the directive will be
>>> useful both directions.
>>
>> http://hg.cython.org/cython-devel/rev/5dfcb27621fe
>
> Tests?

Hmm, thanks for catching this. It looks like there is one important case 
that I didn't consider. How should this behave:

     def func(list L):
         ...

Should builtins continue to accept None or should it behave like other 
extension types? For consistency, I think it should be for the latter, with 
the obvious exception of "object".

Stefan

From dagss at student.matnat.uio.no  Wed Apr 14 18:51:11 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Wed, 14 Apr 2010 18:51:11 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC5EB94.7060009@noaa.gov>
References: <4BC269ED.1070703@canterbury.ac.nz>
	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>
	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>
	<4BC40F9F.6080204@behnel.de>	<4BC4192A.6040106@student.matnat.uio.no>
	<4BC4563A.8020007@behnel.de>	<4BC47F73.4090304@student.matnat.uio.no>
	<4BC489F4.2080609@behnel.de>	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>	<4BC590F0.1090707@behnel.de>
	<4BC5EB94.7060009@noaa.gov>
Message-ID: <4BC5F27F.5090401@student.matnat.uio.no>

Christopher Barker wrote:
> Stefan Behnel wrote:
>   
>> I've written up a CEP for now:
>>
>> http://wiki.cython.org/enhancements/argumentnonechecks
>>     
>
> great, thanks!
>
> A couple comments:
>
>
> """
> Typed argument values that must reject None are more common than those 
> than can safely allow them (without being optional arguments). So the 
> current syntax requires more overhead on the user side than the proposed 
> syntax.
> """
>
> This seems to be very much style dependent. If I followed the 
> conversation right, Lisandro (and I) would use a lot of "not None", 
> whereas Dag would use a lot of "or None". So I think the more compelling 
> argument is that leaving out a "not None" could result in a segfault, 
> whereas leaving out an "or None" would result in an exception, and be 
> more likely to be caught in tests.
>
>
> About "or None": None is a very common and useful case for an extra type 
> one might want, but it's not the only one -- is there a way to tell 
> Cython that you want to accept one of two (or a few) types?
>
>
>
> """Changing the behavior of cdef MyExtType x is not an option as the 
> variable requires an initial value and users must be able to set it to a 
> neutral value to destroy the reference they contain.
> """
>
> This is probably a result of my limited understanding, but I'm still a 
> bit confused about this -- there has got to be a less dangerous way to 
> do that -- after all, people write a lot of code in C without needing it.
>   
I believe you are wrong:

a) Pointers in C can always be NULL regardless of the pointer type.
b) Things (including pointers) are typically initialized to totally 
random garbage in C. This is worse than the current Cython behaviour. At 
least, with the current Cython behaviour, it is predictable that you'll 
in 99.9% of the cases get a segfault, rather than corrupt data.

Also, in Java, references can always be NULL. C++ has a reference type 
which is guaranteed to be correctly initialized, but

a) You cannot ever reassign a reference
b) There's a different constructor syntax from Cython:

class Cls {
private:
    int value;
    int& ref_to_value;
public:
    Cls() : ref_to_value(value) {
        // Setting ref_to_value here is illegal
    }
};

Dag Sverre

>   - Can't you destroy the reference with a "del"?
>   - It doesn't seem onerous to me that an initial value needs to be of 
> the type specified -- that's the cost of static typing.
>
>
> Dag Sverre Seljebotn wrote:
>   
>> But it can still be compared to Cython-with-control-flow-analysis, at 
>> which point I don't have to bother about "or None" or "not None" at all, 
>> meaning one less thing to think about (and learn!) with respect to pure 
>> Python.
>>     
>
> Yes, that will be wonderful -- but are we talking months or years before 
> it's a reality?
>
> -Chris
>
>
>
>
>
>
>
>
>   


From dalcinl at gmail.com  Wed Apr 14 19:23:10 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 14 Apr 2010 14:23:10 -0300
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC5EBE2.5030005@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz>
	<4BC4192A.6040106@student.matnat.uio.no> <4BC4563A.8020007@behnel.de>
	<4BC47F73.4090304@student.matnat.uio.no> <4BC489F4.2080609@behnel.de>
	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>
	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>
	<4BC5D7B5.8090102@behnel.de>
	<m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>
	<4BC5EBE2.5030005@behnel.de>
Message-ID: <u2oe7ba66e41004141023vbcef49a8k68b1313052f12756@mail.gmail.com>

On 14 April 2010 13:22, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Lisandro Dalcin, 14.04.2010 17:52:
>> On 14 April 2010 11:56, Stefan Behnel wrote:
>>> Robert Bradshaw, 13.04.2010 19:44:
>>>> OK, Here's what we're going to do:
>>>>
>>>> 1) We'll make a directive controlling the behavior of typed arguments.
>>>> 2) We'll support the "or None" annotation, so the directive will be
>>>> useful both directions.
>>>
>>> http://hg.cython.org/cython-devel/rev/5dfcb27621fe
>>
>> Tests?
>
> Hmm, thanks for catching this.

It was more than a catch, you actually broke things ;-), and in turn I
broke  it further because I did not noticed my own mistakes by running
the testsuite...

Please hg pull && up, and try, for example, this one:

$ python runtests.py tupleassign


-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From stefan_ml at behnel.de  Wed Apr 14 19:25:20 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 14 Apr 2010 19:25:20 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC5EB94.7060009@noaa.gov>
References: <4BC269ED.1070703@canterbury.ac.nz>
	<4BC2C772.8040000@behnel.de>	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>	<4BC32F89.40808@student.matnat.uio.no>
	<4BC3F639.4070200@behnel.de>	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>	<4BC40CD6.6080206@student.matnat.uio.no>
	<4BC40F9F.6080204@behnel.de>	<4BC4192A.6040106@student.matnat.uio.no>
	<4BC4563A.8020007@behnel.de>	<4BC47F73.4090304@student.matnat.uio.no>
	<4BC489F4.2080609@behnel.de>	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>	<4BC590F0.1090707@behnel.de>
	<4BC5EB94.7060009@noaa.gov>
Message-ID: <4BC5FA80.3020703@behnel.de>

Christopher Barker, 14.04.2010 18:21:
> """
> Typed argument values that must reject None are more common than those
> than can safely allow them (without being optional arguments). So the
> current syntax requires more overhead on the user side than the proposed
> syntax.
> """
>
> This seems to be very much style dependent. If I followed the
> conversation right, Lisandro (and I) would use a lot of "not None",
> whereas Dag would use a lot of "or None".

I still have some doubts that the example that Dag presented comes close 
enough to a majority. Python has optional arguments, so it's extremely rare 
that I need to pass None into a function explicitly. In many, many cases 
where I accept None, there will be a None default value, which the change 
would handle properly (and, BTW, that can already be implemented now, so I 
will do it).

In most Python code I have seen so far, required arguments that received a 
None value eventually resulted in an exception at some point. I doubt that 
this is substantially different in Cython code.


> So I think the more compelling
> argument is that leaving out a "not None" could result in a segfault,
> whereas leaving out an "or None" would result in an exception, and be
> more likely to be caught in tests.

I agree, and that argument is already in the CEP.


> About "or None": None is a very common and useful case for an extra type
> one might want, but it's not the only one -- is there a way to tell
> Cython that you want to accept one of two (or a few) types?

That has been discussed back in the early days but never got us anywhere 
AFAIR. I would expect that such a discussion would be redirected to this 
page nowadays:

http://wiki.cython.org/enhancements/methodtemplates


> """Changing the behavior of cdef MyExtType x is not an option as the
> variable requires an initial value and users must be able to set it to a
> neutral value to destroy the reference they contain.
> """
>
> This is probably a result of my limited understanding, but I'm still a
> bit confused about this -- there has got to be a less dangerous way to
> do that -- after all, people write a lot of code in C without needing it.
>    - Can't you destroy the reference with a "del"?

Not in current Cython. However, this could be faked even without flow 
control analysis by setting the variable to NULL on deletion and adding 
NULL checks (and an appropriate exception) to each access of the variable. 
Since this would only apply to names that are actually del-ed within a 
function, the performance hit wouldn't even be /that/ high.

It's not that trivial to implement, though, because you'd also have to take 
nogil blocks into account etc. Getting proper flow control analysis working 
is certainly a better thing to spend your time on.


>    - It doesn't seem onerous to me that an initial value needs to be of
> the type specified -- that's the cost of static typing.

Python objects are different, though, and None is a *very* handy value that 
you really want to be able to assign to a typed name, believe me.

In some cases, e.g. when the extension type guards certain resources, you 
won't even be able to use an instance of the extension type itself as 
None-like value, because you can't easily instantiate it.


> Dag Sverre Seljebotn wrote:
>> But it can still be compared to Cython-with-control-flow-analysis, at
>> which point I don't have to bother about "or None" or "not None" at all,
>> meaning one less thing to think about (and learn!) with respect to pure
>> Python.
>
> Yes, that will be wonderful -- but are we talking months or years before
> it's a reality?

Since no-one has started working on this yet, and there are no known plans 
to do so in the near future, I'd expect 'years' not to be too far away from 
the truth, although 'months' could be possible with the right combination 
of developer time and interest.

Stefan

From stefan_ml at behnel.de  Wed Apr 14 19:31:36 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 14 Apr 2010 19:31:36 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <u2oe7ba66e41004141023vbcef49a8k68b1313052f12756@mail.gmail.com>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC4192A.6040106@student.matnat.uio.no>
	<4BC4563A.8020007@behnel.de>	<4BC47F73.4090304@student.matnat.uio.no>
	<4BC489F4.2080609@behnel.de>	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>	<4BC5D7B5.8090102@behnel.de>	<m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>	<4BC5EBE2.5030005@behnel.de>
	<u2oe7ba66e41004141023vbcef49a8k68b1313052f12756@mail.gmail.com>
Message-ID: <4BC5FBF8.5030303@behnel.de>

Lisandro Dalcin, 14.04.2010 19:23:
> On 14 April 2010 13:22, Stefan Behnel wrote:
>> Lisandro Dalcin, 14.04.2010 17:52:
>>> On 14 April 2010 11:56, Stefan Behnel wrote:
>>>> Robert Bradshaw, 13.04.2010 19:44:
>>>>> OK, Here's what we're going to do:
>>>>>
>>>>> 1) We'll make a directive controlling the behavior of typed arguments.
>>>>> 2) We'll support the "or None" annotation, so the directive will be
>>>>> useful both directions.
>>>>
>>>> http://hg.cython.org/cython-devel/rev/5dfcb27621fe
>>>
>>> Tests?
>>
>> Hmm, thanks for catching this.
>
> It was more than a catch, you actually broke things ;-)

I know. I don't think I told you yet, but we have a running Hudson-CI 
instance for Cython that automatically builds it against all major Python 
versions after a push to cython-devel, and then runs the test suite over it 
in parallel. I just hadn't taken a look over there yet, and I had only run 
the obviously relevant tests on my side before the commit, not the entire 
test suite. But the test results were just a couple of mouse clicks away 
for me after your hint.

Stefan

From robertwb at math.washington.edu  Wed Apr 14 19:35:13 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Wed, 14 Apr 2010 10:35:13 -0700
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC5EB94.7060009@noaa.gov>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC2C772.8040000@behnel.de>
	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
	<4BC32F89.40808@student.matnat.uio.no> <4BC3F639.4070200@behnel.de>
	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>
	<4BC40CD6.6080206@student.matnat.uio.no>
	<4BC40F9F.6080204@behnel.de>
	<4BC4192A.6040106@student.matnat.uio.no>
	<4BC4563A.8020007@behnel.de>
	<4BC47F73.4090304@student.matnat.uio.no>
	<4BC489F4.2080609@behnel.de>
	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>
	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>
	<4BC590F0.1090707@behnel.de> <4BC5EB94.7060009@noaa.gov>
Message-ID: <BCA66573-2A4D-409C-9F5E-773A9C88CA54@math.washington.edu>

On Apr 14, 2010, at 9:21 AM, Christopher Barker wrote:

> Stefan Behnel wrote:
>> I've written up a CEP for now:
>>
>> http://wiki.cython.org/enhancements/argumentnonechecks
>
> great, thanks!

Yes, thanks!

>
> A couple comments:
>
>
> """
> Typed argument values that must reject None are more common than those
> than can safely allow them (without being optional arguments). So the
> current syntax requires more overhead on the user side than the  
> proposed
> syntax.
> """
>
> This seems to be very much style dependent. If I followed the
> conversation right, Lisandro (and I) would use a lot of "not None",
> whereas Dag would use a lot of "or None". So I think the more  
> compelling
> argument is that leaving out a "not None" could result in a segfault,
> whereas leaving out an "or None" would result in an exception, and be
> more likely to be caught in tests.
>
>
> About "or None": None is a very common and useful case for an extra  
> type
> one might want, but it's not the only one -- is there a way to tell
> Cython that you want to accept one of two (or a few) types?
>
>
>
> """Changing the behavior of cdef MyExtType x is not an option as the
> variable requires an initial value and users must be able to set it  
> to a
> neutral value to destroy the reference they contain.
> """
>
> This is probably a result of my limited understanding, but I'm still a
> bit confused about this -- there has got to be a less dangerous way to
> do that -- after all, people write a lot of code in C without  
> needing it.

This is what happens when people dereference NULL or uninitialized  
pointers. (Under the hood, all Python objects are just PyObject*, in C+ 
+ terms they're all heap allocated.)

>  - Can't you destroy the reference with a "del"?

Not for local variables (yet). What would you put in its place? i.e.

cdef MyType x = ...
del x
print x

The Python interpreter sets it to NULL and checks, on every access, if  
it's NULL (raising an UnboundLocal exception if it is). With control  
flow, we'll be able to do this too without (much, in most cases) cost.

>  - It doesn't seem onerous to me that an initial value needs to be of
> the type specified -- that's the cost of static typing.

It's easy to come up with default values for ints, doubles, pointers,  
etc. (though they may be just garbage). Less so with  
MyRandomExpensiveToCreateClassThatRequiresLotOfArguments.

In C++ one either has default constructors, or the language forces you  
to call one--in other words you can't declare a stack-allocated C++  
object without assigning it. (Not only are Python classes all heap  
allocated, there are several reasons why this is not very Pythonic or  
feasible for Cython, among them the much simpler scoping rules.)

> Dag Sverre Seljebotn wrote:
>> But it can still be compared to Cython-with-control-flow-analysis, at
>> which point I don't have to bother about "or None" or "not None" at  
>> all,
>> meaning one less thing to think about (and learn!) with respect to  
>> pure
>> Python.
>
> Yes, that will be wonderful -- but are we talking months or years  
> before
> it's a reality?

I'd think less than a year. For example, personally, I'll have a lot  
more time to work on Cython once I graduate.

- Robert


From stefan_ml at behnel.de  Wed Apr 14 22:36:54 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 14 Apr 2010 22:36:54 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC5EBE2.5030005@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC40CD6.6080206@student.matnat.uio.no>	<4BC40F9F.6080204@behnel.de>	<4BC4192A.6040106@student.matnat.uio.no>	<4BC4563A.8020007@behnel.de>	<4BC47F73.4090304@student.matnat.uio.no>	<4BC489F4.2080609@behnel.de>	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>	<4BC5D7B5.8090102@behnel.de>	<m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>
	<4BC5EBE2.5030005@behnel.de>
Message-ID: <4BC62766.9030805@behnel.de>

Stefan Behnel, 14.04.2010 18:22:
> Lisandro Dalcin, 14.04.2010 17:52:
>> On 14 April 2010 11:56, Stefan Behnel wrote:
>>> Robert Bradshaw, 13.04.2010 19:44:
>>>> OK, Here's what we're going to do:
>>>>
>>>> 1) We'll make a directive controlling the behavior of typed arguments.
>>>> 2) We'll support the "or None" annotation, so the directive will be
>>>> useful both directions.
>>>
>>> http://hg.cython.org/cython-devel/rev/5dfcb27621fe
>>
>> Tests?
>
> Hmm, thanks for catching this. It looks like there is one important case
> that I didn't consider. How should this behave:
>
>       def func(list L):
>           ...
>
> Should builtins continue to accept None or should it behave like other
> extension types? For consistency, I think it should be for the latter, with
> the obvious exception of "object".

I've extended the CEP to reflect this, please comment.

http://wiki.cython.org/enhancements/argumentnonechecks

Stefan

From greg.ewing at canterbury.ac.nz  Wed Apr 14 23:24:01 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 15 Apr 2010 09:24:01 +1200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC57EB1.3040906@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC2C772.8040000@behnel.de>
	<y2ze7ba66e41004120635xe9c30b58n86ff72c5f25d8020@mail.gmail.com>
	<4BC32F89.40808@student.matnat.uio.no> <4BC3F639.4070200@behnel.de>
	<903ABD12-A3BF-4414-A33C-689C82859E4C@math.washington.edu>
	<4BC40F10.9060300@behnel.de>
	<4428764B-DEDE-4EA7-9FB4-118722CA87C0@math.washington.edu>
	<4BC49CFE.9050408@noaa.gov> <4BC4A18A.4080708@student.matnat.uio.no>
	<4BC50AC6.3000709@canterbury.ac.nz>
	<4BC57648.9000301@student.matnat.uio.no>
	<4BC57EB1.3040906@behnel.de>
Message-ID: <4BC63271.6080505@canterbury.ac.nz>

Stefan Behnel wrote:
> Dag Sverre Seljebotn, 14.04.2010 10:01:
>
>>Yes, that was my point: I'm fallible, I make bugs, sometimes I don't
>>know what I'm doing :-)
> 
> Sure, and the proposed change is targeting one common source of those bugs.

There's another assumption at work here as well. It's
assumed that if you're using Pyrex at all you're already
dabbling in C land, where there are so many ways of
crashing things that misusing None doesn't seem to be
worth singling out for special treatment.

As I said at the beginning, the right thing for Cython
to do might be quite different.

-- 
Greg



From greg.ewing at canterbury.ac.nz  Thu Apr 15 00:38:58 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 15 Apr 2010 10:38:58 +1200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC5EBE2.5030005@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz>
	<4BC40CD6.6080206@student.matnat.uio.no> <4BC40F9F.6080204@behnel.de>
	<4BC4192A.6040106@student.matnat.uio.no> <4BC4563A.8020007@behnel.de>
	<4BC47F73.4090304@student.matnat.uio.no> <4BC489F4.2080609@behnel.de>
	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>
	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>
	<4BC5D7B5.8090102@behnel.de>
	<m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>
	<4BC5EBE2.5030005@behnel.de>
Message-ID: <4BC64402.9010800@canterbury.ac.nz>

Stefan Behnel wrote:

> Should builtins continue to accept None or should it behave like other 
> extension types?

Pyrex treats them just like any other extension type. In
fact the *are* just extension types that happen to be
predeclared.

-- 
Greg

From dalcinl at gmail.com  Thu Apr 15 01:31:09 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 14 Apr 2010 20:31:09 -0300
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC64402.9010800@canterbury.ac.nz>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC4563A.8020007@behnel.de>
	<4BC47F73.4090304@student.matnat.uio.no> <4BC489F4.2080609@behnel.de>
	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>
	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>
	<4BC5D7B5.8090102@behnel.de>
	<m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>
	<4BC5EBE2.5030005@behnel.de> <4BC64402.9010800@canterbury.ac.nz>
Message-ID: <y2ie7ba66e41004141631kc9fcdc5zaf7e907bec50d2c8@mail.gmail.com>

On 14 April 2010 19:38, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Stefan Behnel wrote:
>
>> Should builtins continue to accept None or should it behave like other
>> extension types?
>
> Pyrex treats them just like any other extension type. In
> fact the *are* just extension types that happen to be
> predeclared.
>

I agree with Greg and Stefan, i.e., no special handling for builtin types.


-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From robertwb at math.washington.edu  Thu Apr 15 01:39:41 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Wed, 14 Apr 2010 16:39:41 -0700
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <y2ie7ba66e41004141631kc9fcdc5zaf7e907bec50d2c8@mail.gmail.com>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC4563A.8020007@behnel.de>
	<4BC47F73.4090304@student.matnat.uio.no>
	<4BC489F4.2080609@behnel.de>
	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>
	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>
	<4BC5D7B5.8090102@behnel.de>
	<m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>
	<4BC5EBE2.5030005@behnel.de> <4BC64402.9010800@canterbury.ac.nz>
	<y2ie7ba66e41004141631kc9fcdc5zaf7e907bec50d2c8@mail.gmail.com>
Message-ID: <D8F39763-5C0A-496E-9018-86F4DE2AB89B@math.washington.edu>

On Apr 14, 2010, at 4:31 PM, Lisandro Dalcin wrote:

> On 14 April 2010 19:38, Greg Ewing <greg.ewing at canterbury.ac.nz>  
> wrote:
>> Stefan Behnel wrote:
>>
>>> Should builtins continue to accept None or should it behave like  
>>> other
>>> extension types?
>>
>> Pyrex treats them just like any other extension type. In
>> fact the *are* just extension types that happen to be
>> predeclared.

In Cython, builtin types are a bit special as it checks to make sure  
it is exactly the specified type, not just a subclass. (Otherwise,  
there's little room for optimization and so little point in declaring  
them.)

> I agree with Greg and Stefan, i.e., no special handling for builtin  
> types.

Yes, lets not an an exception to the exception...

- Robert


From dalcinl at gmail.com  Thu Apr 15 02:23:41 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 14 Apr 2010 21:23:41 -0300
Subject: [Cython] unused docstrings
Message-ID: <t2pe7ba66e41004141723zc6bcdabpdba861918c7aaec3@mail.gmail.com>

I've finally managed to get rid of unused docstrings:
http://hg.cython.org/cython-devel/rev/5848ee0eca01

It was not that easy to figure out the right way, as some special
methods are added both to typeobject slots and pymemberdef slots with
METH_COEXISTS... BTW, unrelated to docstrings, perhaps METH_COEXISTS
usage could provides room for further optimizations.

-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From robertwb at math.washington.edu  Thu Apr 15 02:59:05 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Wed, 14 Apr 2010 17:59:05 -0700
Subject: [Cython] unused docstrings
In-Reply-To: <t2pe7ba66e41004141723zc6bcdabpdba861918c7aaec3@mail.gmail.com>
References: <t2pe7ba66e41004141723zc6bcdabpdba861918c7aaec3@mail.gmail.com>
Message-ID: <6D2CBD66-15BC-4300-9D7D-88C51F601B8A@math.washington.edu>

On Apr 14, 2010, at 5:23 PM, Lisandro Dalcin wrote:

> I've finally managed to get rid of unused docstrings:
> http://hg.cython.org/cython-devel/rev/5848ee0eca01

Thanks! You're making great progress on totally warning-free test runs.

> It was not that easy to figure out the right way, as some special
> methods are added both to typeobject slots and pymemberdef slots with
> METH_COEXISTS... BTW, unrelated to docstrings, perhaps METH_COEXISTS
> usage could provides room for further optimizations.

The METH_COEXISTS is what allows us to give any typeslot function an  
optimized wrapper (with doctstring). The reason its not done for  
__contains__, __len__, etc. is that they return non-object values  
(though it should be easy to create our own wrappers, with optimized  
calling conventions and proper docstrings).

For introspection purpose, we could even create "wrappers" for  
__cinit__/__dealloc__ that would simply raise errors saying that these  
are not meant to be called directly (or let them be some kind of non- 
callable object with a docstring). This wouldn't use the METH_COEXISTS  
structure, as there aren't any existing wrapper methods.

- Robert


From izarf.mail at gmail.com  Thu Apr 15 05:20:29 2010
From: izarf.mail at gmail.com (Izarf)
Date: Thu, 15 Apr 2010 05:20:29 +0200
Subject: [Cython] Unrecognized character
Message-ID: <q2ladadc6921004142020nc25c2bbdze2aa7b585ea1326d@mail.gmail.com>

Hi folks!
Seems to be a parser error in the current cersion of cython -- it cannot
parse any of my .pyx files!
Did i mention I use windows and python 2.6? Am I the only one with this
issue?

-- 
-Elias
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20100415/d233cfda/attachment.htm 

From dalcinl at gmail.com  Thu Apr 15 05:34:33 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 15 Apr 2010 00:34:33 -0300
Subject: [Cython] unused docstrings
In-Reply-To: <6D2CBD66-15BC-4300-9D7D-88C51F601B8A@math.washington.edu>
References: <t2pe7ba66e41004141723zc6bcdabpdba861918c7aaec3@mail.gmail.com>
	<6D2CBD66-15BC-4300-9D7D-88C51F601B8A@math.washington.edu>
Message-ID: <l2te7ba66e41004142034nc87e2b23l39fcc506e3623d70@mail.gmail.com>

On 14 April 2010 21:59, Robert Bradshaw <robertwb at math.washington.edu> wrote:
> On Apr 14, 2010, at 5:23 PM, Lisandro Dalcin wrote:
>
>> I've finally managed to get rid of unused docstrings:
>> http://hg.cython.org/cython-devel/rev/5848ee0eca01
>
> Thanks! You're making great progress on totally warning-free test runs.
>

Yep, that's my intention...

>> It was not that easy to figure out the right way, as some special
>> methods are added both to typeobject slots and pymemberdef slots with
>> METH_COEXISTS... BTW, unrelated to docstrings, perhaps METH_COEXISTS
>> usage could provides room for further optimizations.
>
> The METH_COEXISTS is what allows us to give any typeslot function an
> optimized wrapper (with doctstring). The reason its not done for
> __contains__, __len__, etc. is that they return non-object values
> (though it should be easy to create our own wrappers, with optimized
> calling conventions and proper docstrings).
>

Yes, it should be easy... Hopefully, Stefan will beat us :-) ,
though... Do anyone have actual numbers about the gains?

> For introspection purpose, we could even create "wrappers" for
> __cinit__/__dealloc__ that would simply raise errors saying that these
> are not meant to be called directly (or let them be some kind of non-
> callable object with a docstring). This wouldn't use the METH_COEXISTS
> structure, as there aren't any existing wrapper methods.
>

Now that you say it... The name "__dealloc__" always hurts my
eyes...The method does not do any "deallocation", so the name is IMHO
very misleading... I would love to introduce a __cdel__ alias for it,
then we would have near-to-perfect symetry __init__/__del__ and
__cinit__/__cdel__ ...


-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From dalcinl at gmail.com  Thu Apr 15 05:35:51 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 15 Apr 2010 00:35:51 -0300
Subject: [Cython] Unrecognized character
In-Reply-To: <q2ladadc6921004142020nc25c2bbdze2aa7b585ea1326d@mail.gmail.com>
References: <q2ladadc6921004142020nc25c2bbdze2aa7b585ea1326d@mail.gmail.com>
Message-ID: <k2se7ba66e41004142035qaa8d4bf2jdac16d128d1a9ed4@mail.gmail.com>

On 15 April 2010 00:20, Izarf <izarf.mail at gmail.com> wrote:
> Hi folks!
> Seems to be a parser error in the current cersion of cython -- it cannot
> parse any of my .pyx files!
> Did i mention I use windows and python 2.6? Am I the only one with this
> issue?
>

Perhaps this is the newlines-related issue others have already
reported and that is already fixed in cython-devel?


> --
> -Elias
>
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>
>



-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From stefan_ml at behnel.de  Thu Apr 15 07:55:43 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 15 Apr 2010 07:55:43 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <D8F39763-5C0A-496E-9018-86F4DE2AB89B@math.washington.edu>
References: <4BC269ED.1070703@canterbury.ac.nz>
	<4BC4563A.8020007@behnel.de>	<4BC47F73.4090304@student.matnat.uio.no>	<4BC489F4.2080609@behnel.de>	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>	<4BC5D7B5.8090102@behnel.de>	<m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>	<4BC5EBE2.5030005@behnel.de>
	<4BC64402.9010800@canterbury.ac.nz>	<y2ie7ba66e41004141631kc9fcdc5zaf7e907bec50d2c8@mail.gmail.com>
	<D8F39763-5C0A-496E-9018-86F4DE2AB89B@math.washington.edu>
Message-ID: <4BC6AA5F.6060500@behnel.de>

Robert Bradshaw, 15.04.2010 01:39:
> On Apr 14, 2010, at 4:31 PM, Lisandro Dalcin wrote:
>
>> On 14 April 2010 19:38, Greg Ewing wrote:
>>> Stefan Behnel wrote:
>>>
>>>> Should builtins continue to accept None or should it behave like
>>>> other
>>>> extension types?
>>>
>>> Pyrex treats them just like any other extension type. In
>>> fact the *are* just extension types that happen to be
>>> predeclared.
>
> In Cython, builtin types are a bit special as it checks to make sure
> it is exactly the specified type, not just a subclass. (Otherwise,
> there's little room for optimization and so little point in declaring
> them.)
>
>> I agree with Greg and Stefan, i.e., no special handling for builtin
>> types.
>
> Yes, lets not an an exception to the exception...

Ok, so it looks like we agree that this should raise an exception:

     def func(list L not None):

How about this:

     def func(object obj not None):

which should likely behave the same as this:

     def func(obj not None):

I assume that this should also be allowed and behave like the other 
extension types, right?

Stefan

From robertwb at math.washington.edu  Thu Apr 15 08:07:30 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Wed, 14 Apr 2010 23:07:30 -0700
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC6AA5F.6060500@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz>
	<4BC4563A.8020007@behnel.de>	<4BC47F73.4090304@student.matnat.uio.no>	<4BC489F4.2080609@behnel.de>	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>	<4BC5D7B5.8090102@behnel.de>	<m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>	<4BC5EBE2.5030005@behnel.de>
	<4BC64402.9010800@canterbury.ac.nz>	<y2ie7ba66e41004141631kc9fcdc5zaf7e907bec50d2c8@mail.gmail.com>
	<D8F39763-5C0A-496E-9018-86F4DE2AB89B@math.washington.edu>
	<4BC6AA5F.6060500@behnel.de>
Message-ID: <6BC0947A-2619-46C2-A9B7-CB6F3E49D72A@math.washington.edu>

On Apr 14, 2010, at 10:55 PM, Stefan Behnel wrote:

> Robert Bradshaw, 15.04.2010 01:39:
>> On Apr 14, 2010, at 4:31 PM, Lisandro Dalcin wrote:
>>
>>> On 14 April 2010 19:38, Greg Ewing wrote:
>>>> Stefan Behnel wrote:
>>>>
>>>>> Should builtins continue to accept None or should it behave like
>>>>> other
>>>>> extension types?
>>>>
>>>> Pyrex treats them just like any other extension type. In
>>>> fact the *are* just extension types that happen to be
>>>> predeclared.
>>
>> In Cython, builtin types are a bit special as it checks to make sure
>> it is exactly the specified type, not just a subclass. (Otherwise,
>> there's little room for optimization and so little point in declaring
>> them.)
>>
>>> I agree with Greg and Stefan, i.e., no special handling for builtin
>>> types.
>>
>> Yes, lets not an an exception to the exception...
>
> Ok, so it looks like we agree that this should raise an exception:
>
>     def func(list L not None):
>
> How about this:
>
>     def func(object obj not None):
>
> which should likely behave the same as this:
>
>     def func(obj not None):
>
> I assume that this should also be allowed and behave like the other
> extension types, right?

No, I would actually make an exception here--normally

     def func(Type x not None)

throws an error if and only if not isinstance(x, Type). The "object"  
type is different is not really treated like an extension type. If  
something is "[object] obj not None" then I think it should actually  
reject None.

- Robert


From stefan_ml at behnel.de  Thu Apr 15 08:41:59 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 15 Apr 2010 08:41:59 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <6BC0947A-2619-46C2-A9B7-CB6F3E49D72A@math.washington.edu>
References: <4BC269ED.1070703@canterbury.ac.nz>	<4BC4563A.8020007@behnel.de>	<4BC47F73.4090304@student.matnat.uio.no>	<4BC489F4.2080609@behnel.de>	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>	<4BC5D7B5.8090102@behnel.de>	<m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>	<4BC5EBE2.5030005@behnel.de>	<4BC64402.9010800@canterbury.ac.nz>	<y2ie7ba66e41004141631kc9fcdc5zaf7e907bec50d2c8@mail.gmail.com>	<D8F39763-5C0A-496E-9018-86F4DE2AB89B@math.washington.edu>	<4BC6AA5F.6060500@behnel.de>
	<6BC0947A-2619-46C2-A9B7-CB6F3E49D72A@math.washington.edu>
Message-ID: <4BC6B537.7000909@behnel.de>

Robert Bradshaw, 15.04.2010 08:07:
> On Apr 14, 2010, at 10:55 PM, Stefan Behnel wrote:
>
>> Robert Bradshaw, 15.04.2010 01:39:
>>> On Apr 14, 2010, at 4:31 PM, Lisandro Dalcin wrote:
>>>
>>>> On 14 April 2010 19:38, Greg Ewing wrote:
>>>>> Stefan Behnel wrote:
>>>>>
>>>>>> Should builtins continue to accept None or should it behave like
>>>>>> other
>>>>>> extension types?
>>>>>
>>>>> Pyrex treats them just like any other extension type. In
>>>>> fact the *are* just extension types that happen to be
>>>>> predeclared.
>>>
>>> In Cython, builtin types are a bit special as it checks to make sure
>>> it is exactly the specified type, not just a subclass. (Otherwise,
>>> there's little room for optimization and so little point in declaring
>>> them.)
>>>
>>>> I agree with Greg and Stefan, i.e., no special handling for builtin
>>>> types.
>>>
>>> Yes, lets not an an exception to the exception...
>>
>> Ok, so it looks like we agree that this should raise an exception:
>>
>>      def func(list L not None):
>>
>> How about this:
>>
>>      def func(object obj not None):
>>
>> which should likely behave the same as this:
>>
>>      def func(obj not None):
>>
>> I assume that this should also be allowed and behave like the other
>> extension types, right?
>
> No, I would actually make an exception here--normally
>
>       def func(Type x not None)
>
> throws an error if and only if not isinstance(x, Type). The "object"
> type is different is not really treated like an extension type. If
> something is "[object] obj not None" then I think it should actually
> reject None.

;) that's exactly what I meant.

Stefan

From dalcinl at gmail.com  Thu Apr 15 16:30:42 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 15 Apr 2010 11:30:42 -0300
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC6B537.7000909@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC5D7B5.8090102@behnel.de>
	<m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>
	<4BC5EBE2.5030005@behnel.de> <4BC64402.9010800@canterbury.ac.nz>
	<y2ie7ba66e41004141631kc9fcdc5zaf7e907bec50d2c8@mail.gmail.com>
	<D8F39763-5C0A-496E-9018-86F4DE2AB89B@math.washington.edu>
	<4BC6AA5F.6060500@behnel.de>
	<6BC0947A-2619-46C2-A9B7-CB6F3E49D72A@math.washington.edu>
	<4BC6B537.7000909@behnel.de>
Message-ID: <w2ke7ba66e41004150730yefe68fddm470f8967c859f3de@mail.gmail.com>

On 15 April 2010 03:41, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Robert Bradshaw, 15.04.2010 08:07:
>> On Apr 14, 2010, at 10:55 PM, Stefan Behnel wrote:
>>
>>> Robert Bradshaw, 15.04.2010 01:39:
>>>> On Apr 14, 2010, at 4:31 PM, Lisandro Dalcin wrote:
>>>>
>>>>> On 14 April 2010 19:38, Greg Ewing wrote:
>>>>>> Stefan Behnel wrote:
>>>>>>
>>>>>>> Should builtins continue to accept None or should it behave like
>>>>>>> other
>>>>>>> extension types?
>>>>>>
>>>>>> Pyrex treats them just like any other extension type. In
>>>>>> fact the *are* just extension types that happen to be
>>>>>> predeclared.
>>>>
>>>> In Cython, builtin types are a bit special as it checks to make sure
>>>> it is exactly the specified type, not just a subclass. (Otherwise,
>>>> there's little room for optimization and so little point in declaring
>>>> them.)
>>>>
>>>>> I agree with Greg and Stefan, i.e., no special handling for builtin
>>>>> types.
>>>>
>>>> Yes, lets not an an exception to the exception...
>>>
>>> Ok, so it looks like we agree that this should raise an exception:
>>>
>>> ? ? ?def func(list L not None):
>>>
>>> How about this:
>>>
>>> ? ? ?def func(object obj not None):
>>>
>>> which should likely behave the same as this:
>>>
>>> ? ? ?def func(obj not None):
>>>
>>> I assume that this should also be allowed and behave like the other
>>> extension types, right?
>>
>> No, I would actually make an exception here--normally
>>
>> ? ? ? def func(Type x not None)
>>
>> throws an error if and only if not isinstance(x, Type). The "object"
>> type is different is not really treated like an extension type. If
>> something is "[object] obj not None" then I think it should actually
>> reject None.
>
> ;) that's exactly what I meant.
>

Sorry, I'm a bit confused... Does this means that this code:

# cython: allow_none_for_extension_args=False
def func(obj): pass

then I call "func(None)" from Python code and get and exception??



-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From dalcinl at gmail.com  Thu Apr 15 16:59:26 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 15 Apr 2010 11:59:26 -0300
Subject: [Cython] [PATCH] type promotion, last call
Message-ID: <k2ge7ba66e41004150759y2fe3c6d6p2e769f0d3e3d2911@mail.gmail.com>

I've already mailed about this, no one (other than Greg, of course)
commented... This patch fix and  add tests for the C rules of integral
promotions in some interesting corner cases. The algorithm is
basically the one available in latest Pyrex plus the handling of
complex types. As we already discussed with Greg, this algorithm is
not actually correct according to the wording of the C99 standard (and
common sense), because type promotion do depends on the actual type
sizes. But Greg's implementation is IMHO the best we can do from
Pyrex/Cython side.

If no one complains, I'll push the patch.

-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169
-------------- next part --------------
A non-text attachment was scrubbed...
Name: type_promotion.diff
Type: text/x-patch
Size: 3552 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20100415/c2c16a68/attachment.bin 

From Chris.Barker at noaa.gov  Thu Apr 15 18:12:10 2010
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Thu, 15 Apr 2010 09:12:10 -0700
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <6BC0947A-2619-46C2-A9B7-CB6F3E49D72A@math.washington.edu>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC4563A.8020007@behnel.de>
	<4BC47F73.4090304@student.matnat.uio.no> <4BC489F4.2080609@behnel.de>
	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>
	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>
	<4BC5D7B5.8090102@behnel.de>
	<m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>
	<4BC5EBE2.5030005@behnel.de> <4BC64402.9010800@canterbury.ac.nz>
	<y2ie7ba66e41004141631kc9fcdc5zaf7e907bec50d2c8@mail.gmail.com>
	<D8F39763-5C0A-496E-9018-86F4DE2AB89B@math.washington.edu>
	<4BC6AA5F.6060500@behnel.de>
	<6BC0947A-2619-46C2-A9B7-CB6F3E49D72A@math.washington.edu>
Message-ID: <4BC73ADA.4060005@noaa.gov>

>> Ok, so it looks like we agree that this should raise an exception:
>>
>>     def func(list L not None):

yes.

>> How about this:
>>
>>     def func(object obj not None):

This is basically nonsense -- None is an object:

In [7]: isinstance( None, object)
Out[7]: True

so something typedefed to object should be able to hold a None. The 
whole "not None"/"or None" thing simply isn't designed for this.

Given that the user is asking for nonsense, I suppose it doesn't matter 
too much how it's handled, but I'd expect that the "not None" in this 
case would be a no-op.

Unless, we're going to generalize and allow:

def func(object obj not float):

or any other specific type, but we're special casing None for good 
reason here.

> throws an error if and only if not isinstance(x, Type). The "object"  
> type is different is not really treated like an extension type.

and yet the isinstance() check will work anyway, won't it (passing for 
any python object)? Is object already special cased in that way?

> something is "[object] obj not None" then I think it should actually  
> reject None.

That does follow the rule that it is what it says, but I wouldn't make 
any effort in the code to make it work.

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

From Chris.Barker at noaa.gov  Thu Apr 15 18:23:45 2010
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Thu, 15 Apr 2010 09:23:45 -0700
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC73ADA.4060005@noaa.gov>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC4563A.8020007@behnel.de>
	<4BC47F73.4090304@student.matnat.uio.no> <4BC489F4.2080609@behnel.de>
	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>
	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>
	<4BC5D7B5.8090102@behnel.de>
	<m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>
	<4BC5EBE2.5030005@behnel.de> <4BC64402.9010800@canterbury.ac.nz>
	<y2ie7ba66e41004141631kc9fcdc5zaf7e907bec50d2c8@mail.gmail.com>
	<D8F39763-5C0A-496E-9018-86F4DE2AB89B@math.washington.edu>
	<4BC6AA5F.6060500@behnel.de>
	<6BC0947A-2619-46C2-A9B7-CB6F3E49D72A@math.washington.edu>
	<4BC73ADA.4060005@noaa.gov>
Message-ID: <4BC73D91.8040001@noaa.gov>

One more thought:

>>>     def func(object obj not None):
> 
> This is basically nonsense -- None is an object

could it give an error or warning on compilation? That seems the best 
thing to do for a directive that doesn't make sense.

-Chris




-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

From dalcinl at gmail.com  Thu Apr 15 18:26:47 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 15 Apr 2010 13:26:47 -0300
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC73D91.8040001@noaa.gov>
References: <4BC269ED.1070703@canterbury.ac.nz>
	<m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>
	<4BC5EBE2.5030005@behnel.de> <4BC64402.9010800@canterbury.ac.nz>
	<y2ie7ba66e41004141631kc9fcdc5zaf7e907bec50d2c8@mail.gmail.com>
	<D8F39763-5C0A-496E-9018-86F4DE2AB89B@math.washington.edu>
	<4BC6AA5F.6060500@behnel.de>
	<6BC0947A-2619-46C2-A9B7-CB6F3E49D72A@math.washington.edu>
	<4BC73ADA.4060005@noaa.gov> <4BC73D91.8040001@noaa.gov>
Message-ID: <l2te7ba66e41004150926icfc8000pf8f48cccdc3db8ed@mail.gmail.com>

On 15 April 2010 13:23, Christopher Barker <Chris.Barker at noaa.gov> wrote:
> One more thought:
>
>>>> ? ? def func(object obj not None):
>>
>> This is basically nonsense -- None is an object
>

I think Christopher has a very valid point.




-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From robertwb at math.washington.edu  Thu Apr 15 19:36:26 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 15 Apr 2010 10:36:26 -0700
Subject: [Cython] [PATCH] type promotion, last call
In-Reply-To: <k2ge7ba66e41004150759y2fe3c6d6p2e769f0d3e3d2911@mail.gmail.com>
References: <k2ge7ba66e41004150759y2fe3c6d6p2e769f0d3e3d2911@mail.gmail.com>
Message-ID: <5B99118A-88F4-445F-BEBD-6DF22601A5DE@math.washington.edu>

On Apr 15, 2010, at 7:59 AM, Lisandro Dalcin wrote:

> I've already mailed about this, no one (other than Greg, of course)
> commented... This patch fix and  add tests for the C rules of integral
> promotions in some interesting corner cases. The algorithm is
> basically the one available in latest Pyrex plus the handling of
> complex types. As we already discussed with Greg, this algorithm is
> not actually correct according to the wording of the C99 standard (and
> common sense), because type promotion do depends on the actual type
> sizes. But Greg's implementation is IMHO the best we can do from
> Pyrex/Cython side.
>
> If no one complains, I'll push the patch.

I'm fine with it. Basically, you're assuming that the rankings are  
strict, whereas the former code assumed that it was not.  
Unfortunately, most of the places this will come up is int < long =  
long long or int = long < long long, but there's not really a way to  
do better.

- Robert


From dalcinl at gmail.com  Thu Apr 15 20:25:38 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 15 Apr 2010 15:25:38 -0300
Subject: [Cython] [PATCH] type promotion, last call
In-Reply-To: <5B99118A-88F4-445F-BEBD-6DF22601A5DE@math.washington.edu>
References: <k2ge7ba66e41004150759y2fe3c6d6p2e769f0d3e3d2911@mail.gmail.com>
	<5B99118A-88F4-445F-BEBD-6DF22601A5DE@math.washington.edu>
Message-ID: <n2ze7ba66e41004151125u4663b44dm619e76aa07740fb@mail.gmail.com>

On 15 April 2010 14:36, Robert Bradshaw <robertwb at math.washington.edu> wrote:
> On Apr 15, 2010, at 7:59 AM, Lisandro Dalcin wrote:
>
>> I've already mailed about this, no one (other than Greg, of course)
>> commented... This patch fix and ?add tests for the C rules of integral
>> promotions in some interesting corner cases. The algorithm is
>> basically the one available in latest Pyrex plus the handling of
>> complex types. As we already discussed with Greg, this algorithm is
>> not actually correct according to the wording of the C99 standard (and
>> common sense), because type promotion do depends on the actual type
>> sizes. But Greg's implementation is IMHO the best we can do from
>> Pyrex/Cython side.
>>
>> If no one complains, I'll push the patch.
>
> I'm fine with it. Basically, you're assuming that the rankings are
> strict, whereas the former code assumed that it was not.

Yes, I'm assuming all types have different sizes. And no, the former
code actually assumed that: For all signed T1+ unsigned T2 -> unsigned
max(T1,T2), and that's not right, think of T1=longlong and T2=char,
the result should be signed.

> Unfortunately, most of the places this will come up is int < long =
> long long or int = long < long long, but there's not really a way to
> do better.
>

Yep, but only when mixing signed+unsigned and
rank(signed)>rank(unsigned) . I would be nice to genetate Cython
compile warnings in this case...


-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From stefan_ml at behnel.de  Thu Apr 15 22:14:14 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 15 Apr 2010 22:14:14 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <w2ke7ba66e41004150730yefe68fddm470f8967c859f3de@mail.gmail.com>
References: <4BC269ED.1070703@canterbury.ac.nz>
	<4BC5D7B5.8090102@behnel.de>	<m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>	<4BC5EBE2.5030005@behnel.de>
	<4BC64402.9010800@canterbury.ac.nz>	<y2ie7ba66e41004141631kc9fcdc5zaf7e907bec50d2c8@mail.gmail.com>	<D8F39763-5C0A-496E-9018-86F4DE2AB89B@math.washington.edu>	<4BC6AA5F.6060500@behnel.de>	<6BC0947A-2619-46C2-A9B7-CB6F3E49D72A@math.washington.edu>	<4BC6B537.7000909@behnel.de>
	<w2ke7ba66e41004150730yefe68fddm470f8967c859f3de@mail.gmail.com>
Message-ID: <4BC77396.8060605@behnel.de>

Lisandro Dalcin, 15.04.2010 16:30:
> Sorry, I'm a bit confused... Does this means that this code:
>
> # cython: allow_none_for_extension_args=False
> def func(obj): pass
>
> then I call "func(None)" from Python code and get and exception??

No it doesn't.  NoneType is a subtype of object, so None will always be a 
valid argument for a plain Python object parameter, regardless of the 
directive.

Stefan

From stefan_ml at behnel.de  Thu Apr 15 22:26:06 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 15 Apr 2010 22:26:06 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <l2te7ba66e41004150926icfc8000pf8f48cccdc3db8ed@mail.gmail.com>
References: <4BC269ED.1070703@canterbury.ac.nz>	<m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>	<4BC5EBE2.5030005@behnel.de>
	<4BC64402.9010800@canterbury.ac.nz>	<y2ie7ba66e41004141631kc9fcdc5zaf7e907bec50d2c8@mail.gmail.com>	<D8F39763-5C0A-496E-9018-86F4DE2AB89B@math.washington.edu>	<4BC6AA5F.6060500@behnel.de>	<6BC0947A-2619-46C2-A9B7-CB6F3E49D72A@math.washington.edu>	<4BC73ADA.4060005@noaa.gov>
	<4BC73D91.8040001@noaa.gov>
	<l2te7ba66e41004150926icfc8000pf8f48cccdc3db8ed@mail.gmail.com>
Message-ID: <4BC7765E.7060606@behnel.de>

Lisandro Dalcin, 15.04.2010 18:26:
> On 15 April 2010 13:23, Christopher Barker wrote:
>> One more thought:
>>
>>>>>      def func(object obj not None):
>>>
>>> This is basically nonsense -- None is an object
>
> I think Christopher has a very valid point.

It doesn't matter, though. If the code says

    def func(object obj not None):

then it is clear that None should not be allowed, which may or may not make 
sense in a given situation. It certainly doesn't hurt anyone, so why should 
we actively keep users from doing this?

Stefan

From Chris.Barker at noaa.gov  Thu Apr 15 22:49:52 2010
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Thu, 15 Apr 2010 13:49:52 -0700
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC7765E.7060606@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz>
	<m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>
	<4BC5EBE2.5030005@behnel.de> <4BC64402.9010800@canterbury.ac.nz>
	<y2ie7ba66e41004141631kc9fcdc5zaf7e907bec50d2c8@mail.gmail.com>
	<D8F39763-5C0A-496E-9018-86F4DE2AB89B@math.washington.edu>
	<4BC6AA5F.6060500@behnel.de>
	<6BC0947A-2619-46C2-A9B7-CB6F3E49D72A@math.washington.edu>
	<4BC73ADA.4060005@noaa.gov> <4BC73D91.8040001@noaa.gov>
	<l2te7ba66e41004150926icfc8000pf8f48cccdc3db8ed@mail.gmail.com>
	<4BC7765E.7060606@behnel.de>
Message-ID: <4BC77BF0.9070306@noaa.gov>

Stefan Behnel wrote:
> It doesn't matter, though. If the code says
> 
>     def func(object obj not None):
> 
> then it is clear that None should not be allowed, which may or may not make 
> sense in a given situation. It certainly doesn't hurt anyone, so why should 
> we actively keep users from doing this?

well, it's kind of like:

def func(int obj not 5):

which would be a pretty cool feature, but I don't think we're trying to 
support that!

both None and object are special cases, so there may not be any "right" 
thing to do.

 > then it is clear that None should not be allowed

True -- it's probably not a good idea to allow the syntax, but not have 
it do anything, so I'd say either catch it at compile time, or have it 
check for None at run time.

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov

From stefan_ml at behnel.de  Thu Apr 15 22:59:01 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 15 Apr 2010 22:59:01 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC77BF0.9070306@noaa.gov>
References: <4BC269ED.1070703@canterbury.ac.nz>	<m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>	<4BC5EBE2.5030005@behnel.de>
	<4BC64402.9010800@canterbury.ac.nz>	<y2ie7ba66e41004141631kc9fcdc5zaf7e907bec50d2c8@mail.gmail.com>	<D8F39763-5C0A-496E-9018-86F4DE2AB89B@math.washington.edu>	<4BC6AA5F.6060500@behnel.de>	<6BC0947A-2619-46C2-A9B7-CB6F3E49D72A@math.washington.edu>	<4BC73ADA.4060005@noaa.gov>
	<4BC73D91.8040001@noaa.gov>	<l2te7ba66e41004150926icfc8000pf8f48cccdc3db8ed@mail.gmail.com>	<4BC7765E.7060606@behnel.de>
	<4BC77BF0.9070306@noaa.gov>
Message-ID: <4BC77E15.8060904@behnel.de>

Christopher Barker, 15.04.2010 22:49:
> Stefan Behnel wrote:
>> It doesn't matter, though. If the code says
>>
>>      def func(object obj not None):
>>
>> then it is clear that None should not be allowed, which may or may not make
>> sense in a given situation. It certainly doesn't hurt anyone, so why should
>> we actively keep users from doing this?
>
> well, it's kind of like:
>
> def func(int obj not 5):
>
> which would be a pretty cool feature, but I don't think we're trying to
> support that!

The difference is that the 'not None' syntax is already there, so the 
question is: should we bother special casing 'object not None' and thus 
effectively give 'not None' a different meaning than just 'not None'?


>   >  then it is clear that None should not be allowed
>
> True -- it's probably not a good idea to allow the syntax, but not have
> it do anything, so I'd say either catch it at compile time, or have it
> check for None at run time.

The current code now checks for None at runtime on 'object not None'.

Stefan

From greg.ewing at canterbury.ac.nz  Fri Apr 16 03:43:28 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 16 Apr 2010 13:43:28 +1200
Subject: [Cython] unused docstrings
In-Reply-To: <l2te7ba66e41004142034nc87e2b23l39fcc506e3623d70@mail.gmail.com>
References: <t2pe7ba66e41004141723zc6bcdabpdba861918c7aaec3@mail.gmail.com>
	<6D2CBD66-15BC-4300-9D7D-88C51F601B8A@math.washington.edu>
	<l2te7ba66e41004142034nc87e2b23l39fcc506e3623d70@mail.gmail.com>
Message-ID: <4BC7C0C0.9020008@canterbury.ac.nz>

On 15/04/10 15:34, Lisandro Dalcin wrote:
> Now that you say it... The name "__dealloc__" always hurts my
> eyes...The method does not do any "deallocation", so the name is IMHO
> very misleading...

It's called that for three reasons:

* It corresponds to the tp_dealloc type slot.

* It gets called when the object is in the process of
   being deallocated.

* It's the place for *you* to deallocate any resources
   that you have allocated for the object.

The symmetry wouldn't be perfect, because you can
also give an extension type a Python-style __init__
method, but a Python __del__ doesn't work.

-- 
Greg



From greg.ewing at canterbury.ac.nz  Fri Apr 16 03:50:51 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 16 Apr 2010 13:50:51 +1200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC6AA5F.6060500@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC4563A.8020007@behnel.de>
	<4BC47F73.4090304@student.matnat.uio.no> <4BC489F4.2080609@behnel.de>
	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>
	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>
	<4BC5D7B5.8090102@behnel.de>
	<m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>
	<4BC5EBE2.5030005@behnel.de> <4BC64402.9010800@canterbury.ac.nz>
	<y2ie7ba66e41004141631kc9fcdc5zaf7e907bec50d2c8@mail.gmail.com>
	<D8F39763-5C0A-496E-9018-86F4DE2AB89B@math.washington.edu>
	<4BC6AA5F.6060500@behnel.de>
Message-ID: <4BC7C27B.9090706@canterbury.ac.nz>

On 15/04/10 17:55, Stefan Behnel wrote:

> How about this:
>
>       def func(object obj not None):

I'm not completely sure without trying it, but I think
Pyrex will currently treat this as a compile-time error.
No type checking code is generated for arguments of
type 'object', and I would have to go out of my way to
do so for the special case of 'object not None'.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Fri Apr 16 04:03:15 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 16 Apr 2010 14:03:15 +1200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC6B537.7000909@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC4563A.8020007@behnel.de>
	<4BC47F73.4090304@student.matnat.uio.no> <4BC489F4.2080609@behnel.de>
	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>
	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>
	<4BC5D7B5.8090102@behnel.de>
	<m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>
	<4BC5EBE2.5030005@behnel.de> <4BC64402.9010800@canterbury.ac.nz>
	<y2ie7ba66e41004141631kc9fcdc5zaf7e907bec50d2c8@mail.gmail.com>
	<D8F39763-5C0A-496E-9018-86F4DE2AB89B@math.washington.edu>
	<4BC6AA5F.6060500@behnel.de>
	<6BC0947A-2619-46C2-A9B7-CB6F3E49D72A@math.washington.edu>
	<4BC6B537.7000909@behnel.de>
Message-ID: <4BC7C563.3070500@canterbury.ac.nz>

On 15/04/10 18:41, Stefan Behnel wrote:
> Robert Bradshaw, 15.04.2010 08:07:
>> The "object"
>> type is different is not really treated like an extension type. If
>> something is "[object] obj not None" then I think it should actually
>> reject None.

That would make 'object' inconsistent with the new rule
that 'not None' is the default, unless you're going to
make 'or None' required for object arguments that can
be None, too.

-- 
Greg

From stefan_ml at behnel.de  Fri Apr 16 08:25:40 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 16 Apr 2010 08:25:40 +0200
Subject: [Cython] move PyBytes C-API compat to top-level
In-Reply-To: <j2ve7ba66e41004152058j2fe29f67r68e603688305d3a@mail.gmail.com>
References: <j2ve7ba66e41004152058j2fe29f67r68e603688305d3a@mail.gmail.com>
Message-ID: <4BC802E4.8060103@behnel.de>

Lisandro Dalcin, 16.04.2010 05:58:
> Stefan, I would like to push a patch that starts like below... As time
> passes and all us get more and more used to Py3 API, it is very easy
> to forget compatibility with Py<2.6 ... For ejample, one of your
> previous commits broke the testsuite because you used
> PyBytes_GET_SIZE...

Right. I set up test jobs on Hudson for Py2.4 and Py2.5 now. I initially 
wanted to safe processing time, but it looks like we need those, too.


> As the PyBytes_XXX API's are official in Py3 and
> moreover Py2.6 provides the compatibility definitions to PyString_XXX,
> I think we should just exten the PyBytes ->  PyString defs to Py<=2.5
>
> What do you think?
>
>
> diff -r fe69c27824be Cython/Compiler/ModuleNode.py
> --- a/Cython/Compiler/ModuleNode.py	Thu Apr 15 23:18:28 2010 +0200
> +++ b/Cython/Compiler/ModuleNode.py	Fri Apr 16 00:51:19 2010 -0300
> @@ -522,11 +522,19 @@
>
>   #if PY_MAJOR_VERSION>= 3
>     #define PyBaseString_Type            PyUnicode_Type
> +  #define PyStringObject               PyUnicodeObject
>     #define PyString_Type                PyUnicode_Type
> +  #define PyString_Check               PyUnicode_Check
>     #define PyString_CheckExact          PyUnicode_CheckExact
> -#else
> +#elif PY_VERSION_HEX<  0x02060000
> +  #define PyBytesObject                PyStringObject
>     #define PyBytes_Type                 PyString_Type
> +  #define PyBytes_Check                PyString_Check
>     #define PyBytes_CheckExact           PyString_CheckExact
> +  #define PyBytes_FromString           PyString_FromString
> +  #define PyBytes_FromStringAndSize    PyString_FromStringAndSize
> +  #define PyBytes_AsString             PyString_AsString
> +  #define PyBytes_GET_SIZE             PyString_GET_SIZE
>   #endif

We should be aware that any defines that we use at the global level can 
interfere with user code, e.g. if the user (or an imported header file) 
decides to define PyString_Check as a check for plain bytes. The patch 
above shows that we already redefine PyString_CheckExact, so we should 
either switch to defining PyString_Check alike or to replacing everything 
with __Pyx_PyString*.

I think the reason why we need the Py3 defines above in the first place is 
that Cython can end up generating those names for the 'str' type, right? 
Maybe there are other places where we can fix this, so that we do not need 
to define PyString_* at all. If we could use __Pyx_PyString_* in all places 
where we need Cython's 'str' semantics, we could remove all PyString 
definitions above. Similarly, PyBaseString_Type would become 
__Pyx_PyBaseString_Type.

Regarding the last part, I'm all for #defining the PyBytes_* API in Py<=2.5 
and using those names wherever we refer to the bytes type. They are the 
safe path.

Stefan

From binet at cern.ch  Fri Apr 16 10:59:25 2010
From: binet at cern.ch (Sebastien Binet)
Date: Fri, 16 Apr 2010 10:59:25 +0200
Subject: [Cython] json to .pxd - autogen cython declaration code
Message-ID: <1271407753-sup-9628@farnsworth>

hi there,

say I have a way to automatically generate a json structure
representing C++ classes, functions,... (the code to do so uses gccxml[0]
and Reflex[1]. once I migrated it to use the new gcc-4.5 plugin
architecture and/or clang, I'll release it under a BSD-like licence)

this json structure looks like so for the moment (from cython C++
test-case class):

{'bases': [],
 'dso': 'libFoo.so',
 'members': [{'name': 'number', 'offset': 0L, 'size': 8, 'type':
 'double'}],
 'methods': [{'args': [],
              'doc': '',
              'name': '~DoubleKeeper',
              'prototype': 'void DoubleKeeper::~DoubleKeeper()',
              'return': 'void'},
             {'args': [{'default': None,
                        'name': None,
                        'type': 'const DoubleKeeper&'}],
              'doc': '',
              'name': 'operator=',
              'prototype': 'DoubleKeeper& DoubleKeeper::operator=(const DoubleKeeper& )',
              'return': 'DoubleKeeper&'},
             {'args': [{'default': None,
                        'name': None,
                        'type': 'const DoubleKeeper&'}],
              'doc': '',
              'name': 'DoubleKeeper',
              'prototype': 'DoubleKeeper DoubleKeeper::DoubleKeeper(const DoubleKeeper& )',
              'return': 'DoubleKeeper'},
             {'args': [{'default': None,
                        'name': 'number',
                        'type': 'double'}],
              'doc': '',
              'name': 'DoubleKeeper',
              'prototype': 'DoubleKeeper DoubleKeeper::DoubleKeeper(double number)',
              'return': 'DoubleKeeper'},
             {'args': [{'default': None, 
                        'name': 'num', 
                        'type': 'double'}],
              'doc': '',
              'name': 'set_number',
              'prototype': 'void DoubleKeeper::set_number(double num)',
              'return': 'void'},
             {'args': [],
              'doc': '',
              'name': 'get_number',
              'prototype': 'double DoubleKeeper::get_number()',
              'return': 'double'},
             {'args': [{'default': None, 
                        'name': 'value', 
                        'type': 'double'}],
              'doc': '',
              'name': 'transmogrify',
              'prototype': 'double DoubleKeeper::transmogrify(double value)',
              'return': 'double'}],
 'name': 'DoubleKeeper',
 'size': 12,
 'typeinfo': '12DoubleKeeper'}

which should be more or less self-describing.

I am wondering if there is an API or a facility to write the
corresponding .pxd file ?

cheers,
sebastien.

[0]: http://www.gccxml.org/HTML/Index.html
[1]: http://root.cern.ch/drupal/content/reflex
-- 
#########################################
# Dr. Sebastien Binet
# Laboratoire de l'Accelerateur Lineaire
# Universite Paris-Sud XI
# Batiment 200
# 91898 Orsay
#########################################

From dagss at student.matnat.uio.no  Fri Apr 16 12:57:20 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Fri, 16 Apr 2010 12:57:20 +0200
Subject: [Cython] json to .pxd - autogen cython declaration code
In-Reply-To: <1271407753-sup-9628@farnsworth>
References: <1271407753-sup-9628@farnsworth>
Message-ID: <4BC84290.6070206@student.matnat.uio.no>

Sebastien Binet wrote:
> hi there,
>
> say I have a way to automatically generate a json structure
> representing C++ classes, functions,... (the code to do so uses gccxml[0]
> and Reflex[1]. once I migrated it to use the new gcc-4.5 plugin
> architecture and/or clang, I'll release it under a BSD-like licence)
>
> this json structure looks like so for the moment (from cython C++
> test-case class):
>
> {'bases': [],
>  'dso': 'libFoo.so',
>  'members': [{'name': 'number', 'offset': 0L, 'size': 8, 'type':
>  'double'}],
>  'methods': [{'args': [],
>               'doc': '',
>               'name': '~DoubleKeeper',
>               'prototype': 'void DoubleKeeper::~DoubleKeeper()',
>               'return': 'void'},
>              {'args': [{'default': None,
>                         'name': None,
>                         'type': 'const DoubleKeeper&'}],
>               'doc': '',
>               'name': 'operator=',
>               'prototype': 'DoubleKeeper& DoubleKeeper::operator=(const DoubleKeeper& )',
>               'return': 'DoubleKeeper&'},
>              {'args': [{'default': None,
>                         'name': None,
>                         'type': 'const DoubleKeeper&'}],
>               'doc': '',
>               'name': 'DoubleKeeper',
>               'prototype': 'DoubleKeeper DoubleKeeper::DoubleKeeper(const DoubleKeeper& )',
>               'return': 'DoubleKeeper'},
>              {'args': [{'default': None,
>                         'name': 'number',
>                         'type': 'double'}],
>               'doc': '',
>               'name': 'DoubleKeeper',
>               'prototype': 'DoubleKeeper DoubleKeeper::DoubleKeeper(double number)',
>               'return': 'DoubleKeeper'},
>              {'args': [{'default': None, 
>                         'name': 'num', 
>                         'type': 'double'}],
>               'doc': '',
>               'name': 'set_number',
>               'prototype': 'void DoubleKeeper::set_number(double num)',
>               'return': 'void'},
>              {'args': [],
>               'doc': '',
>               'name': 'get_number',
>               'prototype': 'double DoubleKeeper::get_number()',
>               'return': 'double'},
>              {'args': [{'default': None, 
>                         'name': 'value', 
>                         'type': 'double'}],
>               'doc': '',
>               'name': 'transmogrify',
>               'prototype': 'double DoubleKeeper::transmogrify(double value)',
>               'return': 'double'}],
>  'name': 'DoubleKeeper',
>  'size': 12,
>  'typeinfo': '12DoubleKeeper'}
>
> which should be more or less self-describing.
>
> I am wondering if there is an API or a facility to write the
> corresponding .pxd file ?
>   
If you are willing to spend a little time to write a transform, then it 
should be very straightforward:

The json matches very well with the internal code tree which Cython 
uses. Basically, you'd import Cython itself and instantiate the node 
instances used internally (from Nodes.py and ExprNodes.py). Then, use 
the code in CodeWriter.py to dump it to a pxd file. (The CodeWriter may 
need to be extended for some node types, but that is straightforward.)

(The advantage of this approach is that one could very easily hook up 
whatever transform you make Cython understand the json files directly, 
rather than having to use a pxd file.)

There's no docs on the internal Cython tree, but there's a decent "print 
tree" functionality. Basically search for "pipeline" in Main.py and 
insert a line "dumptree," right after parsing. Then run Cython on a pyx 
file containing the kind of tree you want to study, and a representation 
will be dumped to screen.

Dag Sverre

From stefan_ml at behnel.de  Fri Apr 16 13:20:13 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 16 Apr 2010 13:20:13 +0200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC7C563.3070500@canterbury.ac.nz>
References: <4BC269ED.1070703@canterbury.ac.nz>
	<4BC4563A.8020007@behnel.de>	<4BC47F73.4090304@student.matnat.uio.no>
	<4BC489F4.2080609@behnel.de>	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>	<4BC5D7B5.8090102@behnel.de>	<m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>	<4BC5EBE2.5030005@behnel.de>
	<4BC64402.9010800@canterbury.ac.nz>	<y2ie7ba66e41004141631kc9fcdc5zaf7e907bec50d2c8@mail.gmail.com>	<D8F39763-5C0A-496E-9018-86F4DE2AB89B@math.washington.edu>	<4BC6AA5F.6060500@behnel.de>	<6BC0947A-2619-46C2-A9B7-CB6F3E49D72A@math.washington.edu>	<4BC6B537.7000909@behnel.de>
	<4BC7C563.3070500@canterbury.ac.nz>
Message-ID: <4BC847ED.5040105@behnel.de>

Greg Ewing, 16.04.2010 04:03:
> On 15/04/10 18:41, Stefan Behnel wrote:
>> Robert Bradshaw, 15.04.2010 08:07:
>>> The "object"
>>> type is different is not really treated like an extension type. If
>>> something is "[object] obj not None" then I think it should actually
>>> reject None.
>
> That would make 'object' inconsistent with the new rule
> that 'not None' is the default, unless you're going to
> make 'or None' required for object arguments that can
> be None, too.

Ah, right, very good catch.

So the new default is not that "not None" applies, the new default is that 
the type check for a Python argument is strict. So None is a valid value 
for an 'object' typed argument, whereas it's not valid for some 'MyExtType' 
argument.

I'll fix the CEP.

Stefan

From dalcinl at gmail.com  Fri Apr 16 15:08:44 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Fri, 16 Apr 2010 10:08:44 -0300
Subject: [Cython] move PyBytes C-API compat to top-level
In-Reply-To: <4BC802E4.8060103@behnel.de>
References: <j2ve7ba66e41004152058j2fe29f67r68e603688305d3a@mail.gmail.com>
	<4BC802E4.8060103@behnel.de>
Message-ID: <g2ye7ba66e41004160608va133fccfn402b7ab3bfb3c090@mail.gmail.com>

On 16 April 2010 03:25, Stefan Behnel <stefan_ml at behnel.de> wrote:
>
> We should be aware that any defines that we use at the global level can
> interfere with user code, e.g. if the user (or an imported header file)
> decides to define PyString_Check as a check for plain bytes.

Then such code is broken in Python2.6 and 2.7, as the definitions are
already there.

> The patch above
> shows that we already redefine PyString_CheckExact, so we should either
> switch to defining PyString_Check alike or to replacing everything with
> __Pyx_PyString*.
>
> I think the reason why we need the Py3 defines above in the first place is
> that Cython can end up generating those names for the 'str' type, right?
> Maybe there are other places where we can fix this, so that we do not need
> to define PyString_* at all. If we could use __Pyx_PyString_* in all places
> where we need Cython's 'str' semantics, we could remove all PyString
> definitions above. Similarly, PyBaseString_Type would become
> __Pyx_PyBaseString_Type.
>

So in short you say that we are going to uses PyBytes_XXX,
PyUnicode_XXX, and _Pyx_PyString_XXX? That sounds good, any misuse of
PyString in Py3 will be easily catched.

> Regarding the last part, I'm all for #defining the PyBytes_* API in Py<=2.5
> and using those names wherever we refer to the bytes type. They are the safe
> path.
>

Nice. I'll push this part.


-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From greg.ewing at canterbury.ac.nz  Sat Apr 17 02:15:37 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 17 Apr 2010 12:15:37 +1200
Subject: [Cython] [Pyrex] ANN: Pyrex 0.9.9
In-Reply-To: <4BC847ED.5040105@behnel.de>
References: <4BC269ED.1070703@canterbury.ac.nz> <4BC4563A.8020007@behnel.de>
	<4BC47F73.4090304@student.matnat.uio.no> <4BC489F4.2080609@behnel.de>
	<E31A5D1F-E60C-4D21-A297-886465FE6657@math.washington.edu>
	<93D9AD28-A9C0-4708-BCC1-C86797C6B716@math.washington.edu>
	<4BC5D7B5.8090102@behnel.de>
	<m2oe7ba66e41004140852uafdf396bjac43fa26de31bf05@mail.gmail.com>
	<4BC5EBE2.5030005@behnel.de> <4BC64402.9010800@canterbury.ac.nz>
	<y2ie7ba66e41004141631kc9fcdc5zaf7e907bec50d2c8@mail.gmail.com>
	<D8F39763-5C0A-496E-9018-86F4DE2AB89B@math.washington.edu>
	<4BC6AA5F.6060500@behnel.de>
	<6BC0947A-2619-46C2-A9B7-CB6F3E49D72A@math.washington.edu>
	<4BC6B537.7000909@behnel.de> <4BC7C563.3070500@canterbury.ac.nz>
	<4BC847ED.5040105@behnel.de>
Message-ID: <4BC8FDA9.8010207@canterbury.ac.nz>

Stefan Behnel wrote:

> So the new default is not that "not None" applies, the new default is that 
> the type check for a Python argument is strict. So None is a valid value 
> for an 'object' typed argument, whereas it's not valid for some 'MyExtType' 
> argument.

Yes, that's probably the best way of looking at it. The
'or None' and 'not None' are about types, not values.

-- 
Greg

From alex.bw at gmail.com  Sat Apr 17 21:32:15 2010
From: alex.bw at gmail.com (Alex Wiltschko)
Date: Sat, 17 Apr 2010 15:32:15 -0400
Subject: [Cython] Cython output generates linking errors on compile
Message-ID: <47CFD94C-DCE3-45AB-8AC6-CA9DF89D822B@gmail.com>

I've stepped away from Cython for about a year, and returned on a new computer, needing to recompile some older Cython code I wrote. It has run fine previously.

I'm issuing this command to build my code:
: cython mytype_svd.pyx
: gcc -shared -pthread -fPIC -fwrapv -O2 -Wall -fno-strict-aliasing -I/usr/include/python2.6 -o mytype_svd.so mytype_svd.c

I'm getting the following (very long) error code:
http://pastie.org/924882
It seems it's complaining about Python C-api calls not being linked into a rather oddly-named .o file. 

I'm running Mac OS 10.6.3, Cython 0.12.1, Python 2.6.1, gcc 4.2 (bundled with XCode).
Note that I've symlinked the numpy includes folder, as well as the Cython includes folder into the python 2.6 include path.



What's going on here? Am I missing some step?





-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20100417/1d47471f/attachment.htm 

From dalcinl at gmail.com  Sat Apr 17 22:46:45 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Sat, 17 Apr 2010 17:46:45 -0300
Subject: [Cython] Cython output generates linking errors on compile
In-Reply-To: <47CFD94C-DCE3-45AB-8AC6-CA9DF89D822B@gmail.com>
References: <47CFD94C-DCE3-45AB-8AC6-CA9DF89D822B@gmail.com>
Message-ID: <h2le7ba66e41004171346j1c99b8dbk3bf058eff52f95c7@mail.gmail.com>

On 17 April 2010 16:32, Alex Wiltschko <alex.bw at gmail.com> wrote:
> I've stepped away from Cython for about a year, and returned on a new
> computer, needing to recompile some older Cython code I wrote. It has run
> fine previously.
> I'm issuing this command to build my code:
> : cython mytype_svd.pyx
> : gcc -shared -pthread -fPIC -fwrapv -O2 -Wall -fno-strict-aliasing
> -I/usr/include/python2.6 -o mytype_svd.so mytype_svd.c

In general, you should nerver do that, but use distutils instead. For
example, you direct invocation to gcc is wrong, you are missing a
"-lpython2.6" or equivalent. You should write a proper setup.py and
let distutils invoke GCC the right way for your platform.



-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From dagss at student.matnat.uio.no  Sat Apr 17 23:07:20 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Sat, 17 Apr 2010 23:07:20 +0200
Subject: [Cython] Cython output generates linking errors on compile
In-Reply-To: <h2le7ba66e41004171346j1c99b8dbk3bf058eff52f95c7@mail.gmail.com>
References: <47CFD94C-DCE3-45AB-8AC6-CA9DF89D822B@gmail.com>
	<h2le7ba66e41004171346j1c99b8dbk3bf058eff52f95c7@mail.gmail.com>
Message-ID: <4BCA2308.60500@student.matnat.uio.no>

Lisandro Dalcin wrote:
> On 17 April 2010 16:32, Alex Wiltschko <alex.bw at gmail.com> wrote:
>> I've stepped away from Cython for about a year, and returned on a new
>> computer, needing to recompile some older Cython code I wrote. It has run
>> fine previously.
>> I'm issuing this command to build my code:
>> : cython mytype_svd.pyx
>> : gcc -shared -pthread -fPIC -fwrapv -O2 -Wall -fno-strict-aliasing
>> -I/usr/include/python2.6 -o mytype_svd.so mytype_svd.c
> 
> In general, you should nerver do that, but use distutils instead. For
> example, you direct invocation to gcc is wrong, you are missing a
> "-lpython2.6" or equivalent. You should write a proper setup.py and
> let distutils invoke GCC the right way for your platform.
> 

Alternatively, depending on your needs, only use the 
"distutils.sysconfig" module to get the proper flags, and then use a 
decent build system.

-- 
Dag Sverre

From robertwb at math.washington.edu  Sun Apr 18 06:24:41 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 17 Apr 2010 21:24:41 -0700
Subject: [Cython] json to .pxd - autogen cython declaration code
In-Reply-To: <4BC84290.6070206@student.matnat.uio.no>
References: <1271407753-sup-9628@farnsworth>
	<4BC84290.6070206@student.matnat.uio.no>
Message-ID: <BEDCEC58-3CEA-4AD7-BE6D-15F1CC2531A6@math.washington.edu>

On Apr 16, 2010, at 3:57 AM, Dag Sverre Seljebotn wrote:

> Sebastien Binet wrote:
>> hi there,
>>
>> say I have a way to automatically generate a json structure
>> representing C++ classes, functions,... (the code to do so uses  
>> gccxml[0]
>> and Reflex[1]. once I migrated it to use the new gcc-4.5 plugin
>> architecture and/or clang, I'll release it under a BSD-like licence)
>>
>> this json structure looks like so for the moment (from cython C++
>> test-case class):
>>
>> {'bases': [],
>> 'dso': 'libFoo.so',
>> 'members': [{'name': 'number', 'offset': 0L, 'size': 8, 'type':
>> 'double'}],
>> 'methods': [{'args': [],
>>              'doc': '',
>>              'name': '~DoubleKeeper',
>>              'prototype': 'void DoubleKeeper::~DoubleKeeper()',
>>              'return': 'void'},
>>             {'args': [{'default': None,
>>                        'name': None,
>>                        'type': 'const DoubleKeeper&'}],
>>              'doc': '',
>>              'name': 'operator=',
>>              'prototype': 'DoubleKeeper&  
>> DoubleKeeper::operator=(const DoubleKeeper& )',
>>              'return': 'DoubleKeeper&'},
>>             {'args': [{'default': None,
>>                        'name': None,
>>                        'type': 'const DoubleKeeper&'}],
>>              'doc': '',
>>              'name': 'DoubleKeeper',
>>              'prototype': 'DoubleKeeper  
>> DoubleKeeper::DoubleKeeper(const DoubleKeeper& )',
>>              'return': 'DoubleKeeper'},
>>             {'args': [{'default': None,
>>                        'name': 'number',
>>                        'type': 'double'}],
>>              'doc': '',
>>              'name': 'DoubleKeeper',
>>              'prototype': 'DoubleKeeper  
>> DoubleKeeper::DoubleKeeper(double number)',
>>              'return': 'DoubleKeeper'},
>>             {'args': [{'default': None,
>>                        'name': 'num',
>>                        'type': 'double'}],
>>              'doc': '',
>>              'name': 'set_number',
>>              'prototype': 'void DoubleKeeper::set_number(double  
>> num)',
>>              'return': 'void'},
>>             {'args': [],
>>              'doc': '',
>>              'name': 'get_number',
>>              'prototype': 'double DoubleKeeper::get_number()',
>>              'return': 'double'},
>>             {'args': [{'default': None,
>>                        'name': 'value',
>>                        'type': 'double'}],
>>              'doc': '',
>>              'name': 'transmogrify',
>>              'prototype': 'double DoubleKeeper::transmogrify(double  
>> value)',
>>              'return': 'double'}],
>> 'name': 'DoubleKeeper',
>> 'size': 12,
>> 'typeinfo': '12DoubleKeeper'}
>>
>> which should be more or less self-describing.
>>
>> I am wondering if there is an API or a facility to write the
>> corresponding .pxd file ?
>>
> If you are willing to spend a little time to write a transform, then  
> it
> should be very straightforward:
>
> The json matches very well with the internal code tree which Cython
> uses. Basically, you'd import Cython itself and instantiate the node
> instances used internally (from Nodes.py and ExprNodes.py). Then, use
> the code in CodeWriter.py to dump it to a pxd file. (The CodeWriter  
> may
> need to be extended for some node types, but that is straightforward.)
>
> (The advantage of this approach is that one could very easily hook up
> whatever transform you make Cython understand the json files directly,
> rather than having to use a pxd file.)
>
> There's no docs on the internal Cython tree, but there's a decent  
> "print
> tree" functionality. Basically search for "pipeline" in Main.py and
> insert a line "dumptree," right after parsing. Then run Cython on a  
> pyx
> file containing the kind of tree you want to study, and a  
> representation
> will be dumped to screen.

I'd like to add that automatic generation of .pxd files from .h and .c  
files has been on our "we'd love to have that" list for a long time,  
so if you could put something like this together we'd love to ship it  
with Cython.

- Robert



From gabriele.lanaro at gmail.com  Sun Apr 18 22:19:45 2010
From: gabriele.lanaro at gmail.com (Gabriele Lanaro)
Date: Sun, 18 Apr 2010 22:19:45 +0200
Subject: [Cython] Extension types and reference counting
Message-ID: <h2tb0364aa51004181319o86623f93y7d553ed82839f70d@mail.gmail.com>

I've a C where some relationship between "object" exists, like this
pseudocode:

c_structA {
  c_structB *bstructs;
}

c_structB ...

I've defined some methods to handle the memory management of these
structures:

new_c_structA
new_c_strucB
free_c_structA
free_c_structB

And also other functions (mehods) like:
c_structA_append(c_structA *a, c_structB *b)


I've to wrap these structures in cython but I encounter some problems with
the reference counting and the garbage collector.


class A:
   cdef c_structA *this

   def append(self, b):
       c_structA_append(self.this, b.this)

class B:
   ...


My problem is in particular, when I call append from the class A, the
reference count of b doesn't increment (since the aggregation is done in C),
so b is garbage collected and I lost any reference of c_structB (b.this)
causing a segfault when accessing it. One solution would be for example
this:

   def append(self, b):
       c_structA_append(self.this, b.this)
       self.blist.append(b)

However it seems to be a dirty solution that brings some code duplication,
what do you think about it?

Are there any best practice to handle this kind of situations?

( I'm sorry for the bad example and bad naming I choose, if you don't
understand the question I can rewrite it).

Thank you!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20100418/2ba0391a/attachment.htm 

From stefan_ml at behnel.de  Sun Apr 18 22:45:37 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 18 Apr 2010 22:45:37 +0200
Subject: [Cython] Extension types and reference counting
In-Reply-To: <h2tb0364aa51004181319o86623f93y7d553ed82839f70d@mail.gmail.com>
References: <h2tb0364aa51004181319o86623f93y7d553ed82839f70d@mail.gmail.com>
Message-ID: <4BCB6F71.1070204@behnel.de>

Gabriele Lanaro, 18.04.2010 22:19:
> I've a C where some relationship between "object" exists, like this
> pseudocode:
>
> c_structA {
>    c_structB *bstructs;
> }
>
> c_structB ...
>
> I've defined some methods to handle the memory management of these
> structures:
>
> new_c_structA
> new_c_strucB
> free_c_structA
> free_c_structB
>
> And also other functions (mehods) like:
> c_structA_append(c_structA *a, c_structB *b)
>
>
> I've to wrap these structures in cython but I encounter some problems with
> the reference counting and the garbage collector.
>
>
> class A:
>     cdef c_structA *this
>
>     def append(self, b):
>         c_structA_append(self.this, b.this)
>
> class B:
>     ...
>
>
> My problem is in particular, when I call append from the class A, the
> reference count of b doesn't increment (since the aggregation is done in C),
> so b is garbage collected and I lost any reference of c_structB (b.this)
> causing a segfault when accessing it. One solution would be for example
> this:
>
>     def append(self, b):
>         c_structA_append(self.this, b.this)
>         self.blist.append(b)
>
> However it seems to be a dirty solution

Not at all.


> Are there any best practice to handle this kind of situations?

You have two choices: let A handle all the memory allocation, or keep a 
Python reference from A to the B instances it references. You were not very 
explicit about the meaning and the relationship of the two structs, so 
depending on what they do and why they do it, either of the two solutions 
can be better.

Stefan

From gabriele.lanaro at gmail.com  Mon Apr 19 00:55:04 2010
From: gabriele.lanaro at gmail.com (Gabriele Lanaro)
Date: Mon, 19 Apr 2010 00:55:04 +0200
Subject: [Cython] Extension types and reference counting
In-Reply-To: <4BCB6F71.1070204@behnel.de>
References: <h2tb0364aa51004181319o86623f93y7d553ed82839f70d@mail.gmail.com>
	<4BCB6F71.1070204@behnel.de>
Message-ID: <g2lb0364aa51004181555l8fc2e2a5o98371cb03f3bea4c@mail.gmail.com>

The relationship was something generic, one holds a reference of the other.
I will hold a Python reference to let python handle all the reference
counting, after all it isn't so bad!

Thank you very much.

- Gabriele

2010/4/18 Stefan Behnel <stefan_ml at behnel.de>

> Gabriele Lanaro, 18.04.2010 22:19:
> > I've a C where some relationship between "object" exists, like this
> > pseudocode:
> >
> > c_structA {
> >    c_structB *bstructs;
> > }
> >
> > c_structB ...
> >
> > I've defined some methods to handle the memory management of these
> > structures:
> >
> > new_c_structA
> > new_c_strucB
> > free_c_structA
> > free_c_structB
> >
> > And also other functions (mehods) like:
> > c_structA_append(c_structA *a, c_structB *b)
> >
> >
> > I've to wrap these structures in cython but I encounter some problems
> with
> > the reference counting and the garbage collector.
> >
> >
> > class A:
> >     cdef c_structA *this
> >
> >     def append(self, b):
> >         c_structA_append(self.this, b.this)
> >
> > class B:
> >     ...
> >
> >
> > My problem is in particular, when I call append from the class A, the
> > reference count of b doesn't increment (since the aggregation is done in
> C),
> > so b is garbage collected and I lost any reference of c_structB (b.this)
> > causing a segfault when accessing it. One solution would be for example
> > this:
> >
> >     def append(self, b):
> >         c_structA_append(self.this, b.this)
> >         self.blist.append(b)
> >
> > However it seems to be a dirty solution
>
> Not at all.
>
>
> > Are there any best practice to handle this kind of situations?
>
> You have two choices: let A handle all the memory allocation, or keep a
> Python reference from A to the B instances it references. You were not very
> explicit about the meaning and the relationship of the two structs, so
> depending on what they do and why they do it, either of the two solutions
> can be better.
>
> Stefan
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20100419/b43e876f/attachment.htm 

From baihaoyu at gmail.com  Mon Apr 19 17:57:32 2010
From: baihaoyu at gmail.com (Haoyu Bai)
Date: Mon, 19 Apr 2010 23:57:32 +0800
Subject: [Cython] json to .pxd - autogen cython declaration code
In-Reply-To: <1271407753-sup-9628@farnsworth>
References: <1271407753-sup-9628@farnsworth>
Message-ID: <k2j3eb99a591004190857v34cafa54ofe19e3ef966b98f5@mail.gmail.com>

On Fri, Apr 16, 2010 at 4:59 PM, Sebastien Binet <binet at cern.ch> wrote:
> hi there,
>
> say I have a way to automatically generate a json structure
> representing C++ classes, functions,... (the code to do so uses gccxml[0]
> and Reflex[1]. once I migrated it to use the new gcc-4.5 plugin
> architecture and/or clang, I'll release it under a BSD-like licence)
>
> this json structure looks like so for the moment (from cython C++
> test-case class):
>
> {'bases': [],
> ?'dso': 'libFoo.so',
> ?'members': [{'name': 'number', 'offset': 0L, 'size': 8, 'type':
> ?'double'}],
> ?'methods': [{'args': [],
> ? ? ? ? ? ? ?'doc': '',
> ? ? ? ? ? ? ?'name': '~DoubleKeeper',
> ? ? ? ? ? ? ?'prototype': 'void DoubleKeeper::~DoubleKeeper()',
> ? ? ? ? ? ? ?'return': 'void'},
> ? ? ? ? ? ? {'args': [{'default': None,
> ? ? ? ? ? ? ? ? ? ? ? ?'name': None,
> ? ? ? ? ? ? ? ? ? ? ? ?'type': 'const DoubleKeeper&'}],
> ? ? ? ? ? ? ?'doc': '',
> ? ? ? ? ? ? ?'name': 'operator=',
> ? ? ? ? ? ? ?'prototype': 'DoubleKeeper& DoubleKeeper::operator=(const DoubleKeeper& )',
> ? ? ? ? ? ? ?'return': 'DoubleKeeper&'},
> ? ? ? ? ? ? {'args': [{'default': None,
> ? ? ? ? ? ? ? ? ? ? ? ?'name': None,
> ? ? ? ? ? ? ? ? ? ? ? ?'type': 'const DoubleKeeper&'}],
> ? ? ? ? ? ? ?'doc': '',
> ? ? ? ? ? ? ?'name': 'DoubleKeeper',
> ? ? ? ? ? ? ?'prototype': 'DoubleKeeper DoubleKeeper::DoubleKeeper(const DoubleKeeper& )',
> ? ? ? ? ? ? ?'return': 'DoubleKeeper'},
> ? ? ? ? ? ? {'args': [{'default': None,
> ? ? ? ? ? ? ? ? ? ? ? ?'name': 'number',
> ? ? ? ? ? ? ? ? ? ? ? ?'type': 'double'}],
> ? ? ? ? ? ? ?'doc': '',
> ? ? ? ? ? ? ?'name': 'DoubleKeeper',
> ? ? ? ? ? ? ?'prototype': 'DoubleKeeper DoubleKeeper::DoubleKeeper(double number)',
> ? ? ? ? ? ? ?'return': 'DoubleKeeper'},
> ? ? ? ? ? ? {'args': [{'default': None,
> ? ? ? ? ? ? ? ? ? ? ? ?'name': 'num',
> ? ? ? ? ? ? ? ? ? ? ? ?'type': 'double'}],
> ? ? ? ? ? ? ?'doc': '',
> ? ? ? ? ? ? ?'name': 'set_number',
> ? ? ? ? ? ? ?'prototype': 'void DoubleKeeper::set_number(double num)',
> ? ? ? ? ? ? ?'return': 'void'},
> ? ? ? ? ? ? {'args': [],
> ? ? ? ? ? ? ?'doc': '',
> ? ? ? ? ? ? ?'name': 'get_number',
> ? ? ? ? ? ? ?'prototype': 'double DoubleKeeper::get_number()',
> ? ? ? ? ? ? ?'return': 'double'},
> ? ? ? ? ? ? {'args': [{'default': None,
> ? ? ? ? ? ? ? ? ? ? ? ?'name': 'value',
> ? ? ? ? ? ? ? ? ? ? ? ?'type': 'double'}],
> ? ? ? ? ? ? ?'doc': '',
> ? ? ? ? ? ? ?'name': 'transmogrify',
> ? ? ? ? ? ? ?'prototype': 'double DoubleKeeper::transmogrify(double value)',
> ? ? ? ? ? ? ?'return': 'double'}],
> ?'name': 'DoubleKeeper',
> ?'size': 12,
> ?'typeinfo': '12DoubleKeeper'}
>
> which should be more or less self-describing.
>
> I am wondering if there is an API or a facility to write the
> corresponding .pxd file ?
>
> cheers,
> sebastien.
>
> [0]: http://www.gccxml.org/HTML/Index.html
> [1]: http://root.cern.ch/drupal/content/reflex
> --
> #########################################
> # Dr. Sebastien Binet
> # Laboratoire de l'Accelerateur Lineaire
> # Universite Paris-Sud XI
> # Batiment 200
> # 91898 Orsay
> #########################################
Hi,

I just found a project on GitHub which might do the thing you want:

http://github.com/cournape/cython-codegen


-- 
Haoyu BAI
School of Computing,
National University of Singapore.

From binet at cern.ch  Mon Apr 19 18:19:25 2010
From: binet at cern.ch (Sebastien Binet)
Date: Mon, 19 Apr 2010 18:19:25 +0200
Subject: [Cython] json to .pxd - autogen cython declaration code
In-Reply-To: <k2j3eb99a591004190857v34cafa54ofe19e3ef966b98f5@mail.gmail.com>
References: <1271407753-sup-9628@farnsworth>
	<k2j3eb99a591004190857v34cafa54ofe19e3ef966b98f5@mail.gmail.com>
Message-ID: <1271692944-sup-1243@farnsworth>

Excerpts from Haoyu Bai's message of 2010-04-19 17:57:32 +0200:
> Hi,
> 
> I just found a project on GitHub which might do the thing you want:
> 
> http://github.com/cournape/cython-codegen
> 

yes, I am aware of this project from (prolific!) David.
he actually went thru the "pain" of generating by hand all of the .pxd
code from the ctypeslib informations one can extract from a gccxml
produced xml file.

the problem is that ctypeslib has limited semantics when comes to C++
constructs (namespace, class inheritance,... to name a few)
...and the (LHC) experiment I will provide that code for is heavily
using C++.

I will probably re-use parts of the code which generates the .pxd
stuff as a short term solution (I am still trying to wrap my brain
against the Cython.Compiler.xyz classes to be completely confident to
be able to go the way advertised earlier in this thread)

depending of how much I reap off from David's work, I might consider
directly merge my stuff into his.

cheers,
sebastien.
-- 
#########################################
# Dr. Sebastien Binet
# Laboratoire de l'Accelerateur Lineaire
# Universite Paris-Sud XI
# Batiment 200
# 91898 Orsay
#########################################

From dagss at student.matnat.uio.no  Mon Apr 19 19:29:24 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Mon, 19 Apr 2010 19:29:24 +0200
Subject: [Cython] json to .pxd - autogen cython declaration code
In-Reply-To: <1271692944-sup-1243@farnsworth>
References: <1271407753-sup-9628@farnsworth>	<k2j3eb99a591004190857v34cafa54ofe19e3ef966b98f5@mail.gmail.com>
	<1271692944-sup-1243@farnsworth>
Message-ID: <4BCC92F4.8090505@student.matnat.uio.no>

Sebastien Binet wrote:
> Excerpts from Haoyu Bai's message of 2010-04-19 17:57:32 +0200:
>> Hi,
>>
>> I just found a project on GitHub which might do the thing you want:
>>
>> http://github.com/cournape/cython-codegen
>>
> 
> yes, I am aware of this project from (prolific!) David.
> he actually went thru the "pain" of generating by hand all of the .pxd
> code from the ctypeslib informations one can extract from a gccxml
> produced xml file.
> 
> the problem is that ctypeslib has limited semantics when comes to C++
> constructs (namespace, class inheritance,... to name a few)
> ...and the (LHC) experiment I will provide that code for is heavily
> using C++.
> 
> I will probably re-use parts of the code which generates the .pxd
> stuff as a short term solution (I am still trying to wrap my brain
> against the Cython.Compiler.xyz classes to be completely confident to
> be able to go the way advertised earlier in this thread)

Please don't hesitate to ask questions about this aspect.


> 
> depending of how much I reap off from David's work, I might consider
> directly merge my stuff into his.
> 
> cheers,
> sebastien.


-- 
Dag Sverre

From stefan_ml at behnel.de  Tue Apr 20 08:18:56 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 20 Apr 2010 08:18:56 +0200
Subject: [Cython] Allow analyse_types() to replace the current node?
Message-ID: <4BCD4750.7090006@behnel.de>

Hi,

I ran into this more than once already, so I think this is worth 
discussing. In a couple of cases, an analyse_types() method finds out that 
the current node isn't suitable for what is actually happening. There are a 
couple of different hacks that deal with that, ranging from __class__ 
replacement to a partial reconfiguration of the node together with a 
subsequent transform that replaces it.

Could we change the semantics of analyse_types() to return the current node 
instead, so that it is free to return a different node if it needs to? 
Since this isn't happening in a transform, this would mean that we need to 
change a lot of code to get the old node reference replaced wherever 
analyse_types() is called, but these places shouldn't be too hard to find 
at least.

I know that this interferes with the idea of transforms, but at least in 
some cases, it's easier to replace a node directly after finding out that 
it needs to be replaced, than to do part of the analysis now and to rely on 
a later transform to do the replacement and the correct setup of the new 
node, especially when it may have an impact on some of the surrounding 
nodes (e.g. because the new node returns a completely different type).

What do you think?

Stefan

From dagss at student.matnat.uio.no  Tue Apr 20 08:46:33 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 20 Apr 2010 08:46:33 +0200
Subject: [Cython] Allow analyse_types() to replace the current node?
In-Reply-To: <4BCD4750.7090006@behnel.de>
References: <4BCD4750.7090006@behnel.de>
Message-ID: <4BCD4DC9.2060509@student.matnat.uio.no>

Stefan Behnel wrote:
> Hi,
>
> I ran into this more than once already, so I think this is worth 
> discussing. In a couple of cases, an analyse_types() method finds out that 
> the current node isn't suitable for what is actually happening. There are a 
> couple of different hacks that deal with that, ranging from __class__ 
> replacement to a partial reconfiguration of the node together with a 
> subsequent transform that replaces it.
>
> Could we change the semantics of analyse_types() to return the current node 
> instead, so that it is free to return a different node if it needs to? 
> Since this isn't happening in a transform, this would mean that we need to 
> change a lot of code to get the old node reference replaced wherever 
> analyse_types() is called, but these places shouldn't be too hard to find 
> at least.
>
> I know that this interferes with the idea of transforms, but at least in 
> some cases, it's easier to replace a node directly after finding out that 
> it needs to be replaced, than to do part of the analysis now and to rely on 
> a later transform to do the replacement and the correct setup of the new 
> node, especially when it may have an impact on some of the surrounding 
> nodes (e.g. because the new node returns a completely different type).
>
> What do you think?
>   
+1. I don't think it really interferes with the idea of transforms; it's 
more like going only half the way because it is less likely to break the 
code.

If somebody is preparing 0.13 for release then that somebody will have 
to comment on whether it should wait until after the release though.

Dag Sverre

From efiring at hawaii.edu  Tue Apr 20 09:04:12 2010
From: efiring at hawaii.edu (Eric Firing)
Date: Mon, 19 Apr 2010 21:04:12 -1000
Subject: [Cython] bug in numpy.pxd, with patch
Message-ID: <4BCD51EC.2060800@hawaii.edu>

Two declarations in numpy.pxd are missing type specifiers; a patch is 
attached.

Eric
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cython.diff
Type: text/x-patch
Size: 771 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20100419/f14c6404/attachment.bin 

From dagss at student.matnat.uio.no  Tue Apr 20 10:17:29 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 20 Apr 2010 10:17:29 +0200
Subject: [Cython] bug in numpy.pxd, with patch
In-Reply-To: <4BCD51EC.2060800@hawaii.edu>
References: <4BCD51EC.2060800@hawaii.edu>
Message-ID: <4BCD6319.6050806@student.matnat.uio.no>

Eric Firing wrote:
> Two declarations in numpy.pxd are missing type specifiers; a patch is 
> attached.
Thanks! Pushed:

http://hg.cython.org/cython-devel/rev/47d0f9226ea3

Dag Sverre

From cournape at gmail.com  Tue Apr 20 12:06:17 2010
From: cournape at gmail.com (David Cournapeau)
Date: Tue, 20 Apr 2010 19:06:17 +0900
Subject: [Cython] json to .pxd - autogen cython declaration code
In-Reply-To: <k2j3eb99a591004190857v34cafa54ofe19e3ef966b98f5@mail.gmail.com>
References: <1271407753-sup-9628@farnsworth>
	<k2j3eb99a591004190857v34cafa54ofe19e3ef966b98f5@mail.gmail.com>
Message-ID: <h2i5b8d13221004200306m72bcc0cs3e41ba0c5ff49c38@mail.gmail.com>

On Tue, Apr 20, 2010 at 12:57 AM, Haoyu Bai <baihaoyu at gmail.com> wrote:

> Hi,
>
> I just found a project on GitHub which might do the thing you want:
>
> http://github.com/cournape/cython-codegen

That code is horrible though :) It worked for me to wrap big header
files, but that's not really usable. Gccxml is horrible to work with,
I think a much better solution would be to reuse clang - I started
looking at it, and it looked doable, but I don't have the time to work
on it ATM,

http://amnoid.de/tmp/clangtut/tut.html

cheers,

David

From binet at cern.ch  Tue Apr 20 12:53:04 2010
From: binet at cern.ch (Sebastien Binet)
Date: Tue, 20 Apr 2010 12:53:04 +0200
Subject: [Cython] json to .pxd - autogen cython declaration code
In-Reply-To: <h2i5b8d13221004200306m72bcc0cs3e41ba0c5ff49c38@mail.gmail.com>
References: <1271407753-sup-9628@farnsworth>
	<k2j3eb99a591004190857v34cafa54ofe19e3ef966b98f5@mail.gmail.com>
	<h2i5b8d13221004200306m72bcc0cs3e41ba0c5ff49c38@mail.gmail.com>
Message-ID: <1271760467-sup-7955@farnsworth>

Excerpts from David Cournapeau's message of 2010-04-20 12:06:17 +0200:
> On Tue, Apr 20, 2010 at 12:57 AM, Haoyu Bai <baihaoyu at gmail.com> wrote:
> 
> > Hi,
> >
> > I just found a project on GitHub which might do the thing you want:
> >
> > http://github.com/cournape/cython-codegen
> 
> That code is horrible though :)

don't worry about that, I am dealing with high energy physicists (C++
and python) code on a daily basis :)
...and I am guilty of a few programming "gems" too.

> It worked for me to wrap big header
> files, but that's not really usable. Gccxml is horrible to work with,
> I think a much better solution would be to reuse clang - I started
> looking at it, and it looked doable, but I don't have the time to work
> on it ATM,
> 
> http://amnoid.de/tmp/clangtut/tut.html

thanks for the link.
llvm/clang was on my radar (I played a bit w/ llvm-py) but this
tutorial will most definitely help !

cheers,
sebastien.
-- 
#########################################
# Dr. Sebastien Binet
# Laboratoire de l'Accelerateur Lineaire
# Universite Paris-Sud XI
# Batiment 200
# 91898 Orsay
#########################################

From wing1127aishi at gmail.com  Tue Apr 20 18:42:56 2010
From: wing1127aishi at gmail.com (Leon Sit)
Date: Tue, 20 Apr 2010 11:42:56 -0500
Subject: [Cython] Cannot compile cython on Windows with visual studio
Message-ID: <k2q3af1998d1004200942v8dc70aa2z28f8f8b398b0ce8b@mail.gmail.com>

I was trying to compile cython on windows and I get the following error

c:\Users\leon.sit\Desktop>c:\Python26\python.exe
cython-8bff3332e34f\setup.py install
running install
running build
running build_py
file pyximport/__init__.py (for module pyximport/__init__) not found
file pyximport/pyximport.py (for module pyximport/pyximport) not found
file pyximport/pyxbuild.py (for module pyximport/pyxbuild) not found
file cython.py (for module cython) not found
package init file 'Cython\__init__.py' not found (or not a regular file)
creating build
creating build\lib.win32-2.6
creating build\lib.win32-2.6\Cython
copying Cython\cython.py -> build\lib.win32-2.6\Cython
copying Cython\runtests.py -> build\lib.win32-2.6\Cython
copying Cython\setup.py -> build\lib.win32-2.6\Cython
copying Cython\setupegg.py -> build\lib.win32-2.6\Cython
error: package directory 'Cython\Compiler' does not exist
--------------------------------------------------------------------------------------

I am using

Python 2.6.2 (r262:71605, Apr 14 2009, 22:40:02) [MSC v.1500 32 bit
(Intel)] on win32

and I have VS2005 and VS2008 on my computer.

Any pointer is appreciated.


Best Regards,

Leon Sit
Graduate Student in Statistics at Rutgers University
wing1127aishi at gmail.com
http://eden.rutgers.edu/~wingsit/

From stefan_ml at behnel.de  Tue Apr 20 19:45:05 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 20 Apr 2010 19:45:05 +0200
Subject: [Cython] Cannot compile cython on Windows with visual studio
In-Reply-To: <k2q3af1998d1004200942v8dc70aa2z28f8f8b398b0ce8b@mail.gmail.com>
References: <k2q3af1998d1004200942v8dc70aa2z28f8f8b398b0ce8b@mail.gmail.com>
Message-ID: <4BCDE821.1070609@behnel.de>

Leon Sit, 20.04.2010 18:42:
> I was trying to compile cython on windows and I get the following error
>
> c:\Users\leon.sit\Desktop>c:\Python26\python.exe
> cython-8bff3332e34f\setup.py install

Change to the directory "cython-8bff3332e34f" and run the setup.py script 
from there.

Stefan

From wing1127aishi at gmail.com  Tue Apr 20 20:44:10 2010
From: wing1127aishi at gmail.com (Leon Sit)
Date: Tue, 20 Apr 2010 13:44:10 -0500
Subject: [Cython] Cython-dev Digest, Vol 28, Issue 12
In-Reply-To: <mailman.2053.1271785525.6215.cython-dev@codespeak.net>
References: <mailman.2053.1271785525.6215.cython-dev@codespeak.net>
Message-ID: <m2g3af1998d1004201144p78363f92l92cf19051b627d9@mail.gmail.com>

Thanks. That works. Didn't know that setup.py takes absolute path from
command line instead of location of script.

Best Regards,

Leon Sit
Graduate Student in Statistics at Rutgers University
wing1127aishi at gmail.com
http://eden.rutgers.edu/~wingsit/

> Message: 15
> Date: Tue, 20 Apr 2010 19:45:05 +0200
> From: Stefan Behnel <stefan_ml at behnel.de>
> Subject: Re: [Cython] Cannot compile cython on Windows with visual
> ? ? ? ?studio
> To: cython-dev at codespeak.net
> Message-ID: <4BCDE821.1070609 at behnel.de>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Leon Sit, 20.04.2010 18:42:
>> I was trying to compile cython on windows and I get the following error
>>
>> c:\Users\leon.sit\Desktop>c:\Python26\python.exe
>> cython-8bff3332e34f\setup.py install
>
> Change to the directory "cython-8bff3332e34f" and run the setup.py script
> from there.
>
> Stefan
>
>
> ------------------------------
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>
>
> End of Cython-dev Digest, Vol 28, Issue 12
> ******************************************
>

From izarf.mail at gmail.com  Wed Apr 21 10:22:53 2010
From: izarf.mail at gmail.com (Izarf)
Date: Wed, 21 Apr 2010 10:22:53 +0200
Subject: [Cython] Compiling extensions with cl.exe
Message-ID: <o2madadc6921004210122za1bfeb7av170764d83494072a@mail.gmail.com>

All examples I have seen in compiling has been using gcc.
Could some one please give me an example compiling the generated c-code with
visual studios cl.exe? I also need to include a library (dokan.lib).

like: cl.exe generated-extension-code.c [link with dokan.lib] [make to DLL?]
-- 
-Elias
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20100421/aa61d4c4/attachment.htm 

From dalcinl at gmail.com  Wed Apr 21 16:18:21 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 21 Apr 2010 11:18:21 -0300
Subject: [Cython] Compiling extensions with cl.exe
In-Reply-To: <o2madadc6921004210122za1bfeb7av170764d83494072a@mail.gmail.com>
References: <o2madadc6921004210122za1bfeb7av170764d83494072a@mail.gmail.com>
Message-ID: <n2se7ba66e41004210718t58c30d02n86953d4f79184c43@mail.gmail.com>

On 21 April 2010 05:22, Izarf <izarf.mail at gmail.com> wrote:
> All examples I have seen in compiling has been using gcc.
> Could some one please give me an example compiling the generated c-code with
> visual studios cl.exe? I also need to include a library (dokan.lib).
>
> like: cl.exe generated-extension-code.c [link with dokan.lib] [make to DLL?]

Write a proper setup.py file, and Python distutils will pick MSVC for
you and handle the gory details. Refer to the distutils documentation
to figure out how to add define macros, include dirs, libraries,
library dirs and all that.

PS: This question is more appropriate for cython-users


-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From dalcinl at gmail.com  Thu Apr 22 03:41:38 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 21 Apr 2010 22:41:38 -0300
Subject: [Cython] push related to ticket #399
Message-ID: <u2re7ba66e41004211841se539a4bcu47dca43789efdb39@mail.gmail.com>

I've not closed the ticket, because I would like to know your
opinions. For your convenience, you have below my comments.

Pushed: http://hg.cython.org/cython-devel/rev/1a9bfb4ff18a
Tested: Linux32/64 and Windows32 (MSVC and MinGW)

Comments: 'ssize_t' is not a C99 standard type. If available, core
Python(>2.4) defines Py_ssize_t to ssize_t. If ssize_t is not
available, then any Cython code using ssize_t will fail at C compile
time. I do not think we should #define or typedef ssize_t if it is
missing, but raise your voice if you do not agree. For Py>=2.5,
detecting a missing ssize_t is trivial (macro available pyconfig.h)
and then we could conditionally "#define ssize_t Py_ssize_t" (or
ptrdiff_t).

For older Python(<2.5), the to_py/from_py conversions will end-up
being performed with the PyInt_{As|From}Long() functions, so things
will be fine for Linux{32|64}, Windows32, and not sure what happens on
OS X 32/64.

PS: From the list of mayor platforms, Win64 is the only one where
things are broken for Py<2.5 . Provided that full 64bits support was
introduced in Py2.5, I do not actually care too much. If anyone
complains about this, I'll politely ask her to donate a Win64 license
;-) in order to support Cython development.


-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From dalcinl at gmail.com  Thu Apr 22 03:48:10 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 21 Apr 2010 22:48:10 -0300
Subject: [Cython] help me to make "runtests.py --compiler=mingw32" work
Message-ID: <k2me7ba66e41004211848n2c01b32ewd401c5202d187492@mail.gmail.com>

Could any of you help me to complete the patch?

diff -r 1a9bfb4ff18a runtests.py
--- a/runtests.py	Wed Apr 21 21:47:54 2010 -0300
+++ b/runtests.py	Wed Apr 21 22:45:19 2010 -0300
@@ -213,6 +213,7 @@
         self.test_directory = test_directory
         self.workdir = workdir
         self.module = module
+        self.compiler = None#'mingw32'
         self.language = language
         self.expect_errors = expect_errors
         self.annotate = annotate
@@ -334,10 +335,13 @@
         os.chdir(workdir)
         try:
             build_extension = build_ext(distutils_distro)
+            build_extension.initialize_options()
             build_extension.include_dirs = INCLUDE_DIRS[:]
             if incdir:
                 build_extension.include_dirs.append(incdir)
             build_extension.finalize_options()
+            if self.compiler:
+                build_extension.compiler = self.compiler
             ext_include_dirs = []
             for match, get_additional_include_dirs in EXT_DEP_INCLUDES:
                 if match(module):


-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From cournape at gmail.com  Thu Apr 22 05:09:43 2010
From: cournape at gmail.com (David Cournapeau)
Date: Thu, 22 Apr 2010 12:09:43 +0900
Subject: [Cython] json to .pxd - autogen cython declaration code
In-Reply-To: <1271760467-sup-7955@farnsworth>
References: <1271407753-sup-9628@farnsworth>
	<k2j3eb99a591004190857v34cafa54ofe19e3ef966b98f5@mail.gmail.com>
	<h2i5b8d13221004200306m72bcc0cs3e41ba0c5ff49c38@mail.gmail.com>
	<1271760467-sup-7955@farnsworth>
Message-ID: <x2p5b8d13221004212009jb32485a7wbc4a043edd15c686@mail.gmail.com>

On Tue, Apr 20, 2010 at 7:53 PM, Sebastien Binet <binet at cern.ch> wrote:
> Excerpts from David Cournapeau's message of 2010-04-20 12:06:17 +0200:
>> On Tue, Apr 20, 2010 at 12:57 AM, Haoyu Bai <baihaoyu at gmail.com> wrote:
>>
>> > Hi,
>> >
>> > I just found a project on GitHub which might do the thing you want:
>> >
>> > http://github.com/cournape/cython-codegen
>>
>> That code is horrible though :)
>
> don't worry about that, I am dealing with high energy physicists (C++
> and python) code on a daily basis :)

I love the  researchers/C++ combination, always full of surprises :)

I mostly meant that something reusable should not be based on this
code at all - neither the implementation nor the idea of reusing
gccxml makes much sense for something meant to be used in the long
term. Gccxml is not that great, because it is not really maintained, a
PITA to build, and you cannot force C mode (which was the biggest
issue in my case, and I guess an issue for many cython users).

>
> thanks for the link.
> llvm/clang was on my radar (I played a bit w/ llvm-py) but this
> tutorial will most definitely help !

Yes, unfortunately, there is no python API, and llvm is not as mature
as I had hoped (the API has significantly changed since this tutorial,
for example). OTOH, for C++, llvm code is relatively readable, and
there are also an ocaml interface and a haskell one
(http://augustss.blogspot.com/) - maybe a good excuse to learn
haskell.

It took me half a day to be able to implement a basic pre-processor in
C++ with llvm, so anyone more familiar with C++ (and parsing
techniques) should be able to implement something useful in a short
amount of time,

cheers,

David

From stefan_ml at behnel.de  Thu Apr 22 07:37:03 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 22 Apr 2010 07:37:03 +0200
Subject: [Cython] coercion of char/Py_UNICODE to Python objects - string or
	integer?
In-Reply-To: <g2je7ba66e41004211426s3ab87024iedcf5c8e0d8be19a@mail.gmail.com>
References: <g2je7ba66e41004211426s3ab87024iedcf5c8e0d8be19a@mail.gmail.com>
Message-ID: <4BCFE07F.70307@behnel.de>

Lisandro Dalcin, 21.04.2010 23:26:
> What do you think?
>
> diff -r 2701901737d4 Cython/Compiler/PyrexTypes.py
> --- a/Cython/Compiler/PyrexTypes.py	Wed Apr 21 15:36:27 2010 +0200
> +++ b/Cython/Compiler/PyrexTypes.py	Wed Apr 21 18:25:42 2010 -0300
> @@ -871,7 +871,7 @@
>       # to integers here.  The maximum value for a Py_UNICODE is
>       # 1114111, so PyInt_FromLong() will do just fine here.
>
> -    to_py_function = "PyInt_FromLong"
> +    to_py_function = "PyUnicode_FromOrdinal"
>
>       def sign_and_name(self):
>           return "Py_UNICODE"

I didn't know about that function, even though I had looked for it in the 
CPython docs. It's available in all relevant CPython versions, and it's 
pretty efficient, too.

This would let Py_UNICODE values turn into a single character unicode 
string when coercing to a Python object. I had also thought about this, and 
wasn't sure what I wanted. In current Cython, 'char' doesn't coerce to a 
single character 'bytes' object but to an integer. My thinking was that 
Py_UNICODE should behave the same.

This is a bit inconsistent in itself, given that single character strings 
can coerce to their C ordinal value, e.g. on comparison with 
char/Py_UNICODE, but not so much of an inconsistency to break backwards 
compatibility. I'm really not sure what the 'expected' behaviour is here, 
although I'm leaning slightly towards the char/bytes and Py_UNICODE/unicode 
coercion.

It's certainly easier to write

     cdef Py_UNICODE cval = some_c_integer

     py_object = <long>cval

to get a Python integer value, than to find, import and call 
PyUnicode_FromOrdinal() to get a unicode string. There doesn't seem to be 
an equivalent PyBytes function, so I guess the PyBytes conversion would use

     py_bytes = PyBytes_FromStringAndSize(&char_val, 1)

which isn't exactly beautiful either, and certainly less so than the opposite

     py_integer = <int>char_val

This would also speak in favour of letting char and Py_UNICODE coerce to 
Python strings by default, although the above would go away if we special 
cased the builtin chr() function to output exactly the above code for each 
input type.

Another option is to consider Py_UNICODE more special (and more specific) 
than the somewhat generic 'char', and to accept the inconsistency of 
coercing one to a unicode string and the other to an integer.

What do the others think?

Stefan

From craigcitro at gmail.com  Thu Apr 22 08:54:34 2010
From: craigcitro at gmail.com (Craig Citro)
Date: Wed, 21 Apr 2010 23:54:34 -0700
Subject: [Cython] Allow analyse_types() to replace the current node?
In-Reply-To: <4BCD4DC9.2060509@student.matnat.uio.no>
References: <4BCD4750.7090006@behnel.de>
	<4BCD4DC9.2060509@student.matnat.uio.no>
Message-ID: <t2xa8c4ff4c1004212354u98cb8824ka8f63adbcdbce3ef@mail.gmail.com>

>> What do you think?
>>
> +1. I don't think it really interferes with the idea of transforms; it's
> more like going only half the way because it is less likely to break the
> code.
>

I'm also +1 on this -- in general, I'm happier to see the tree get
modified when you know it needs to be rather than try to "leave a
note" and do the right thing later. Also, there currently seems to be
a mix of behavior for analyse_types (i.e. most seem to just use
side-effects to get the job done, but a few return values) -- it would
be nice if these ended up being uniform.

> If somebody is preparing 0.13 for release then that somebody will have
> to comment on whether it should wait until after the release though.
>

I'm currently actively working on the 0.13 release. Stefan, have you
started already writing the code? If not, or if you're just starting,
I'd say we should wait until after the release ...

-cc

From stefan_ml at behnel.de  Thu Apr 22 09:06:57 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 22 Apr 2010 09:06:57 +0200
Subject: [Cython] Allow analyse_types() to replace the current node?
In-Reply-To: <t2xa8c4ff4c1004212354u98cb8824ka8f63adbcdbce3ef@mail.gmail.com>
References: <4BCD4750.7090006@behnel.de>	<4BCD4DC9.2060509@student.matnat.uio.no>
	<t2xa8c4ff4c1004212354u98cb8824ka8f63adbcdbce3ef@mail.gmail.com>
Message-ID: <4BCFF591.3070002@behnel.de>

Craig Citro, 22.04.2010 08:54:
>>> What do you think?
>>>
>> +1. I don't think it really interferes with the idea of transforms; it's
>> more like going only half the way because it is less likely to break the
>> code.
>
> I'm also +1 on this -- in general, I'm happier to see the tree get
> modified when you know it needs to be rather than try to "leave a
> note" and do the right thing later. Also, there currently seems to be
> a mix of behavior for analyse_types (i.e. most seem to just use
> side-effects to get the job done, but a few return values) -- it would
> be nice if these ended up being uniform.
 >
>> If somebody is preparing 0.13 for release then that somebody will have
>> to comment on whether it should wait until after the release though.
>
> I'm currently actively working on the 0.13 release. Stefan, have you
> started already writing the code? If not, or if you're just starting,
> I'd say we should wait until after the release ...

Nothing started yet, so it looks like the plan is to release 0.13 and do 
the refactoring after that.

Stefan

From dagss at student.matnat.uio.no  Thu Apr 22 09:52:34 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Thu, 22 Apr 2010 09:52:34 +0200
Subject: [Cython] push related to ticket #399
In-Reply-To: <u2re7ba66e41004211841se539a4bcu47dca43789efdb39@mail.gmail.com>
References: <u2re7ba66e41004211841se539a4bcu47dca43789efdb39@mail.gmail.com>
Message-ID: <4BD00042.4040002@student.matnat.uio.no>

Lisandro Dalcin wrote:
> I've not closed the ticket, because I would like to know your
> opinions. For your convenience, you have below my comments.
>
> Pushed: http://hg.cython.org/cython-devel/rev/1a9bfb4ff18a
> Tested: Linux32/64 and Windows32 (MSVC and MinGW)
>
> Comments: 'ssize_t' is not a C99 standard type. If available, core
> Python(>2.4) defines Py_ssize_t to ssize_t. If ssize_t is not
> available, then any Cython code using ssize_t will fail at C compile
> time. I do not think we should #define or typedef ssize_t if it is
> missing, but raise your voice if you do not agree. For Py>=2.5,
> detecting a missing ssize_t is trivial (macro available pyconfig.h)
> and then we could conditionally "#define ssize_t Py_ssize_t" (or
> ptrdiff_t).
>   
Hmm.

My vote is in favor of simply always making "ssize_t" in Cython always 
mean Py_ssize_t in C.

This is because I want some Py_ssize_t without the special index 
behaviour, so that I can start recommend using it in numerical loops. 
Py_ssize_t is the "correct" type to use for indexing NumPy arrays, but 
has problems when using it in mathematical expressions because of the 
special index coercion behaviour.

Dag SVrre

From stefan_ml at behnel.de  Thu Apr 22 13:19:24 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 22 Apr 2010 13:19:24 +0200
Subject: [Cython] push related to ticket #399
In-Reply-To: <u2re7ba66e41004211841se539a4bcu47dca43789efdb39@mail.gmail.com>
References: <u2re7ba66e41004211841se539a4bcu47dca43789efdb39@mail.gmail.com>
Message-ID: <4BD030BC.2090309@behnel.de>

Lisandro Dalcin, 22.04.2010 03:41:
> PS: From the list of mayor platforms, Win64 is the only one where
> things are broken for Py<2.5 . Provided that full 64bits support was
> introduced in Py2.5, I do not actually care too much.

The rule of thumb here is to make sure it works with 32bit on Py<2.5 and 
support full 64bit from Py2.5 onwards. Cython requires a working CPython 
runtime for the platform at hand, so trying to do anything more than that 
is futile.

Stefan

From cournape at gmail.com  Fri Apr 23 03:52:03 2010
From: cournape at gmail.com (David Cournapeau)
Date: Fri, 23 Apr 2010 10:52:03 +0900
Subject: [Cython] push related to ticket #399
In-Reply-To: <4BD030BC.2090309@behnel.de>
References: <u2re7ba66e41004211841se539a4bcu47dca43789efdb39@mail.gmail.com>
	<4BD030BC.2090309@behnel.de>
Message-ID: <o2m5b8d13221004221852w31705b8v2699c867c4e674ab@mail.gmail.com>

On Thu, Apr 22, 2010 at 8:19 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Lisandro Dalcin, 22.04.2010 03:41:
>> PS: From the list of mayor platforms, Win64 is the only one where
>> things are broken for Py<2.5 . Provided that full 64bits support was
>> introduced in Py2.5, I do not actually care too much.
>
> The rule of thumb here is to make sure it works with 32bit on Py<2.5 and
> support full 64bit from Py2.5 onwards. Cython requires a working CPython
> runtime for the platform at hand, so trying to do anything more than that
> is futile.

I would not even care for python 2.5 for windows 64, as the 64 bits MS
toolsuite used in 2.5 (VS 2005) is a PITA to use and setup.

David

From dalcinl at gmail.com  Fri Apr 23 05:06:52 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Fri, 23 Apr 2010 00:06:52 -0300
Subject: [Cython] push related to ticket #399
In-Reply-To: <o2m5b8d13221004221852w31705b8v2699c867c4e674ab@mail.gmail.com>
References: <u2re7ba66e41004211841se539a4bcu47dca43789efdb39@mail.gmail.com>
	<4BD030BC.2090309@behnel.de>
	<o2m5b8d13221004221852w31705b8v2699c867c4e674ab@mail.gmail.com>
Message-ID: <g2ue7ba66e41004222006vebdbec8an8a5feff99258beec@mail.gmail.com>

On 22 April 2010 22:52, David Cournapeau <cournape at gmail.com> wrote:
> On Thu, Apr 22, 2010 at 8:19 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
>> Lisandro Dalcin, 22.04.2010 03:41:
>>> PS: From the list of mayor platforms, Win64 is the only one where
>>> things are broken for Py<2.5 . Provided that full 64bits support was
>>> introduced in Py2.5, I do not actually care too much.
>>
>> The rule of thumb here is to make sure it works with 32bit on Py<2.5 and
>> support full 64bit from Py2.5 onwards. Cython requires a working CPython
>> runtime for the platform at hand, so trying to do anything more than that
>> is futile.
>
> I would not even care for python 2.5 for windows 64, as the 64 bits MS
> toolsuite used in 2.5 (VS 2005) is a PITA to use and setup.
>

Thanks for your comments. BTW, if any of you have access to a Win64
machine, please try to run the testsuite. I have no idea how Cython
behaves in that platform.


-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From cournape at gmail.com  Fri Apr 23 05:44:24 2010
From: cournape at gmail.com (David Cournapeau)
Date: Fri, 23 Apr 2010 12:44:24 +0900
Subject: [Cython] push related to ticket #399
In-Reply-To: <g2ue7ba66e41004222006vebdbec8an8a5feff99258beec@mail.gmail.com>
References: <u2re7ba66e41004211841se539a4bcu47dca43789efdb39@mail.gmail.com>
	<4BD030BC.2090309@behnel.de>
	<o2m5b8d13221004221852w31705b8v2699c867c4e674ab@mail.gmail.com>
	<g2ue7ba66e41004222006vebdbec8an8a5feff99258beec@mail.gmail.com>
Message-ID: <s2l5b8d13221004222044k589a7af8s26ef4cf9bb8361d5@mail.gmail.com>

On Fri, Apr 23, 2010 at 12:06 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:

>
> Thanks for your comments. BTW, if any of you have access to a Win64
> machine, please try to run the testsuite. I have no idea how Cython
> behaves in that platform.

Cygwin is limited to 32 bits, both as a host and target platform
(using the autotools convention of build/host/target), and I don't
think you can even launch 64 bits .exe from it. That's why building
ATLAS is not possible BTW,

cheers,

David

From berthold at despammed.com  Fri Apr 23 11:49:46 2010
From: berthold at despammed.com (Berthold Hoellmann)
Date: Fri, 23 Apr 2010 11:49:46 +0200
Subject: [Cython] bug in numpy.pxd, with patch
References: <4BCD51EC.2060800@hawaii.edu>
	<4BCD6319.6050806@student.matnat.uio.no>
Message-ID: <seiq7isdl1.fsf@pc081634.dhcp.germanlloyd.org>

Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> writes:

> Eric Firing wrote:
>> Two declarations in numpy.pxd are missing type specifiers; a patch is 
>> attached.
> Thanks! Pushed:
>
> http://hg.cython.org/cython-devel/rev/47d0f9226ea3
>
> Dag Sverre

Why is it

    bint PyArray_ZEROS(ndarray m, npy_intp* dims, int type, int fortran)

and not 

    object PyArray_ZEROS(int nd, npy_intp* dims, int type, int fortran)


?

Berthold H?llmann
-- 
                            __   Address:
                         G /  \ L Germanischer Lloyd
phone: +49-40-36149-7374 -+----+- Brooktorkai 18
fax  : +49-40-36149-7320   \__/   D-20457 Hamburg


From stefan_ml at behnel.de  Fri Apr 23 12:52:00 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 23 Apr 2010 12:52:00 +0200
Subject: [Cython] bug in numpy.pxd, with patch
In-Reply-To: <seiq7isdl1.fsf@pc081634.dhcp.germanlloyd.org>
References: <4BCD51EC.2060800@hawaii.edu>	<4BCD6319.6050806@student.matnat.uio.no>
	<seiq7isdl1.fsf@pc081634.dhcp.germanlloyd.org>
Message-ID: <4BD17BD0.10406@behnel.de>

Berthold Hoellmann, 23.04.2010 11:49:
> Dag Sverre Seljebotn writes:
>
>> Eric Firing wrote:
>>> Two declarations in numpy.pxd are missing type specifiers; a patch is
>>> attached.
>> Thanks! Pushed:
>>
>> http://hg.cython.org/cython-devel/rev/47d0f9226ea3
>>
>> Dag Sverre
>
> Why is it
>
>      bint PyArray_ZEROS(ndarray m, npy_intp* dims, int type, int fortran)
>
> and not
>
>      object PyArray_ZEROS(int nd, npy_intp* dims, int type, int fortran)

Thanks. Fixed also for PyArray_EMPTY().

Stefan

From dalcinl at gmail.com  Fri Apr 23 18:05:38 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Fri, 23 Apr 2010 13:05:38 -0300
Subject: [Cython] support arbitrarily sized typedef integral types
Message-ID: <y2ve7ba66e41004230905g8c5f2a95y8c9cb2cc2ff2b277@mail.gmail.com>

http://hg.cython.org/cython-devel/rev/473fa96caa93

BTW, If some one could review the C++ class I had to write to make
proper testing, that would be great.

-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From vinjvinj at gmail.com  Fri Apr 23 19:14:24 2010
From: vinjvinj at gmail.com (Vineet Jain)
Date: Fri, 23 Apr 2010 13:14:24 -0400
Subject: [Cython] Question on generator support
Message-ID: <s2w70a7f84f1004231014oa9605faei9674c0c12c8cfecd@mail.gmail.com>

Any ideas on when generator support is going to be available? On trac, it's
classified under the 1.0 milestone release. Is it safe to assume we are not
going to have generators in the  0.13 release?

Thanks,

Vineet
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20100423/910c34a7/attachment.htm 

From craigcitro at gmail.com  Fri Apr 23 20:01:10 2010
From: craigcitro at gmail.com (Craig Citro)
Date: Fri, 23 Apr 2010 11:01:10 -0700
Subject: [Cython] Question on generator support
In-Reply-To: <s2w70a7f84f1004231014oa9605faei9674c0c12c8cfecd@mail.gmail.com>
References: <s2w70a7f84f1004231014oa9605faei9674c0c12c8cfecd@mail.gmail.com>
Message-ID: <q2ga8c4ff4c1004231101o39084e4do274e48c045c4e6d0@mail.gmail.com>

> Any ideas on when generator support is going to be available? On trac, it's
> classified under the 1.0 milestone release. Is it safe to assume we are not
> going to have generators in the ?0.13 release?
>

I think it's a safe bet that generators won't be in 0.13 -- I'm still
hopeful that closures will be, but that's still a toss-up. I think
that there's going to be a flurry of activity come late May/June --
Robert will be done with his PhD, and I'll be done teaching/moving,
and I think Dag and Stefan will also have more free time coming soon.

-cc

From wing1127aishi at gmail.com  Fri Apr 23 20:33:21 2010
From: wing1127aishi at gmail.com (Leon Sit)
Date: Fri, 23 Apr 2010 13:33:21 -0500
Subject: [Cython] Wiki is down?
Message-ID: <m2n3af1998d1004231133sef49f9f1l8684732124891c11@mail.gmail.com>

When I try to access wiki, I get the message

"Warning:
You triggered the wiki's surge protection by doing too many requests
in a short time.
Please make a short break reading the stuff you already got.
When you restart doing requests AFTER that, slow down or you might get
locked out for a longer time!"

I havent accessed the wiki for few days already. Is there other places
that I can access the http://wiki.cython.org/WrappingCPlusPlus page?

Thanks

Best Regards,

Leon Sit
Graduate Student in Statistics at Rutgers University
wing1127aishi at gmail.com
http://eden.rutgers.edu/~wingsit/

From dsurviver at gmail.com  Fri Apr 23 22:45:08 2010
From: dsurviver at gmail.com (Danilo Freitas)
Date: Fri, 23 Apr 2010 17:45:08 -0300
Subject: [Cython] Wiki is down?
In-Reply-To: <m2n3af1998d1004231133sef49f9f1l8684732124891c11@mail.gmail.com>
References: <m2n3af1998d1004231133sef49f9f1l8684732124891c11@mail.gmail.com>
Message-ID: <i2z45239151004231345s5feef397of51fc7e1b27dcf49@mail.gmail.com>

Here it is working fine.

There's a similiar page [0], with some examples. It's not really a
tutorial, but should help for now.

Note that these examples won't work with Cython 0.12.1. They'll work
only with cython-devel branch.

[0]: http://wiki.cython.org/gsoc09/daniloaf/progress
[1]: http://hg.cython.org/cython-devel/

2010/4/23 Leon Sit <wing1127aishi at gmail.com>:
> When I try to access wiki, I get the message
>
> "Warning:
> You triggered the wiki's surge protection by doing too many requests
> in a short time.
> Please make a short break reading the stuff you already got.
> When you restart doing requests AFTER that, slow down or you might get
> locked out for a longer time!"
>
> I havent accessed the wiki for few days already. Is there other places
> that I can access the http://wiki.cython.org/WrappingCPlusPlus page?
>
> Thanks
>
> Best Regards,
>
> Leon Sit
> Graduate Student in Statistics at Rutgers University
> wing1127aishi at gmail.com
> http://eden.rutgers.edu/~wingsit/
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
- Danilo Freitas

From dalcinl at gmail.com  Fri Apr 23 23:22:53 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Fri, 23 Apr 2010 18:22:53 -0300
Subject: [Cython] Cython&Pyrex: grammar for function declarators
Message-ID: <q2he7ba66e41004231422j2676fcb0z55d3e89c93cd0fb0@mail.gmail.com>

Consider this declaration, where any of the [nogil], [except*], [with
gil] bits could be present or absent::

    ctypedef int (*foo)() nogil except* with gil

The corresponding parser lines are the one below::

	nogil = p_nogil(s)
	exc_val, exc_check = p_exception_value_clause(s)
	with_gil = p_with_gil(s)


It is somewhat unintuitive/contrived ::

(1) What "int(*foo)() nogil with gil" does actually mean?

(2)  "int(*foo)() nogil with gil" is redundant, as "int(*foo)() with
gil" would be enough (i.e., implies "nogil").

(3) I "int (*foo)() except* nogil" is illegal syntax, but "int
(*foo)() except* with gil" is legal.


>From that list, what really annoys me is (3). What do you think?

Greg, Is there a rationale for all that?


-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From zstone at gmail.com  Sat Apr 24 00:19:16 2010
From: zstone at gmail.com (Zak Stone)
Date: Fri, 23 Apr 2010 18:19:16 -0400
Subject: [Cython] Wiki is down?
In-Reply-To: <m2n3af1998d1004231133sef49f9f1l8684732124891c11@mail.gmail.com>
References: <m2n3af1998d1004231133sef49f9f1l8684732124891c11@mail.gmail.com>
Message-ID: <o2r6629e85c1004231519ze67cbe91w132c2eea9ea8f21f@mail.gmail.com>

The wiki is also down for me -- I received the "surge protection"
message on my first visit today and every attempt thereafter.

Zak


On Fri, Apr 23, 2010 at 2:33 PM, Leon Sit <wing1127aishi at gmail.com> wrote:
> When I try to access wiki, I get the message
>
> "Warning:
> You triggered the wiki's surge protection by doing too many requests
> in a short time.
> Please make a short break reading the stuff you already got.
> When you restart doing requests AFTER that, slow down or you might get
> locked out for a longer time!"
>
> I havent accessed the wiki for few days already. Is there other places
> that I can access the http://wiki.cython.org/WrappingCPlusPlus page?
>
> Thanks
>
> Best Regards,
>
> Leon Sit
> Graduate Student in Statistics at Rutgers University
> wing1127aishi at gmail.com
> http://eden.rutgers.edu/~wingsit/
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>

From craigcitro at gmail.com  Sat Apr 24 01:03:19 2010
From: craigcitro at gmail.com (Craig Citro)
Date: Fri, 23 Apr 2010 16:03:19 -0700
Subject: [Cython] Wiki is down?
In-Reply-To: <o2r6629e85c1004231519ze67cbe91w132c2eea9ea8f21f@mail.gmail.com>
References: <m2n3af1998d1004231133sef49f9f1l8684732124891c11@mail.gmail.com> 
	<o2r6629e85c1004231519ze67cbe91w132c2eea9ea8f21f@mail.gmail.com>
Message-ID: <r2ua8c4ff4c1004231603y6f1faa35yd71b1dd6108b6184@mail.gmail.com>

> The wiki is also down for me -- I received the "surge protection"
> message on my first visit today and every attempt thereafter.
>

I just fixed this -- in fact, I turned off surge protection for good.
If anyone has more trouble, let me know.

-cc

From vinjvinj at gmail.com  Sat Apr 24 05:56:33 2010
From: vinjvinj at gmail.com (Vineet Jain)
Date: Fri, 23 Apr 2010 23:56:33 -0400
Subject: [Cython] Question on generator support
In-Reply-To: <q2ga8c4ff4c1004231101o39084e4do274e48c045c4e6d0@mail.gmail.com>
References: <s2w70a7f84f1004231014oa9605faei9674c0c12c8cfecd@mail.gmail.com>
	<q2ga8c4ff4c1004231101o39084e4do274e48c045c4e6d0@mail.gmail.com>
Message-ID: <g2q70a7f84f1004232056rbe3b4268k5fe95cd8bf01f39b@mail.gmail.com>

Thanks for the update and all the hardwork on Cython.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20100423/0ef40c86/attachment.htm 

From greg.ewing at canterbury.ac.nz  Sat Apr 24 06:59:18 2010
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 24 Apr 2010 16:59:18 +1200
Subject: [Cython] Cython&Pyrex: grammar for function declarators
In-Reply-To: <q2he7ba66e41004231422j2676fcb0z55d3e89c93cd0fb0@mail.gmail.com>
References: <q2he7ba66e41004231422j2676fcb0z55d3e89c93cd0fb0@mail.gmail.com>
Message-ID: <4BD27AA6.3040209@canterbury.ac.nz>

Lisandro Dalcin wrote:

> (1) What "int(*foo)() nogil with gil" does actually mean?

It means that the function is safe to call without the
gil, and that it will acquire the gil before entering the
body of the function.

> (2)  "int(*foo)() nogil with gil" is redundant, as "int(*foo)() with
> gil" would be enough (i.e., implies "nogil").

Yes, it's redundant, but the parser would have to go out
of its way to disallow it, and I can't see a strong reason
to do that.

> (3) I "int (*foo)() except* nogil" is illegal syntax, but "int
> (*foo)() except* with gil" is legal.

That could be improved. The options really ought to be
accepted in any order. Feel free to submit a patch. :-)

-- 
Greg

From stefan_ml at behnel.de  Sat Apr 24 20:00:18 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 24 Apr 2010 20:00:18 +0200
Subject: [Cython] good news: cython-closures is all green again
Message-ID: <4BD331B2.5090505@behnel.de>

Hi,

I managed to track down the ref-counting bug in the cython-closures branch. 
Function arguments that participated in the closure were not always 
INCREFed, which induced several problems with the cleanup of the closure 
class instance as it expected that any non-NULL field could safely be 
DECREF-ed. Especially errors during argument conversion were pretty deadly.

After this fix, all tests in the cython-closures pass, and even the 
failures in the Python regression suite start to get interesting.

So, unless there are still objections from others, my vote is for getting 
it merged back into mainline (it's already up-to-date with cython-devel) 
and pushing out a 0.13 alpha. (Last I heard, Sage was still failing to 
build, but that doesn't have to keep us from getting an alpha version out, 
does it?)

Stefan

From craigcitro at gmail.com  Sat Apr 24 21:23:20 2010
From: craigcitro at gmail.com (Craig Citro)
Date: Sat, 24 Apr 2010 12:23:20 -0700
Subject: [Cython] good news: cython-closures is all green again
In-Reply-To: <4BD331B2.5090505@behnel.de>
References: <4BD331B2.5090505@behnel.de>
Message-ID: <o2qa8c4ff4c1004241223pce64b231ke835c13280753e3a@mail.gmail.com>

Hey Stefan,

> I managed to track down the ref-counting bug in the cython-closures branch.
> Function arguments that participated in the closure were not always
> INCREFed, which induced several problems with the cleanup of the closure
> class instance as it expected that any non-NULL field could safely be
> DECREF-ed. Especially errors during argument conversion were pretty deadly.
>

Sweet! That's interesting ... I won't get time to sit down and look at
the patch until tonight at the earliest, but I'm looking forward to
seeing what the issue was.

> After this fix, all tests in the cython-closures pass, and even the
> failures in the Python regression suite start to get interesting.
>
> So, unless there are still objections from others, my vote is for getting
> it merged back into mainline (it's already up-to-date with cython-devel)
> and pushing out a 0.13 alpha. (Last I heard, Sage was still failing to
> build, but that doesn't have to keep us from getting an alpha version out,
> does it?)
>

This sounds reasonable enough, with one caveat: if something in the
closures branch causes huge issues with Sage, we'll either need to
keep delaying 0.13 or revert. I can't imagine why it would, but those
are famous last words. :)

-cc

From stefan_ml at behnel.de  Sat Apr 24 21:57:52 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 24 Apr 2010 21:57:52 +0200
Subject: [Cython] good news: cython-closures is all green again
In-Reply-To: <o2qa8c4ff4c1004241223pce64b231ke835c13280753e3a@mail.gmail.com>
References: <4BD331B2.5090505@behnel.de>
	<o2qa8c4ff4c1004241223pce64b231ke835c13280753e3a@mail.gmail.com>
Message-ID: <4BD34D40.3090902@behnel.de>

Craig Citro, 24.04.2010 21:23:
 > Stefan Behnel wrote:
>> (Last I heard, Sage was still failing to
>> build, but that doesn't have to keep us from getting an alpha version out,
>> does it?)
>
> This sounds reasonable enough, with one caveat: if something in the
> closures branch causes huge issues with Sage, we'll either need to
> keep delaying 0.13 or revert. I can't imagine why it would, but those
> are famous last words. :)

To find out, it should be enough to run a build with both branches and 
check if the problems look similar. ;)

There certainly are changes in the branch that may affect existing code, 
but I would expect that the other cython-devel changes have a much larger 
potential to break code than the closures feature itself. Most of the new 
code in the cython-closures branch won't get triggered by current Cython code.

Stefan

From robertwb at math.washington.edu  Sun Apr 25 06:39:00 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 24 Apr 2010 21:39:00 -0700
Subject: [Cython] good news: cython-closures is all green again
In-Reply-To: <4BD331B2.5090505@behnel.de>
References: <4BD331B2.5090505@behnel.de>
Message-ID: <B84F0756-B51B-43E3-8A57-A9100556F55E@math.washington.edu>

On Apr 24, 2010, at 11:00 AM, Stefan Behnel wrote:

> Hi,
>
> I managed to track down the ref-counting bug in the cython-closures  
> branch.
> Function arguments that participated in the closure were not always
> INCREFed, which induced several problems with the cleanup of the  
> closure
> class instance as it expected that any non-NULL field could safely be
> DECREF-ed. Especially errors during argument conversion were pretty  
> deadly.
>
> After this fix, all tests in the cython-closures pass, and even the
> failures in the Python regression suite start to get interesting.

Awesome!

> So, unless there are still objections from others, my vote is for  
> getting
> it merged back into mainline (it's already up-to-date with cython- 
> devel)
> and pushing out a 0.13 alpha. (Last I heard, Sage was still failing to
> build, but that doesn't have to keep us from getting an alpha  
> version out,
> does it?)


I think an alpha should be coming any day now, but the closures should  
be able to go in before release.

- Robert


From robertwb at math.washington.edu  Sun Apr 25 06:47:31 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 24 Apr 2010 21:47:31 -0700
Subject: [Cython] Wiki is down?
In-Reply-To: <r2ua8c4ff4c1004231603y6f1faa35yd71b1dd6108b6184@mail.gmail.com>
References: <m2n3af1998d1004231133sef49f9f1l8684732124891c11@mail.gmail.com>
	<o2r6629e85c1004231519ze67cbe91w132c2eea9ea8f21f@mail.gmail.com>
	<r2ua8c4ff4c1004231603y6f1faa35yd71b1dd6108b6184@mail.gmail.com>
Message-ID: <FB09B951-64E2-4816-A6C8-C09224542ACF@math.washington.edu>

On Apr 23, 2010, at 4:03 PM, Craig Citro wrote:

>> The wiki is also down for me -- I received the "surge protection"
>> message on my first visit today and every attempt thereafter.
>>
>
> I just fixed this -- in fact, I turned off surge protection for good.
> If anyone has more trouble, let me know.

Thanks! This has hit us before too.

I think surge protection might have been triggered on the IP, and  
since it's sitting in a VM behind a proxy server, surge protection, if  
triggered, impacted everyone.

- Robert


From robertwb at math.washington.edu  Sun Apr 25 06:57:48 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 24 Apr 2010 21:57:48 -0700
Subject: [Cython] push related to ticket #399
In-Reply-To: <4BD00042.4040002@student.matnat.uio.no>
References: <u2re7ba66e41004211841se539a4bcu47dca43789efdb39@mail.gmail.com>
	<4BD00042.4040002@student.matnat.uio.no>
Message-ID: <A77003C3-8A43-45C1-A990-E225A6C8960D@math.washington.edu>

On Apr 22, 2010, at 12:52 AM, Dag Sverre Seljebotn wrote:

> Lisandro Dalcin wrote:
>> I've not closed the ticket, because I would like to know your
>> opinions. For your convenience, you have below my comments.
>>
>> Pushed: http://hg.cython.org/cython-devel/rev/1a9bfb4ff18a
>> Tested: Linux32/64 and Windows32 (MSVC and MinGW)
>>
>> Comments: 'ssize_t' is not a C99 standard type. If available, core
>> Python(>2.4) defines Py_ssize_t to ssize_t. If ssize_t is not
>> available, then any Cython code using ssize_t will fail at C compile
>> time. I do not think we should #define or typedef ssize_t if it is
>> missing, but raise your voice if you do not agree. For Py>=2.5,
>> detecting a missing ssize_t is trivial (macro available pyconfig.h)
>> and then we could conditionally "#define ssize_t Py_ssize_t" (or
>> ptrdiff_t).
>>
> Hmm.
>
> My vote is in favor of simply always making "ssize_t" in Cython always
> mean Py_ssize_t in C.
>
> This is because I want some Py_ssize_t without the special index
> behaviour, so that I can start recommend using it in numerical loops.
> Py_ssize_t is the "correct" type to use for indexing NumPy arrays, but
> has problems when using it in mathematical expressions because of the
> special index coercion behaviour.

This sounds very reasonable to me.

- Robert



From robertwb at math.washington.edu  Sun Apr 25 06:59:45 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 24 Apr 2010 21:59:45 -0700
Subject: [Cython] Allow analyse_types() to replace the current node?
In-Reply-To: <4BCD4750.7090006@behnel.de>
References: <4BCD4750.7090006@behnel.de>
Message-ID: <3A0D60B7-4944-4821-BFDA-CB2AC9EDBA6B@math.washington.edu>

On Apr 19, 2010, at 11:18 PM, Stefan Behnel wrote:

> Hi,
>
> I ran into this more than once already, so I think this is worth
> discussing. In a couple of cases, an analyse_types() method finds  
> out that
> the current node isn't suitable for what is actually happening.  
> There are a
> couple of different hacks that deal with that, ranging from __class__
> replacement to a partial reconfiguration of the node together with a
> subsequent transform that replaces it.
>
> Could we change the semantics of analyse_types() to return the  
> current node
> instead, so that it is free to return a different node if it needs to?
> Since this isn't happening in a transform, this would mean that we  
> need to
> change a lot of code to get the old node reference replaced wherever
> analyse_types() is called, but these places shouldn't be too hard to  
> find
> at least.
>
> I know that this interferes with the idea of transforms, but at  
> least in
> some cases, it's easier to replace a node directly after finding out  
> that
> it needs to be replaced, than to do part of the analysis now and to  
> rely on
> a later transform to do the replacement and the correct setup of the  
> new
> node, especially when it may have an impact on some of the surrounding
> nodes (e.g. because the new node returns a completely different type).
>
> What do you think?

+1, that's a very good idea. It's a big change, but I don't think  
it'll introduce bugs as it'll be easy to find where things need to be  
fixed.

- Robert


From craigcitro at gmail.com  Sun Apr 25 07:03:39 2010
From: craigcitro at gmail.com (Craig Citro)
Date: Sat, 24 Apr 2010 22:03:39 -0700
Subject: [Cython] Wiki is down?
In-Reply-To: <FB09B951-64E2-4816-A6C8-C09224542ACF@math.washington.edu>
References: <m2n3af1998d1004231133sef49f9f1l8684732124891c11@mail.gmail.com> 
	<o2r6629e85c1004231519ze67cbe91w132c2eea9ea8f21f@mail.gmail.com> 
	<r2ua8c4ff4c1004231603y6f1faa35yd71b1dd6108b6184@mail.gmail.com> 
	<FB09B951-64E2-4816-A6C8-C09224542ACF@math.washington.edu>
Message-ID: <w2na8c4ff4c1004242203nd952a939sc2e2bcb38c641732@mail.gmail.com>

> Thanks! This has hit us before too.
>
> I think surge protection might have been triggered on the IP, and
> since it's sitting in a VM behind a proxy server, surge protection, if
> triggered, impacted everyone.
>

Yep -- I think surge protection was turned off on the Sage wiki for
the same reason (i.e. more trouble than it's worth with the VM
routing). I think they're still getting more traffic than us by a bit,
so they can be our canary. ;)

-cc

From robertwb at math.washington.edu  Sun Apr 25 07:05:47 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 24 Apr 2010 22:05:47 -0700
Subject: [Cython] coercion of char/Py_UNICODE to Python objects - string
	or integer?
In-Reply-To: <4BCFE07F.70307@behnel.de>
References: <g2je7ba66e41004211426s3ab87024iedcf5c8e0d8be19a@mail.gmail.com>
	<4BCFE07F.70307@behnel.de>
Message-ID: <6F4F2615-4B5B-4DAC-9089-15776CFECE29@math.washington.edu>

On Apr 21, 2010, at 10:37 PM, Stefan Behnel wrote:

> Lisandro Dalcin, 21.04.2010 23:26:
>> What do you think?
>>
>> diff -r 2701901737d4 Cython/Compiler/PyrexTypes.py
>> --- a/Cython/Compiler/PyrexTypes.py	Wed Apr 21 15:36:27 2010 +0200
>> +++ b/Cython/Compiler/PyrexTypes.py	Wed Apr 21 18:25:42 2010 -0300
>> @@ -871,7 +871,7 @@
>>      # to integers here.  The maximum value for a Py_UNICODE is
>>      # 1114111, so PyInt_FromLong() will do just fine here.
>>
>> -    to_py_function = "PyInt_FromLong"
>> +    to_py_function = "PyUnicode_FromOrdinal"
>>
>>      def sign_and_name(self):
>>          return "Py_UNICODE"
>
> I didn't know about that function, even though I had looked for it  
> in the
> CPython docs. It's available in all relevant CPython versions, and  
> it's
> pretty efficient, too.
>
> This would let Py_UNICODE values turn into a single character unicode
> string when coercing to a Python object. I had also thought about  
> this, and
> wasn't sure what I wanted. In current Cython, 'char' doesn't coerce  
> to a
> single character 'bytes' object but to an integer. My thinking was  
> that
> Py_UNICODE should behave the same.
>
> This is a bit inconsistent in itself, given that single character  
> strings
> can coerce to their C ordinal value, e.g. on comparison with
> char/Py_UNICODE, but not so much of an inconsistency to break  
> backwards
> compatibility. I'm really not sure what the 'expected' behaviour is  
> here,
> although I'm leaning slightly towards the char/bytes and Py_UNICODE/ 
> unicode
> coercion.
>
> It's certainly easier to write
>
>     cdef Py_UNICODE cval = some_c_integer
>
>     py_object = <long>cval
>
> to get a Python integer value, than to find, import and call
> PyUnicode_FromOrdinal() to get a unicode string. There doesn't seem  
> to be
> an equivalent PyBytes function, so I guess the PyBytes conversion  
> would use
>
>     py_bytes = PyBytes_FromStringAndSize(&char_val, 1)
>
> which isn't exactly beautiful either, and certainly less so than the  
> opposite
>
>     py_integer = <int>char_val
>
> This would also speak in favour of letting char and Py_UNICODE  
> coerce to
> Python strings by default, although the above would go away if we  
> special
> cased the builtin chr() function to output exactly the above code  
> for each
> input type.
>
> Another option is to consider Py_UNICODE more special (and more  
> specific)
> than the somewhat generic 'char', and to accept the inconsistency of
> coercing one to a unicode string and the other to an integer.
>
> What do the others think?


I think char -> bytes and Py_UNICODE -> unicode make a lot of sense,  
my only concern would be backwards incompatibility.

- Robert


From robertwb at math.washington.edu  Sun Apr 25 07:17:11 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 24 Apr 2010 22:17:11 -0700
Subject: [Cython] int complex failure in C++ with MSVC
In-Reply-To: <v2qe7ba66e41004121416p7292fb48p7423ca9e999d9a43@mail.gmail.com>
References: <q2oe7ba66e41004111650w35b057c7x26018d385d244207@mail.gmail.com>
	<2EA9CDD9-34BC-47D4-A3C1-0B5018ADB0E7@math.washington.edu>
	<r2me7ba66e41004121342h8e59de80n4ea99d865f60fbe@mail.gmail.com>
	<520D0D09-16AA-4B1E-8401-36C104A3B836@math.washington.edu>
	<v2qe7ba66e41004121416p7292fb48p7423ca9e999d9a43@mail.gmail.com>
Message-ID: <9B7702BD-07FF-42EF-BEFB-F1239D6C166E@math.washington.edu>

On Apr 12, 2010, at 2:16 PM, Lisandro Dalcin wrote:

> On 12 April 2010 18:04, Robert Bradshaw  
> <robertwb at math.washington.edu> wrote:
>> On Apr 12, 2010, at 1:42 PM, Lisandro Dalcin wrote:
>>
>>> On 12 April 2010 14:14, Robert Bradshaw
>>> <robertwb at math.washington.edu> wrote:
>>>> On Apr 11, 2010, at 4:50 PM, Lisandro Dalcin wrote:
>>>>
>>>>> See this...
>>>>>
>>>>> ----------------------------------------------------------------------
>>>>> File "C:\Users\dalcinl\DEVEL\cython-devel\BUILD\run\cpp
>>>>> \complex_int_T446.pyd",
>>>>> line ?, in complex_int_T446.__test__.test_arith (line 3)
>>>>> Failed example:
>>>>>    test_arith(29+11j, 5+7j)
>>>>> Expected:
>>>>>    ((-29-11j), (34+18j), (24+4j), (68+258j), (3-2j))
>>>>> Got:
>>>>>    ((-29-11j), (34+18j), (24+4j), (68+258j), (1-4j))
>>>>>
>>>>> It seems that std::complex<int> in MSVC does not work well for
>>>>> integrals, operator/() was written with floating types in mind.
>>>>>
>>>>> What should we do?
>>>>
>>>> Hmm... I would be OK with complex_int_T446 not testing division  
>>>> (with
>>>> a note).
>>>
>>> Do you mean a little comment saying this is broken with MSVC?
>>
>>
>> Yep. For non-exact division I could see calling the result undefined,
>> but that's not the case here.
>>
>
> Mmm.. Wait a minute... Now this makes me think a bit different ... In
> general, we cannot trust std::complex implementations [except GCC, of
> course :-)] ... So what about switching to struct-based complexes if
> the real-type is integral? Or perhaps better, use an ad-hoc quot()  
> implementation?

I think the situation with complex types is complicated enough--adding  
another case for integral vs. floating point types (or just their  
division) will make things even messier. What we could do is let C++  
behave like C and use the structs unless <complex> is involved.

Note that the user can control this with the CYTHON_CCOMPLEX macro as  
well.

- Robert


From stefan_ml at behnel.de  Sun Apr 25 07:28:40 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 25 Apr 2010 07:28:40 +0200
Subject: [Cython] coercion of char/Py_UNICODE to Python objects - string
 or integer?
In-Reply-To: <6F4F2615-4B5B-4DAC-9089-15776CFECE29@math.washington.edu>
References: <g2je7ba66e41004211426s3ab87024iedcf5c8e0d8be19a@mail.gmail.com>	<4BCFE07F.70307@behnel.de>
	<6F4F2615-4B5B-4DAC-9089-15776CFECE29@math.washington.edu>
Message-ID: <4BD3D308.6090509@behnel.de>

Robert Bradshaw, 25.04.2010 07:05:
> On Apr 21, 2010, at 10:37 PM, Stefan Behnel wrote:
>
>> Lisandro Dalcin, 21.04.2010 23:26:
>>> What do you think?
>>>
>>> diff -r 2701901737d4 Cython/Compiler/PyrexTypes.py
>>> --- a/Cython/Compiler/PyrexTypes.py	Wed Apr 21 15:36:27 2010 +0200
>>> +++ b/Cython/Compiler/PyrexTypes.py	Wed Apr 21 18:25:42 2010 -0300
>>> @@ -871,7 +871,7 @@
>>>       # to integers here.  The maximum value for a Py_UNICODE is
>>>       # 1114111, so PyInt_FromLong() will do just fine here.
>>>
>>> -    to_py_function = "PyInt_FromLong"
>>> +    to_py_function = "PyUnicode_FromOrdinal"
>>>
>>>       def sign_and_name(self):
>>>           return "Py_UNICODE"
>>
>> I didn't know about that function, even though I had looked for it
>> in the
>> CPython docs. It's available in all relevant CPython versions, and
>> it's
>> pretty efficient, too.
>>
>> This would let Py_UNICODE values turn into a single character unicode
>> string when coercing to a Python object. I had also thought about
>> this, and
>> wasn't sure what I wanted. In current Cython, 'char' doesn't coerce
>> to a
>> single character 'bytes' object but to an integer. My thinking was
>> that
>> Py_UNICODE should behave the same.
>>
>> This is a bit inconsistent in itself, given that single character
>> strings
>> can coerce to their C ordinal value, e.g. on comparison with
>> char/Py_UNICODE, but not so much of an inconsistency to break
>> backwards
>> compatibility. I'm really not sure what the 'expected' behaviour is
>> here,
>> although I'm leaning slightly towards the char/bytes and Py_UNICODE/
>> unicode
>> coercion.
>>
>> It's certainly easier to write
>>
>>      cdef Py_UNICODE cval = some_c_integer
>>
>>      py_object =<long>cval
>>
>> to get a Python integer value, than to find, import and call
>> PyUnicode_FromOrdinal() to get a unicode string. There doesn't seem
>> to be
>> an equivalent PyBytes function, so I guess the PyBytes conversion
>> would use
>>
>>      py_bytes = PyBytes_FromStringAndSize(&char_val, 1)
>>
>> which isn't exactly beautiful either, and certainly less so than the
>> opposite
>>
>>      py_integer =<int>char_val
>>
>> This would also speak in favour of letting char and Py_UNICODE
>> coerce to
>> Python strings by default, although the above would go away if we
>> special
>> cased the builtin chr() function to output exactly the above code
>> for each
>> input type.
>>
>> Another option is to consider Py_UNICODE more special (and more
>> specific)
>> than the somewhat generic 'char', and to accept the inconsistency of
>> coercing one to a unicode string and the other to an integer.
>>
>> What do the others think?
>
> I think char ->  bytes and Py_UNICODE ->  unicode make a lot of sense,
> my only concern would be backwards incompatibility.

It occurred to me that the alternative is actually simpler. We can support 
the explicit coercions

     cdef Py_UNICODE uval = ...
     cdef unicode u = uval
     s = <unicode>uval

so the question is just if we want

     py_int_val = uval
     py_ustr_val = <unicode>uval

or

     py_int_val = <int>uval
     py_ustr_val = uval

My gut feeling is that the coercion to strings would be more straight 
forward. It would also clean up the compiler code a bit as implicit 
coercions (e.g. for comparisons) would then work out-of-the-box in both 
ways. Currently, "Py_UNICODE in unicode" must be special cased (which it 
still would in the future, but only for optimisation purposes, not to make 
it work at all).

Stefan

From stefan_ml at behnel.de  Sun Apr 25 07:40:18 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 25 Apr 2010 07:40:18 +0200
Subject: [Cython] push related to ticket #399
In-Reply-To: <A77003C3-8A43-45C1-A990-E225A6C8960D@math.washington.edu>
References: <u2re7ba66e41004211841se539a4bcu47dca43789efdb39@mail.gmail.com>	<4BD00042.4040002@student.matnat.uio.no>
	<A77003C3-8A43-45C1-A990-E225A6C8960D@math.washington.edu>
Message-ID: <4BD3D5C2.5090609@behnel.de>

Robert Bradshaw, 25.04.2010 06:57:
> On Apr 22, 2010, at 12:52 AM, Dag Sverre Seljebotn wrote:
>> Lisandro Dalcin wrote:
>>> I've not closed the ticket, because I would like to know your
>>> opinions. For your convenience, you have below my comments.
>>>
>>> Pushed: http://hg.cython.org/cython-devel/rev/1a9bfb4ff18a
>>> Tested: Linux32/64 and Windows32 (MSVC and MinGW)
>>>
>>> Comments: 'ssize_t' is not a C99 standard type. If available, core
>>> Python(>2.4) defines Py_ssize_t to ssize_t. If ssize_t is not
>>> available, then any Cython code using ssize_t will fail at C compile
>>> time. I do not think we should #define or typedef ssize_t if it is
>>> missing, but raise your voice if you do not agree. For Py>=2.5,
>>> detecting a missing ssize_t is trivial (macro available pyconfig.h)
>>> and then we could conditionally "#define ssize_t Py_ssize_t" (or
>>> ptrdiff_t).
>>
>> My vote is in favor of simply always making "ssize_t" in Cython always
>> mean Py_ssize_t in C.
>>
>> This is because I want some Py_ssize_t without the special index
>> behaviour, so that I can start recommend using it in numerical loops.
>> Py_ssize_t is the "correct" type to use for indexing NumPy arrays, but
>> has problems when using it in mathematical expressions because of the
>> special index coercion behaviour.
>
> This sounds very reasonable to me.

+1. If CPython defines one as the other anyway, it won't make a difference 
in Py2.5+, and older Python versions a) have 64 bit issues anyway and b) 
are already out of maintenance and thus will die out rather sooner than later.

Stefan

From craigcitro at gmail.com  Sun Apr 25 08:15:23 2010
From: craigcitro at gmail.com (Craig Citro)
Date: Sat, 24 Apr 2010 23:15:23 -0700
Subject: [Cython] rss/atom feeds
Message-ID: <u2ta8c4ff4c1004242315j5de86c1cy70909ef0198e1e05@mail.gmail.com>

Hey all,

I don't know if anyone but me had ever run into it, but the RSS/Atom
feeds on hg.cython.org were slightly broken -- the URLs they provided
for changesets were all referring to "http://sagemath/" instead of
"http://hg.cython.org/" (this is an artifact of the way the sites are
hosted). Anyway, I tried to fix this via apache, but that was dicey
for various reasons. I fixed this another (slightly more hackish) way
-- I think it should be fine (my random testing shows no trouble, but
that doesn't mean much). If anyone runs into problems with the hg web
service, or hg.cython.org at all, give me a shout.

-cc

From stefan_ml at behnel.de  Sun Apr 25 10:30:14 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 25 Apr 2010 10:30:14 +0200
Subject: [Cython] coercion of char/Py_UNICODE to Python objects - string
 or integer?
In-Reply-To: <4BD3D308.6090509@behnel.de>
References: <g2je7ba66e41004211426s3ab87024iedcf5c8e0d8be19a@mail.gmail.com>	<4BCFE07F.70307@behnel.de>	<6F4F2615-4B5B-4DAC-9089-15776CFECE29@math.washington.edu>
	<4BD3D308.6090509@behnel.de>
Message-ID: <4BD3FD96.7080803@behnel.de>

Stefan Behnel, 25.04.2010 07:28:
> the question is just if we want
>
>       py_int_val = uval
>       py_ustr_val =<unicode>uval
>
> or
>
>       py_int_val =<int>uval
>       py_ustr_val = uval
>
> My gut feeling is that the coercion to strings would be more straight
> forward. It would also clean up the compiler code a bit as implicit
> coercions (e.g. for comparisons) would then work out-of-the-box in both
> ways. Currently, "Py_UNICODE in unicode" must be special cased (which it
> still would in the future, but only for optimisation purposes, not to make
> it work at all).

Given that only char->bytes breaks backwards compatibility, since 
Py_UNICODE wasn't supported at all until now - could we agree on making 
Py_UNICODE->unicode the default behaviour and leaving char->PyInt as is for 
the time being? We can still decide to break backwards compatibility later 
on, and we can always support the explicit

     py_s = <bytes>char_val
     py_i = <int>pyunicode_val

safely, so users can just rely on that if they need safe behaviour.

The 'reasoning' would be that a plain 'char' is a bit too generic to map it 
to a Python bytes string (note that 'char*' isn't), whereas Py_UNICODE is 
something that only really makes sense in the context of a Python unicode 
string, so it's a lot less surprising if it also maps to a Python unicode 
string.

Stefan

From dalcinl at gmail.com  Mon Apr 26 01:41:48 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Sun, 25 Apr 2010 20:41:48 -0300
Subject: [Cython] push related to ticket #399
In-Reply-To: <4BD3D5C2.5090609@behnel.de>
References: <u2re7ba66e41004211841se539a4bcu47dca43789efdb39@mail.gmail.com>
	<4BD00042.4040002@student.matnat.uio.no>
	<A77003C3-8A43-45C1-A990-E225A6C8960D@math.washington.edu>
	<4BD3D5C2.5090609@behnel.de>
Message-ID: <m2qe7ba66e41004251641p23b1d7a3xd6cc9b4592761781@mail.gmail.com>

On 25 April 2010 02:40, Stefan Behnel <stefan_ml at behnel.de> wrote:
>>>
>>> My vote is in favor of simply always making "ssize_t" in Cython always
>>> mean Py_ssize_t in C.
>>>
>>
>> This sounds very reasonable to me.
>
> +1. If CPython defines one as the other anyway,

Yes, but only if the ssize_t is available.

> it won't make a difference
> in Py2.5+, and older Python versions a) have 64 bit issues anyway and b)
> are already out of maintenance and thus will die out rather sooner than later.
>

I'm still not sure that defining ssize_t is a good idea. As the type
is missing, we can expect that other API's could also define it. Then
you #include a  header, and get conflicting definitions.

Other way I would not object so strongly is that ssize_t in Cython
code actually emit Py_ssize_t in C code. Or perhaps better, invent our
own __Pyx_ssize_t, and then:

#ifndef __Pyx_ssize_t
#define __Pyx_ssize_t Py_ssize_t
#endif

then people has a last chance to hack on our definition at C compile time.

What do you think about this?

-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From stefan_ml at behnel.de  Mon Apr 26 06:39:53 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 26 Apr 2010 06:39:53 +0200
Subject: [Cython] push related to ticket #399
In-Reply-To: <m2qe7ba66e41004251641p23b1d7a3xd6cc9b4592761781@mail.gmail.com>
References: <u2re7ba66e41004211841se539a4bcu47dca43789efdb39@mail.gmail.com>	<4BD00042.4040002@student.matnat.uio.no>	<A77003C3-8A43-45C1-A990-E225A6C8960D@math.washington.edu>	<4BD3D5C2.5090609@behnel.de>
	<m2qe7ba66e41004251641p23b1d7a3xd6cc9b4592761781@mail.gmail.com>
Message-ID: <4BD51919.2030608@behnel.de>

Lisandro Dalcin, 26.04.2010 01:41:
> On 25 April 2010 02:40, Stefan Behnel wrote:
>>>>
>>>> My vote is in favor of simply always making "ssize_t" in Cython always
>>>> mean Py_ssize_t in C.
>>>>
>>>
>>> This sounds very reasonable to me.
>>
>> +1. If CPython defines one as the other anyway,
>
> Yes, but only if the ssize_t is available.
>
>> it won't make a difference
>> in Py2.5+, and older Python versions a) have 64 bit issues anyway and b)
>> are already out of maintenance and thus will die out rather sooner than later.
>
> Other way I would not object so strongly is that ssize_t in Cython
> code actually emit Py_ssize_t in C code.

That's what I meant. CPython already does the checking for us and defines 
Py_ssize_t as ssize_t if it's there. If it's not there, Py_ssize_t will 
still have the right type, regardless of any additional definitions at C 
compile time, and guaranteed to be compatible with them.


> Or perhaps better, invent our own __Pyx_ssize_t, and then:
>
> #ifndef __Pyx_ssize_t
> #define __Pyx_ssize_t Py_ssize_t
> #endif

-1, Py_ssize_t already does that for us, so that would just add another 
level of superfluous indirection.

Stefan

From stefan_ml at behnel.de  Mon Apr 26 07:16:51 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 26 Apr 2010 07:16:51 +0200
Subject: [Cython] coercion of char/Py_UNICODE to Python objects - string
 or integer?
In-Reply-To: <6F4F2615-4B5B-4DAC-9089-15776CFECE29@math.washington.edu>
References: <g2je7ba66e41004211426s3ab87024iedcf5c8e0d8be19a@mail.gmail.com>	<4BCFE07F.70307@behnel.de>
	<6F4F2615-4B5B-4DAC-9089-15776CFECE29@math.washington.edu>
Message-ID: <4BD521C3.1030104@behnel.de>

Robert Bradshaw, 25.04.2010 07:05:
> I think char ->  bytes and Py_UNICODE ->  unicode make a lot of sense,
> my only concern would be backwards incompatibility.

There is another thing regarding the char/bytes case, which hits in Py3. 
Here, indexing returns integer values, i.e.

     b"abcdefg"[4]

returns 'e' (str) in Py2 and 101 (int) in Py3. With my recent changes, the 
following now works in both environments:

     cdef bytes s = b"abcdefg"
     cdef char c = s[4]

and (efficiently) returns 101 for c. The Py2/3 bytes difference also hits 
in other places, though, and Cython doesn't hide it. For example, the 
bytes() constructor:

     Py2:   bytes(3)  ==  b'3'
     Py3:   bytes(3)  ==  b'\0'*3

so this will not do the right thing in any case:

     cdef char c = b'e'
     s = bytes(c)

neither will this in Py3, where it returns a unicode string:

     s = chr(c)

But at least this can be made to work in Cython:

     s = <bytes>c   # s == b'e'

We may still have to do more to make the bytes type really usable in Cython 
in a portable way...

Stefan

From dagss at student.matnat.uio.no  Mon Apr 26 08:25:02 2010
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Mon, 26 Apr 2010 08:25:02 +0200
Subject: [Cython] push related to ticket #399
In-Reply-To: <m2qe7ba66e41004251641p23b1d7a3xd6cc9b4592761781@mail.gmail.com>
References: <u2re7ba66e41004211841se539a4bcu47dca43789efdb39@mail.gmail.com>	<4BD00042.4040002@student.matnat.uio.no>	<A77003C3-8A43-45C1-A990-E225A6C8960D@math.washington.edu>	<4BD3D5C2.5090609@behnel.de>
	<m2qe7ba66e41004251641p23b1d7a3xd6cc9b4592761781@mail.gmail.com>
Message-ID: <4BD531BE.1090801@student.matnat.uio.no>

Lisandro Dalcin wrote:
> On 25 April 2010 02:40, Stefan Behnel <stefan_ml at behnel.de> wrote:
>>>> My vote is in favor of simply always making "ssize_t" in Cython always
>>>> mean Py_ssize_t in C.
>>>>
>>> This sounds very reasonable to me.
>> +1. If CPython defines one as the other anyway,
> 
> Yes, but only if the ssize_t is available.
> 
>> it won't make a difference
>> in Py2.5+, and older Python versions a) have 64 bit issues anyway and b)
>> are already out of maintenance and thus will die out rather sooner than later.
>>
> 
> I'm still not sure that defining ssize_t is a good idea. As the type
> is missing, we can expect that other API's could also define it. Then
> you #include a  header, and get conflicting definitions.
> 
> Other way I would not object so strongly is that ssize_t in Cython
> code actually emit Py_ssize_t in C code. Or perhaps better, invent our

This is what I was suggesting.

> own __Pyx_ssize_t, and then:
> 
> #ifndef __Pyx_ssize_t
> #define __Pyx_ssize_t Py_ssize_t
> #endif
> 
> then people has a last chance to hack on our definition at C compile time.


-- 
Dag Sverre

From trevorbtippetts at gmail.com  Mon Apr 26 17:21:41 2010
From: trevorbtippetts at gmail.com (Trevor Tippetts)
Date: Mon, 26 Apr 2010 16:21:41 +0100
Subject: [Cython] numerics bug?: pure python and buffer+ndim
Message-ID: <w2y2d4ecf241004260821u18e29ff0u3c3dd38d9f0d0aa7@mail.gmail.com>

Hi,

I have recently uncovered some mysterious behaviour and Dag Sverre
suggested on the cython-users list that I file a bug report. Since I
don't have an account, I'm sending this to the dev list in the hope that
this is a simple bug that can be fixed easily.

 I've been trying to go through the cython numpy tutorial (http://
wiki.cython.org/tutorials/numpy) and reproduce the speed-ups with pure
python mode (http://wiki.cython.org/pure), either with or without an
accompanying .pxd file.

The sticking point for me is specifying the dimensions of the numpy
array args. I can't find any way to do it without breaking python
compatibility in the .py file. Oddly, a cython.locals decorator in
a .pxd file should work, but it has no effect whatsoever on the
generated .c file. For example, with a class defined to set ndim
(without using a comma inside brackets in the .pxd, which will fail to
compile):
cdef class MyArray:
    cdef __cythonbufferdefaults__ = {"ndim": 2, "mode": "strided"}

A non-python def line in .py works fine.
def naive_convolve(MyArray[DTYPE_t] f, MyArray[DTYPE_t] g):

But setting the type with cython.locals in a .pxd, like this,
@cython.locals(f=MyArray[DTYPE_t],g=MyArray[DTYPE_t])
cpdef naive_convolve(f, g)

produces a .c file that is identical to not using the decorator at
all.
#... at cython.locals(f=MyArray[DTYPE_t],g=MyArray[DTYPE_t])
cpdef naive_convolve(f, g)

Any idea why cython.locals has no effect? Every other way I can think
of to set ndim in pure python mode fails to compile. I've tried this
with cython 0.12, with both python 2.5 and 2.6 (enthought epd-6.0.1-
rh5-x86). I really want to preserve python compatibility, but setting
ndim makes the biggest performance improvement. Thanks for any help.

Trevor

From robertwb at math.washington.edu  Mon Apr 26 19:24:29 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 26 Apr 2010 10:24:29 -0700
Subject: [Cython] push related to ticket #399
In-Reply-To: <4BD531BE.1090801@student.matnat.uio.no>
References: <u2re7ba66e41004211841se539a4bcu47dca43789efdb39@mail.gmail.com>	<4BD00042.4040002@student.matnat.uio.no>	<A77003C3-8A43-45C1-A990-E225A6C8960D@math.washington.edu>	<4BD3D5C2.5090609@behnel.de>
	<m2qe7ba66e41004251641p23b1d7a3xd6cc9b4592761781@mail.gmail.com>
	<4BD531BE.1090801@student.matnat.uio.no>
Message-ID: <B9DA4BC5-9A5E-44D8-BBA1-C4ABB60AC6F9@math.washington.edu>

On Apr 25, 2010, at 11:25 PM, Dag Sverre Seljebotn wrote:

> Lisandro Dalcin wrote:
>> On 25 April 2010 02:40, Stefan Behnel <stefan_ml at behnel.de> wrote:
>>>>> My vote is in favor of simply always making "ssize_t" in Cython  
>>>>> always
>>>>> mean Py_ssize_t in C.
>>>>>
>>>> This sounds very reasonable to me.
>>> +1. If CPython defines one as the other anyway,
>>
>> Yes, but only if the ssize_t is available.
>>
>>> it won't make a difference
>>> in Py2.5+, and older Python versions a) have 64 bit issues anyway  
>>> and b)
>>> are already out of maintenance and thus will die out rather sooner  
>>> than later.
>>>
>>
>> I'm still not sure that defining ssize_t is a good idea. As the type
>> is missing, we can expect that other API's could also define it. Then
>> you #include a  header, and get conflicting definitions.
>>
>> Other way I would not object so strongly is that ssize_t in Cython
>> code actually emit Py_ssize_t in C code. Or perhaps better, invent  
>> our
>
> This is what I was suggesting.

That's what I thought. Let's do it.

- Robert


From stefan_ml at behnel.de  Tue Apr 27 20:27:00 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 27 Apr 2010 20:27:00 +0200
Subject: [Cython] Serious constant folding bug fixed in cython-devel
Message-ID: <4BD72C74.6040903@behnel.de>

Hi,

there was a pretty serious bug in the constant folding code that made

    if 1 == 1:

turn into a false condition result. The above may not seem very likely to 
appear in real world code, but the same applies to constant DEF values, for 
example, which is a lot more likely to show up. I wouldn't be surprised if 
this had an effect on the Sage build, that's why I though I'd better 
mention it here.

Stefan

From robertwb at math.washington.edu  Tue Apr 27 21:45:55 2010
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 27 Apr 2010 12:45:55 -0700
Subject: [Cython] Haoyu Bai's application accepted for GSoC
Message-ID: <D100655C-0D9A-47F1-BB34-94338B6D6873@math.washington.edu>

The list of accepted applications for Google Summer of Code 2010 were  
just announced yesterday, and Haoyu Bai's Cython application was  
officially accepted. Congratulations, and we're looking forward to a  
lot of great stuff happing this summer.

- Robert


From f_krull at gmx.de  Wed Apr 28 00:02:45 2010
From: f_krull at gmx.de (fk)
Date: Wed, 28 Apr 2010 00:02:45 +0200
Subject: [Cython]  Two bugs with Py3 support
In-Reply-To: <w2y2d4ecf241004260821u18e29ff0u3c3dd38d9f0d0aa7@mail.gmail.com>
References: <w2y2d4ecf241004260821u18e29ff0u3c3dd38d9f0d0aa7@mail.gmail.com>
Message-ID: <4BD75F05.10602@gmx.de>

Hi,

so, I stumbled about some funny behaviour with Python 3 support. Pretty 
sure those are bugs, however as I don?t have a Trac account, I?ll report 
them here; hope that?s OK.

i) As you may know, support for __cmp__ is gone in Python 3 - the 
relevant field in the type object structure has turned from ``cmpfunc 
tp_compare`` into ``void *reserved``. However, the Cython compiler even 
on Python 3 tries to put the comparison function into that slot:

   0, /*tp_setattr*/
   __pyx_pf_6bullet_9datatypes_7vector3___cmp__, /*tp_compare*/
   __pyx_pf_6bullet_9datatypes_7vector3___repr__, /*tp_repr*/

That works fine with C, however it doesn?t with C++:

bullet/datatypes.cpp:4835: error: invalid conversion from ?int 
(*)(PyObject*, PyObject*)? to ?void*?

ii) The header files that get generated when you use ``public`` in the 
Cython code use the DL_IMPORT (and DL_EXPORT?) macro, which, again, 
seems to have disappeared with Python 3 (it has been deprecated since 
2.3 already). The endorsed replacement, it appears, are the PyAPI_* macros.

So, I hope it?s clear what I mean. For the record, I?m using Ubuntu 
10.04 64-bit, Python 2.6.5 and 3.1.2, GCC 4.4.3 and Cython tip, 
changeset 3adaf1ef25b9 as I?m writing.

Regards,
Felix

From dalcinl at gmail.com  Wed Apr 28 01:21:36 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Tue, 27 Apr 2010 20:21:36 -0300
Subject: [Cython] Two bugs with Py3 support
In-Reply-To: <4BD75F05.10602@gmx.de>
References: <w2y2d4ecf241004260821u18e29ff0u3c3dd38d9f0d0aa7@mail.gmail.com>
	<4BD75F05.10602@gmx.de>
Message-ID: <j2he7ba66e41004271621z13112061nd6df7bd39ee1c931@mail.gmail.com>

On 27 April 2010 19:02, fk <f_krull at gmx.de> wrote:
> Hi,
>
> so, I stumbled about some funny behaviour with Python 3 support. Pretty
> sure those are bugs, however as I don?t have a Trac account, I?ll report
> them here; hope that?s OK.
>
> i) As you may know, support for __cmp__ is gone in Python 3 - the
> relevant field in the type object structure has turned from ``cmpfunc
> tp_compare`` into ``void *reserved``. However, the Cython compiler even
> on Python 3 tries to put the comparison function into that slot:
>
> ? 0, /*tp_setattr*/
> ? __pyx_pf_6bullet_9datatypes_7vector3___cmp__, /*tp_compare*/
> ? __pyx_pf_6bullet_9datatypes_7vector3___repr__, /*tp_repr*/
>
> That works fine with C, however it doesn?t with C++:
>
> bullet/datatypes.cpp:4835: error: invalid conversion from ?int
> (*)(PyObject*, PyObject*)? to ?void*?
>

Below there is a quick fix:

diff -r 3adaf1ef25b9 Cython/Compiler/TypeSlots.py
--- a/Cython/Compiler/TypeSlots.py	Tue Apr 27 21:36:49 2010 +0200
+++ b/Cython/Compiler/TypeSlots.py	Tue Apr 27 20:14:34 2010 -0300
@@ -637,7 +637,7 @@
     EmptySlot("tp_print"), #MethodSlot(printfunc, "tp_print", "__print__"),
     EmptySlot("tp_getattr"),
     EmptySlot("tp_setattr"),
-    MethodSlot(cmpfunc, "tp_compare", "__cmp__"),
+    MethodSlot(cmpfunc, "tp_compare", "__cmp__", py3k = '<RESERVED>'),
     MethodSlot(reprfunc, "tp_repr", "__repr__"),

     SuiteSlot(PyNumberMethods, "PyNumberMethods", "tp_as_number"),

BTW, Should I push this?

We could try to synthesize __richcmp__ from __cmp__, but perhaps I'm
missing something.

In the mean time, I recommend you to move to use __richcmp__

> ii) The header files that get generated when you use ``public`` in the
> Cython code use the DL_IMPORT (and DL_EXPORT?) macro, which, again,
> seems to have disappeared with Python 3 (it has been deprecated since
> 2.3 already). The endorsed replacement, it appears, are the PyAPI_* macros.
>

The handling of public/api is something in my radar, but I'm not sure
what to do. Discussing here have not been helpful. BTW, the PyAPI_***
macros also have issues, as they depend on Py_ENABLE_SHARED. We should
find a solution that work as expected even in the case of a Python
build with static libraries (BTW, this is the default for official
Python tarballs).

-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From f_krull at gmx.de  Wed Apr 28 12:29:02 2010
From: f_krull at gmx.de (fk)
Date: Wed, 28 Apr 2010 12:29:02 +0200
Subject: [Cython] Two bugs with Py3 support
In-Reply-To: <j2he7ba66e41004271621z13112061nd6df7bd39ee1c931@mail.gmail.com>
References: <w2y2d4ecf241004260821u18e29ff0u3c3dd38d9f0d0aa7@mail.gmail.com>	<4BD75F05.10602@gmx.de>
	<j2he7ba66e41004271621z13112061nd6df7bd39ee1c931@mail.gmail.com>
Message-ID: <4BD80DEE.1060907@gmx.de>

> Below there is a quick fix:
>
> diff -r 3adaf1ef25b9 Cython/Compiler/TypeSlots.py
> --- a/Cython/Compiler/TypeSlots.py	Tue Apr 27 21:36:49 2010 +0200
> +++ b/Cython/Compiler/TypeSlots.py	Tue Apr 27 20:14:34 2010 -0300
> @@ -637,7 +637,7 @@
>       EmptySlot("tp_print"), #MethodSlot(printfunc, "tp_print", "__print__"),
>       EmptySlot("tp_getattr"),
>       EmptySlot("tp_setattr"),
> -    MethodSlot(cmpfunc, "tp_compare", "__cmp__"),
> +    MethodSlot(cmpfunc, "tp_compare", "__cmp__", py3k = '<RESERVED>'),
>       MethodSlot(reprfunc, "tp_repr", "__repr__"),
>
>       SuiteSlot(PyNumberMethods, "PyNumberMethods", "tp_as_number"),
>
> BTW, Should I push this?

Well, fixes it for me at least.

> We could try to synthesize __richcmp__ from __cmp__, but perhaps I'm
> missing something.
>
> In the mean time, I recommend you to move to use __richcmp__

Yeah, I was going to do that anyway; but it?s still a bug ;)

> The handling of public/api is something in my radar, but I'm not sure
> what to do. Discussing here have not been helpful. BTW, the PyAPI_***
> macros also have issues, as they depend on Py_ENABLE_SHARED. We should
> find a solution that work as expected even in the case of a Python
> build with static libraries (BTW, this is the default for official
> Python tarballs).

But it would be a fix until that gets taken care of, would it not? It 
doesn?t happen for my code anymore, I?m now using the *_api.h headers 
since I realized the relevant code is not in fact in the same module, 
but still ...

I think I even spotted that discussion, or a part of it, when going 
through the archives looking for information on 'public' and 'api'. I?m 
not really into all this shared library stuff, though what the 'api' 
keyword does is probably *immensely* more useful for cross-module stuff; 
shared-linking to Python modules doesn?t seem like a good idea.

Oh, and by the way, amazing project you have going here. Writing Python 
extension modules (for whatever reason) has never been more fun.
Not that I?ve written any without Cython.

Regards,
Felix

From stefan_ml at behnel.de  Wed Apr 28 14:51:23 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 28 Apr 2010 14:51:23 +0200
Subject: [Cython] Two bugs with Py3 support
In-Reply-To: <j2he7ba66e41004271621z13112061nd6df7bd39ee1c931@mail.gmail.com>
References: <w2y2d4ecf241004260821u18e29ff0u3c3dd38d9f0d0aa7@mail.gmail.com>	<4BD75F05.10602@gmx.de>
	<j2he7ba66e41004271621z13112061nd6df7bd39ee1c931@mail.gmail.com>
Message-ID: <4BD82F4B.40209@behnel.de>

Lisandro Dalcin, 28.04.2010 01:21:
> diff -r 3adaf1ef25b9 Cython/Compiler/TypeSlots.py
> --- a/Cython/Compiler/TypeSlots.py	Tue Apr 27 21:36:49 2010 +0200
> +++ b/Cython/Compiler/TypeSlots.py	Tue Apr 27 20:14:34 2010 -0300
> @@ -637,7 +637,7 @@
>       EmptySlot("tp_print"), #MethodSlot(printfunc, "tp_print", "__print__"),
>       EmptySlot("tp_getattr"),
>       EmptySlot("tp_setattr"),
> -    MethodSlot(cmpfunc, "tp_compare", "__cmp__"),
> +    MethodSlot(cmpfunc, "tp_compare", "__cmp__", py3k = '<RESERVED>'),
>       MethodSlot(reprfunc, "tp_repr", "__repr__"),
>
>       SuiteSlot(PyNumberMethods, "PyNumberMethods", "tp_as_number"),
>
> BTW, Should I push this?

Absolutely.


> We could try to synthesize __richcmp__ from __cmp__

We shouldn't. __richcmp__ is the right thing to implement. If we consider 
emulating it from something, that would be the eq/lt/gt/... special methods.

Stefan

From dalcinl at gmail.com  Wed Apr 28 17:08:39 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 28 Apr 2010 12:08:39 -0300
Subject: [Cython] Two bugs with Py3 support
In-Reply-To: <4BD82F4B.40209@behnel.de>
References: <w2y2d4ecf241004260821u18e29ff0u3c3dd38d9f0d0aa7@mail.gmail.com>
	<4BD75F05.10602@gmx.de>
	<j2he7ba66e41004271621z13112061nd6df7bd39ee1c931@mail.gmail.com>
	<4BD82F4B.40209@behnel.de>
Message-ID: <o2he7ba66e41004280808w21e845f3h3a4913c06c796015@mail.gmail.com>

On 28 April 2010 09:51, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Lisandro Dalcin, 28.04.2010 01:21:
>> diff -r 3adaf1ef25b9 Cython/Compiler/TypeSlots.py
>> --- a/Cython/Compiler/TypeSlots.py ? ?Tue Apr 27 21:36:49 2010 +0200
>> +++ b/Cython/Compiler/TypeSlots.py ? ?Tue Apr 27 20:14:34 2010 -0300
>> @@ -637,7 +637,7 @@
>> ? ? ? EmptySlot("tp_print"), #MethodSlot(printfunc, "tp_print", "__print__"),
>> ? ? ? EmptySlot("tp_getattr"),
>> ? ? ? EmptySlot("tp_setattr"),
>> - ? ?MethodSlot(cmpfunc, "tp_compare", "__cmp__"),
>> + ? ?MethodSlot(cmpfunc, "tp_compare", "__cmp__", py3k = '<RESERVED>'),
>> ? ? ? MethodSlot(reprfunc, "tp_repr", "__repr__"),
>>
>> ? ? ? SuiteSlot(PyNumberMethods, "PyNumberMethods", "tp_as_number"),
>>
>> BTW, Should I push this?
>
> Absolutely.
>

I went further and added support for __bool__ .

http://hg.cython.org/cython-devel/rev/07dd946da241

>From all that patch that has some cleanup, the important bits are this ones:

http://hg.cython.org/cython-devel/rev/07dd946da241#l2.80 (class MethodSlot)
http://hg.cython.org/cython-devel/rev/07dd946da241#l2.175 (__nonzero__/__bool__)
http://hg.cython.org/cython-devel/rev/07dd946da241#l2.223 (__cmp__)


>> We could try to synthesize __richcmp__ from __cmp__
>
> We shouldn't. __richcmp__ is the right thing to implement.

OK, you are definitely right.

> If we consider
> emulating it from something, that would be the eq/lt/gt/... special methods.
>

Even more taking into account that in many cases all you actually what
is __eq__ and __neq__


-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From dalcinl at gmail.com  Wed Apr 28 17:57:40 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 28 Apr 2010 12:57:40 -0300
Subject: [Cython] constant folding issue?
Message-ID: <j2ve7ba66e41004280857m1e6c9957o75b4d40109e00edd@mail.gmail.com>

Stefan, as you know I'm always fighting against GCC warnigns.
Suddenly, this one started to show up:

compile_time_unraisable_T370.c:365: warning:
'__pyx_f_28compile_time_unraisable_T370_raiseit' defined but not used

$ cat tests/errors/compile_time_unraisable_T370.pyx

cdef int raiseit():
    raise IndexError
if False: raiseit()

_ERRORS = u"""
FIXME: provide a good error message here.
"""

Inspecting the generated C code, the "if False: raiseit()" is skipped.

Is this an intended side effect of your recent work on constant folding?

-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From stefan_ml at behnel.de  Wed Apr 28 18:40:27 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 28 Apr 2010 18:40:27 +0200
Subject: [Cython] constant folding issue?
In-Reply-To: <j2ve7ba66e41004280857m1e6c9957o75b4d40109e00edd@mail.gmail.com>
References: <j2ve7ba66e41004280857m1e6c9957o75b4d40109e00edd@mail.gmail.com>
Message-ID: <4BD864FB.8090406@behnel.de>

Lisandro Dalcin, 28.04.2010 17:57:
> Stefan, as you know I'm always fighting against GCC warnigns.
> Suddenly, this one started to show up:
>
> compile_time_unraisable_T370.c:365: warning:
> '__pyx_f_28compile_time_unraisable_T370_raiseit' defined but not used
>
> $ cat tests/errors/compile_time_unraisable_T370.pyx
>
> cdef int raiseit():
>      raise IndexError
> if False: raiseit()
>
> _ERRORS = u"""
> FIXME: provide a good error message here.
> """
>
> Inspecting the generated C code, the "if False: raiseit()" is skipped.
>
> Is this an intended side effect of your recent work on constant folding?

I wouldn't call it a "side" effect, it's rather the intended behaviour. 
Code inside of unreachable conditional blocks no longer emits C code. I 
already fixed a couple of tests, and this one needs fixing, too.

It actually needs fixing in two ways - it also lacks an error message that 
is tested for. IIRC, we agreed to emit a compiler warning for this, right? 
I don't think there's a way to test for warnings yet, and it looks like the 
"unraisable" runtime message passes the doctest output catcher.

Stefan

From stefan_ml at behnel.de  Wed Apr 28 18:55:53 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 28 Apr 2010 18:55:53 +0200
Subject: [Cython] Haoyu Bai's application accepted for GSoC
In-Reply-To: <D100655C-0D9A-47F1-BB34-94338B6D6873@math.washington.edu>
References: <D100655C-0D9A-47F1-BB34-94338B6D6873@math.washington.edu>
Message-ID: <4BD86899.6090402@behnel.de>

Robert Bradshaw, 27.04.2010 21:45:
> The list of accepted applications for Google Summer of Code 2010 were
> just announced yesterday, and Haoyu Bai's Cython application was
> officially accepted. Congratulations, and we're looking forward to a
> lot of great stuff happing this summer.

Happy to read this. Congratulations!

Stefan

From craigcitro at gmail.com  Wed Apr 28 19:14:00 2010
From: craigcitro at gmail.com (Craig Citro)
Date: Wed, 28 Apr 2010 10:14:00 -0700
Subject: [Cython] Haoyu Bai's application accepted for GSoC
In-Reply-To: <4BD86899.6090402@behnel.de>
References: <D100655C-0D9A-47F1-BB34-94338B6D6873@math.washington.edu> 
	<4BD86899.6090402@behnel.de>
Message-ID: <t2ua8c4ff4c1004281014x21a4ae69m273adf07911d3a53@mail.gmail.com>

>> The list of accepted applications for Google Summer of Code 2010 were
>> just announced yesterday, and Haoyu Bai's Cython application was
>> officially accepted. Congratulations, and we're looking forward to a
>> lot of great stuff happing this summer.
>
> Happy to read this. Congratulations!
>

Ditto -- between this and several of us having other responsibilities
off our shoulders, it should be an exciting summer for Cython. ;)

-cc

From dalcinl at gmail.com  Wed Apr 28 20:42:25 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 28 Apr 2010 15:42:25 -0300
Subject: [Cython] constant folding issue?
In-Reply-To: <4BD864FB.8090406@behnel.de>
References: <j2ve7ba66e41004280857m1e6c9957o75b4d40109e00edd@mail.gmail.com>
	<4BD864FB.8090406@behnel.de>
Message-ID: <t2ye7ba66e41004281142l25ef295ez30062e6d3de021ef@mail.gmail.com>

On 28 April 2010 13:40, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Lisandro Dalcin, 28.04.2010 17:57:
>> Stefan, as you know I'm always fighting against GCC warnigns.
>> Suddenly, this one started to show up:
>>
>> compile_time_unraisable_T370.c:365: warning:
>> '__pyx_f_28compile_time_unraisable_T370_raiseit' defined but not used
>>
>> $ cat tests/errors/compile_time_unraisable_T370.pyx
>>
>> cdef int raiseit():
>> ? ? ?raise IndexError
>> if False: raiseit()
>>
>> _ERRORS = u"""
>> FIXME: provide a good error message here.
>> """
>>
>> Inspecting the generated C code, the "if False: raiseit()" is skipped.
>>
>> Is this an intended side effect of your recent work on constant folding?
>
> I wouldn't call it a "side" effect, it's rather the intended behaviour.
> Code inside of unreachable conditional blocks no longer emits C code. I
> already fixed a couple of tests, and this one needs fixing, too.
>

OK. Then I'll have to work a bit more to silent GCC.



-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From dalcinl at gmail.com  Wed Apr 28 23:08:14 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 28 Apr 2010 18:08:14 -0300
Subject: [Cython] all GCC warnings silenced
Message-ID: <j2ie7ba66e41004281408p8c0f40d5ub4cbaa9d4598dddb@mail.gmail.com>

This is the fist time since I've started contributing to Cython that running

$ CPPFLAGS='-Wextra -Wno-unused-parameter' python runtests.py

finalized without a single warning.

Please contribute to maintain this.

-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From stefan_ml at behnel.de  Thu Apr 29 06:11:20 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 29 Apr 2010 06:11:20 +0200
Subject: [Cython] all GCC warnings silenced
In-Reply-To: <j2ie7ba66e41004281408p8c0f40d5ub4cbaa9d4598dddb@mail.gmail.com>
References: <j2ie7ba66e41004281408p8c0f40d5ub4cbaa9d4598dddb@mail.gmail.com>
Message-ID: <4BD906E8.7030005@behnel.de>

Lisandro Dalcin, 28.04.2010 23:08:
> This is the fist time since I've started contributing to Cython that running
>
> $ CPPFLAGS='-Wextra -Wno-unused-parameter' python runtests.py
>
> finalized without a single warning.

Thanks a lot for going through this.


> Please contribute to maintain this.

I added "--std=c99 -Wall -Wextra -Wno-unused-parameter" to the CFLAGS on 
the Hudson-CI server, so that we can look up the current warnings in the 
build logs. (does -Wextra imply -Wall? "man gcc" doesn't tell me...)

Stefan

From dalcinl at gmail.com  Thu Apr 29 16:05:53 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 29 Apr 2010 11:05:53 -0300
Subject: [Cython] all GCC warnings silenced
In-Reply-To: <4BD906E8.7030005@behnel.de>
References: <j2ie7ba66e41004281408p8c0f40d5ub4cbaa9d4598dddb@mail.gmail.com>
	<4BD906E8.7030005@behnel.de>
Message-ID: <l2we7ba66e41004290705n8465ff5bi3c3ddf05b31f0be6@mail.gmail.com>

On 29 April 2010 01:11, Stefan Behnel <stefan_ml at behnel.de> wrote:
>
> I added "--std=c99 -Wall -Wextra -Wno-unused-parameter" to the CFLAGS on
> the Hudson-CI server, so that we can look up the current warnings in the
> build logs.

BTW, is there any way to see these build logs?

>
> (does -Wextra imply -Wall? "man gcc" doesn't tell me...)
>

I don't know. But using both will not harm, right?


-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From dalcinl at gmail.com  Fri Apr 30 20:50:16 2010
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Fri, 30 Apr 2010 15:50:16 -0300
Subject: [Cython] push related to ticket #399
In-Reply-To: <B9DA4BC5-9A5E-44D8-BBA1-C4ABB60AC6F9@math.washington.edu>
References: <u2re7ba66e41004211841se539a4bcu47dca43789efdb39@mail.gmail.com>
	<4BD00042.4040002@student.matnat.uio.no>
	<A77003C3-8A43-45C1-A990-E225A6C8960D@math.washington.edu>
	<4BD3D5C2.5090609@behnel.de>
	<m2qe7ba66e41004251641p23b1d7a3xd6cc9b4592761781@mail.gmail.com>
	<4BD531BE.1090801@student.matnat.uio.no>
	<B9DA4BC5-9A5E-44D8-BBA1-C4ABB60AC6F9@math.washington.edu>
Message-ID: <v2ie7ba66e41004301150w97c13b6ch62a1341b64c5d9a1@mail.gmail.com>

On 26 April 2010 14:24, Robert Bradshaw <robertwb at math.washington.edu> wrote:
> On Apr 25, 2010, at 11:25 PM, Dag Sverre Seljebotn wrote:
>
>> Lisandro Dalcin wrote:
>>> On 25 April 2010 02:40, Stefan Behnel <stefan_ml at behnel.de> wrote:
>>>>>> My vote is in favor of simply always making "ssize_t" in Cython
>>>>>> always
>>>>>> mean Py_ssize_t in C.
>>>>>>
>>>>> This sounds very reasonable to me.
>>>> +1. If CPython defines one as the other anyway,
>
> That's what I thought. Let's do it.
>

Pushed: http://hg.cython.org/cython-devel/rev/bc1e09f2fef7 (and #399
resolved as fixed)



-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169