[Cython] Draft for operator overloading

[Robert Bradshaw]

On Mar 16, 2008, at 3:42 AM, Dag Sverre Seljebotn wrote:

>>> http://wiki.cython.org/enhancements/operators/ambitious
>>
>> Essentially, you still will have a transformation that converts the
>> binary operations to a function call (which may or may not be
>> inlined, either by Cython or by gcc (I would say the latter). The
>> function body comes from the .pxd file. This is a bit odd, but I
>> think the right place to put it.
>
> I'll tackle who inlines at the bottom, but I'll agree it should  
> usually be
> gcc.
>
> Bear in mind that I don't have much experience wrapping C code with
> Cython, I've read the Pyrex docs but I didn't understand everything  
> fully.
>
> I don't believe it has to be in the .pxd file but because I know  
> little
> about the internals there it was a fuzzy point. Basically, the C++- 
> style
> operators should be in the PXD file as they only are declarations and
> don't contain any code. However the Python-style operators should  
> be in
> .pyx.

The .pxd files should contain everything needed to cimport a class.  
This is the whole point of .pxd files, and since it is the module  
*using* the code that needs to know how to set/get items, etc. it  
needs to be in the .pxd file.

An example that makes this clear is numpy--there isn't even a .pyx  
file to put the code into (just the .c files, and a .pxd file that  
goes along with them).

> Yes, a transform that translates operators to function calls is  
> required.
> In the case that a C-style operator is declared (this will only  
> happen on
> C++ classes and the native types) it is simply let through to C  
> code (so
> it replaces whatever is currently syntax checking operator use), in  
> the
> case that a Python-operator is declared it is translated to
> .__operator__(), which in turn gets translated to a normal C function
> call.

And then the default __operator__() would get translated to the  
Python operation (i.e. PyNumber_Add)?

> I'm not sure whether I want to extend the struct so that it is  
> possible to
> declare member functions (non-polymorphic!, basically all they  
> would do is
> pass the subject as self, and make sure the function is mangled and
> implemented); or whether I want to require NumPy to basically  
> implement a
> Python class (with fancy operators) wrapping the C type (without
> operators).

Not quite following you here.

>> If the given type is a parameterized type, how would the "body" of
>> the method refer to this compile-time data? You had a typeof(x)
>> pseudo function that might work--i.e. typeof(x).foo() would call the
>> method foo of the type x (as known at compile time), call foo() on it
>> which by would return a note that would be inserted into the tree at
>> that point.
>
> I don't believe it is necesarry wth type-only functions. It is all
> standard Python stuff. However for a type with arguments one would  
> want to
> (in a Python operator) access the type arguments, which could be  
> done by
> type(self).arg).

The compile-time type and runtime type are two distinct objects, and  
will have different methods and members. There may not even be a 1-1  
correspondence. And it would be unsafe at compile time to evaluate  
type(x) for most x (e.g. one could get a subclass).

> Then there's the issue of dealing with C++ templates (because "T"  
> might be
> a type name in the C++-style operator+ declaration) but I think that
> should actually come in the form of native C++ template support and
> special keywords for that.

We'll have to deal with this before treating templates--if a cython  
array is type float, the return value of indexing should be the same.

> About who inlines then:
>
> NumPy is an excemption because GCC isn't smart enough (at least on - 
> O3).
> Consider accessing a numpy array in a tight for-loop. The []  
> operator must
> contain lookup and division of the stride which can be cached  
> outside of
> the for-loop; however with
>
> If Cython inlines, then it might be possible for another transform  
> coming
> *after* the inlining to "lift" out the recomputed variables (detect  
> that
> they cannot change within the loop). But, it might be a bit far- 
> fetched.
> Anyway, inlining would be controlled with a decorator.

This is a good point--but I'm not sure if we should worry about this  
too much for the first iteration.

> An alternative is a new functionality, I'll call it "scope", but  
> I'll have
> to write a new draft about it I think.

Sounds a lot like control flow analysis we kicked around back in  
December/January. At the time I thought it'd be too ambitious, but it  
seems more reasonable now.

- Robert