[Cython] -dagss merge? + what next

[Dag Sverre Seljebotn]

Robert Bradshaw wrote:
> On Jul 29, 2008, at 3:57 AM, Dag Sverre Seljebotn wrote:
> 
>> Robert Bradshaw wrote:
>>> On Jul 29, 2008, at 2:25 AM, Dag Sverre Seljebotn wrote:
>>>
>>>> Those are the big tasks, but there are also a few smaller nice- 
>>>> haves I
>>>> can go for (a "cython.shape" builtin that allows you to retrieve the
>>>> shape information of a buffer, and so on).
>>> I saw you mention shape (and other "magic" functions), and I'm
>>> curious. Could you elaborate?
>> When acquiring a buffer, you get hold of a lot of information that is
>> not currently exported -- shape is most important. For NumPy this  
>> is ok
>> as ndarray exports its shape field for efficient access, but without
>> another mechanism you can't write generic algorithms that work with  
>> all
>> buffers without passing the shape explicitly. This even leads to less
>> clean test case code...
>>
>> I haven't thought much about the list (add as needed) but something  
>> like:
>>
>> - shape(buf) -- Returns a tuple containing the shape of the buffer.
>> However, shape(buf)[0] should compile directly to  
>> __pyx_bshape0_buf. (I
>> thought about shape(buf, 0) as well but decided against, how about  
>> you?)
>>
>> This would be the primary for-loop mechanism:
>>
>> from cython import shape
>> ...
>> cdef int i
>> for i in range(shape(buf)[0]): buf[i] = 9
>>
>> - buffer(buf), strides(buf), suboffsets(buf) -- Could have these to
>> allow low-level access (buffer gets a void* directly, strides and
>> suboffsets probably). For contiguous arrays this could be more  
>> efficient
>> (the proper high-level approach for such things are iterators). The  
>> idea
>> is that even if you write low-level code, you could use the buffer
>> syntax so that you don't need to worry about acquisition.
>>
>> - ndim(buf) -- If the ndim-at-compile-time-restriction is lifted
>> (because you have low-level access or iterators you can use instead of
>> indexing) then this will be useful.
>>
>> Finally, if I get as far as iterators (I won't!), cython.ndenumerate
>> could create very efficient iteration code. (Probably a NumPy-specific
>> iterator could be less work though...if I can't just copy the approach
>> of NumPy iterators directly...NumPy comes with a seperate n- 
>> dimensional
>> iterator class that allows you to write ndim-generic code.)
> 
> Ah, I see. These could be very useful, though I'm not sure we should  
> be augmenting the namespace to provide them (perhaps make them magic  
> methods of a buffer object instead?)

What are the disadvantages of adding a cimportable cython namespace for 
this? I don't think we can get around it personally -- in fact I was 
thinking of using it for the pragma code [1].

Meanwhile, attributes can always collide. Especially having 
"object[int]" not do an attribute lookup for every attribute access 
would be a big loss I feel.

What about adding syntax in "cdef extern class ndarray" to provide these 
functions in addition, so that the class author is in control and can 
make the decision.

That is in line with another feature I'd like feedback on, starting 
another thread.

[1]:

- cython.unchecked_getitem(buf, i, j) and cython.unchecked_cast(...)
- with cython.noboundschecking: ...
- @cython.noboundschecking
   def f(): ...

etc. Also it could be a primary vehicle (in a future far off) of making 
Cython more namespace-compatible with Python:

with cython.nogil:

-- 
Dag Sverre