[Cython] Cython array type: Summary, introducing CEP 518

[Dag Sverre Seljebotn]

Dag Sverre Seljebotn wrote:
> Stefan Behnel wrote:
>> Dag Sverre Seljebotn wrote:
>>> Stefan Behnel wrote:
>>>> Dag Sverre Seljebotn wrote:
>>>>> Stefan Behnel wrote:
>>>>>> I also read in CEP 517 that resizing is not to be supported. Why not?
>>>>>> It
>>>>>> could just fail with an exception when it notices that there are live
>>>>>> buffers on it.
>>>>> It just seemed kind of useless if only Cython-allocated memory can be
>>>>> resized, and not C arrays or memory in other Python objects. But we
>>>>> could do it for Cython-allocated memory, perhaps by extending PEP 3118
>>>>> with some Cython-specific flags etc.
>>>> I don't understand what resizing has to do with PEP 3118 at all. You
>>>> obviously wouldn't resize a buffer view but the memory object itself.
>>>> The
>>>> only link is that resizing would fail if you reduced the size of a
>>>> memory
>>>> object while there is a live buffer view on it.
>>> I've been struggling to find the right terms for this the entire spring
>>> -- perhaps the right term for CEP 517 is "typed memoryviews", although
>>> which can allocate new memory when necesarry.
>> You keep adding features as we keep going. ;)
>>
>> Then we have three types:
>>
>> 1) a dynamic array type
>> - allocates memory on creation
>> - reallocates on (explicit) resizing, e.g. a .resize() method
>> - supports PEP 3118 (and disables shrinking with live buffers)
>> - returns a typed value on indexing
>> - returns a typed array copy on slicing
>> - behaves like a tuple otherwise
>>
>> 2) a typed memory view
>> - created on top of a buffer (or array)
>> - never allocates memory (for data, that is)
>> - creates a new view object on slicing
>> - behaves like an array otherwise
> 
> This last point is dangerous as we seem to disagree about what an array is.
> 
>> 3) a SIMD memory view
>> - created on top of a buffer, array or memory view
>> - supports parallel per-item arithmetic
>> - behaves like a memory view otherwise
> 
> Good summary. Starting from this: I want int[:,:] to be the combination 
> of 2) and 3) (the catch being I can't promise to implement the 3-part 
> myself this time around, but I think somebody will pick it up eventually).
> 
> An issue which is not covered here is that when doing
> 
> cdef your-type-3) a, b, c
> # assign b and c
> a = b + c
> 
> then new memory must be created as the target of "b + c", and then 3) 
> will be hard-coded to allocate something (which could likely be 1), but 
> there will be no references to it anywhere it except through 3), so no 
> resize can happen).

While I'm at it, another confusing thing is that I proposed

cdef int[:,:] a = int[:100,:100]()

to mean "create an object of the type in your 1), then assign it to a 
view of the type in 2)&3)".

I'm quite ready to pull that part of the proposal if it seems confusing, 
requiring one to do

cdef int[:,:] a = cython.somearray[int]((100,100))

instead.

-- 
Dag Sverre