[Cython] Cython array type: Summary, introducing CEP 518

[Dag Sverre Seljebotn]

Stefan Behnel wrote:
> Dag Sverre Seljebotn wrote:
>> Stefan Behnel wrote:
>>> Dag Sverre Seljebotn wrote:
>>>> Stefan Behnel wrote:
>>>>> I also read in CEP 517 that resizing is not to be supported. Why not?
>>>>> It
>>>>> could just fail with an exception when it notices that there are live
>>>>> buffers on it.
>>>> It just seemed kind of useless if only Cython-allocated memory can be
>>>> resized, and not C arrays or memory in other Python objects. But we
>>>> could do it for Cython-allocated memory, perhaps by extending PEP 3118
>>>> with some Cython-specific flags etc.
>>> I don't understand what resizing has to do with PEP 3118 at all. You
>>> obviously wouldn't resize a buffer view but the memory object itself.
>>> The
>>> only link is that resizing would fail if you reduced the size of a
>>> memory
>>> object while there is a live buffer view on it.
>> I've been struggling to find the right terms for this the entire spring
>> -- perhaps the right term for CEP 517 is "typed memoryviews", although
>> which can allocate new memory when necesarry.
> 
> You keep adding features as we keep going. ;)
> 
> Then we have three types:
> 
> 1) a dynamic array type
> - allocates memory on creation
> - reallocates on (explicit) resizing, e.g. a .resize() method
> - supports PEP 3118 (and disables shrinking with live buffers)
> - returns a typed value on indexing
> - returns a typed array copy on slicing
> - behaves like a tuple otherwise
> 
> 2) a typed memory view
> - created on top of a buffer (or array)
> - never allocates memory (for data, that is)
> - creates a new view object on slicing
> - behaves like an array otherwise

This last point is dangerous as we seem to disagree about what an array is.

> 
> 3) a SIMD memory view
> - created on top of a buffer, array or memory view
> - supports parallel per-item arithmetic
> - behaves like a memory view otherwise

Good summary. Starting from this: I want int[:,:] to be the combination 
of 2) and 3) (the catch being I can't promise to implement the 3-part 
myself this time around, but I think somebody will pick it up eventually).

An issue which is not covered here is that when doing

cdef your-type-3) a, b, c
# assign b and c
a = b + c

then new memory must be created as the target of "b + c", and then 3) 
will be hard-coded to allocate something (which could likely be 1), but 
there will be no references to it anywhere it except through 3), so no 
resize can happen).

-- 
Dag Sverre