[Cython] Cython array type: Summary, introducing CEP 518

[Dag Sverre Seljebotn]

Dag Sverre Seljebotn wrote:
> Sorry about the medium-sized length, but I'd like this to be close to my 
> last email on the subject. I'd just refer to Robert's mail, but I guess 
> some more explanation about NumPy semantics is in order for the benefit 
> of non-NumPy-users, so I've made a summary of that.
> 
> Stefan Behnel wrote:
>> Dag Sverre Seljebotn wrote:
>>> Stefan Behnel wrote:
>>>> we have three types:
>>>>
>>>> 1) a dynamic array type
>>>> - allocates memory on creation
>>>> - reallocates on (explicit) resizing, e.g. a .resize() method
>>>> - supports PEP 3118 (and disables shrinking with live buffers)
>>>> - returns a typed value on indexing
>>>> - returns a typed array copy on slicing
>>>> - behaves like a tuple otherwise
>>>>
>>>> 2) a typed memory view
>>>> - created on top of a buffer (or array)
>>>> - never allocates memory (for data, that is)
>>>> - creates a new view object on slicing
>>>> - behaves like an array otherwise
>>> This last point is dangerous as we seem to disagree about what an array
>>> is.
>> It's what I described under 1).
>>
>>
>>>> 3) a SIMD memory view
>>>> - created on top of a buffer, array or memory view
>>>> - supports parallel per-item arithmetic
>>>> - behaves like a memory view otherwise
>>> Good summary. Starting from this: I want int[:,:] to be the combination
>>> of 2) and 3)
>> You mean "3) and not 2)", right? Could you explain why you need a syntax
>> for this if it's only a view?
> 
> I suppose I meant some variation of 3) with some extra bullet points 
> (slicing in particular). We need a syntax because SIMD operations must 
> be handled as a special-case compile-time.
> 
> Robert put it well; what I want is the core NumPy array semantics on a 
> view to any array memory -- builtin, so that it can be optimized 
> compile-time. We need to return to that; trying to distill something 
> else and more generic out of this seems to only bring confusion.
> 
> (This is about 1) and 2) in Robert's mail only though.)
> 
> I'll make a list of what we mean by NumPy semantics below. At the very 
> bottom is some things which I think should *not* be included.
> 
> First:
> 
> 1) Nobody is claming this is elegant or Pythonic. It is catering for a 
> numerical special interest, nothing more nor less.
> 
> 2) As Robert put it: He won't use it himself, but the rest of Cython 
> indirectly benefits from all the Cython interest from the numerical users.
> 
> 3) The proposed semantics below are really not up for in-detail 
> discussion, what I'm really after is a "yes" or "no" -- I just don't 
> have the time, and NumPy is the de facto standard for Python numerics 
> and what everybody expects anyway. I don't want to invent something 
> entirely new.
> 

> # Some ways of multiplying all elements with 2
> x *= 2
> x[...] *= 2
> x[:,:] *= 2
> x += x
> x[...] += x

OK I'll make an exception here -- I'm willing to discuss whether we 
should depart from NumPy semantics here and let

x2 = x
x *= 2

allocate new memory, so that x2 is not modified, being consistent with a 
direct transformation to "x = x * 2". One can always write

x[...] *= 2

if one wishes to modify original memory.

-- 
Dag Sverre