[Cython] Buffer passing implementation details

[Dag Sverre Seljebotn]

(Kurt is the major recipient of this.)

I've been thinking some more on buffer passing. In the end (and perhaps 
even in summer; I could perhaps have a go at it working alongside you) 
the scenario we are looking at is something like

external_set_foo_array(foo_handle, self.arr[2::2])

with fast operations all the way.

That is
a) Efficient slicing without Python overhead (#178)
b) Storing acquired buffers in cdef class fields (#301)
c) External functions can keep a reference to the buffer (not really 
necesarry, but it is necesarry for internal Cython "cdef" functions, and 
it would be nice to treat them the same)
d) The base pointer may have to be moved (again slicing can do this)

This seems to make it clear that
a) The Py_buffer is not suitable as the primary vessel of our buffer 
data, e.g. to the external function. It can't work that well with 
slicing as we must maintain the original Py_buffer data when releasing it.
b) We need a reference count. E.g. doing a slice, or assigning to 
self.arr, would increase this count. This must go on the heap; and so we 
might as well put the entire Py_buffer on the heap.

I have made a new ticket outlining the thoughts in detail in #311 and 
added a comment in #299. #311 is not really in your direct interest for 
GSoC but it is very tightly coupled with #299 so it would be good to 
keep in the back of your mind anyway.

Thoughts?

It sheems a shame to let go of a neatly PEP-defined Py_buffer for 
passing to external functions, but I think it won't be too bad if we 
with each Cython version ship nice C header and Fortran include files 
containing the appropriate structs and access macros.

-- 
Dag Sverre