[Cython] canonical way of converting numpy arrays to C arrays

[Dag Sverre Seljebotn]

Ondrej Certik wrote:
> On Mon, Oct 13, 2008 at 6:33 PM, Dag Sverre Seljebotn
>
>   
>> Note that this is non-trivial since any copy made must also not be
>> deallocated, so that iarray_d would have to return a reference which
>> must be held on to for the duration of using the buffer data.
>>
>> If you need to accept ndarrays coming from the user, a better idiom is
>> likely to take a callback function which will be called with the buffer
>> as an argument, or similar -- returning the buffer through a double**
>> (or np.float64_t**) is rather error-prone when it comes to reference
>> counting.
>>     
>
> Yeah, I thought pretty much along the same lines.
>
> Basically, I am fine if the function raises an exception on
> noncontiguous arrays and/or copy the array automatically, so that's
> not an issue.
>
> The big problem is with the reference counting, i.e. who is
> reposnsible for freeing the double** memory. Basically, the memory is
> automatically freeed when the numpy array goes out of scope, right? So
> basically, the rule is: either do something with the double** array
> immediatelly (in C/C++) on the fly, or make a copy of it if you need
> it longer. Another option would be to Py_IncRef/DecRef the numpy array
> to make sure the C array is not recycled.
>   
Pretty much. To hold a reference, simply assign it to a Cython "object" 
variable, don't bother manually reference counting it.

I think the general rule should be "keep the data in the numpy array for 
as long as possible". I must admit that I cannot imagine many situations 
where it would be good to store the double* for future reference rather 
than the entire Python array object. They're just pointers, it's not 
like it is "heavier" to store the ndarray :-)

I suppose one case is if you have a stateful C library like this:

clib.set_data(mydata)
clib.perform(...)
clib.release_data()

and you want to create a 1:1 wrapper in Cython. Yes, in that case you 
need to mirror the state info on the Cython-side and store a reference 
to the ndarray in the wrapper as well as feeding the pointer to C.

Dag Sverre