[Cython] array.array and PIL directly ?

[Brent Pedersen]

On Thu, Jun 11, 2009 at 2:01 AM, Dag Sverre
Seljebotn<dagss at student.matnat.uio.no> wrote:
> Brent Pedersen wrote:
>> On Wed, Jun 10, 2009 at 3:23 PM, Brent Pedersen<bpederse at gmail.com> wrote:
>>> On Wed, Jun 10, 2009 at 12:08 PM, Dag Sverre
>>> Seljebotn<dagss at student.matnat.uio.no> wrote:
>>>> Brent Pedersen wrote:
>>>>> On Thu, May 14, 2009 at 9:05 AM, Robert<kxroberto at googlemail.com> wrote:
>>>>>> Dag Sverre Seljebotn wrote:
>>>>>>> Robert wrote:
>>>>>>>> How to deal with Python's array.array directly - as with numpy.pxd
>>>>>>>> is there a array.pxd ?
>>>>>>> Which Python version?
>>>>>> mainly 2.6; and 2.3
>>>>>>
>>>>>>> Under Python 3 this should probably happen automatically, try:
>>>>>>>
>>>>>>> cdef object[int, ndim=1, mode="c"] arr = yourarray
>>>>>>>
>>>>>>> Not sure about Python 2.6+
>>>>>> 2.6.2 didn't:
>>>>>> File "calc_c.pyx", line 32, in calc_c.test1 (calc_c.c:808)
>>>>>>     cdef object[float, ndim=1, mode="c"] b = pyarray
>>>>>> TypeError: 'array.array' does not have the buffer interface
>>>>>>
>>>>>> in Python "buffer(myarray)" also behaves strange
>>>>>>
>>>>>>> For Python 2.5- an array.pxd must be written. It is not difficult, one
>>>>>>> simply follows the pattern in numpy.pxd (by implementing __getbuffer__
>>>>>>> and filling in the Py_buffer struct).
>>>>>>>
>>>>>>> If somebody ends up doing this, please submit it for inclusion in
>>>>>>> Python.
>>>>>> I've put a array.pxd here:
>>>>>> http://trac.cython.org/cython_trac/ticket/314
>>>>> hi, what's the status on this? it seemed very useful.
>>>>> if it needs tests, i could try to write some given some info on where
>>>>> to start, what to
>>>>> cover or maybe some cython-numpy tests to from which to crib?
>>>>> thanks,
>>>>> -brent
>>>> Great!
>>>>
>>>> The main issue with the patch as I see it is that it tries to hack on
>>>> multi-dimensionality. That is very easily done simply by writing a
>>>> subclass of array instead, and so doesn't belong in the pxd like this.
>>>>
>>>> If you could just remove the multi-dimensional stuff from it I'd be
>>>> happy to accept it. Accompanying tests are strongly preferred though.
>>>>
>>>> There's a section on writing tests here:
>>>> http://wiki.cython.org/HackerGuide
>>>>
>>>> I don't expect the numpy tests to be too useful (though they are in
>>>> tests/run), just make sure the basics work with a couple of different
>>>> datatypes. I'll be happy to suggest improvements if you make a first
>>>> iteration anyway.
>>>>
>>> thanks for the pointers,
>>> i have a start on this, but what should i do with the arrayarray.h?
>>> there doesnt seem to be any cases where a header is included with cython itself.
>>>
>>> -brent
>>>
>>
>> here's a first iteration with tests for some feedback.
>> i'm still not sure what to do with the .h file. currently
>
> Hmm. If it wasn't for the #ifdefs it could have been ported to Cython
> code and inserted in the pxd I think.
>
> Anyway, there's not much to do -- but you can have a look at runtests.py
> to make it add Cython/Includes to the C include path automatically?

i updated the patch to do that, but then what about when running outside
of tests? perhaps this is better as its own python module? robert?
i'm attaching updated patch here which adjusts the runtests.py hopefully
someone can add to the tests or suggest more. they're surely not complete
as i have only a superficial understanding of this.

-brent

>
> I'm a bit uncertain about this though. Is the faster creation etc.
> really necesarry? It seems like adding a lot of complexity (basically
> one needs to keep up with all Python versions and verify that the header
> still works for each new Python release). So I'm not quite sure about
> it...thoughts? But if it is deemed important I think it can go in.
>
> Tests seems fine -- I only glanced superficially, I just see that they
> are there and trust the submitter of the patch to know the subject
> matter better than me.
>
> Finally, to properly support the PEP 3118, one should make sure that
> memory is not deallocated between a call to getbuffer and release. I.e.
> when the array is resized, one should not free the memory, but
> releasebuffer should do that.
>
> I realize that this is likely impossible to do though (the array can be
> resized through Python calls, right?) so we'll just have to drop that
> and make a big warning not to resize while a buffer is acquired.
>
> --
> Dag Sverre
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: array.export.patch
Type: text/x-patch
Size: 16652 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20090611/58272a94/attachment-0001.bin