[Cython] [Pyrex] Language stability

[Robert Bradshaw]

On May 14, 2008, at 6:27 AM, Stefan Behnel wrote:

> Robert Bradshaw wrote:
>> I'm not sure how much this would help (it would
>> some for sure), as I believe many of these iterators are already
>> written in C and it's a series of C calls. I think the biggest
>> savings is that
>>
>>      cdef int i
>>      for i, k in enumerate(L):
>>          ...
>>
>> could increment i as a c int, and avoid all the tuple packing/
>> unpacking.
>
> ... which is a rather big overhead for such a trivial iterator. The  
> thing
> is, many iterators really do extremely simple things, but require  
> all the
> tuple packing and function call indirection *on each iteration*.  
> That's
> different from functions like max() or the Py2 zip(), which are called
> once and the do loads of stuff in C.
>
> Just look at these:
>
> http://docs.python.org/lib/itertools-functions.html
>
> The really simple ones are: enumerate, range, chain, count, islice,  
> repeat
>
> count() and range() are actually equivalent in their iterable  
> versions.
> But I think enumerate() and islice(), and maybe chain() might be worth
> optimising.

I am agreeing with you that this is worth doing, I was just pointing  
out that your nesting proposal didn't save the tuple packing/ 
unpacking step which I believe is the most significant overhead.

>> Likewise, the tuple packing/unpacking could be avoided for
>> zip (assuming the number of targets is equal to the number of
>> arguments).
>
> Which would be part of the compiler decision if this should run in  
> plain C
> code or not. One thing to note: zip() is an iterator in Py3, which  
> changes
> its semantics in that you must now call list(zip()) if you want to  
> modify
> its arguments in place. And zip() is actually less trivial than the  
> above
> mentioned ones. Maybe zip() is not worth being optimised.

I use zip more than anything but range... However, I don't think  
there's much if any savings to optimize these except in the context  
of a loop, and in that case zip is pretty easy to do.

- Robert