[Cython] Parallel loop annotations proposal

[Robert Bradshaw]

On Jun 23, 2008, at 8:40 AM, Brian Granger wrote:

> Here is my two cents on this...
>
> Something like OpenMP has already been tried in Python, although it
> really hasn't made it out into the wild.  Here is the link:
>
> http://www.cs.utk.edu/~luszczek/pubs/plw-200605.pdf
>
> Using markup or directives has never really become a very popular way
> of handling parallelism.  I am not sure why, but OpenMP has never
> really caught on like MPI even though MPI is more painful for many
> things.
>
> I think it would be much better to use the language (Python) itself to
> handle these things as the language is already extremely powerful.
>
> But, I think the actual parallel constructs should be kept out of
> Cython proper.  In general I think it is much better to develop
> libraries, not new language syntax.  Obviously, Cython itself is a
> distinct language, but I think considerable effort must be made to
> prevent feature creep.  I am not against new features, but it is easy
> to get carried away with these things.  It is much more important to
> make the existing features and syntax more robust.  This is why most
> successful languages develop in a rather conservative manner.  I would
> suggest a good way of evaluating new ideas is to ask to question "what
> would Guido say?"  Of course, Cython is not Python, but we all hope
> the two don't stray far from each other.
>
> Lots of projects have begun using Cython lately, which is really
> great.  This makes it more important than ever to focus on the core
> language rather than wild new features.  Plus, there are a number of
> other projects that are already having great success in tackling
> parallelism in Python.  If people are interested in parallelism, I
> highly recommend getting involved in those existing efforts.
>
> Cheers,
>
> Brian


These were *exactly* my thoughts when I started reading this thread.  
I'm not sure if the OpenMP paradigm can be supported without adding  
features to the language, which I would like to avoid.

In terms of syntax proposals, decorated inline functions (closures)  
seem the best, but I don't know how to avoid the performance hit. I  
think the for loop, as in proposal 6, is the best semantic fit.  
(Executing the body of a with statement multiple times seems a bit  
odd (it doesn't have anything to do with contexts) and would be  
difficult (if even possible?) to emulate in CPython.) Another  
(unfinished) alternative is to consider

for i in ...
for i from ...
for i xxxx ... # if we can come up with a good keyword here, the rest  
would be the same as "in."

- Robert


>