[Cython] Fwd: Lisp inspired transforms

[David J. C. Beach]

I've been experimenting with using Cython using a simple python  
preprocessor that I wrote.  It would seem that everything I've done  
with my preprocessor could potentially be done with "Lisp inspired  
transforms".  I've been particularly interested in creating a math  
library for small matrices and vectors which uses explicit loop- 
unrolling at compile time.

I believe that the idea behind Cython calls for this type of  
metaprogramming more than Python itself.  The reason for this is that  
Cython is capable of running sophisticated algorithms at C-speed, and  
this means that compile-time inlining, unrolling, and policy-based  
abstractions (a la Alexandrescu) become somewhat compelling.  It also  
seems possible to implement generator functions in Pyrex using these  
code transforms.

I do, however, understand the hesitance to add a Cython feature that  
falls completely outside both Python and C heritage.

I have 11 years experience with Python and 13 with C++.  The kind of  
hatred I've developed for C++ is the slow-building kind that grows  
over years of watching superior alternatives foregone because "C++ is  
an industry standard".  I implore you *not* to adopt the C++ template  
meta-programming syntax!

FWIW, I like options 3, 2, and 1, in that order.  C++ is an absurdly  
hard-to-parse language.  I fear that #1 could derail the project.

Just my $0.02.

David



On Apr 8, 2008, at 7:46 AM, Martin C. Martin wrote:

> To recap the motivation for having it in base Cython:
>
> - C++ template metaprogramming shows the value of compile time
> metaprogramming, i.e. writing programs that run at compile time and  
> that
> produce programs as their output.
>
> - However, C++ templates are essentially a separate, purely functional
> programming language embedded in C++, with different syntax and
> semantics than C+.  Of course, they were never really intended or
> designed for metaprogramming.  But now that they're being used that  
> way,
> it's clear that the substitution model of templates is awkward and
> unnatural.  If we're going to write little functions to run at compile
> time, it makes sense to use the syntax and semantics of the base  
> language.
>
> Compile time metaprogramming doesn't exist in Python, so adding it to
> Cython means extending Cython beyond what Python has.  There are a
> couple options:
>
> 1. Add a way to generate C++ templates, and use that for
> metaprogramming.  It keeps Cython as "writing both Python and C++ with
> an extended Python syntax."
>
> 2. Don't add metaprogramming, even C++ templates.  Keeps Cython  
> close to
> Python and C and C++, including all the good & bad parts.
>
> 3. Add a way to specify Python code for the transformation.  This
> recognizes that metaprogramming is a valuable activity that Cython
> developers will want to do; that the existing way to do it in C++ is
> more-or-less not up to the task; and that it's better to provide a  
> new,
> cleaner mechanism using what we've learned in hindsight.
>
> Of course, metaprogramming in an imperative, stateful language,  
> opens a
> can of worms, e.g. it will be valuable to modify Python data at  
> compile
> time, and have that serialized once all transforming is done, then
> loaded at the start of runtime.  I don't think any of these problems  
> are
> particularly difficult though.
>
> So, what do people this is the best way forward for Cython?
>
> Best,
> Martin
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev

--
David Beach
<beach at verinet.com>