[Cython] Lisp inspired transforms

[Robert Bradshaw]

On Apr 6, 2008, at 5:46 AM, Martin C. Martin wrote:
> Hi all,
>
> I've been doing some thinking and prototyping of a transform system  
> inspired by Common Lisp macros.  You can see the results as the  
> newest CEP:
>
> http://wiki.cython.org/enhancements/metaprogramming
>
> Briefly, it allows you to define a transform in the Cython source  
> code.  The transform runs at compile time, and takes the *parse  
> trees* of its arguments.  In the examples, I define a simple  
> symbolic differentiator
> "deriv" which means you can write:
>
> def eggs(a, b):
>    return deriv(5*a+b, a)
>
> and it is translated at compile time to:
>
> def eggs(a, b):
>    return (((5 * 1) + (0 * a)) + 0)
>
> You can use this, for example, with numerical optimization  
> techniques like Newton's method.  An example which differentiates 5  
> * x**3 - 10 and finds it's root is given on the above wiki page,  
> here's the output:
>
> >>> TestTrans.myfunct(10.0)
> (4990.0, 1500.0)
> >>> TestTrans.newtons(TestTrans.myfunct, 10.0)
> f( 10.0 ) = 4990.0 , f'( 10.0 ) = 1500.0
> f( 6.67333333333 ) = 1475.93037185 , f'( 6.67333333333 ) =  
> 668.000666667
> f( 4.46385893383 ) = 434.735082042 , f'( 4.46385893383 ) =  
> 298.890548717
> f( 3.00936301916 ) = 126.267956666 , f'( 3.00936301916 ) =  
> 135.843986716
> f( 2.07985587115 ) = 34.9852072625 , f'( 2.07985587115 ) =  
> 64.8870066716
> f( 1.54068464016 ) = 8.28568621842 , f'( 1.54068464016 ) =  
> 35.6056374064
> f( 1.3079774954 ) = 1.18847303518 , f'( 1.3079774954 ) = 25.6620769269
> f( 1.26166506953 ) = 0.0415843883547 , f'( 1.26166506953 ) =  
> 23.8769812152
> f( 1.25992345957 ) = 5.73769235945e-05 , f'( 1.25992345957 ) =  
> 23.8111068596
> f( 1.2599210499 ) = 1.09734443754e-10 , f'( 1.2599210499 ) =  
> 23.8110157797
> f( 1.25992104989 ) = 0.0 , f'( 1.25992104989 ) = 23.8110157795
> 1.2599210498948732
>
> It's also useful for moving computation from run time to compile  
> time, and makes it easy for the programmer to specify what should  
> happen where.
>
> There are many other use cases as well.
>
> I'm attaching a patch that implements the examples.  It's very  
> proof-of-concept, which means its really rough.  :)  I also  
> implemented the bar minimum to get the examples working, but it  
> gives you an idea.
>
> Best,
> Martin

Thanks. This looks very interesting. Also, I much prefer this type of  
"macro" to the text-substitution C-style macros that have  
occasionally been suggested.

One thought that kept going through my head as I was reading this,  
however, is that one of the current defects (in my mind) is its  
inconsistency with actual Python, and the associated learning curve  
to read and write Python. Some of this inconsistency (e.g. (minimal)  
static typing information) is necessary to achieve the kind of  
speedups we need, but I think the focus should be on features that  
narrow the gap between Cython and Python, not those that widen it.

On the other hand, features like the one above can be very useful,  
and it'd be a shame to deny their availability to "power  
users" (though it would take away one thing that I really like about  
Cython--almost any Python programmer can at least read a Cython  
function without any additional knowledge).

- Robert