[Cython] Lisp inspired transforms

[Martin C. Martin]

Stefan Behnel wrote:
> Robert Bradshaw wrote:
>> Though it is tempting to head down the language development path,
>> adding little (or big) features that make it more powerful than
>> Python itself, I think doing so will actually be counterproductive to
>> the goals stated above. Perhaps there could be a Cython++ that is a
>> proper superset of the Cython language with more powerful features
>> (though I'd hope not near the gap of C vs. C++) but in the near term
>> we should be focusing on things like being able to compile all of
>> Python as it is.
> 
> I agree with Robert. As long as Cython does not support closures, for
> example, it cannot come close enough to being a real option for speeding
> up existing (non-static) Python code and making it easy to use for
> non-C-but-Python programmers.

Both your and Robert's thoughts are very wise.

> For the time being, we should try to
> 
> a) implement as many Python (3?) language features as possible, keeping in
> mind that a correct implementation is more important than a fast one,
> especially for dynamic features that do not have a direct counterpart in
> the C world.
> 
> b) get a well-designed and well-integrated compile-time code
> transformation infrastructure in place, thus allowing to provide pluggable
> language enhancements *later* and independent of the core compiler, which
> could then become an advanced Cython++ distribution (or make it back into
> mainstream). I see the major focus here on adjusting the line between
> compile-time and runtime code evaluation, and maybe some additional AOP
> features (as the ones Martin described).
> 
> I think it would help Cython a *lot* to have a stable core language
> feature set that is well based in the Python language, *before* we start
> extending the language with all sorts of 'cool' new features that may
> already have some sort of (run-time) equivalent in Python. For a
> programming language, stability is a very valuable feature of its own -
> and there should (preferrably) be "one way to do it". That's why I like
> Martin's transformers, for example, they look like plain Python but run at
> compile time. I think that's the way to do it.

Do you see deftrans as the "well-integrated compile-time code 
transformation infrastructure," and thus a target for (base) Cython, 
rather than Cython++?  Or by infrastructure, do you mean a change to the 
compiler to support Python-coded transforms, without saying where these 
transforms come from?  In the latter, deftrans (a syntatical convention 
for defining the transforms) becomes part of Cython++, and if the 
experiment works out, is incorporated back into Cython.

Best,
Martin