[Cython] Temp framework proposal

[Robert Bradshaw]

On Jul 23, 2008, at 2:59 PM, Dag Sverre Seljebotn wrote:

> I'd like some feedback on this idea.

Sure.

> The idea is simple, it is only the explanation that is complex.
>
> I am becoming very traditional in my code :-), but am making one
> non-traditional turn though, in how I use temps. I believe the  
> following
> temp refactoring is a) conceptually simple and not a big break with
> current ways of doing things, b) can be made very incrementally  
> (though
> there may be inconsistencies in "how things are done" along the  
> way), c)
> is useful (at least to me).
>
> Seperating out the analyse_types phase in a seperate pipeline phase  
> would
> have been an option; however this alternative proposal has the  
> advantage
> of being more transform friendly -- allowing temps to be allocated and
> used at any time.
>
> Proposal (implementation cannot be uploaded until the end of week  
> because
> of internet connection restriction but I have it locally):
>
> - There will be two temp systems in parallell for now. This means each
> function can have at worst twice the number of temps as the temp
> allocation algorithms would indicate, which I consider OK myself. (See
> below for how this can be remedied).

I'm fine for a transitional overlap. Sure beats trying to rip out the  
old one and fix everything at once :-).

> - Temps can be retrieved as a typed Entry instance, using the  
> following
> call: Symtab.new_temp(type). No releasing is needed, and it can be  
> used as
> a regular Entry. *However*, it is required that each node registers  
> the
> temp entries it is using in a list self.temps. (If you wondered, this
> requires that the temp currently stored in result_code for temp  
> exprnodes
> would show up in the temps list of the _parent_. However we can add  
> more
> conventions so we don't have to do this (like, a combination of
> self.is_temp and self.result_entry would count as a registration in
> parent(self).temps, all that is required is that the transform  
> mentioned
> below can somehow understand what is going on.)

This is starting to sound a little confusing (not that the current  
system is totally straightforward, and I really don't like the fact  
that we're just passing strings around). Also, in general I'd rather  
have things worry about going down the tree instead of up.

> - The "cname" on the temps is NOT available before code generation  
> time.
> The idea is that you hold on to and pass around the entry until code
> generation phase and then look at cname. Neither are the temps  
> available
> for lookup in any Scope, they merely exist by convention in the  
> scope of
> the node using them (though if this really is a problem this could be
> fixed).
>
> - An "AnchorTemp" (need better name) transform is run right prior  
> to code
> generation. This transform is responsible for figuring out what temps
> should actually be generated (it sets the cnames, registers the needed
> amount of temps in the scope etc.; currently it does this simply by  
> wiping
> out free_temp_entries and using allocate/release_temps on the  
> "node.temps"
> attributes, very simple code). Also (importantly), it modifies
> TryExceptNode and friends to make them clean up these temps.
>
> - An immediate consequence is that except clauses get an exact list of
> temps to decref (within the set of temps using the new system)  
> rather than
> the current shotgun approach which is suboptimal (as if it  
> mattered...)

Right now it's pretty tight, it only tries to decref the temps that  
could have been set in the block, depending on where the error is.

> if you have a deep nested expression right prior to a try block  
> with only
> simple expressions (and to get it to the theoretical minimum one could
> start decrefing the exact used temps on the raise statements rather  
> than
> in the except/finally clause...leaving that for now though).

This idea has been brought up before. Trying to dynamically keep  
track of what temps need to be cleared isn't going to be faster than  
using xdecref, and putting specialized special code under each raise  
will do really bad things to code size (an important consideration-- 
and its much better to optimized the non-error path).

> The current implementation is already useful to me, however it  
> cannot be
> used in most cases as a lot of current code store temps in  
> result_code at
> analysis time (at which point cname is not available). This can be  
> fixed
> either by refactoring or by the following hack: Let cname be easily
> recognizeable magic strings ("$!idx!$", and have Code.py run a
> substitution on all input strings, when the cname is available (or  
> is this
> too ugly?

IMHO, too ugly.

> It would bring closures closer as well I think, and I think it
> is ok if we agree that result_code should be refactored at some later
> point if one gets the time and resources; i.e. one stores  
> "result_entry"
> and other entry references instead at analysis time and only calculate
> result_code at code generation time).
>
> Given such a hack or refactor, this temp system can easily be used by
> TreeFragment so that WithStatementTransform automatically stop using
> non-temp local variables.
>
> (This has been in my drawer for some time, I am focusing on GSoC  
> now, but
> I thought I should get it out there now that it was mentioned.)

Sure.

You've probably thought about this, but I'm not seeing in your  
proposal how you're planing on handling deep expressions.

- Robert