[Cython] transform efficiency

[Dag Sverre Seljebotn]

Speaking of that, visitchildren (which I unfortunately named contrary to conventions) is actually a bit (or possibly very) inefficient at the moment.

One could try to clone and modify it into a fast an a simple version, where the simple drops the attribute exclude filters. The advanced functionality is used in so few places that one can just change those.

As I don't have profiling etc. set up I'll leave it as an idea for now...an extra list lookup per node per transform...don't know what of a difference it would make...

(as for gcc vs. Cython running time it appears to be a simple function of how much .h vs .pxd you are using. Makes sense.)


Dag Sverre Seljebotn
-----Original Message-----
From: Robert Bradshaw <robertwb at math.washington.edu>
Date: Monday, Sep 8, 2008 6:53 pm
Subject: Re: [Cython] transform efficiency
To: cython-dev at codespeak.netReply-To: cython-dev at codespeak.net

On Sep 8, 2008, at 9:46 AM, Dag Sverre Seljebotn wrote:
>
>> Robert Bradshaw wrote:
>> On Sep 8, 2008, at 7:41 AM, Lisandro Dalcin wrote:
>>
>>
>>> I have some working code following Dag advices (BTW, that transform
>>> stuff its a real win!).
>>>
>>
>> Yes, it certainly is! One thing that worries me is that adding a
>> bunch of tree traversals starts to slow things down--at the very
>> least we should have a transform that doesn't descend into function
>> bodies for speed reasons for stuff like this.
>>
> Lisandro simply shouldn't call self.visitchildren in  
> visit_FuncDefNode,
> no new transform type is needed.
>
>> For the record (re: transforms):
> 1) If they enable us to more quickly develop to a point where  
> Cython can
> self-compile, then that likely win backs what we temporarily loose  
> (more
> tree-traversals is only a constant-time overhead after all, and quick
> function dispatches during traversals will likely win back what we
> loose?). (A compiler slowdown for about a year is perhaps not
> "temporary" enough though...)
>
>> 2) The transform framework could be extended to be able to merge
> transforms. For instance:
>
>> pipeline = [PostParseTransform(...), ..., MergeNonConflicting(A, B, C,
> D), ...]
>
>> where you know that A, B, C and D are not interfering with one another
> and can be executed "simultaneously" in one sense (the restriction  
> here
> is that a B for a given node only depends on the previous siblings and
> ancestors with respect to what is done by A). So MergeNonConflicting
> simply takes the result of visit_FuncDefNode of A and feeds it  
> directly
> to visit_FuncDefNode of B.
>
>Yep, this might be a savings if we have lots of little stuff.
>
>> Many nodes are leaves and the tree is shallow so perhaps it doesn't  
> win
> that much; but the visitor dispatch only needs to be used once  
> (i.e. the
> types of the nodes are looked at less times) so perhaps there is some
> potential here *shrug*.
>
>> For my own uses, gcc still uses *a lot* more running-time than Cython,
> so I don't have an itch to scratch here -- how are things in SAGE?
>
>The opposite, Cython takes longer than gcc. Some of this might be due  
>to lots of re-parsing of .pxd files, which I hope to be able to  
>resolve in a better way (someday...). On profiling visitChildren is  
>up there at the top.
>
>I am bringing this up as food for though on how to make things  
>faster, but for the record I think that transforms are awesome and  
>well worth the additional (so far minor) cost in speed.
>
>- Robert
>
>
>_______________________________________________
>Cython-dev mailing list
>Cython-dev at codespeak.net
>http://codespeak.net/mailman/listinfo/cython-dev
>