[Cython] -dagss merge? + what next

[Dag Sverre Seljebotn]

I'm at a stage where I think we can consider a new merge of -dagss. What 
I say below applies to revision 876:5df94de34ff7; the -dagss branch will 
promptly go unstable again so make sure you pull this one.

Buffers are basically working (I am reasonably confident it is working 
but I still have ideas for testcases that I haven't written so the 
testsuite is not exhaustive). In particular I'd like to put it out as an 
"available beta component" in the next Cython release (I can write some 
release notes for it).

What next? Comments welcome on this:
- I'd like to leave buffers in the global scope or as attributes out of 
it for now. It is really non-trivial and I think it would be wrong to 
prioritize it above all the other stuff I could do.
- Given that, I think these three tasks are important:
    - Supporting structs (currently only ints and floats are supported). 
Complex numbers are part of that.
    - Temp allocation gets quite hairy with buffers and I'd like to 
implement Greg Ewing's proposal to clean it up. But there is a potential 
for cleaning it more than it is within the existing framework as well.
    - "final" functions in pxd file, in order to implement 
numpy.ndarray.__getbuffer__ in the pxd file. (I won't do any inlining in 
Cython, it is just a way of having code automatically copied from the 
pxd to the pyx (and potentially skip vtable lookups if I get around to it)).

Those are the big tasks, but there are also a few smaller nice-haves I 
can go for (a "cython.shape" builtin that allows you to retrieve the 
shape information of a buffer, and so on).

Notes:

- Includes is now always included. (Though I am not any longer using the 
__cython__ namespace, my test cases will like to cimport things).

- Get/ReleaseBuffer is no longer defined when not needed

- Temp allocation is rather nasty at the moment (using my prototype 
which we ended up rejecting!). I am thinking that I might rather spend 
half a day to a day implementing Greg Ewing's ideas and use those than 
fix it. (I'm thinking along the lines of first implementing 
"code.fork()" which forks the output stream, using a cStringIO "tree" so 
that no buffer copying is needed).

But it is working, and even though I'll definitely remove it I don't 
think it is blocking a merge.

- Overall strategy: I've modified FuncDefNode, NameNode and IndexNode to 
do what I need to do. No new nodes. I ended up thinking this was much 
cleaner than inserting new nodes:
   - IndexNode already supports two different concepts (C arrays and 
Python Get/SetItem), adding a third seemed to be consistent with this.
   - NameNode: The way things are implemented, buffer auxiliary 
variables (a lot of them) are declared and connected to the local 
variable that is of the buffer type. So a typecasting-style node that 
primarily work through temporaries would have been less clean. NameNode 
also already support many different kinds of operations and so adding 
another one seemed ok.

I'm ok with splitting into seperate buffer nodes later, but I think it 
should be done while also splitting the rest, i.e. transform IndexNode 
into CArrayIndexNode, PythonIndexNode and BufferIndexNode and so on. 
(But, seeing as that is only for abstract purity at the moment I 
definitely won't touch it.)

-- 
Dag Sverre