[Cython] Builtin declarations + should BufferException be backported?

[Robert Bradshaw]

On Sep 23, 2008, at 1:08 AM, Dag Sverre Seljebotn wrote:

> Robert Bradshaw wrote:
>> On Sep 17, 2008, at 3:57 AM, Dag Sverre Seljebotn wrote:
>>
>>
>>> Slightly related, look at this code in Symtab.py:
>>> def declare_builtin(self, name, pos):
>>>     if not hasattr(__builtin__, name):
>>>
>>> If I'm not entirely wrong, this means that BufferError (and other  
>>> new
>>> builtin symbols) will be available only if you run Cython under  
>>> Python
>>> 2.6+. I find the fact that results of a compilation depends on the
>>> Python version used for running Cython somewhat disturbing. Should
>>> perhaps a literal list of accepted Cython builtins be present in
>>> Symtab.py instead? (Or couple it to Builtins.py somehow?)
>>>
>>
>> I think maintaining such a list would be less elegant than looking
>> them up at runtime, especially as it is fairly stable and
>> monotonically increasing. If one wants to use the code with, say,
>> Python 2.3, then one simply has to avoid using newer symbols in ones
>> code (just as with Python), and it is easy to check (just compile
>> with Python 2.3). This allows people to write code with newer
>> builtins if one wants without having to maintain a list of when every
>> symbol was introduced.
>>
> I might have misunderstood some earlier discussions then. So, in  
> effect,
> one has to run cython.py using an interpreter that has at least the  
> same
> Python version as the builtins you are using.

More precisely, it is possible to get a runtime error rather than a  
compile time error if there is a mismatch. For example, if one uses  
the reversed builtin from 2.4, then it will complain if you compile  
with 2.3, and if you compile with 2.4+ it won't run under 2.3. In  
this sense it is just as compatible as .py files.

> Well, I can certainly live with that.

Me too, especially as the set of builtins is rather stable. (At least  
it was for 2.3-2.6, perhaps someone could comment on how disruptive  
3.0 is).

> But I think it kind of contradicts
> what Stefan has tried to achieve with 3.0 compatability etc. (i.e one
> then has to use Python 2.6 if one wants to compile Cython code for
> Python 3.0 using BufferError...)

Buffers are a special case, because we decided to backport them  
(mainly at the request of the NumPy folks, who support back to 2.3).

> Can't this just be deferred until run-time anyway? I.e. have the  
> list in
> question be the set of all strings :-) I don't think there's a strong
> reason we have to have a compile-time error here? Then  
> sys.version_info
> etc. could be used to create code that runs on multiple targets (and
> compile, using Cython, under multiple targets!) just like in Python.


Yes, we could. This is exactly what Pyrex does--anything that it  
doesn't recognize is treated as a builtin. But catching this kind of  
stuff early is very nice, especially as the edit-compile-run cycle is  
necessarily longer with Cython code than with Python. I find it  
catches all sorts of typos and other errors way more often then it  
gets in the way (and other people on this list have said the same--my  
favorite is someone who complained that Cython wouldn't let him raise  
ValueErr anymore...). It also makes import * feasible. We could have  
an option to turn it off, but I don't know that it'd do much good  
(it'd only be useful if you want to write code for a platform you  
don't have access to, and even then you could put "from __builtin__  
import foo" at the top of the file to manually declare it).

- Robert