[pypy-dev] Application-level helpers

[Armin Rigo]

Hello everybody,

Holger came up with a nice idea about application-level helpers, which he 
experimented on an svn branch:

    http://codespeak.net/svn/pypy/branch/pypy-noappfile/pypy/

The idea is already a few days old and I promized to write about it earlier, 
but I'm a bit busy until tomorrow so I'll keep it short :-)

Holger clarified the role and definition of what is currently found in the
_app.py files. We are trying to do several things at once there, and it is
true that we often only need a convenient way to write code without constantly
wrapping and unwrapping objects.

The plan is to be able to just write plain Python functions in the code, and
call them "at application-level". There should be no need for the functions to 
be in a separate _app file if it is clear that it is application-level. For 
example, dictobject.py currently contains the magic code:

def dict_update__Dict_Dict(space, w_self, w_other):
    w_self.space.gethelper(applicationfile).call("dict_update", [w_self, 
w_other])

Instead it could directly contain:

def app_dict_update(d,o):
    for k in o.keys():
        d[k] = o[k]
dict_update__Dict_Dict = appfunction(app_dict_update)

This means that app_dict_update() is a helper function that you write at
application-level (hence the app_ prefix), whereas dict_update__Dict_Dict is a
wrapper around it that you can call from interpreter-level with an extra
"space" first argument.

The change seems to be minor, but consider that helpers would now be together
with the place they really belong to. They could be used directly as methods
in classes, too. In short they make the bridge between interpreter-level and
application-level code much smaller. The application-level code can also be
allowed to call not only other application-level code, but interpreter-level
one, all transparently (because they essentially live in the same globals).

This clarification also means that it is an independent issue to know what we
should write by strictly sticking to RPython or not. Application-level code is
essentially just a way to avoid wraps. All interpreter-level code should be in
RPython, but then some application-level code could be easy enough for the
analyzer to understand too. If they are, the code generator can make efficient
code from them too (cool!), i.e. it would not matter if the code was defined
at interpreter-level or application-level. For very complex helpers, there is
always the option to fall back to emitting frozen bytecode and interpreting
it.

I believe that this also clarifies some of the issues about the
AnnotationObjSpace.

More about it later...


A bientôt,

Armin.