[pypy-dev] Sprint wonderings

[Armin Rigo]

Hi,

For a number of us, the sprint was a full week of running translate_pypy.py 
and wait for the next crash, then debugging and thinking and fixing and 
workarounding the issue.  Then start again at step #1.  In the last days it 
generally took some time (and lots of memory) before it ran into a problem and 
crashed.  It also reported increasingly large amounts of warnings... well, 
more precisely, we turned increasingly many errors into warnings :-)  These 
ones report problems that should eventually be fixed.  This mostly occupied 
Marius, Holger, Michael and myself.  (Cheers to Marius, btw, for having caught 
up so quickly with PyPy!)

Some time in the middle of it all, the basic annotation-less 
'translate_pypy.py -no-a' was able to compile and run (-6)*(-7) and 
successfully gave us the ultimate answer.  I guess the next step here is to 
try to change the 'entry_point' so that 'translate_pypy' can actually compile 
and run the interactive command-line interpreter.

Bob worked on more specific pieces of code: the user interface of the Pygame 
graph viewer (Marius did bits of that too), and the genc.py C code generation, 
which now generates code with tons of debugging features.  The best one is 
probably that if the generated C code fails with a Python exception, you can 
use pdb.pm() to debug it!  You see the C frames with the content of their 
variables, with line numbers set correctly.  pdb will display extracts of the 
C source code in the tracebacks :-)

Christian worked on using the flow graph analysis to understand 
application-level code and transform them to interpreter-level code.  For 
example, each + in the source becomes an add operation in the flow graph 
(using the existing almost-unmodified flow object space), which then (this is 
Christian's work) gets translated as a space.add() by genrpy.py.  This is 
useful for application-level helper code, for example string formatting, 
which is currently written in _formatting.py as regular, app-level Python 
code.  The goal is to turn this nice piece of code into its obfuscated 
interp-level equivalent automatically.  This is essentially an optimization 
only, which will allow the code to run a bit faster because PyPy's own 
interpreter doesn't have to interpret the app-level _formatting.py every time 
it has to do a string formatting operation.  It also allows future more 
advanced optimizations.

Jacob and Laura, who didn't feel like digging into the PyPy internals too 
deepdy, focused on reimplementing some of the C functionality in plain Python.  
We now have MD5 and SHA-1 and a 'class file' replacement based on low-level 
primitives (generally the 'os' module).  You can play with the 'file' class by 
importing 'appspace/_file.py' in CPython.  The general guidelines to integrate 
this kind of pure Python module with PyPy is: (1) try the module in py.py when 
it works in CPython, and (2) plug it -- for the 'file' class, it's done by
inserting 'from _file import file' in 'module/__builtin__module.py'.
Warning, this will probably be quite slow :-)

Finally, we got news from the EU.  In all likelyhood the project will really 
start on the 1st of December.  Yippee!  End of the phase 1 of the 
administrative overload!

Now we have to plan for more closely-packed sprints.  The next one might take 
place somewhere around the end of January in Switzerland.

Last but not least, thanks to the Programmers of Vilnius for the excellent 
sprint organization!


See you soon,

Armin.