3.1.1   The py.py interpreter

To start interpreting Python with PyPy, use Python 2.4 or greater:

cd pypy
python bin/py.py

After a few seconds (remember: this is running on top of CPython), you should be at the PyPy prompt, which is the same as the Python prompt, but with an extra ">".

Now you are ready to start running Python code. Most Python modules should work if they don't involve CPython extension modules. Here is an example of determining PyPy's performance in pystones:

>>>> from test import pystone
>>>> pystone.main(10)

The parameter is the number of loops to run through the test. The default is 50000, which is far too many to run in a non-translated PyPy version (i.e. when PyPy's interpreter itself is being interpreted by CPython).

3.1.2   py.py options

To list the PyPy interpreter command line options, type:

cd pypy
python bin/py.py --help

py.py supports most of the options that CPython supports too (in addition to a large amount of options that can be used to customize py.py). As an example of using PyPy from the command line, you could type:

python py.py -c "from test import pystone; pystone.main(10)"

Alternatively, as with regular Python, you can simply give a script name on the command line:

python py.py ../../lib-python/2.4.1/test/pystone.py 10

See our configuration sections for details about what all the commandline options do.

3.2.1   Interpreter-level console

There are quite a few extra features of the PyPy console: If you press <Ctrl-C> on the console you enter the interpreter-level console, a usual CPython console. You can then access internal objects of PyPy (e.g. the object space) and any variables you have created on the PyPy prompt with the prefix w_:

>>>> a = 123
>>>> <Ctrl-C>
*** Entering interpreter-level console ***
>>> w_a
W_IntObject(123)

Note that the prompt of the interpreter-level console is only '>>>' since it runs on CPython level. If you want to return to PyPy, press <Ctrl-D> (under Linux) or <Ctrl-Z>, <Enter> (under Windows).

You may be interested in reading more about the distinction between interpreter-level and app-level.

3.2.2   Tracing bytecode and operations on objects

You can use the trace object space to monitor the interpretation of bytecodes in connection with object space operations. To enable it, set __pytrace__=1 on the interactive PyPy console:

>>>> __pytrace__ = 1
Tracing enabled
>>>> a = 1 + 2
|- <<<< enter <inline>a = 1 + 2 @ 1 >>>>
|- 0    LOAD_CONST    0 (W_IntObject(1))
|- 3    LOAD_CONST    1 (W_IntObject(2))
|- 6    BINARY_ADD
  |-    add(W_IntObject(1), W_IntObject(2))   -> W_IntObject(3)
|- 7    STORE_NAME    0 (a)
  |-    hash(W_StringObject('a'))   -> W_IntObject(-468864544)
  |-    int_w(W_IntObject(-468864544))   -> -468864544
|-10    LOAD_CONST    2 (<W_NoneObject()>)
|-13    RETURN_VALUE
|- <<<< leave <inline>a = 1 + 2 @ 1 >>>>

3.2.3   Lazily computed objects

One of the original features provided by PyPy is the "thunk" object space, providing lazily-computed objects in a fully transparent manner:

cd pypy
python bin/py.py -o thunk

>>>> from __pypy__ import thunk
>>>> def longcomputation(lst):
....     print "computing..."
....     return sum(lst)
....
>>>> x = thunk(longcomputation, range(5))
>>>> y = thunk(longcomputation, range(10))

From the application perspective, x and y represent exactly the objects being returned by the longcomputation() invocations. You can put these objects into a dictionary without triggering the computation:

>>>> d = {5: x, 10: y}
>>>> result = d[5]
>>>> result
computing...
10
>>>> type(d[10])
computing...
<type 'int'>
>>>> d[10]
45

It is interesting to note that this lazy-computing Python extension is solely implemented in a small objspace/thunk.py file consisting of around 200 lines of code. Since the 0.8.0 release you can translate PyPy with the thunk object space.

3.2.4   More

If you know more about pypy-exclusive features, choose one from the following, go to objspace proxies document.

3.2.5   Running the tests

The PyPy project uses test-driven-development. Right now, there are a couple of different categories of tests which you can run. To run all the unit tests:

cd pypy
python test_all.py

(this is not recommended, since it takes hours and uses huge amounts of RAM). Alternatively, you may run subtests by going to the correct subdirectory and running them individually:

python test_all.py interpreter/test/test_pyframe.py

test_all.py is actually just a synonym for py.test which is our external testing tool. If you have installed that you can as well just issue py.test DIRECTORY_OR_FILE in order to perform test runs or simply start it without arguments to run all tests below the current directory.

Finally, there are the CPython regression tests which you can run like this (this will take hours and hours and hours):

cd lib-python/2.4.1/test
python ../../../pypy/test_all.py

or if you have installed py.test then you simply say:

py.test -E

from the lib-python/2.4.1/test directory. Running one of the above commands tells you how to proceed.

3.2.6   Demos

The demo/ directory contains examples of various aspects of PyPy, ranging from running regular Python programs (that we used as compliance goals) over experimental distribution mechanisms to examples translating sufficiently static programs into low level code.

3.3.1   Trying out the type annotator

We have a type annotator that can completely infer types for functions like is_perfect_number (as well as for much larger examples):

>>> t.annotate([int])
>>> t.view()

Move the mouse over variable names (in red) to see their inferred types.

3.3.2   Translating the flow graph to C code

The graph can be turned into C code:

>>> t.rtype()
>>> f = t.compile_c()

The first command replaces the operations with other low level versions that only use low level types that are available in C (e.g. int). To try out the compiled version:

>>> f(5)
False
>>> f(6)
True

3.3.3   Translating the flow graph to LLVM code

The LLVM or low level virtual machine project has, among other things, defined a statically typed portable assembly language and a set of tools that optimize and compile this assembly for a variety of platforms. As such, this assembly is a natural target for PyPy's translator.

To translate to LLVM assembly you must first have LLVM version 1.9 installed - the how to install LLVM page provides some helpful hints.

The LLVM backend is not as flexible as the C backend, and for example only supports one garbage collection strategy. Calling compiled LLVM code from CPython is more restrictive than the C backend - the return type and the arguments of the entry function must be ints, floats or bools - as the emphasis of the LLVM backend is to compile standalone executables.

Here is a simple example to try:

>>> t = Translation(snippet.my_gcd)
>>> a = t.annotate([int, int])
>>> t.rtype()
>>> f = t.compile_llvm()
>>> f(15, 10)
5

3.3.4   Translating the flow graph to JavaScript code

The JavaScript backend is still experimental but was heavily improved during 2006 Google summer of code. It contains support for the document object model and a good integration with PyPy's unittesting framework. Code can be tested with the Spidermonkey commandline JavaScript interpreter in addition to a multitude of JavaScript capable browsers. The emphasis of the JavaScript backend is to compile RPython code into JavaScript snippets that can be used in a range of browsers. The goal is to make it more and more capable to produce full featured web applications. Please see the pypy/translator/js/test directory for example unittests.

Here is a simple example to try:

>>> t = Translation(snippet.my_gcd)
>>> a = t.annotate([int, int])
>>> source = t.source_js()

If you want to know more about the JavaScript backend please refer to the JavaScript docs.

3.3.5   Translating the flow graph to CLI code

Use the CLI backend to translate the flow graphs into .NET executables: gencli is quite mature now and can also compile the whole interpreter. You can try out the CLI backend from the interactive translator shell:

>>> def myfunc(a, b): return a+b
...
>>> t = Translation(myfunc)
>>> t.annotate([int, int])
>>> f = t.compile_cli()
>>> f(4, 5)
9

The object returned by compile_cli is a wrapper around the real executable: the parameters are passed as command line arguments, and the returned value is read from the standard output.

Once you have compiled the snippet, you can also try to launch the executable directly from the shell; you can find the executable in one of the /tmp/usession-* directories:

$ mono /tmp/usession-<username>/main.exe 4 5
9

To translate and run for CLI you must have the SDK installed: Windows users need the .NET Framework SDK 2.0, while Linux and Mac users can use Mono.

3.3.6   Translating RPystone to JVM code

Using the interactive shell with the JVM is basically the same as any other backend. For example, you might enter:

>>> def myfunc(a, b):
...     return a+b
>>> t = Translation(myfunc)
>>> t.annotate([int, int])
>>> f = t.compile_jvm()
>>> f(4, 5)
9

As with the CLI, the object returned by compile_jvm is a wrapper around the real executable. When invoked, it actually starts a JVM process, passing the parameters as command line arguments and the result as a string over stdout. Therefore, the interface works best with simple types like integers and strings.

If you prefer to run the compiled code directly, you will find it in one of the /tmp/usession-* directories. You can run it like so:

$java -cp /tmp/usession-<username>/pypy/ pypy.Main 4 5 9

Note that the main entrypoint is always pypy.Main.

To successfully use the JVM you will need to have both a JDK installed (at least version 5), and the Jasmin assembler. Furthermore, you need a script on your path called jasmin which runs the Jasmin jar file, something like the following:

$ cat which jasmin #!/bin/bash java -jar $PATH_TO_JASMIN_JAR "$@"

3.3.7   A slightly larger example

There is a small-to-medium demo showing the translator and the annotator:

cd demo
python bpnn.py

This causes bpnn.py to display itself as a call graph and class hierarchy. Clicking on functions shows the flow graph of the particular function. Clicking on a class shows the attributes of its instances. All this information (call graph, local variables' types, attributes of instances) is computed by the annotator.

As soon as you close the PyGame window, the function is turned into C code, compiled and executed.

3.4.1   Translating with the thunk object space

It is also possible to experimentally translate a PyPy version using the "thunk" object space:

cd pypy/translator/goal
python translate.py targetpypystandalone.py --objspace=thunk

the examples in lazily computed objects should work in the translated result.

3.4.2   Translating using the LLVM backend

To create a standalone executable using the experimental LLVM compiler infrastructure:

./translate.py --text --batch --backend=llvm targetpypystandalone.py

3.4.3   Translating using the CLI backend

To create a standalone .NET executable using the CLI backend:

./translate.py --text --batch --backend=cli targetpypystandalone.py

The executable and all its dependecies will be stored in the ./pypy-cli-data directory. To run pypy.NET, you can run ./pypy-cli-data/main.exe. If you are using Linux or Mac, you can use the convenience ./pypy-cli script:

$ ./pypy-cli
debug: entry point starting
debug:  argv ->
debug: importing code
debug: calling code.interact()
Python 2.4.1 (pypy 0.9.0 build 38134) on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
 >>>> 1 + 1
 2
 >>>>

Unfortunately the interactive interpreter will not work with Mono versions older than 1.1.17.2 due to a Mono bug. Everything else will work fine, but not the interactive shell.

Moreover, at the moment it's not possible to do the full translation using only the tools provided by the Microsoft .NET SDK, since ilasm crashes when trying to assemble the pypy-cli code due to its size. Microsoft .NET SDK 2.0.50727.42 is affected by this bug; other version could be affected as well: if you find a version of the SDK that works, please tell us.

Windows users that want to compile their own pypy-cli can install Mono: if a Mono installation is detected the translation toolchain will automatically use its ilasm2 tool to assemble the executables.

>>>> import clr
>>>> ArrayList = clr.load_cli_class('System.Collections', 'ArrayList')
>>>> obj = ArrayList()
>>>> obj.Add(1)
0
>>>> obj.Add(2)
1
>>>> obj.Add("foo")
2
>>>> print obj[0], obj[1], obj[2]
1 2 foo
>>>> print obj.Count
3

3.6.1   graphviz & pygame for flow graph viewing (highly recommended)

graphviz and pygame are both necessary if you want to look at generated flow graphs:

graphviz: http://www.graphviz.org/Download.php

pygame: http://www.pygame.org/download.shtml

3.6.2   CTypes (highly recommended)

ctypes (version 0.9.9.6 or later) is required if you want to run low-level tests. See the download page of ctypes.

3.6.3   py.test and the py lib

The py library is used for supporting PyPy development and running our tests against code and documentation as well as compliance tests. You don't need to install the py library because it ships with PyPy and pypy/test_all.py is an alias for py.test but if you want to have the py.test tool generally in your path, you might like to visit:

http://codespeak.net/py/dist/download.html