A selection of hacks
This is a random selection of hacks I did, vaguely sorted in the ones
that do something out of the box ("tools", "games") and the ones that
are only maybe useful to other programmers ("libraries"). Nowadays I
put all my hacks in my user directory of the Codespeak repository at
http://codespeak.net/svn/user/arigo/hack/
You need Subversion to access it. There are nice instructions here
for PyPy that should get you started with Subversion in general.
All this is MIT License unless otherwise specified (for the
case of patches of a previous program with a more restrictive
license like GPL).
- Subversion utilities: merge3.py is a history-preserving merge
tool. svnwcrevert.py cleans up all foreign files and changes in a
working copy (to bring it back to the same state as a fresh checkout).
svncleancopy.py makes a clean copy of a working copy; it uses hard
links to transparently share parts of the administrative area between
the two copies.
- A graph viewer for GraphViz dot, based on Pygame. The goal is
to fluently and readably display and navigate around very large
graphs. It does not support all dot features though (limited set of
node shapes, ...). It supports displaying remotely (the viewer and
the producer of the data are on different machines). (This project
was part of PyPy and got many contributions from many people. See
also the reference tracker.)
- For the Python developer: my start-up script adding automatic
imports in the interactive prompts (so that you don't have to remember
to import a module each time you want to play around with it) and a
sitecustomize.py that prevents Python from importing .pyc
files after the corresponding .py file was deleted.
- Patches for Python 2.4:
- a transparent method cache that makes CPython 10% to 20% faster
- prevent dict lookups from eating all exceptions (a perfect way to
make programs really hard to debug) (included in 2.5)
- shuffle the dictionary order, in order to expose order-dependency
bugs in your programs.
- Pseudo file systems in pure Python, interesting if you want to see
files and directories on your local file systems that are actually
presented and controlled by a Python script. You can e.g. read and
present files from another source like a .zip file or a remote
machine, or create a file-like view from scratch on any kind of data.
- On a modern Linux, a deamon for FUSE is the best way to do it.
(This is a custom deamon talking directly to the kernel via a TCP
pipe, which you can use as a library from your own programs too; the
standard FUSE-Python binding tries to embed a Python interpreter in
the C daemon, which is a mess to compile and is much less flexible.)
- On older Linuxes, you can also try a similar daemon for lufs.
- Even more standard, a pure Python NFS server. The drawback is that
the NFS protocol is a mess for presenting custom data.
- Unpolished and with a number of dependencies: crappyline and
udpcrappyline allow you to set up a reliable TCP-like connection over
a crappy Internet connection. I hope to be able to eventually start
a larger project that allows this in a more transparent way, using
multiple techniques.
- A bash patch to add a history of visited directories. Use the Page Up
and Page Down keys to change the current directory back to one from
the history, even in the middle of typing the next command. Patch
for bash 3.0 or 3.1. You may also
have to fix the new rl_bind_keyseq() calls because Page Up and
Page Down send different escape commands on different terminals; the
ones in the patch work on Rxvt and xterm.
- rxvt-metadata is Rxvt, an X terminal, in which I added two
features: showing inline images, and adding metadata behind some text.
An example of usage is in combination with py.test, all failing
tests appear as a red F on which we can double-click at any time
to see a window with the full traceback.
- An X terminal turned to AJAX over the web... What's that? See our
demo. This is based on the code here.
- Capture the audio played by any program on Linux. This creates a dump
of all the sounds sent to the audio card in raw format. All you need
to achieve this with ALSA is a 14-lines obsure "program" in your
.asoundrc... As it took me quite a while to arrive there (including
a lot of unsuccessful googling) I figure I should share it. Here is
a wrapper script.
- Trivial shell scripts that I use all the time.
- Bub-n-bros deserves its own page.
- Puzzle.py: push pieces around to reassemble the image. The catch
is that pieces stick together as soon as they touch their expected
neighbors. You can quickly end up with larger pieces that you can no
longer move around each other. I check in all possible boards I play,
and I'll make a more newbie-friendly interface when I have 100 :-)
- Langton's Ant: this is the ant walking on a three-colors
checkerboard, as described e.g. by Ian Stewart as the Ant 100,
also known as Ant RLL (right-left-left). This program is as optimized
as I could imagine, running each ant step in about 7 CPU cycles on
a modern Pentium machine. Let it run a few days and you get dozens
of trillions of steps. For what it's worth this ant still behaves
chaotically after about 150 trillions of steps (9 days of computation,
1.5GB of RAM used).
- Greenlets: lightweight microthreads for concurrent programming.
Similar to the tasklets of Stackless Python, but as a C extension
module for the regular Python. See also the experimental libraries
built on top of greenlets: greensock2, limelets.
- pysvn is a library reimplementing a Subversion client. It gives
Python programs direct access to the clean and high-level protocol
that the server understands, without having to operate via the rather
more messy svn client or the very low-level subversion bindings.
- Simple efficient database-ish files. They are optimized for
storing large amounts of data in a statically-typed format. They work
well for example to store in a single file a hashtable mapping SHA
checksums to the block of data with that checksum, supporting
e.g. checksum-based file servers.
- Forward Error-correction Codes allow you to split a file in N
pieces of, say, 100KB each, and generate a larger number M of pieces
from it, still 100KB each. The pieces can be stored or sent on the
Internet or whatever; the property is that any N out of the M pieces
is enough to completely rebuild the original file. See the test
file for a step-by-step guide.
- autoctypes is a layer on top of ctypes that (mostly) allows you to
access C libraries without having to write declarations first,
provided a compiler and the header files are installed.
- completepytest: tab-completion for the shell for py.test -k test_<tab>
- pairtype: very simple multimethods for Python.
- Miscellanous: fish around.