How Python solves my daily problems
===================================

`Bruce Eckel loves Python`_.  Not very surprisingly I agree with most
of his points in this presentation.  Recently I've come to truly
appreciate one of them: Python makes automation easy.  Bruce writes
that of course "it's possible to write programs to automate every
task, but you don't.  Python makes it easy enough."

.. _Bruce Eckel loves Python: http://www.pythoncriticalmass.com/downloads/pub/eckel/LovePython.zip

Python indeed lowers the bar enough to make it worth thinking about
automating so many things that you would never dream of writing a C or
Java program for.  With Python as my favourite hammer, so many
problems have started looking like nails.  If you've grown up with
Unix, your favourite hammers might also include make, sed, awk, and
bash.  But these are all *terrible* scripting languages.  If I want to
script things, I rather use Python.  I know it better than, say, bash,
and I can do more with it.

Let me give two examples, one from my personal and one from my
professional life.

Garfield
--------

I'm a big Garfield fan.  I don't always have time to read the newest
strips during a normal work day, so I like reading them on weekends,
e.g. when traveling on the train.  Having figured out the date pattern
in the strips' URLs, a short Python script can generate a list of URLs
that can be fed to wget::

    from datetime import date, timedelta

    base = 'http://images.ucomics.com/comics/ga/'
    d = timedelta(1)
    start = date(2006, 7, 23)
    end = date(2006, 8, 22)

    day = start
    while day <= end:
        print base + str(day.year) + '/ga' + day.strftime("%y%m%d") + '.gif'
        day += d

Thanks to the date arithmetic from the ``datetime`` module, this works
across months, new years, leap years, etc.  How long would you have
needed to do this with bash?  You probably wouldn't have bothered and
just saved the strips manually...

Now, I actually like reading Garfield strips more than once, so I'd
like to store them for a little while.  Having a huge amount of strips
in one directory can be confusing.  Sort them manually? Nah::

    import os, glob

    garfield_files = glob.glob(os.path.join(os.getcwd(), "ga*.gif"))
    for fullpath in garfield_files:
        path, filename = os.path.split(fullpath)
        year = filename[2:4]
        if int(year) < 78:
            year = '20' + year
        else:
            year = '19' + year
        fullyear = os.path.join(path, year)
        if not os.path.exists(fullyear):
            os.mkdir(fullyear)
        os.rename(fullpath, os.path.join(fullyear, filename))

If I would've had to do this one in bash, it might've been easier than
the previous task because there's no complicated date arithmetic
involved.  Still, why bother with bash?

ezmerge
-------

The three Zope projects that I do most of the work with involve a lot
of branches.  `Zope 3`_ typically has two active branches, the trunk
and the current release branch (e.g. 3.3).  `Zope 2`_ has three to
four active branches, the trunk, the current release branch
(e.g. 2.10) and one or two maintenance branches (2.9 and currently
also 2.8).  Five_ is part of Zope 2 but typically has intermediate
releases as well (1.0 is part of Zope 2.8, 1.3 is part of Zope 2.9,
1.5 is part of Zope 2.10).

.. _Zope 3: http://svn.zope.org/Zope3
.. _Zope 2: http://svn.zope.org/Zope
.. _Five: http://svn.zope.org/Products.Five

When I fix a bug in Five, for example, I fix it in the oldest
maintenance branch, 1.2.  Then the fix needs to be propagated all the
way up to 1.3, 1.4, 1.5 and the trunk.  You can probably tell that I'm
merging *a lot*.

We're using svn for all Zope projects and I'm happy with it.  But
merging with svn is a pain:

1. You have to know the repository URL.  With the Zope repository,
   that's ``svn+ssh://philikon@svn.zope.org/repos/main/``.  Of course,
   you could save that in an environment variable and always refer to
   it in your commands.  But what if I'm in a different SVN
   repository, e.g. codespeak_ (where Five used to be hosted)?

2. You have to know the relative path inside the repository.  Say I
   have a working copy of ``zope.testing`` which could either be a
   working copy of the project, e.g. ``zope.testing/trunk``, or just
   the package, ``zope.testing/trunk/src/zope/testing``.  I first have
   to check ``svn info`` so that I can specify the right relative path
   from which I want to merge.

3. You have to specify a revision range.  90% of time, I just want to
   merge a single revision.  But then I still have to say ``-r
   (n-1):n``.

.. _codespeak: http://codespeak.net/svn

So, after all I end up with a merge command like this::

  svn merge -r (n-1):n svn+ssh://philikon@svn.zope.org/repos/main/Zope3/branches/3.3 .

When I'm already in a working copy of
``svn+ssh://philikon@zope.org/repos/main/Zope3`` (e.g. the trunk), the
only useful information contained in that command is

* the revision number ``n``

* the branch I'm merging from, e.g. ``3.3``

The rest is boilerplate.  So, I wrote a tool in Python that constructs
an ``svn merge`` command for me based on that information::

  ~$ cd Zope3-trunk
  ~/Zope3-trunk$ ~/ezmerge.py 42353 3.3
  svn merge -r 42352:42353 svn+ssh://philikon@svn.zope.org/repos/main/Zope3/branches/3.3 .
  U  src/zope/component/README.txt
  U  src/zope/component/registry.py

As you can see, ezmerge_ takes a revision number and a branch as
argument and constructs the command.  How does ezmerge_ know the
repository path and the relative path within the repository? It does
an ``svn info`` in the current directory (which is to be assumed a
working copy, otherwise ``svn merge`` wouldn't make sense) and grabs
it from there.  ``trunk`` becomes ``branches/<branch>`` and viceversa.
That way, I can actually be deep down in the directory structure and
ezmerge_ will figure it all out for me::

  ~$ cd Zope3-trunk/src/zope
  ~/Zope3-trunk/src/zope$ ~/ezmerge.py 42353 3.3
  svn merge -r 42352:42353 svn+ssh://philikon@svn.zope.org/repos/main/Zope3/branches/3.3/src/zope .
  U  component/README.txt
  U  component/registry.py

ezmerge_ is currently 140 LOC (including doctests!).  The amount of
time it took me to write (and test) it: perhaps 10 minutes.  The
amount of pain it has prevented: priceless.

It gets even better.  I recently found that it couldn't do reverse
merges (where you want to back out a revision).  It took me only two
minutes to extend ezmerge_, even though I hadn't touched it in months.

.. _ezmerge: http://codespeak.net/svn/user/philikon/ezmerge.py