std a complementary second standard library holger krekel @ europython 2004 motivation for the second 'std' complementary library ----------------------------------------------------- - current "batteries included" are very useful, but - some of them are written in a pretty much java-like style, especially the unittest-framework - they directly expose a great number of implementation details which make later improvements and refactoring hard - "how many names and semantics do you have to remember today?" - the best API is one that doesn't exist - most important: where do we want to go tomorrow? focus of this talk: the issue of testing with python ---------------------------------------------------- - testing python code is a *great* thing and wonderfully complements the dynamicness of python - statically typed languages don't test behaviour etc.pp. - a testing package should require as few boilerplate code as possible and offer much flexibility - it should provide premium quality tracebacks and debugging aid - note that there are innumerable extensions and hacks around unittest.py which are usually incompatible with each other --> there is hardly a standard way to run tests within any given application std.utest: how to write assertions ---------------------------------- - first of all ... forget about limited "assertXYZ APIs" and use the real thing, e.g.: assert x == y - this works with plain python but you get unhelpful "assertion failed" errors with no information - std.utest (magic!) actually reinterprets the assertion expression and offers detailed information about underlying values - testing for exceptions happens with one of the following raises(ValueError, int, 'hello') raises(ValueError, "int('hello')") [example of failing demo.py] std.utest: collecting test units --------------------------------- - std.utest collects tests with "restartable iterators", called "collectors" - collectors go over directories, files and the contents of files like functions and classes - testing immediately starts as the first units pop in - there is a "collecting only" mode to let you check that your collection of tests gets done right - the "Runner" drives the collection and execution of test units --> collectors produce "Units" std.utest: Units of control --------------------------- - Units allow total control of executing test methods - by providing custom collectors you can easily produce your own unit class and instances - unitinstance.execute() gets called in order to execute a test - you can provide custom paramters (a db-connection, some initialized application entity, whatever) and you can initialize and finalize resources if neccessary - lift some of the arbitrary restrictions of unittest.py: allow test functions and allow classes to just be "grouping" for tests std.utest: current architecture picture --------------------------------------- ___________________ | | | Collector | |___________________| / \ | unit.execute(runner) | / collect/get units / | /calls for each | / ___________________/ ________________ | | send events | | | Runner |----------------------->| Reporter | |___________________| |________________| ....................... . "utest.conf" . . cmdline options . ....................... std.utest: some more features ----------------------------- - provides nice "focused" tracebacks ... - "utest.conf" allow to specify default options and python versions - more reporters on the way ... - still lots of little nice things to be added! - developers and integrators welcome! what else does 'std' currently provide? --------------------------------------- - a uniform filesystem and subversion path abstraction (listdir, mtime, visit, ...) - selective imports - a 'magic' subpackage providing some nice things i don't want to talk about much here. --> above all: an extensive test-suite std.path: path abstractions --------------------------- - a uniform API to local file system, svn-url and svn-working copy resources - quick example, uniformly traversing a tree: filefilter = path.checker(file=1, endswith='.py', link=0) def recfilter(p): if p.check(dotfile=1, link=0): if p.basename()!='test': return True for x in pathinstance.visit(filefilter, recfilter): try: l = x.readines() except path.Denied: pass # will only iterate non-linked python source files which # are not in a linked, test or '.*' directory - direct exceptions: NotFound, Denied, NoDirectory std.initpkg: controling and virtualizing import/export ------------------------------------------------------ - usually in python packages expose more or less the file-system hierarchy and all names from their modules, making refactoring hard! - with 'std.initpkg' you can control your package namespaces, your mypkg/__init__.py could contain: from std import initpkg initpkg(__name__, exportdefs = { 'mysub.callable' : './path/to/file.somefunc', 'mysub.Class' : './some/path.Classname', }) - and then 'from mypkg.mysub import Class' just works. --> No namespace pollution, more obvious API-entry-points! std: "where do we want to go tomorrow?" --------------------------------------- - 'we' is currently Armin Rigo and me - complete what we currently have and test more on windows - implement "compiling c-packages on the fly" - integrate Greenlets (lightweight stackless-like extension module) - focus on Pythonic small and clean (re-)implementations - experiment with concepts of "std applications" at some point - get more developers! ------> doing a 1.0 release as soon as possible! std starting points ------------------- today, you have to do svn co http://codespeak.net/svn/std/trunk/src/std and put the resulting 'std' directory in your PYTHONPATH. Alternatively, from a unix shell you can do: eval `python path/to/std/env.py` and you will have a 'utest' binary along with correct PYTHONPATHs. yes, there will be a distutils install pretty soon :-)