py lib
[impl-test]
modified Aug 21, 2008 by holger krekel,,,

Implementation and Customization of py.test

1   Collecting and running tests / implementation remarks

In order to customize py.test it's good to understand its basic architure (WARNING: these are not guaranteed yet to stay the way they are now!):

 ___________________
|                   |
|    Collector      |
|___________________|
       / \
        |                Item.run()
        |               ^
 receive test Items    /
        |             /execute test Item
        |            /
 ___________________/
|                   |
|     Session       |
|___________________|

                    .............................
                    . conftest.py configuration .
                    . cmdline options           .
                    .............................

The Session basically receives test Items from a Collector, and executes them via the Item.run() method. It monitors the outcome of the test and reports about failures and successes.

1.1   Collectors and the test collection process

The collecting process is iterative, i.e. the session traverses and generates a collector tree. Here is an example of such a tree, generated with the command py.test --collectonly py/xmlobj:

<Directory 'xmlobj'>
    <Directory 'testing'>
        <Module 'test_html.py' (py.__.xmlobj.testing.test_html)>
            <Function 'test_html_name_stickyness'>
            <Function 'test_stylenames'>
            <Function 'test_class_None'>
            <Function 'test_alternating_style'>
        <Module 'test_xml.py' (py.__.xmlobj.testing.test_xml)>
            <Function 'test_tag_with_text'>
            <Function 'test_class_identity'>
            <Function 'test_tag_with_text_and_attributes'>
            <Function 'test_tag_with_subclassed_attr_simple'>
            <Function 'test_tag_nested'>
            <Function 'test_tag_xmlname'>

By default all directories not starting with a dot are traversed, looking for test_*.py and *_test.py files. Those files are imported under their package name.

The Module collector looks for test functions and test classes and methods. Test functions and methods are prefixed test by default. Test classes must start with a capitalized Test prefix.

1.2   test items are collectors as well

To make the reporting life simple for the session object items offer a run() method as well. In fact the session distinguishes "collectors" from "items" solely by interpreting their return value. If it is a list, then we recurse into it, otherwise we consider the "test" as passed.

1.3   constructing the package name for test modules

Test modules are imported under their fully qualified name. Given a filesystem fspath it is constructed as follows:

  • walk the directories up to the last one that contains an __init__.py file.
  • perform sys.path.insert(0, basedir).
  • import the root package as root
  • determine the fully qualified name for fspath by either:
    • calling root.__pkg__.getimportname(fspath) if the __pkg__ exists.` or
    • otherwise use the relative path of the module path to the base dir and turn slashes into dots and strike the trailing .py.

2   Customizing the testing process

2.1   writing conftest.py files

You may put conftest.py files containing project-specific configuration in your project's root directory, it's usually best to put it just into the same directory level as your topmost __init__.py. In fact, py.test performs an "upwards" search starting from the directory that you specify to be tested and will lookup configuration values right-to-left. You may have options that reside e.g. in your home directory but note that project specific settings will be considered first. There is a flag that helps you debugging your conftest.py configurations:

py.test --traceconfig

2.1.1   adding custom options

To register a project-specific command line option you may have the following code within a conftest.py file:

import py
Option = py.test.config.Option
option = py.test.config.addoptions("pypy options",
    Option('-V', '--view', action="store_true", dest="view", default=False,
           help="view translation tests' flow graphs with Pygame"),
)

and you can then access option.view like this:

if option.view:
    print "view this!"

The option will be available if you type py.test -h Note that you may only register upper case short options. py.test reserves all lower case short options for its own cross-project usage.

2.2   customizing the collecting and running process

To introduce different test items you can create one or more conftest.py files in your project. When the collection process traverses directories and modules the default collectors will produce custom Collectors and Items if they are found in a local conftest.py file.

2.2.1   example: perform additional ReST checks

With your custom collectors or items you can completely derive from the standard way of collecting and running tests in a localized manner. Let's look at an example. If you invoke py.test --collectonly py/documentation then you get:

<DocDirectory 'documentation'>
    <DocDirectory 'example'>
        <DocDirectory 'pytest'>
            <Module 'test_setup_flow_example.py' (test_setup_flow_example)>
                <Class 'TestStateFullThing'>
                    <Instance '()'>
                        <Function 'test_42'>
                        <Function 'test_23'>
    <ReSTChecker 'TODO.txt'>
        <ReSTSyntaxTest 'TODO.txt'>
        <LinkCheckerMaker 'checklinks'>
    <ReSTChecker 'api.txt'>
        <ReSTSyntaxTest 'api.txt'>
        <LinkCheckerMaker 'checklinks'>
            <CheckLink 'getting-started.html'>
    ...

In py/documentation/conftest.py you find the following customization:

class DocDirectory(py.test.collect.Directory):

    def run(self):
        results = super(DocDirectory, self).run()
        for x in self.fspath.listdir('*.txt', sort=True):
                results.append(x.basename)
        return results

    def join(self, name):
        if not name.endswith('.txt'):
            return super(DocDirectory, self).join(name)
        p = self.fspath.join(name)
        if p.check(file=1):
            return ReSTChecker(p, parent=self)

Directory = DocDirectory

The existence of the 'Directory' name in the pypy/documentation/conftest.py module makes the collection process defer to our custom "DocDirectory" collector. We extend the set of collected test items by ReSTChecker instances which themselves create ReSTSyntaxTest and LinkCheckerMaker items. All of this instances (need to) follow the collector API.

2.3   Customizing the collection process in a module

REPEATED WARNING: details of the collection and running process are still subject to refactorings and thus details will change. If you are customizing py.test at "Item" level then you definitely want to be subscribed to the py-dev mailing list to follow ongoing development.

If you have a module where you want to take responsibility for collecting your own test Items and possibly even for executing a test then you can provide generative tests that yield callables and possibly arguments as a tuple. This is especially useful for calling application test machinery with different parameter sets but counting each of the calls as a separate tests.

The other extension possibility is about specifying a custom test Item class which is responsible for setting up and executing an underlying test. Or you can extend the collection process for a whole directory tree by putting Items in a conftest.py configuration file. The collection process dynamically consults the chain of conftest.py modules to determine collectors and items at Directory, Module, Class, Function or Generator level respectively.

2.4   Customizing execution of Functions

  • py.test.collect.Function test items control execution of a test function. function.run() will get called by the session in order to actually run a test. The method is responsible for performing proper setup/teardown ("Test Fixtures") for a Function test.
  • Function.execute(target, *args) methods are invoked by the default Function.run() to actually execute a python function with the given (usually empty set of) arguments.