From memedough at gmail.com Mon Jul 5 12:17:52 2010 From: memedough at gmail.com (meme dough) Date: Mon, 5 Jul 2010 20:17:52 +1000 Subject: [py-dev] pytest-cov multiple reports poll Message-ID: Hi, When I first wrote pytest-cov I thought that when dist=each it should output a coverage report for each slave. However it seems that people want it to combine so they can see the overall coverage. I am thinking of removing it to make it simpler and just always combine and produce single report for both dist=load and dist=each. Would anyone not want this to happen? :) From holger at merlinux.eu Mon Jul 5 12:38:17 2010 From: holger at merlinux.eu (holger krekel) Date: Mon, 5 Jul 2010 12:38:17 +0200 Subject: [py-dev] pytest-cov multiple reports poll In-Reply-To: References: Message-ID: <20100705103817.GK14601@trillke.net> Hi Meme, On Mon, Jul 05, 2010 at 20:17 +1000, meme dough wrote: > Hi, > > When I first wrote pytest-cov I thought that when dist=each it should > output a coverage report for each slave. > > However it seems that people want it to combine so they can see the > overall coverage. > > I am thinking of removing it to make it simpler and just always > combine and produce single report for both dist=load and dist=each. > > Would anyone not want this to happen? > > :) Not sure about the potential double negation - i think it makes sense to always combine reports. It's still possible to get target-specific coverage for a --dist-each situation by just specifying a single-target run. And for --dist=load it hardly makes sense to have separate random coverage. best, holger From prologic at shortcircuit.net.au Thu Jul 8 14:36:04 2010 From: prologic at shortcircuit.net.au (James Mills) Date: Thu, 8 Jul 2010 22:36:04 +1000 Subject: [py-dev] pytest-cov Message-ID: Hi, ~/circuits $ make clean tests ============================= test session starts ============================== platform linux2 -- Python 2.6.5 -- pytest-1.3.1 -- /home/prologic/bin/python test object 1: tests/ [gw0] popen//python=/usr/bin/python2.5 -- platform linux2, Python 2.5.5-final-0 cwd: /home/prologic/work/circuits [gw1] popen//python=/usr/bin/python2.6 -- platform linux2, Python 2.6.5-final-0 cwd: /home/prologic/work/circuits [master] starting full item collection ... /home/prologic/work/circuits/circuits/net/sockets.py:22: UserWarning: No SSL support available. Using python-2.5 ? Try isntalling the ssl module: http://pypi.python.org/pypi/ssl/ warnings.warn("No SSL support available. Using python-2.5 ? Try isntalling the ssl module: http://pypi.python.org/pypi/ssl/") [master] collected 154 items [gw0] txnode ready to receive tests [gw1] txnode ready to receive tests tests/app/test_daemon.py FException in thread App (most likely raised during interpreter shutdown): Traceback (most recent call last): File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner File "/usr/lib/python2.6/threading.py", line 484, in run File "/home/prologic/work/circuits/circuits/core/manager.py", line 539, in run File "/home/prologic/work/circuits/circuits/core/manager.py", line 124, in __len__ : 'NoneType' object is not callable Exception in thread App (most likely raised during interpreter shutdown): Traceback (most recent call last): File "/usr/lib/python2.5/threading.py", line 486, in __bootstrap_inner File "/usr/lib/python2.5/threading.py", line 446, in run File "/home/prologic/work/circuits/circuits/core/manager.py", line 539, in run File "/home/prologic/work/circuits/circuits/core/manager.py", line 124, in __len__ : 'NoneType' object is not callable =================================== FAILURES =================================== _____________________________________ test _____________________________________ [gw1] popen//python=/usr/bin/python2.6 -- platform linux2, Python 2.6.5-final-0 tmpdir = local('/tmp/pytest-76/popen/slave-1/test0') cov = def test(tmpdir, cov): tmpdir.ensure("app.pid") pidpath = tmpdir.join("app.pid") pidfile = str(pidpath) args = ["python", app.__file__, pidfile] cmd = " ".join(args) p = Popen(cmd, shell=True) status = p.wait() > assert status == 0 E assert 1 == 0 tests/app/test_daemon.py:19: AssertionError ------------------------------- Captured stderr -------------------------------- RuntimeError: Bad magic number in .pyc file =========================== short test summary info ============================ FAIL tests/app/test_daemon.py::test !!!!!!!!!!!!!!!!!!!! Interrupted: stopping after 1 failures !!!!!!!!!!!!!!!!!!!! ========================== 1 failed in 20.36 seconds =========================== pastebin session-log: http://paste.pocoo.org/show/234968 make: *** [tests] Error 2 The above ^^^ is a sample test run of my project circuits (1) Because I don't have anything more to add I'll just paste/show it here and ask if anyone has any thoughts... >From what I can tell the xdist plugin might have to re-compile the .pyc files in question (or something). I'm not sure :) cheers James -- -- James Mills -- -- "Problems are solved by method" From holger at merlinux.eu Thu Jul 8 14:40:23 2010 From: holger at merlinux.eu (holger krekel) Date: Thu, 8 Jul 2010 14:40:23 +0200 Subject: [py-dev] pytest-cov In-Reply-To: References: Message-ID: <20100708124022.GP14601@trillke.net> Hi James, does it help if you set the environment variable PYTHONDOWNWRITEBYTECODE=1 before invoking your tests? (i think python2.5 does not support it but a python2.6/python2.5 run should not intermingle wrt to pyc files anymore) holger On Thu, Jul 08, 2010 at 22:36 +1000, James Mills wrote: > Hi, > > ~/circuits > $ make clean tests > ============================= test session starts ============================== > platform linux2 -- Python 2.6.5 -- pytest-1.3.1 -- /home/prologic/bin/python > test object 1: tests/ > [gw0] popen//python=/usr/bin/python2.5 -- platform linux2, Python > 2.5.5-final-0 cwd: /home/prologic/work/circuits > [gw1] popen//python=/usr/bin/python2.6 -- platform linux2, Python > 2.6.5-final-0 cwd: /home/prologic/work/circuits > [master] starting full item collection ... > /home/prologic/work/circuits/circuits/net/sockets.py:22: UserWarning: > No SSL support available. Using python-2.5 ? Try isntalling the ssl > module: http://pypi.python.org/pypi/ssl/ > warnings.warn("No SSL support available. Using python-2.5 ? Try > isntalling the ssl module: http://pypi.python.org/pypi/ssl/") > [master] collected 154 items > [gw0] txnode ready to receive tests > [gw1] txnode ready to receive tests > > tests/app/test_daemon.py FException in thread App (most likely raised > during interpreter shutdown): > Traceback (most recent call last): > File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner > File "/usr/lib/python2.6/threading.py", line 484, in run > File "/home/prologic/work/circuits/circuits/core/manager.py", line 539, in run > File "/home/prologic/work/circuits/circuits/core/manager.py", line > 124, in __len__ > : 'NoneType' object is not callable > Exception in thread App (most likely raised during interpreter shutdown): > Traceback (most recent call last): > File "/usr/lib/python2.5/threading.py", line 486, in __bootstrap_inner > File "/usr/lib/python2.5/threading.py", line 446, in run > File "/home/prologic/work/circuits/circuits/core/manager.py", line 539, in run > File "/home/prologic/work/circuits/circuits/core/manager.py", line > 124, in __len__ > : 'NoneType' object is not callable > > > =================================== FAILURES =================================== > _____________________________________ test _____________________________________ > [gw1] popen//python=/usr/bin/python2.6 -- platform linux2, Python 2.6.5-final-0 > > tmpdir = local('/tmp/pytest-76/popen/slave-1/test0') > cov = > > def test(tmpdir, cov): > tmpdir.ensure("app.pid") > pidpath = tmpdir.join("app.pid") > pidfile = str(pidpath) > > args = ["python", app.__file__, pidfile] > cmd = " ".join(args) > p = Popen(cmd, shell=True) > status = p.wait() > > > assert status == 0 > E assert 1 == 0 > > tests/app/test_daemon.py:19: AssertionError > ------------------------------- Captured stderr -------------------------------- > RuntimeError: Bad magic number in .pyc file > =========================== short test summary info ============================ > FAIL tests/app/test_daemon.py::test > !!!!!!!!!!!!!!!!!!!! Interrupted: stopping after 1 failures !!!!!!!!!!!!!!!!!!!! > ========================== 1 failed in 20.36 seconds =========================== > pastebin session-log: http://paste.pocoo.org/show/234968 > make: *** [tests] Error 2 > > > The above ^^^ is a sample test run of my project circuits (1) > > Because I don't have anything more to add I'll just paste/show > it here and ask if anyone has any thoughts... > > >From what I can tell the xdist plugin might have to re-compile > the .pyc files in question (or something). I'm not sure :) > > cheers > James > > > -- > -- James Mills > -- > -- "Problems are solved by method" > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > -- From prologic at shortcircuit.net.au Thu Jul 8 14:44:50 2010 From: prologic at shortcircuit.net.au (James Mills) Date: Thu, 8 Jul 2010 22:44:50 +1000 Subject: [py-dev] pytest-cov In-Reply-To: <20100708124022.GP14601@trillke.net> References: <20100708124022.GP14601@trillke.net> Message-ID: On Thu, Jul 8, 2010 at 10:40 PM, holger krekel wrote: > does it help if you set the environment variable > PYTHONDOWNWRITEBYTECODE=1 No, this didn't seem to affect anything. cheers James -- -- James Mills -- -- "Problems are solved by method" From holger at merlinux.eu Thu Jul 8 17:31:35 2010 From: holger at merlinux.eu (holger krekel) Date: Thu, 8 Jul 2010 17:31:35 +0200 Subject: [py-dev] py.test 1.3.2: API and reporting refinements & fixes Message-ID: <20100708153135.GQ14601@trillke.net> The pylib/py.test 1.3.2 release brings many bug fixes and some new features. It was refined for and tested against the recently released Python2.7 and remains compatibile to the usual armada of interpreters (Python2.4 through to Python3.1.2, Jython and PyPy). Note that for using distributed testing features you'll need to upgrade to the jointly released pytest-xdist-1.4 because of some internal refactorings. See http://pytest.org for general documentation and below for a inlined detailed CHANGELOG. cheers & particular thanks to Benjamin Peterson, Ronny Pfannschmidt and all issue and patch contributors, holger krekel Changes between 1.3.1 and 1.3.2 ================================================== New features ++++++++++++++++++ - fix issue103: introduce py.test.raises as context manager, examples:: with py.test.raises(ZeroDivisionError): x = 0 1 / x with py.test.raises(RuntimeError) as excinfo: call_something() # you may do extra checks on excinfo.value|type|traceback here (thanks Ronny Pfannschmidt) - Funcarg factories can now dynamically apply a marker to a test invocation. This is for example useful if a factory provides parameters to a test which are expected-to-fail:: def pytest_funcarg__arg(request): request.applymarker(py.test.mark.xfail(reason="flaky config")) ... def test_function(arg): ... - improved error reporting on collection and import errors. This makes use of a more general mechanism, namely that for custom test item/collect nodes ``node.repr_failure(excinfo)`` is now uniformly called so that you can override it to return a string error representation of your choice which is going to be reported as a (red) string. - introduce '--junitprefix=STR' option to prepend a prefix to all reports in the junitxml file. Bug fixes / Maintenance ++++++++++++++++++++++++++ - make tests and the ``pytest_recwarn`` plugin in particular fully compatible to Python2.7 (if you use the ``recwarn`` funcarg warnings will be enabled so that you can properly check for their existence in a cross-python manner). - refine --pdb: ignore xfailed tests, unify its TB-reporting and don't display failures again at the end. - fix assertion interpretation with the ** operator (thanks Benjamin Peterson) - fix issue105 assignment on the same line as a failing assertion (thanks Benjamin Peterson) - fix issue104 proper escaping for test names in junitxml plugin (thanks anonymous) - fix issue57 -f|--looponfail to work with xpassing tests (thanks Ronny) - fix issue92 collectonly reporter and --pastebin (thanks Benjamin Peterson) - fix py.code.compile(source) to generate unique filenames - fix assertion re-interp problems on PyPy, by defering code compilation to the (overridable) Frame.eval class. (thanks Amaury Forgeot) - fix py.path.local.pyimport() to work with directories - streamline py.path.local.mkdtemp implementation and usage - don't print empty lines when showing junitxml-filename - add optional boolean ignore_errors parameter to py.path.local.remove - fix terminal writing on win32/python2.4 - py.process.cmdexec() now tries harder to return properly encoded unicode objects on all python versions - install plain py.test/py.which scripts also for Jython, this helps to get canonical script paths in virtualenv situations - make path.bestrelpath(path) return ".", note that when calling X.bestrelpath the assumption is that X is a directory. - make initial conftest discovery ignore "--" prefixed arguments - fix resultlog plugin when used in an multicpu/multihost xdist situation (thanks Jakub Gustak) - perform distributed testing related reporting in the xdist-plugin rather than having dist-related code in the generic py.test distribution - fix homedir detection on Windows - ship distribute_setup.py version 0.6.13 Changes between 1.3.0 and 1.3.1 ================================================== New features ++++++++++++++++++ - issue91: introduce new py.test.xfail(reason) helper to imperatively mark a test as expected to fail. Can be used from within setup and test functions. This is useful especially for parametrized tests when certain configurations are expected-to-fail. In this case the declarative approach with the @py.test.mark.xfail cannot be used as it would mark all configurations as xfail. - issue102: introduce new --maxfail=NUM option to stop test runs after NUM failures. This is a generalization of the '-x' or '--exitfirst' option which is now equivalent to '--maxfail=1'. Both '-x' and '--maxfail' will now also print a line near the end indicating the Interruption. - issue89: allow py.test.mark decorators to be used on classes (class decorators were introduced with python2.6) and also allow to have multiple markers applied at class/module level by specifying a list. - improve and refine letter reporting in the progress bar: . pass f failed test s skipped tests (reminder: use for dependency/platform mismatch only) x xfailed test (test that was expected to fail) X xpassed test (test that was expected to fail but passed) You can use any combination of 'fsxX' with the '-r' extended reporting option. The xfail/xpass results will show up as skipped tests in the junitxml output - which also fixes issue99. - make py.test.cmdline.main() return the exitstatus instead of raising SystemExit and also allow it to be called multiple times. This of course requires that your application and tests are properly teared down and don't have global state. Fixes / Maintenance ++++++++++++++++++++++ - improved traceback presentation: - improved and unified reporting for "--tb=short" option - Errors during test module imports are much shorter, (using --tb=short style) - raises shows shorter more relevant tracebacks - --fulltrace now more systematically makes traces longer / inhibits cutting - improve support for raises and other dynamically compiled code by manipulating python's linecache.cache instead of the previous rather hacky way of creating custom code objects. This makes it seemlessly work on Jython and PyPy where it previously didn't. - fix issue96: make capturing more resilient against Control-C interruptions (involved somewhat substantial refactoring to the underlying capturing functionality to avoid race conditions). - fix chaining of conditional skipif/xfail decorators - so it works now as expected to use multiple @py.test.mark.skipif(condition) decorators, including specific reporting which of the conditions lead to skipping. - fix issue95: late-import zlib so that it's not required for general py.test startup. - fix issue94: make reporting more robust against bogus source code (and internally be more careful when presenting unexpected byte sequences) Changes between 1.2.1 and 1.3.0 ================================================== - deprecate --report option in favour of a new shorter and easier to remember -r option: it takes a string argument consisting of any combination of 'xfsX' characters. They relate to the single chars you see during the dotted progress printing and will print an extra line per test at the end of the test run. This extra line indicates the exact position or test ID that you directly paste to the py.test cmdline in order to re-run a particular test. - allow external plugins to register new hooks via the new pytest_addhooks(pluginmanager) hook. The new release of the pytest-xdist plugin for distributed and looponfailing testing requires this feature. - add a new pytest_ignore_collect(path, config) hook to allow projects and plugins to define exclusion behaviour for their directory structure - for example you may define in a conftest.py this method:: def pytest_ignore_collect(path): return path.check(link=1) to prevent even a collection try of any tests in symlinked dirs. - new pytest_pycollect_makemodule(path, parent) hook for allowing customization of the Module collection object for a matching test module. - extend and refine xfail mechanism: ``@py.test.mark.xfail(run=False)`` do not run the decorated test ``@py.test.mark.xfail(reason="...")`` prints the reason string in xfail summaries specifiying ``--runxfail`` on command line virtually ignores xfail markers - expose (previously internal) commonly useful methods: py.io.get_terminal_with() -> return terminal width py.io.ansi_print(...) -> print colored/bold text on linux/win32 py.io.saferepr(obj) -> return limited representation string - expose test outcome related exceptions as py.test.skip.Exception, py.test.raises.Exception etc., useful mostly for plugins doing special outcome interpretation/tweaking - (issue85) fix junitxml plugin to handle tests with non-ascii output - fix/refine python3 compatibility (thanks Benjamin Peterson) - fixes for making the jython/win32 combination work, note however: jython2.5.1/win32 does not provide a command line launcher, see http://bugs.jython.org/issue1491 . See pylib install documentation for how to work around. - fixes for handling of unicode exception values and unprintable objects - (issue87) fix unboundlocal error in assertionold code - (issue86) improve documentation for looponfailing - refine IO capturing: stdin-redirect pseudo-file now has a NOP close() method - ship distribute_setup.py version 0.6.10 - added links to the new capturelog and coverage plugins Changes between 1.2.1 and 1.2.0 ===================================== - refined usage and options for "py.cleanup":: py.cleanup # remove "*.pyc" and "*$py.class" (jython) files py.cleanup -e .swp -e .cache # also remove files with these extensions py.cleanup -s # remove "build" and "dist" directory next to setup.py files py.cleanup -d # also remove empty directories py.cleanup -a # synonym for "-s -d -e 'pip-log.txt'" py.cleanup -n # dry run, only show what would be removed - add a new option "py.test --funcargs" which shows available funcargs and their help strings (docstrings on their respective factory function) for a given test path - display a short and concise traceback if a funcarg lookup fails - early-load "conftest.py" files in non-dot first-level sub directories. allows to conveniently keep and access test-related options in a ``test`` subdir and still add command line options. - fix issue67: new super-short traceback-printing option: "--tb=line" will print a single line for each failing (python) test indicating its filename, lineno and the failure value - fix issue78: always call python-level teardown functions even if the according setup failed. This includes refinements for calling setup_module/class functions which will now only be called once instead of the previous behaviour where they'd be called multiple times if they raise an exception (including a Skipped exception). Any exception will be re-corded and associated with all tests in the according module/class scope. - fix issue63: assume <40 columns to be a bogus terminal width, default to 80 - fix pdb debugging to be in the correct frame on raises-related errors - update apipkg.py to fix an issue where recursive imports might unnecessarily break importing - fix plugin links Changes between 1.2 and 1.1.1 ===================================== - moved dist/looponfailing from py.test core into a new separately released pytest-xdist plugin. - new junitxml plugin: --junitxml=path will generate a junit style xml file which is processable e.g. by the Hudson CI system. - new option: --genscript=path will generate a standalone py.test script which will not need any libraries installed. thanks to Ralf Schmitt. - new option: --ignore will prevent specified path from collection. Can be specified multiple times. - new option: --confcutdir=dir will make py.test only consider conftest files that are relative to the specified dir. - new funcarg: "pytestconfig" is the pytest config object for access to command line args and can now be easily used in a test. - install 'py.test' and `py.which` with a ``-$VERSION`` suffix to disambiguate between Python3, python2.X, Jython and PyPy installed versions. - new "pytestconfig" funcarg allows access to test config object - new "pytest_report_header" hook can return additional lines to be displayed at the header of a test run. - (experimental) allow "py.test path::name1::name2::..." for pointing to a test within a test collection directly. This might eventually evolve as a full substitute to "-k" specifications. - streamlined plugin loading: order is now as documented in customize.html: setuptools, ENV, commandline, conftest. also setuptools entry point names are turned to canonical namees ("pytest_*") - automatically skip tests that need 'capfd' but have no os.dup - allow pytest_generate_tests to be defined in classes as well - deprecate usage of 'disabled' attribute in favour of pytestmark - deprecate definition of Directory, Module, Class and Function nodes in conftest.py files. Use pytest collect hooks instead. - collection/item node specific runtest/collect hooks are only called exactly on matching conftest.py files, i.e. ones which are exactly below the filesystem path of an item - change: the first pytest_collect_directory hook to return something will now prevent further hooks to be called. - change: figleaf plugin now requires --figleaf to run. Also change its long command line options to be a bit shorter (see py.test -h). - change: pytest doctest plugin is now enabled by default and has a new option --doctest-glob to set a pattern for file matches. - change: remove internal py._* helper vars, only keep py._pydir - robustify capturing to survive if custom pytest_runtest_setup code failed and prevented the capturing setup code from running. - make py.test.* helpers provided by default plugins visible early - works transparently both for pydoc and for interactive sessions which will regularly see e.g. py.test.mark and py.test.importorskip. - simplify internal plugin manager machinery - simplify internal collection tree by introducing a RootCollector node - fix assert reinterpreation that sees a call containing "keyword=..." - fix issue66: invoke pytest_sessionstart and pytest_sessionfinish hooks on slaves during dist-testing, report module/session teardown hooks correctly. - fix issue65: properly handle dist-testing if no execnet/py lib installed remotely. - skip some install-tests if no execnet is available - fix docs, fix internal bin/ script generation Changes between 1.1.1 and 1.1.0 ===================================== - introduce automatic plugin registration via 'pytest11' entrypoints via setuptools' pkg_resources.iter_entry_points - fix py.test dist-testing to work with execnet >= 1.0.0b4 - re-introduce py.test.cmdline.main() for better backward compatibility - svn paths: fix a bug with path.check(versioned=True) for svn paths, allow '%' in svn paths, make svnwc.update() default to interactive mode like in 1.0.x and add svnwc.update(interactive=False) to inhibit interaction. - refine distributed tarball to contain test and no pyc files - try harder to have deprecation warnings for py.compat.* accesses report a correct location Changes between 1.1.0 and 1.0.2 ===================================== * adjust and improve docs * remove py.rest tool and internal namespace - it was never really advertised and can still be used with the old release if needed. If there is interest it could be revived into its own tool i guess. * fix issue48 and issue59: raise an Error if the module from an imported test file does not seem to come from the filepath - avoids "same-name" confusion that has been reported repeatedly * merged Ronny's nose-compatibility hacks: now nose-style setup_module() and setup() functions are supported * introduce generalized py.test.mark function marking * reshuffle / refine command line grouping * deprecate parser.addgroup in favour of getgroup which creates option group * add --report command line option that allows to control showing of skipped/xfailed sections * generalized skipping: a new way to mark python functions with skipif or xfail at function, class and modules level based on platform or sys-module attributes. * extend py.test.mark decorator to allow for positional args * introduce and test "py.cleanup -d" to remove empty directories * fix issue #59 - robustify unittest test collection * make bpython/help interaction work by adding an __all__ attribute to ApiModule, cleanup initpkg * use MIT license for pylib, add some contributors * remove py.execnet code and substitute all usages with 'execnet' proper * fix issue50 - cached_setup now caches more to expectations for test functions with multiple arguments. * merge Jarko's fixes, issue #45 and #46 * add the ability to specify a path for py.lookup to search in * fix a funcarg cached_setup bug probably only occuring in distributed testing and "module" scope with teardown. * many fixes and changes for making the code base python3 compatible, many thanks to Benjamin Peterson for helping with this. * consolidate builtins implementation to be compatible with >=2.3, add helpers to ease keeping 2 and 3k compatible code * deprecate py.compat.doctest|subprocess|textwrap|optparse * deprecate py.magic.autopath, remove py/magic directory * move pytest assertion handling to py/code and a pytest_assertion plugin, add "--no-assert" option, deprecate py.magic namespaces in favour of (less) py.code ones. * consolidate and cleanup py/code classes and files * cleanup py/misc, move tests to bin-for-dist * introduce delattr/delitem/delenv methods to py.test's monkeypatch funcarg * consolidate py.log implementation, remove old approach. * introduce py.io.TextIO and py.io.BytesIO for distinguishing between text/unicode and byte-streams (uses underlying standard lib io.* if available) * make py.unittest_convert helper script available which converts "unittest.py" style files into the simpler assert/direct-test-classes py.test/nosetests style. The script was written by Laura Creighton. * simplified internal localpath implementation Changes between 1.0.1 and 1.0.2 ===================================== * fixing packaging issues, triggered by fedora redhat packaging, also added doc, examples and contrib dirs to the tarball. * added a documentation link to the new django plugin. Changes between 1.0.0 and 1.0.1 ===================================== * added a 'pytest_nose' plugin which handles nose.SkipTest, nose-style function/method/generator setup/teardown and tries to report functions correctly. * capturing of unicode writes or encoded strings to sys.stdout/err work better, also terminalwriting was adapted and somewhat unified between windows and linux. * improved documentation layout and content a lot * added a "--help-config" option to show conftest.py / ENV-var names for all longopt cmdline options, and some special conftest.py variables. renamed 'conf_capture' conftest setting to 'option_capture' accordingly. * fix issue #27: better reporting on non-collectable items given on commandline (e.g. pyc files) * fix issue #33: added --version flag (thanks Benjamin Peterson) * fix issue #32: adding support for "incomplete" paths to wcpath.status() * "Test" prefixed classes are *not* collected by default anymore if they have an __init__ method * monkeypatch setenv() now accepts a "prepend" parameter * improved reporting of collection error tracebacks * simplified multicall mechanism and plugin architecture, renamed some internal methods and argnames Changes between 1.0.0b9 and 1.0.0 ===================================== * more terse reporting try to show filesystem path relatively to current dir * improve xfail output a bit Changes between 1.0.0b8 and 1.0.0b9 ===================================== * cleanly handle and report final teardown of test setup * fix svn-1.6 compat issue with py.path.svnwc().versioned() (thanks Wouter Vanden Hove) * setup/teardown or collection problems now show as ERRORs or with big "E"'s in the progress lines. they are reported and counted separately. * dist-testing: properly handle test items that get locally collected but cannot be collected on the remote side - often due to platform/dependency reasons * simplified py.test.mark API - see keyword plugin documentation * integrate better with logging: capturing now by default captures test functions and their immediate setup/teardown in a single stream * capsys and capfd funcargs now have a readouterr() and a close() method (underlyingly py.io.StdCapture/FD objects are used which grew a readouterr() method as well to return snapshots of captured out/err) * make assert-reinterpretation work better with comparisons not returning bools (reported with numpy from thanks maciej fijalkowski) * reworked per-test output capturing into the pytest_iocapture.py plugin and thus removed capturing code from config object * item.repr_failure(excinfo) instead of item.repr_failure(excinfo, outerr) Changes between 1.0.0b7 and 1.0.0b8 ===================================== * pytest_unittest-plugin is now enabled by default * introduced pytest_keyboardinterrupt hook and refined pytest_sessionfinish hooked, added tests. * workaround a buggy logging module interaction ("closing already closed files"). Thanks to Sridhar Ratnakumar for triggering. * if plugins use "py.test.importorskip" for importing a dependency only a warning will be issued instead of exiting the testing process. * many improvements to docs: - refined funcargs doc , use the term "factory" instead of "provider" - added a new talk/tutorial doc page - better download page - better plugin docstrings - added new plugins page and automatic doc generation script * fixed teardown problem related to partially failing funcarg setups (thanks MrTopf for reporting), "pytest_runtest_teardown" is now always invoked even if the "pytest_runtest_setup" failed. * tweaked doctest output for docstrings in py modules, thanks Radomir. Changes between 1.0.0b3 and 1.0.0b7 ============================================= * renamed py.test.xfail back to py.test.mark.xfail to avoid two ways to decorate for xfail * re-added py.test.mark decorator for setting keywords on functions (it was actually documented so removing it was not nice) * remove scope-argument from request.addfinalizer() because request.cached_setup has the scope arg. TOOWTDI. * perform setup finalization before reporting failures * apply modified patches from Andreas Kloeckner to allow test functions to have no func_code (#22) and to make "-k" and function keywords work (#20) * apply patch from Daniel Peolzleithner (issue #23) * resolve issue #18, multiprocessing.Manager() and redirection clash * make __name__ == "__channelexec__" for remote_exec code Changes between 1.0.0b1 and 1.0.0b3 ============================================= * plugin classes are removed: one now defines hooks directly in conftest.py or global pytest_*.py files. * added new pytest_namespace(config) hook that allows to inject helpers directly to the py.test.* namespace. * documented and refined many hooks * added new style of generative tests via pytest_generate_tests hook that integrates well with function arguments. Changes between 0.9.2 and 1.0.0b1 ============================================= * introduced new "funcarg" setup method, see doc/test/funcarg.txt * introduced plugin architecuture and many new py.test plugins, see doc/test/plugins.txt * teardown_method is now guaranteed to get called after a test method has run. * new method: py.test.importorskip(mod,minversion) will either import or call py.test.skip() * completely revised internal py.test architecture * new py.process.ForkedFunc object allowing to fork execution of a function to a sub process and getting a result back. XXX lots of things missing here XXX Changes between 0.9.1 and 0.9.2 =============================== * refined installation and metadata, created new setup.py, now based on setuptools/ez_setup (thanks to Ralf Schmitt for his support). * improved the way of making py.* scripts available in windows environments, they are now added to the Scripts directory as ".cmd" files. * py.path.svnwc.status() now is more complete and uses xml output from the 'svn' command if available (Guido Wesdorp) * fix for py.path.svn* to work with svn 1.5 (Chris Lamb) * fix path.relto(otherpath) method on windows to use normcase for checking if a path is relative. * py.test's traceback is better parseable from editors (follows the filenames:LINENO: MSG convention) (thanks to Osmo Salomaa) * fix to javascript-generation, "py.test --runbrowser" should work more reliably now * removed previously accidentally added py.test.broken and py.test.notimplemented helpers. * there now is a py.__version__ attribute Changes between 0.9.0 and 0.9.1 =============================== This is a fairly complete list of changes between 0.9 and 0.9.1, which can serve as a reference for developers. * allowing + signs in py.path.svn urls [39106] * fixed support for Failed exceptions without excinfo in py.test [39340] * added support for killing processes for Windows (as well as platforms that support os.kill) in py.misc.killproc [39655] * added setup/teardown for generative tests to py.test [40702] * added detection of FAILED TO LOAD MODULE to py.test [40703, 40738, 40739] * fixed problem with calling .remove() on wcpaths of non-versioned files in py.path [44248] * fixed some import and inheritance issues in py.test [41480, 44648, 44655] * fail to run greenlet tests when pypy is available, but without stackless [45294] * small fixes in rsession tests [45295] * fixed issue with 2.5 type representations in py.test [45483, 45484] * made that internal reporting issues displaying is done atomically in py.test [45518] * made that non-existing files are igored by the py.lookup script [45519] * improved exception name creation in py.test [45535] * made that less threads are used in execnet [merge in 45539] * removed lock required for atomical reporting issue displaying in py.test [45545] * removed globals from execnet [45541, 45547] * refactored cleanup mechanics, made that setDaemon is set to 1 to make atexit get called in 2.5 (py.execnet) [45548] * fixed bug in joining threads in py.execnet's servemain [45549] * refactored py.test.rsession tests to not rely on exact output format anymore [45646] * using repr() on test outcome [45647] * added 'Reason' classes for py.test.skip() [45648, 45649] * killed some unnecessary sanity check in py.test.collect [45655] * avoid using os.tmpfile() in py.io.fdcapture because on Windows it's only usable by Administrators [45901] * added support for locking and non-recursive commits to py.path.svnwc [45994] * locking files in py.execnet to prevent CPython from segfaulting [46010] * added export() method to py.path.svnurl * fixed -d -x in py.test [47277] * fixed argument concatenation problem in py.path.svnwc [49423] * restore py.test behaviour that it exits with code 1 when there are failures [49974] * don't fail on html files that don't have an accompanying .txt file [50606] * fixed 'utestconvert.py < input' [50645] * small fix for code indentation in py.code.source [50755] * fix _docgen.py documentation building [51285] * improved checks for source representation of code blocks in py.test [51292] * added support for passing authentication to py.path.svn* objects [52000, 52001] * removed sorted() call for py.apigen tests in favour of [].sort() to support Python 2.3 [52481] From memedough at gmail.com Fri Jul 9 10:39:16 2010 From: memedough at gmail.com (meme dough) Date: Fri, 9 Jul 2010 18:39:16 +1000 Subject: [py-dev] pytest-cov In-Reply-To: References: <20100708124022.GP14601@trillke.net> Message-ID: Hi, Did you try deleting all .pyc and .pyo files in /home/prologic/work/circuits recursively and then trying? Since it looks like python 2.6 will create the pyc files in the master process and then the popen to python 2.5 barfs on them. As Holger pointed out PYTHONDOWNWRITEBYTECODE should turn off the master python 2.6 creating pyc files. But if you've still got old ones lying around then the slave python 2.5 will no doubt barf. Another solution to this maybe to give each test process it's own area. So change the working dir of each tx and specify the rsync option to get the proper code to each test area (I assume the rsync doesn't push pyc files). Also that way if your code relies on it's own resources in the dir tree (writing out to a log file for example) then they will be separate and not clash. :) On 8 July 2010 22:44, James Mills wrote: > On Thu, Jul 8, 2010 at 10:40 PM, holger krekel wrote: >> does it help if you set the environment variable >> PYTHONDOWNWRITEBYTECODE=1 > > No, this didn't seem to affect anything. > > cheers > James > > > -- > -- James Mills > -- > -- "Problems are solved by method" > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > From prologic at shortcircuit.net.au Fri Jul 9 10:55:53 2010 From: prologic at shortcircuit.net.au (James Mills) Date: Fri, 9 Jul 2010 18:55:53 +1000 Subject: [py-dev] pytest-cov In-Reply-To: References: <20100708124022.GP14601@trillke.net> Message-ID: On Fri, Jul 9, 2010 at 6:39 PM, meme dough wrote: > Did you try deleting all .pyc and .pyo files in > /home/prologic/work/circuits recursively and then trying? Yes, tried and same result. > Since it looks like python 2.6 will create the pyc files in the master > process and then the popen to python 2.5 barfs on them. ?As Holger > pointed out PYTHONDOWNWRITEBYTECODE should turn off the master python > 2.6 creating pyc files. ?But if you've still got old ones lying around > then the slave python 2.5 will no doubt barf. Also tried, same result. > Another solution to this maybe to give each test process it's own > area. ?So change the working dir of each tx and specify the rsync > option to get the proper code to each test area (I assume the rsync > doesn't push pyc files). ?Also that way if your code relies on it's > own resources in the dir tree (writing out to a log file for example) > then they will be separate and not clash. Could you show an example tx config that does thsi ? cheers James -- -- James Mills -- -- "Problems are solved by method" From prologic at shortcircuit.net.au Fri Jul 9 10:57:30 2010 From: prologic at shortcircuit.net.au (James Mills) Date: Fri, 9 Jul 2010 18:57:30 +1000 Subject: [py-dev] execnet wrt doing sysadmin work against a cluster of nodes Message-ID: Hi all, Would someone be able to show how execnet could be used to manage a cluster of nodes over ssh ? Just a simple example of executing a remote shell command (eg: uptime) against a cluster of nodes (host1, host2, host3). cheers James -- -- James Mills -- -- "Problems are solved by method" From memedough at gmail.com Fri Jul 9 12:47:53 2010 From: memedough at gmail.com (meme dough) Date: Fri, 9 Jul 2010 20:47:53 +1000 Subject: [py-dev] pytest-cov In-Reply-To: References: <20100708124022.GP14601@trillke.net> Message-ID: Hi, Invoke distributed testing with 2 test process both of which are on the same host but have different dirs: py.test --dist=each --tx=popen//chdir=/tmp/testarea1//python=/usr/bin/python2.5 --tx=popen//chdir=/tmp/testarea2//python=/usr/bin/python2.6 --rsyncdir=circuits The 2 test process are started by popen, and each one changes to it's respective dir. The /tmp/testarea1 & 2 get created. The circuits dir should be rsynced to the test areas. py.test will automatically send what it needs to work (py and xdist packages). Then you can see what happens in each test area. Look at what it rsyncs, does it send over the pyc files? Since each is separate then any generation of new pyc files should not conflict between the processes. This works for me but I'm not getting the same errors as you. :) On 9 July 2010 18:55, James Mills wrote: > On Fri, Jul 9, 2010 at 6:39 PM, meme dough wrote: >> Did you try deleting all .pyc and .pyo files in >> /home/prologic/work/circuits recursively and then trying? > > Yes, tried and same result. > >> Since it looks like python 2.6 will create the pyc files in the master >> process and then the popen to python 2.5 barfs on them. ?As Holger >> pointed out PYTHONDOWNWRITEBYTECODE should turn off the master python >> 2.6 creating pyc files. ?But if you've still got old ones lying around >> then the slave python 2.5 will no doubt barf. > > Also tried, same result. > >> Another solution to this maybe to give each test process it's own >> area. ?So change the working dir of each tx and specify the rsync >> option to get the proper code to each test area (I assume the rsync >> doesn't push pyc files). ?Also that way if your code relies on it's >> own resources in the dir tree (writing out to a log file for example) >> then they will be separate and not clash. > > Could you show an example tx config that does thsi ? > > cheers > James > > -- > -- James Mills > -- > -- "Problems are solved by method" > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > From memedough at gmail.com Sat Jul 10 17:14:25 2010 From: memedough at gmail.com (meme dough) Date: Sun, 11 Jul 2010 01:14:25 +1000 Subject: [py-dev] pytest-xdist limitations in interactions between different versions of python Message-ID: Hi Holger, I have a few questions for py.test with xdist and I'm not sure if these are known and I missed the something. When I kick off py.test with dist=each mode I've noticed 3 limitations in interactions between different versions of python. First if start py.test with python2.x then can specify any 2.x txes, so the following is fine: py.test --tb=short -r sfxX --dist=each --tx=popen//python=/usr/local/python246/bin/python --tx=popen//python=/usr/local/python255/bin/python --tx=popen//python=/usr/local/python265/bin/python However problems if txes for 3.0: py.test -v --tb=short -r sfxX --dist=each --tx=popen//python=/usr/local/python246/bin/python --tx=popen//python=/usr/local/python255/bin/python --tx=popen//python=/usr/local/python265/bin/python --tx=popen//python=/usr/local/python301/bin/python ========================= test session starts ========================= platform linux2 -- Python 2.6.4 -- pytest-1.3.2 -- /usr/bin/python test path 1: /home/memedough/work/projects/pytest-cov [gw0] popen//python=/usr/local/python246/bin/python -- platform linux2, Python 2.4.6-final-0 cwd: /home/memedough/work/projects/pytest-cov- /usr/local/python246/bin/python [gw1] popen//python=/usr/local/python255/bin/python -- platform linux2, Python 2.5.5-final-0 cwd: /home/memedough/work/projects/pytest-cov- /usr/local/python255/bin/python [gw2] popen//python=/usr/local/python265/bin/python -- platform linux2, Python 2.6.5-final-0 cwd: /home/memedough/work/projects/pytest-cov- /usr/local/python265/bin/python [gw3] popen//python=/usr/local/python301/bin/python -- platform linux2, Python 3.0.1-final-0 cwd: /home/memedough/work/projects/pytest-cov- /usr/local/python301/bin/python [master] starting full item collection ... [master] collected 7 items [gw0] txnode ready to receive tests [gw3] node down, error: [gw2] txnode ready to receive tests [gw1] txnode ready to receive tests Then I have to ctrl-c. Also problems if txes for 3.1: py.test -v --tb=short -r sfxX --dist=each --tx=popen//python=/usr/local/python246/bin/python --tx=popen//python=/usr/local/python255/bin/python --tx=popen//python=/usr/local/python265/bin/python --tx=popen//python=/usr/local/python312/bin/python ========================= test session starts ========================= platform linux2 -- Python 2.6.4 -- pytest-1.3.2 -- /usr/bin/python test path 1: /home/memedough/work/projects/pytest-cov [gw0] popen//python=/usr/local/python246/bin/python -- platform linux2, Python 2.4.6-final-0 cwd: /home/memedough/work/projects/pytest-cov- /usr/local/python246/bin/python [gw1] popen//python=/usr/local/python255/bin/python -- platform linux2, Python 2.5.5-final-0 cwd: /home/memedough/work/projects/pytest-cov- /usr/local/python255/bin/python [gw2] popen//python=/usr/local/python265/bin/python -- platform linux2, Python 2.6.5-final-0 cwd: /home/memedough/work/projects/pytest-cov- /usr/local/python265/bin/python [gw3] popen//python=/usr/local/python312/bin/python -- platform linux2, Python 3.1.2-final-0 cwd: /home/memedough/work/projects/pytest-cov- /usr/local/python312/bin/python [master] starting full item collection ... [master] collected 7 items [gw3] node down, error: ========================== in 1.79 seconds =========================== I didn't have to ctrl-c then, it varies I think. Second, if start py.test with python 3.0 then can specify 3.0 / 3.1, so the following is fine: py.test --tb=short -r sfxX --dist=each --tx=popen//python=/usr/local/python301/bin/python --tx=popen//python=/usr/local/python312/bin/python However problems if txes for 2.x: py.test -v --tb=short -r sfxX --dist=each --tx=popen//python=/usr/local/python301/bin/python --tx=popen//python=/usr/local/python265/bin/python =========================== test session starts =========================== platform linux2 -- Python 3.0.1 -- pytest-1.3.2 -- /usr/local/python301/bin/python test path 1: /home/memedough/work/projects/pytest-cov [gw0] popen//python=/usr/local/python301/bin/python -- platform linux2, Python 3.0.1-final-0 cwd: /home/memedough/work/projects/pytest-cov- /usr/local/python301/bin/python [gw1] popen//python=/usr/local/python265/bin/python -- platform linux2, Python 2.6.5-final-0 cwd: /home/memedough/work/projects/pytest-cov- /usr/local/python265/bin/python [master] starting full item collection ... [master] collected 7 items [gw1] node down, error: Traceback (most recent call last): File "/usr/local/python301/lib/python3.0/site-packages/execnet-1.0.7-py3.0.egg/execnet/gateway_base.py", line 725, in executetask do_exec(co, loc) File "", line 1, in do_exec File "", line 9, in File "/usr/local/python301/lib/python3.0/site-packages/pytest_xdist-1.4-py3.0.egg/xdist/mypickle.py", line 142, in receive return self._unpickle(pickled_obj) File "/usr/local/python301/lib/python3.0/site-packages/pytest_xdist-1.4-py3.0.egg/xdist/mypickle.py", line 147, in _unpickle return self._ipickle.loads(pickled_obj) File "/usr/local/python301/lib/python3.0/site-packages/pytest_xdist-1.4-py3.0.egg/xdist/mypickle.py", line 103, in loads res = unpickler.load() File "/usr/local/python265/lib/python2.6/pickle.py", line 858, in load dispatch[key](self) File "/usr/local/python265/lib/python2.6/pickle.py", line 1090, in load_global klass = self.find_class(module, name) File "/usr/local/python265/lib/python2.6/pickle.py", line 1124, in find_class __import__(module) ImportError: No module named copyreg ============================ in 0.29 seconds ============================= Third, if start py.test with python 3.1 then can specify 3.1, so the following is fine: py.test --tb=short -r sfxX --dist=each --tx=popen//python=/usr/local/python312/bin/python --tx=popen//python=/usr/local/python312/bin/python However problems if txes for 3.0 or 2.x: py.test -v --tb=short -r sfxX --dist=each --tx=popen//python=/usr/local/python312/bin/python --tx=popen//python=/usr/local/python301/bin/python ========================== test session starts ========================== platform linux2 -- Python 3.1.2 -- pytest-1.3.2 -- /usr/local/python312/bin/python test path 1: /home/memedough/work/projects/pytest-cov [gw0] popen//python=/usr/local/python312/bin/python -- platform linux2, Python 3.1.2-final-0 cwd: /home/memedough/work/projects/pytest-cov- /usr/local/python312/bin/python [gw1] popen//python=/usr/local/python301/bin/python -- platform linux2, Python 3.0.1-final-0 cwd: /home/memedough/work/projects/pytest-cov- /usr/local/python301/bin/python [master] starting full item collection ... /usr/local/python312/lib/python3.1/site-packages/py-1.3.2-py3.1.egg/py/_test/pluginmanager.py:100: UserWarning: Module pytest_cov_init was already imported from /usr/local/python312/lib/python3.1/site-packages/pytest_cov-0.16a1-py3.1.egg/pytest_cov_init.py, but /home/memedough/work/projects/pytest-cov is being added to sys.path from pkg_resources import iter_entry_points [master] collected 7 items [gw1] node down, error: Traceback (most recent call last): File "/usr/local/python312/lib/python3.1/site-packages/execnet-1.0.7-py3.1.egg/execnet/gateway_base.py", line 725, in executetask do_exec(co, loc) File "", line 1, in do_exec File "", line 9, in File "/usr/local/python312/lib/python3.1/site-packages/pytest_xdist-1.4-py3.1.egg/xdist/mypickle.py", line 142, in receive return self._unpickle(pickled_obj) File "/usr/local/python312/lib/python3.1/site-packages/pytest_xdist-1.4-py3.1.egg/xdist/mypickle.py", line 147, in _unpickle return self._ipickle.loads(pickled_obj) File "/usr/local/python312/lib/python3.1/site-packages/pytest_xdist-1.4-py3.1.egg/xdist/mypickle.py", line 103, in loads res = unpickler.load() File "/usr/local/python301/lib/python3.0/pickle.py", line 813, in load dispatch[key[0]](self) File "/usr/local/python301/lib/python3.0/pickle.py", line 1058, in load_global klass = self.find_class(module, name) File "/usr/local/python301/lib/python3.0/pickle.py", line 1092, in find_class __import__(module, level=0) ImportError: No module named copy_reg =========================== in 0.34 seconds ============================ The 2.x problem is: py.test -v --tb=short -r sfxX --dist=each --tx=popen//python=/usr/local/python312/bin/python --tx=popen//python=/usr/local/python265/bin/python ========================== test session starts ========================== platform linux2 -- Python 3.1.2 -- pytest-1.3.2 -- /usr/local/python312/bin/python test path 1: /home/memedough/work/projects/pytest-cov [gw0] popen//python=/usr/local/python312/bin/python -- platform linux2, Python 3.1.2-final-0 cwd: /home/memedough/work/projects/pytest-cov- /usr/local/python312/bin/python [gw1] popen//python=/usr/local/python265/bin/python -- platform linux2, Python 2.6.5-final-0 cwd: /home/memedough/work/projects/pytest-cov- /usr/local/python265/bin/python [master] starting full item collection ... /usr/local/python312/lib/python3.1/site-packages/py-1.3.2-py3.1.egg/py/_test/pluginmanager.py:100: UserWarning: Module pytest_cov_init was already imported from /usr/local/python312/lib/python3.1/site-packages/pytest_cov-0.16a1-py3.1.egg/pytest_cov_init.py, but /home/memedough/work/projects/pytest-cov is being added to sys.path from pkg_resources import iter_entry_points [master] collected 7 items [gw0] txnode ready to receive tests [gw1] node down, error: pickled_obj = "(S'slaveready'\np162\n(t(dp164\ntp166\n." def unpickle_callback(pickled_obj): if pickled_obj is uniqueendmarker: return callback(endmarker) try: > obj = self._unpickle(pickled_obj) callback = > endmarker = -1 pickled_obj = "(S'slaveready'\np162\n(t(dp164\ntp166\n." self = uniqueendmarker =