From pedronis at openend.se Tue Apr 6 17:36:24 2010 From: pedronis at openend.se (Samuele Pedroni) Date: Tue, 06 Apr 2010 17:36:24 +0200 Subject: [py-dev] ANN: oejskit 0.8.8 JavaScript in-browser testing with py.test plugin and unittest.py glue Message-ID: <4BBB54F8.4060500@openend.se> Hi, I just pushed a new release of OE jskit 0.8.8 to PyPI. Main points of interest: * the code to check for the presence of browsers locally has been improved, browser specifications can now with confidence list absent browsers and the respective runs/tests will be skipped * added a workaround to a bug in Firefox 3.5 that interferes with the global variable leak detection code, this is simply turned off for FF3.5. The bug itself is fixed in FF3.6 About OE jskit: jskit contains infrastructure and in particular a py.test plugin to enable running unit tests for JavaScript code inside browsers. It contains also glue code to run JavaScript tests from unittest.py based test suites. The approach also enables to write integration tests such that the JavaScript code is tested against server-side Python code mocked as necessary. Any server-side framework that can already be exposed through WSGI can play. The plugin requires py.test 1.1.1 and should also work with current trunk. More information and downloading at: http://pypi.python.org/pypi/oejskit including a changelog, documentation and the talk I gave at Europython 2009. jskit was initially developed by Open End AB and is released under the MIT license. In various incarnations it has been in use and useful at Open End for more than two years, we are quite happy to share it. Samuele Pedroni for Open End _______________________________________________ py-dev mailing list py-dev at codespeak.net http://codespeak.net/mailman/listinfo/py-dev From simongsmith at gmail.com Wed Apr 7 22:42:39 2010 From: simongsmith at gmail.com (simongsmith at gmail.com) Date: Wed, 07 Apr 2010 20:42:39 -0000 Subject: [py-dev] using oejskit to test hosted html Message-ID: I've been exploring the examples of using oejskit (from [http://www2.openend.se/~pedronis/oejskit/talk/examples.tgz: ]) and they work fine for me. I wanted to change the test_open_eval.py to access the same foo.html via an apache website I'm hosting (it is static in the original). I'm envoking with "py.test test" and am using python2.6, py.test 1.2.1 and oejskit 0.8.8 on a Mac running OS X. Original code (test/test_open_eval.py) from oejskit.testing import BrowserTestClass, JsFailed class TestOpenEval(BrowserTestClass): jstests_browser_kind = 'supported' def test_open_eval(self): pg = self.open('/static/foo.html') title = pg.eval('document.title') assert title == 'Foo' I changed the open call: pg = self.open('http://mysite.net/foo.html') When I re-run the test, the foo.html is loaded into the browser that is launched as a result of the test, and it is identical (via "View Frame Source") as the original, but I get the following error: self = js = 'document.title', variant = 'eval' def eval(self, js, variant='eval'): outcome = self._execute(variant, js, self.root, self.timeout) if outcome.get('error'): > raise JsFailed('[%s] %s' % (self.label, js), outcome['error']) E JsFailed: [[http://ctier.arcode.com:8888/foo.html: ]] document.title: FAILED: E message: Result of expression 'frameWin.eval' [undefined] is not a function. E line: 169 E sourceId: 457014824 E sourceURL: [http://localhost:62737/browser_testing/rt/InBrowserTesting.js: ] E expressionBeginOffset: 4985 E expressionCaretOffset: 4998 E expressionEndOffset: 5004 E name: TypeError ../../../arcode/python/arcode/js- infrastructure/oejskit/browser_ctl.py:263: JsFailed I've dug around in the oejskit code and I don't understand what's going wrong and I can't find any documentation on this? has anyone successfully used oejskit to assert against html hosted on a remote apache? Or does anyone have any suggestions as to what I might be doing incorrectly? foo.html follows below. Thanks!!! Simon Smith [simongsmith at gmail.com: ] foo.html: Foo<title> <script> </script> </head> <body> </body> </html> -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/py-dev/attachments/20100407/26b05693/attachment.htm From pedronis at openend.se Thu Apr 8 13:47:33 2010 From: pedronis at openend.se (Samuele Pedroni) Date: Thu, 08 Apr 2010 13:47:33 +0200 Subject: [py-dev] using oejskit to test hosted html In-Reply-To: <TGmx3i_RRTv0XKAG6Ejdc-2IGXlQdAvMVKkYvDbAjhzwtsJFc@smtp.gmail.com> References: <TGmx3i_RRTv0XKAG6Ejdc-2IGXlQdAvMVKkYvDbAjhzwtsJFc@smtp.gmail.com> Message-ID: <4BBDC255.3020206@openend.se> simongsmith at gmail.com wrote: > > I changed the open call: > > > > pg = self.open('http://mysite.net/foo.html') > > When I re-run the test, the foo.html is loaded into the browser that > is launched as a result of the test, and it is identical (via "View > Frame Source") as the original, but I get the following error: > > > > self = <oejskit.browser_ctl.PageContext object at 0x156dc90> > js = 'document.title', variant = 'eval' > > def eval(self, js, variant='eval'): > outcome = self._execute(variant, js, self.root, self.timeout) > if outcome.get('error'): > > raise JsFailed('[%s] %s' % (self.label, js), outcome['error']) > E JsFailed: [http://ctier.arcode.com:8888/foo.html] > document.title: FAILED: > E message: Result of expression 'frameWin.eval' [undefined] > is not a function. > E line: 169 > E sourceId: 457014824 > E sourceURL: > http://localhost:62737/browser_testing/rt/InBrowserTesting.js > E expressionBeginOffset: 4985 > E expressionCaretOffset: 4998 > E expressionEndOffset: 5004 > E name: TypeError > > ../../../arcode/python/arcode/js-infrastructure/oejskit/browser_ctl.py:263: > JsFailed > > > > > Hi, there are two levels of possible issues here: - in some browsers it seems (as foo.html does) one needs at least an empty script tag to have eval() available on the window object - more importantly while it is possible to use oejskit to do functional testing (we have used it here that way) because of javascript security the tested pages/app and testing infrastructure need to come from the same domain, this can be arranged with proxying but there's no example right now how to do this and this depends on how the pages/app themselves are served. Basically the second url: http://localhost:62737/browser_testing/rt/InBrowserTesting.js would really need to look like: http://ctier.arcode.com:8888/ <http://ctier.arcode.com:8888/foo.html>browser_testing/rt/InBrowserTesting.js <http://localhost:62737/browser_testing/rt/InBrowserTesting.js> regards, Samuele Pedroni From lac at openend.se Fri Apr 9 01:14:03 2010 From: lac at openend.se (Laura Creighton) Date: Fri, 09 Apr 2010 01:14:03 +0200 Subject: [py-dev] oesjkit In-Reply-To: Message from Samuele Pedroni <pedronis@openend.se> of "Thu, 08 Apr 2010 13:47:33 +0200." <4BBDC255.3020206@openend.se> References: <TGmx3i_RRTv0XKAG6Ejdc-2IGXlQdAvMVKkYvDbAjhzwtsJFc@smtp.gmail.com> <4BBDC255.3020206@openend.se> Message-ID: <201004082314.o38NE3aN010030@theraft.openend.se> Any idea how to get it listed here? http://pycheesecake.org/wiki/PythonTestingToolsTaxonomy#MockTestingTools Laura (still ill, but reading some mail now at any rate) From phil at freehackers.org Mon Apr 12 12:24:12 2010 From: phil at freehackers.org (Philippe Fremy) Date: Mon, 12 Apr 2010 12:24:12 +0200 Subject: [py-dev] py.test : setup / teardown at the directory level In-Reply-To: <20091104172342.GU29141@trillke.net> References: <49F6EA81.2070106@freehackers.org> <20090428115556.GJ11963@trillke.net> <49F71C0E.3000800@freehackers.org> <20090428180200.GO11963@trillke.net> <4AE80A42.5030009@freehackers.org> <20091104172342.GU29141@trillke.net> Message-ID: <4BC2F4CC.9040408@freehackers.org> An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/py-dev/attachments/20100412/5307d5db/attachment.htm From phil at freehackers.org Mon Apr 12 12:37:36 2010 From: phil at freehackers.org (Philippe Fremy) Date: Mon, 12 Apr 2010 12:37:36 +0200 Subject: [py-dev] order of setup / teardown Message-ID: <4BC2F7F0.1030603@freehackers.org> Hi, I am not sure if this has been brought before, but I am surprised to see some calls inside the test "hooks" after the test session has finished. I am referring here to teardown_class, revealed by a quick example: pytest_sessionstart setup_class setup_method test1 teardown_method setup_method test2 teardown_method pytest_sessionfinish teardown_class This is creating problems for me because I am assuming that all resources are setup in sesssionstart, and torn down in sessionfinish. And teardown_class is using one of those resources. Is it a bug or intended behavior ? cheers, Philippe From holger at merlinux.eu Mon Apr 12 13:13:08 2010 From: holger at merlinux.eu (holger krekel) Date: Mon, 12 Apr 2010 13:13:08 +0200 Subject: [py-dev] order of setup / teardown In-Reply-To: <4BC2F7F0.1030603@freehackers.org> References: <4BC2F7F0.1030603@freehackers.org> Message-ID: <20100412111308.GY26514@trillke.net> Hi Philippe, (note to all: i am back from a long and fun vacation and will see to answer pending issues in the next days) On Mon, Apr 12, 2010 at 12:37 +0200, Philippe Fremy wrote: > Hi, > > I am not sure if this has been brought before, but I am surprised to see > some calls inside the test "hooks" after the test session has finished. > > I am referring here to teardown_class, revealed by a quick example: > > pytest_sessionstart > > setup_class > setup_method > > test1 > > teardown_method > > setup_method > test2 > > teardown_method > > pytest_sessionfinish > > teardown_class > > This is creating problems for me because > I am assuming that all resources are setup in sesssionstart, > and torn down in sessionfinish. And teardown_class is using one of > those resources. > > Is it a bug or intended behavior ? setup and teardown of python test functions are implemented through the builtin py/_plugin/pytest_runner.py plugin. It itself implements hooks called from the "session" in order to provide the setup/teardown semantics. There, it only tears down class/module state if the "next" test does not need it and this is decided at setup time of said next test. This means that after the last test has run, some resources might still require teardown and currently this is done through implementing it in a pytest_sessionfinish hook. The latter, however, is invoked after all user-provided hooks have been invoked. So much for the analysis. Also related to your other "directory-setup" issue i am not entirely happy about pointing people to pytest_namespace and sessionstart/sessionfinish for scoped python setup purposes. Maybe more python-specific setup/teardown hook(s) would make more sense? Relatedly, I ponder giving up on the incremental collection of tests and internally immediately produce a fully exploded lists of tests at session start. This way teardown could be done in a more precise manner -- right after the resource/scope is not needed anymore. It would also make randomization and other test re-ordering easy. A downside is that it takes longer for the first test to run. A plus is that it probably eases implementation and provides less surprises (such as the one you are having because pytest_runner would not need to work through sessionfinish anymore). If you don't tell anyone - :) - there also is a immediate work around to your issue i think. You could implement something like def pytest_sessionfinish(__multicall__): __multicall__.execute() # call other sessionfinish implementers first '__multicall__' is the object representing the current ongoing plugin hook call and allows to dynamically call other reamining hook implementers first. It's not documented or advertised (but used sometimes from pytest's own builtin plugins) because i think a more declarative decorator-based way of setting call ordering is saner. Then again, i hope to avoid the need altogether for simplicity sake. cheers, holger From Ronny.Pfannschmidt at gmx.de Tue Apr 13 14:19:11 2010 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Tue, 13 Apr 2010 14:19:11 +0200 Subject: [py-dev] py.test : setup / teardown at the directory level In-Reply-To: <4BC2F4CC.9040408@freehackers.org> References: <49F6EA81.2070106@freehackers.org> <20090428115556.GJ11963@trillke.net> <49F71C0E.3000800@freehackers.org> <20090428180200.GO11963@trillke.net> <4AE80A42.5030009@freehackers.org> <20091104172342.GU29141@trillke.net> <4BC2F4CC.9040408@freehackers.org> Message-ID: <1271161151.9772.4.camel@localhost> On Mon, 2010-04-12 at 12:24 +0200, Philippe Fremy wrote: > > That thread is 6 months old, but I see that it is still relevant. > > holger krekel wrote: > > Hi Philippe, > > > > On Wed, Oct 28, 2009 at 10:09 +0100, Philippe Fremy wrote: > > > > > I had a quick look at the plugin architecture to see if I could > > > implement an equivalent of setup/teardown at directory level, but I > > > don't think it's possible. > > > > > > > Whatever is called for "directory" setup could live in a conftest.py file. > > The question is how to transfer any "directory" setup state > > to a module. At first i thought one could write down: > > > > # ./conftest.py > > def setup_module(mod): > > # will be called for all modules in/below the dir > > mod.something = "value" > > > > but one would expect this to be called for each module and not > > just once for a whole directory. Now one could perform some > > directory-level caching but one conceptual issue remains: > > values would be "magically" injected into the test modules. > > > > Do you have ideas about how you'd like the API to work? > > > > Yes, I now know how I want to transfer the value. I would like to use > the py.test namespace. > > Today, only pytest_namespace() is available to setup an initial > namespace. I would like to see a pytest_update_namespace() to pass a > dict with key/values to put in the namespace. > > cheers, > > Philippe > How about more control over funcargs in particular a) caching on various levels of packages b) exposure of all currently usable funcargs in the py.test namespace -- Ronny From memedough at gmail.com Fri Apr 16 11:09:12 2010 From: memedough at gmail.com (meme dough) Date: Fri, 16 Apr 2010 19:09:12 +1000 Subject: [py-dev] xdist sending info between master and slave - support for client code Message-ID: <g2jcf54a0ac1004160209x3aebf4bak1c3acac2a3ba283d@mail.gmail.com> Hi, I'm using pytest and the xdist plugin. I'd like to get some information from the master to the slave, specifically on the slave side I need to know the master's host, master's topdir and master's rsync roots. Currently I modified xdist to send the required info over the channel in install_slave: master_host = socket.gethostname() master_topdir = config.topdir channel.send((config, basetemp, gateway.id, master_host, master_topdir, roots)) The rsync roots were passed into the TXNode during creation by the NodeManager: node = TXNode(gateway, self.config, putevent, self.roots) That gets the info there but I'm not sure of the best way. The master host / topdir could be added to config.options since that get pickled over the channel, but it's not a command line option so it doesn't seem the best place to put that info. But it would work from a client point of view without modifying xdist. The master rsync roots can't be worked out by client unless it reimplements the NodeManager._getrsyncdirs logic. This happens because DSession does setup which creates NodeManager and setup_nodes which does install_slave without any client hook in between so at no point can client get rsync roots from session.nodemanager.roots and put them in something like config.options to get to slave side. That is the first half. Next I need to send some info back from slave to master when slave has finished. Currently I modified xdist to send back a report object when the slave has finished. The SlaveNode has a report object which any client hook can change or add to if they want to send info back to master: self.report = None I use pytest_sessionfinish hook on slave to set report to dict with info. When slave finish it sendevent with the report: self.sendevent("slavefinished", report=self.report) The TXNode on master gets report and stores the slave report on itself: self.report = kwargs['report'] Then on master use pytest_testnodedown hook to get slave report off TXNode. That gets the info back from slave to master but I'm not sure if that's best way. Is it possible to modify xdist so that client code on master can send info to the slave. Will need access to rsync roots before it goes ahead and installs the slave though. Is it possible to modify xdist so that client code on the slave can send info back to the master. This looks like must modify xdist somehow since it doesn't seem to send any report back at finish. Now that I wrote all that, what is the obvious answer that I didn't see? :) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/py-dev/attachments/20100416/0579940b/attachment.htm From holger at merlinux.eu Fri Apr 16 19:21:44 2010 From: holger at merlinux.eu (holger krekel) Date: Fri, 16 Apr 2010 19:21:44 +0200 Subject: [py-dev] xdist sending info between master and slave - support for client code In-Reply-To: <g2jcf54a0ac1004160209x3aebf4bak1c3acac2a3ba283d@mail.gmail.com> References: <g2jcf54a0ac1004160209x3aebf4bak1c3acac2a3ba283d@mail.gmail.com> Message-ID: <20100416172143.GY26514@trillke.net> Hi meme (correct addressing? :) thanks for the mail. For my understanding, can you mention a bit more context? On Fri, Apr 16, 2010 at 19:09 +1000, meme dough wrote: [...] > Is it possible to modify xdist so that client code on master can send info > to the slave. Will need access to rsync roots before it goes ahead and > installs the slave though. I think we could add a hook called at the master-side which could return/update an extra dictionary to be transfered to the client (in theory even a pickled-object but i'd like to avoid guaranteeing pickleability for the core test distribution mechanism). A slave-side hook (sessionstart?) would get this information. > Is it possible to modify xdist so that client code on the slave can send > info back to the master. This looks like must modify xdist somehow since it > doesn't seem to send any report back at finish. We can add a hook or re-use an existing one for allowing a client to send back (e.g. report) data but the question is: which hook sees this information on the master side? Maybe sessionfinish? So the slave's sessionfinish hook could add whatever extra information and the final sessionfinish sees it. Maybe it's better to design and call specific xdist-prefied hooks and not re-use existing ones? As you have hacked on this topic and for your purposes already - what do you think? > Now that I wrote all that, what is the obvious answer that I didn't see? > > :) I don't have an obvious answer but then again after a long holiday i still need to dive back a bit more; maybe i find some then :) cheers, holger From holger at merlinux.eu Fri Apr 16 19:45:07 2010 From: holger at merlinux.eu (holger krekel) Date: Fri, 16 Apr 2010 19:45:07 +0200 Subject: [py-dev] py.test : setup / teardown at the directory level In-Reply-To: <1271161151.9772.4.camel@localhost> References: <49F6EA81.2070106@freehackers.org> <20090428115556.GJ11963@trillke.net> <49F71C0E.3000800@freehackers.org> <20090428180200.GO11963@trillke.net> <4AE80A42.5030009@freehackers.org> <20091104172342.GU29141@trillke.net> <4BC2F4CC.9040408@freehackers.org> <1271161151.9772.4.camel@localhost> Message-ID: <20100416174507.GZ26514@trillke.net> Hi Philippe, Ronny, On Tue, Apr 13, 2010 at 14:19 +0200, Ronny Pfannschmidt wrote: > On Mon, 2010-04-12 at 12:24 +0200, Philippe Fremy wrote: > > > > That thread is 6 months old, but I see that it is still relevant. > > > > holger krekel wrote: > > > Hi Philippe, > > > > > > On Wed, Oct 28, 2009 at 10:09 +0100, Philippe Fremy wrote: > > > > > > > I had a quick look at the plugin architecture to see if I could > > > > implement an equivalent of setup/teardown at directory level, but I > > > > don't think it's possible. > > > > > > > > > > Whatever is called for "directory" setup could live in a conftest.py file. > > > The question is how to transfer any "directory" setup state > > > to a module. At first i thought one could write down: > > > > > > # ./conftest.py > > > def setup_module(mod): > > > # will be called for all modules in/below the dir > > > mod.something = "value" > > > > > > but one would expect this to be called for each module and not > > > just once for a whole directory. Now one could perform some > > > directory-level caching but one conceptual issue remains: > > > values would be "magically" injected into the test modules. > > > > > > Do you have ideas about how you'd like the API to work? > > > > > > > Yes, I now know how I want to transfer the value. I would like to use > > the py.test namespace. > > > > Today, only pytest_namespace() is available to setup an initial > > namespace. I would like to see a pytest_update_namespace() to pass a > > dict with key/values to put in the namespace. > > > > cheers, > > > > Philippe > > > > How about more control over funcargs > > in particular > > a) caching on various levels of packages > b) exposure of all currently usable funcargs in the py.test namespace Well, funcargs are well defined when used/requested from a test function. Having them globally available blurs the lines a lot and convolutes the py.test namespace. and what would e.g. py.test.tmpdir even be? would it be different when called from a test func, the various setup functions or used at import time etc.? Getting back to Philippe's goal of having per-directory cached values exposed to test functions i wonder if we could a) introduce a directory scope for funcarg-caching b) allow (a subset of) funcargs for setup functions Given this we could write down something like: # ./test_module.py def setup_module(mod, appserver): mod.appserver = appserver # ./conftest.py def pytest_funcarg__appserver(request): return request.cached_setup( setup=lambda: MyAppServer(...), teardown=lambda x: x.shutdown(), scope="directory" ) However, i am not sure if allowing funcargs in setup functions is sane. Also i am not thrilled about encouraging the use of global variables for transfering information. Maybe implementing a "directory" scope for cached_setup would go a long way. Any test function could then directly use the 'appserver' object as a function argument and rely on its per-directory creation/teardown. best, holger From memedough at gmail.com Sat Apr 17 05:54:13 2010 From: memedough at gmail.com (meme dough) Date: Sat, 17 Apr 2010 13:54:13 +1000 Subject: [py-dev] xdist sending info between master and slave - support for client code In-Reply-To: <20100416172143.GY26514@trillke.net> References: <g2jcf54a0ac1004160209x3aebf4bak1c3acac2a3ba283d@mail.gmail.com> <20100416172143.GY26514@trillke.net> Message-ID: <w2wcf54a0ac1004162054jc0fd38a2zdc8f8f2b41f70928@mail.gmail.com> Hi, > Is it possible to modify xdist so that client code on master can send info > > > to the slave. ?Will need access to rsync roots before it goes ahead and > > installs the slave though. > > I think we could add a hook called at the master-side which could return/update > an extra dictionary to be transfered to the client (in theory even a > pickled-object but i'd like to avoid guaranteeing pickleability for the core > test distribution mechanism). ?A slave-side hook (sessionstart?) would > get this information. I created a new hook that gets called just before each test node is installed: def pytest_testnodeinstall(nodemanager, gateway, slavedata): Client code on the master can set anything it needs to send to the slave on the slavedata object. The slavedata is passed over the channel in the same way as config object and is stored on the slave node such that any client code on the slave can access the slavedata. It can do this in any hook that the slave client code needs to implement (provided that it can get access to the session object which is the slave node object which has the slavedata). > > Is it possible to modify xdist so that client code on the slave can send > > info back to the master. ?This looks like must modify xdist somehow since it > > doesn't seem to send any report back at finish. > > We can add a hook or re-use an existing one for allowing a client to send > back (e.g. report) data but the question is: which hook sees this information > on the master side? > > Maybe sessionfinish? ?So the slave's sessionfinish hook could add whatever > extra information and the final sessionfinish sees it. When the slave node is created it sets a slavereport to an empty slavereport object. Client code on the slave can add to the slavereport in any hook that it implements. When the slave finishes it already sends a slavefinished event which now also sends back the slavereport object. The master TXNode stores the slavereport on itself. Client code on the master can implement the pytest_testnodedown hook and access the slavereport. > Maybe it's better to design and call specific xdist-prefied hooks and not re-use > existing ones? ?As you have hacked on this topic and for your purposes > already - what do you think? xdist hooks seem better than the existing ones. Also many slaves send reports back on pytest_testnodedown hook and seems better to deal with just the one immediately rather than build up and pass to pytest_sessionfinish. I implemented and if you think it is good enough would you please be so kind as to pull the changes in and release pytest and xdist. You can change of course if you have a better idea! http://bitbucket.org/memedough/py-trunk/overview http://bitbucket.org/memedough/pytest-xdist/overview :) From holger at merlinux.eu Mon Apr 19 13:04:54 2010 From: holger at merlinux.eu (holger krekel) Date: Mon, 19 Apr 2010 13:04:54 +0200 Subject: [py-dev] xdist sending info between master and slave - support for client code In-Reply-To: <w2wcf54a0ac1004162054jc0fd38a2zdc8f8f2b41f70928@mail.gmail.com> References: <g2jcf54a0ac1004160209x3aebf4bak1c3acac2a3ba283d@mail.gmail.com> <20100416172143.GY26514@trillke.net> <w2wcf54a0ac1004162054jc0fd38a2zdc8f8f2b41f70928@mail.gmail.com> Message-ID: <20100419110454.GC26514@trillke.net> On Sat, Apr 17, 2010 at 13:54 +1000, meme dough wrote: > > Is it possible to modify xdist so that client code on master can send info > > > > > to the slave. ?Will need access to rsync roots before it goes ahead and > > > installs the slave though. > > > > I think we could add a hook called at the master-side which could return/update > > an extra dictionary to be transfered to the client (in theory even a > > pickled-object but i'd like to avoid guaranteeing pickleability for the core > > test distribution mechanism). ?A slave-side hook (sessionstart?) would > > get this information. > > I created a new hook that gets called just before each test node is installed: > > def pytest_testnodeinstall(nodemanager, gateway, slavedata): I think we can just reuse testnodeready and carry a "nodemanager" and a slavedata attribute on each node. Sidenote: at some point i'd like to change the distribution model and thus would like to hide as many xdist-internals as possible for now. The goal for that change is to do test collection on the clients and co-ordinate only their execution and reporting from the master. > Client code on the master can set anything it needs to send to the > slave on the slavedata object. > > The slavedata is passed over the channel in the same way as config > object and is stored on the slave node such that any client code on > the slave can access the slavedata. > It can do this in any hook that > the slave client code needs to implement (provided that it can get > access to the session object which is the slave node object which has > the slavedata). I'd like slavedata to be a simple dictionary so that slave code can access it by "session.slavedata['NAME']". Btw, I actually consider it a bit of a hack that 'session == node' on slaves. But apparently useful :) > > > Is it possible to modify xdist so that client code on the slave can send > > > info back to the master. ?This looks like must modify xdist somehow since it > > > doesn't seem to send any report back at finish. > > > > We can add a hook or re-use an existing one for allowing a client to send > > back (e.g. report) data but the question is: which hook sees this information > > on the master side? > > > > Maybe sessionfinish? ?So the slave's sessionfinish hook could add whatever > > extra information and the final sessionfinish sees it. > > When the slave node is created it sets a slavereport to an empty > slavereport object. > > Client code on the slave can add to the slavereport in any hook that > it implements. only in hooks that know about 'session' or nodes respectively, right? I wonder if we could settle on a one-to-one config<->node relationship on the slave side. Would make the node nicely available. > When the slave finishes it already sends a slavefinished event which > now also sends back the slavereport object. > > The master TXNode stores the slavereport on itself. > > Client code on the master can implement the pytest_testnodedown hook > and access the slavereport. sounds ok enough. > > Maybe it's better to design and call specific xdist-prefied hooks and not re-use > > existing ones? ?As you have hacked on this topic and for your purposes > > already - what do you think? > > xdist hooks seem better than the existing ones. Also many slaves send > reports back on pytest_testnodedown hook and seems better to deal with > just the one immediately rather than build up and pass to > pytest_sessionfinish. Can you finally briefly describe your use case - i.e. for what are you using the new master<->slave data exchange? > I implemented and if you think it is good enough would you please be > so kind as to pull the changes in and release pytest and xdist. You > can change of course if you have a better idea! > > http://bitbucket.org/memedough/py-trunk/overview > > http://bitbucket.org/memedough/pytest-xdist/overview Could you see to implement my above suggestions and see if it still satisfies your use case? (reuse testnodeready, slavedata dict, config<->node relationship on slaves) If i see it right that'd confine changes to the xdist code in which case i'd be willing to release it soon. I'd be happy if you could also add tests for the new functionality. cheers & thanks for the patch and explanations, holger > :) > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > -- From prologic at shortcircuit.net.au Wed Apr 21 04:43:44 2010 From: prologic at shortcircuit.net.au (James Mills) Date: Wed, 21 Apr 2010 12:43:44 +1000 Subject: [py-dev] Testing and Coverage against different python interpreter versions Message-ID: <m2te1a84d571004201943x1fd749eam5e7c4c316d7140bc@mail.gmail.com> Hi All, I am in need of a solution regarding the use of py.test and pytest-coverage (the coverage plugin I help maintain for py.test). The problem is being able to run a test suite against 2 or more different python interpreter versions (eg: 2.5 and 2.6) AND combining their coverage data. Without the above behavior, it would not be possible to achieve 100% code coverage (by tests). Often (at least I do) in situations where you wish to support 2 or more different versions of python you might have: try: import foo # 2.6 import except ImportError: import bar # 2.5 import and if sys.version_info[:2] == (2, 6): # 2.6 code else: # 2.5 code Currently I have no solution for this, I'm hoping others might have some ideas or are already working on something similar that might achieve this. cheers James -- -- "Problems are solved by method" From memedough at gmail.com Wed Apr 21 10:15:51 2010 From: memedough at gmail.com (meme dough) Date: Wed, 21 Apr 2010 18:15:51 +1000 Subject: [py-dev] Testing and Coverage against different python interpreter versions In-Reply-To: <m2te1a84d571004201943x1fd749eam5e7c4c316d7140bc@mail.gmail.com> References: <m2te1a84d571004201943x1fd749eam5e7c4c316d7140bc@mail.gmail.com> Message-ID: <k2hcf54a0ac1004210115k6ef8cb05zd7bd557a36fe7ee4@mail.gmail.com> Hi, I have written a new pytest coverage plugin from the ground up to fully support distributed testing since both pytest and coverage provide good support for doing this. Plus it gives access to pretty much everything that coverage has that I know about. The use case that you state is interesting. Setting dist=load will cause pytest to spread the tests out to many slaves with a single coverage report from their combined data. However each node is running different tests so setting them to different python / platform may not give full coverage. Setting dist=each will cause pytest to gives all tests to each slave with a report from each slave. However each report will just be for the one slave. I had not thought of the case you suggest where it may be desirable to instruct the plugin to actually combine the results from each slave where each slave has run all tests. I will consider this further. :) On 21 April 2010 12:43, James Mills <prologic at shortcircuit.net.au> wrote: > Hi All, > > I am in need of a solution regarding the use of py.test and pytest-coverage > (the coverage plugin I help maintain for py.test). > > The problem is being able to run a test suite against 2 or more different > python interpreter versions (eg: 2.5 and 2.6) AND combining their > coverage data. > > Without the above behavior, it would not be possible to achieve 100% code > coverage (by tests). Often (at least I do) in situations where you > wish to support > 2 or more different versions of python you might have: > > try: > ? import foo # 2.6 import > except ImportError: > ? import bar # 2.5 import > > and > > if sys.version_info[:2] == (2, 6): > ? # 2.6 code > else: > ? # 2.5 code > > Currently I have no solution for this, I'm hoping others might have > some ideas or are already working on something similar that might > achieve this. > > cheers > James > > -- > -- "Problems are solved by method" > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > From prologic at shortcircuit.net.au Wed Apr 21 10:23:37 2010 From: prologic at shortcircuit.net.au (James Mills) Date: Wed, 21 Apr 2010 18:23:37 +1000 Subject: [py-dev] Testing and Coverage against different python interpreter versions In-Reply-To: <k2hcf54a0ac1004210115k6ef8cb05zd7bd557a36fe7ee4@mail.gmail.com> References: <m2te1a84d571004201943x1fd749eam5e7c4c316d7140bc@mail.gmail.com> <k2hcf54a0ac1004210115k6ef8cb05zd7bd557a36fe7ee4@mail.gmail.com> Message-ID: <u2ve1a84d571004210123gb5a9349fhe760e981730d5312@mail.gmail.com> On Wed, Apr 21, 2010 at 6:15 PM, meme dough <memedough at gmail.com> wrote: > I have written a new pytest coverage plugin from the ground up to > fully support distributed testing since both pytest and coverage > provide good support for doing this. ?Plus it gives access to pretty > much everything that coverage has that I know about. Do you have a link to this at all ? Do you intend to maintain and support this new plugin ? I'm happy to drop the one I maintain on bitbucket in favor of yours if need be. > I had not thought of the case you suggest where it may be desirable to > instruct the plugin to actually combine the results from each slave > where each slave has run all tests. > > I will consider this further. Please do as this is an important use-case ihmo (and others agree). (Maybe not for distributed testing, but certainly for testing a library against more than one supported version of python). Look forward to using your work, cheers James From holger at merlinux.eu Wed Apr 21 10:34:02 2010 From: holger at merlinux.eu (holger krekel) Date: Wed, 21 Apr 2010 10:34:02 +0200 Subject: [py-dev] Testing and Coverage against different python interpreter versions In-Reply-To: <u2ve1a84d571004210123gb5a9349fhe760e981730d5312@mail.gmail.com> References: <m2te1a84d571004201943x1fd749eam5e7c4c316d7140bc@mail.gmail.com> <k2hcf54a0ac1004210115k6ef8cb05zd7bd557a36fe7ee4@mail.gmail.com> <u2ve1a84d571004210123gb5a9349fhe760e981730d5312@mail.gmail.com> Message-ID: <20100421083402.GG26514@trillke.net> On Wed, Apr 21, 2010 at 18:23 +1000, James Mills wrote: > On Wed, Apr 21, 2010 at 6:15 PM, meme dough <memedough at gmail.com> wrote: > > I have written a new pytest coverage plugin from the ground up to > > fully support distributed testing since both pytest and coverage > > provide good support for doing this. ?Plus it gives access to pretty > > much everything that coverage has that I know about. > > Do you have a link to this at all ? > Do you intend to maintain and support this new plugin ? > > I'm happy to drop the one I maintain on bitbucket in favor of yours if need be. Let's discuss and review things when they are there. > > I had not thought of the case you suggest where it may be desirable to > > instruct the plugin to actually combine the results from each slave > > where each slave has run all tests. > > > > I will consider this further. > > Please do as this is an important use-case ihmo (and others agree). I often run tests on multiple platforms and i'd appreciate a combined coverage report. Moreover, i'd like to see how to then only report skips when they were skipped on all platforms. This way it'd be easy to identify which platform/interpreter is missing in the testing mix by looking at remaining skips. cheers, holger > (Maybe not for distributed testing, but certainly for testing a library > against more than one supported version of python). > > Look forward to using your work, > > cheers > James > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev -- From memedough at gmail.com Wed Apr 21 14:47:25 2010 From: memedough at gmail.com (meme dough) Date: Wed, 21 Apr 2010 22:47:25 +1000 Subject: [py-dev] xdist sending info between master and slave - support for client code In-Reply-To: <20100419110454.GC26514@trillke.net> References: <g2jcf54a0ac1004160209x3aebf4bak1c3acac2a3ba283d@mail.gmail.com> <20100416172143.GY26514@trillke.net> <w2wcf54a0ac1004162054jc0fd38a2zdc8f8f2b41f70928@mail.gmail.com> <20100419110454.GC26514@trillke.net> Message-ID: <m2kcf54a0ac1004210547pdcee4817ybd9e25fe96af09ee@mail.gmail.com> Hi, > Could you see to implement my above suggestions and see > if it still satisfies your use case? > (reuse testnodeready, slavedata dict, > config<->node relationship on slaves) > > If i see it right that'd confine changes to the xdist code > in which case i'd be willing to release it soon. I changed to use slaveready and changed both the slavedata and slavereport to be a simple dict. Not sure with the relationships on slaves. The hooks just access the session / slave object to get config / slavedata / slavereport. Seems clear to me atm, but maybe I misunderstood. I added test which works, but could maybe be better when it fails. Wasn't sure how to best make a test of pytest with a hook running on a slave fail. What do you think? http://bitbucket.org/memedough/pytest-xdist-slaveready/overview :) From holger at merlinux.eu Wed Apr 21 21:27:24 2010 From: holger at merlinux.eu (holger krekel) Date: Wed, 21 Apr 2010 21:27:24 +0200 Subject: [py-dev] xdist sending info between master and slave - support for client code In-Reply-To: <m2kcf54a0ac1004210547pdcee4817ybd9e25fe96af09ee@mail.gmail.com> References: <g2jcf54a0ac1004160209x3aebf4bak1c3acac2a3ba283d@mail.gmail.com> <20100416172143.GY26514@trillke.net> <w2wcf54a0ac1004162054jc0fd38a2zdc8f8f2b41f70928@mail.gmail.com> <20100419110454.GC26514@trillke.net> <m2kcf54a0ac1004210547pdcee4817ybd9e25fe96af09ee@mail.gmail.com> Message-ID: <20100421192724.GI26514@trillke.net> Hi Meme, On Wed, Apr 21, 2010 at 22:47 +1000, meme dough wrote: > > Could you see to implement my above suggestions and see > > if it still satisfies your use case? > > (reuse testnodeready, slavedata dict, > > config<->node relationship on slaves) > > > > If i see it right that'd confine changes to the xdist code > > in which case i'd be willing to release it soon. > > I changed to use slaveready and changed both the slavedata and > slavereport to be a simple dict. thanks. > Not sure with the relationships on slaves. The hooks just access the > session / slave object to get config / slavedata / slavereport. Seems > clear to me atm, but maybe I misunderstood. Only sessionstart/sessionfinish get a 'session' (node) object while all other hooks can not easily obtain a session reference. However, almost all hooks receive or can easily get to a 'config' object. Therefore i think it's better to stick the slavedata object to the config object. > I added test which works, but could maybe be better when it fails. > Wasn't sure how to best make a test of pytest with a hook running on a > slave fail. > > What do you think? > > http://bitbucket.org/memedough/pytest-xdist-slaveready/overview The test looks fine. Looking at your patch and at things some more i wonder if it isn't eventually better to introduce a new hook - even if it requires a new py.test release. Something like: # runs only on the master side def pytest_xdist_configure_node(node): node.slaveinput[NAME] = value # any hook or test can do the following manipulation if hasattr(config, 'slavedata'): # means we are running slave side config.slaveinput[NAME] # access data sent from the master config.slaveoutput['result'] = 42 # set slave output # runs only on the master side def pytest_xdist_nodedown(node, slaveoutput): node.slaveoutput['result'] # access result from slave I think both internal implementation and usage becomes simpler. What do you think? When doing the a new py.test release i'd like to look into allowing to add per-plugin hooks and systematically name xdist-hooks. If you'd like to push ahead before i finish this you can just add the hooks statically and i patch it out later. cheers, holger From holger at merlinux.eu Thu Apr 22 12:05:25 2010 From: holger at merlinux.eu (holger krekel) Date: Thu, 22 Apr 2010 12:05:25 +0200 Subject: [py-dev] xdist sending info between master and slave - support for client code In-Reply-To: <20100421192724.GI26514@trillke.net> References: <g2jcf54a0ac1004160209x3aebf4bak1c3acac2a3ba283d@mail.gmail.com> <20100416172143.GY26514@trillke.net> <w2wcf54a0ac1004162054jc0fd38a2zdc8f8f2b41f70928@mail.gmail.com> <20100419110454.GC26514@trillke.net> <m2kcf54a0ac1004210547pdcee4817ybd9e25fe96af09ee@mail.gmail.com> <20100421192724.GI26514@trillke.net> Message-ID: <20100422100525.GM26514@trillke.net> On Wed, Apr 21, 2010 at 21:27 +0200, holger krekel wrote: > When doing the a new py.test release i'd like to look into > allowing to add per-plugin hooks and systematically name xdist-hooks. Just did checkins to py-trunk and xdist implementing a first go at this, bumping internal version numbers to 1.2.2 and 1.2 respectively. (hudson testruns are ongoing at http://hudson.testrun.org/view/pytest/) > If you'd like to push ahead before i finish this you can just add the > hooks statically and i patch it out later. It should be easy to add new hooks now to xdist/newhooks.py holger From memedough at gmail.com Thu Apr 22 14:35:12 2010 From: memedough at gmail.com (meme dough) Date: Thu, 22 Apr 2010 22:35:12 +1000 Subject: [py-dev] xdist sending info between master and slave - support for client code In-Reply-To: <20100422100525.GM26514@trillke.net> References: <g2jcf54a0ac1004160209x3aebf4bak1c3acac2a3ba283d@mail.gmail.com> <20100416172143.GY26514@trillke.net> <w2wcf54a0ac1004162054jc0fd38a2zdc8f8f2b41f70928@mail.gmail.com> <20100419110454.GC26514@trillke.net> <m2kcf54a0ac1004210547pdcee4817ybd9e25fe96af09ee@mail.gmail.com> <20100421192724.GI26514@trillke.net> <20100422100525.GM26514@trillke.net> Message-ID: <y2pcf54a0ac1004220535obe97edabh7c5593b936d5b924@mail.gmail.com> Hi, > Just did checkins to py-trunk and xdist implementing a first go at this, > bumping internal version numbers to 1.2.2 and 1.2 respectively. > (hudson testruns are ongoing at http://hudson.testrun.org/view/pytest/) > >> If you'd like to push ahead before i finish this you can just add the >> hooks statically and i patch it out later. > > It should be easy to add new hooks now to xdist/newhooks.py I've pulled in changes and looks good. Another commit to xdist for putting slaveinput / slaveoutput on the config object for slaves. For master remains on the node since many nodes, this also allows checking if running as slave as you pointed out. I renamed to slaveinput / slaveoutput as had in previous email. I also changed the test so that instead of asserts in the hooks on slave or master it will output something at summary and the test process can check. http://bitbucket.org/memedough/pytest-xdist-slaveready-config/changeset/5257145dbd4b An issue though: Last execnet commit is 1.0.6 version bump but not tagged or released on pypi. Hence can't install or develop pytest-xdist. I changed locally but didn't put in this changeset. Not sure if you want to down xdist to point 1.0.5 or release execnet 1.0.6. :) From memedough at gmail.com Thu Apr 22 15:50:57 2010 From: memedough at gmail.com (meme dough) Date: Thu, 22 Apr 2010 23:50:57 +1000 Subject: [py-dev] xdist sending info between master and slave - support for client code In-Reply-To: <20100421192724.GI26514@trillke.net> References: <g2jcf54a0ac1004160209x3aebf4bak1c3acac2a3ba283d@mail.gmail.com> <20100416172143.GY26514@trillke.net> <w2wcf54a0ac1004162054jc0fd38a2zdc8f8f2b41f70928@mail.gmail.com> <20100419110454.GC26514@trillke.net> <m2kcf54a0ac1004210547pdcee4817ybd9e25fe96af09ee@mail.gmail.com> <20100421192724.GI26514@trillke.net> Message-ID: <t2zcf54a0ac1004220650kdb7c7eb5pfb5a912449259685@mail.gmail.com> Hi, > The test looks fine. ?Looking at your patch and at things some more i > wonder if it isn't eventually better to introduce a new hook - even > if it requires a new py.test release. ?Something like: > > ? ?# runs only on the master side > ? ?def pytest_xdist_configure_node(node): > ? ? ? ?node.slaveinput[NAME] = value > > ? ?# any hook or test can do the following manipulation > ? ? ? ?if hasattr(config, 'slavedata'): # means we are running slave side > ? ? ? ? ? ?config.slaveinput[NAME] ? ? ?# access data sent from the master > ? ? ? ? ? ?config.slaveoutput['result'] = 42 ?# set slave output > > ? ?# runs only on the master side > ? ?def pytest_xdist_nodedown(node, slaveoutput): > ? ? ? ?node.slaveoutput['result'] # access result from slave > > I think both internal implementation and usage becomes simpler. > What do you think? It looks good. The hook names can be made consistent as well as consistent args. :) From holger at merlinux.eu Thu Apr 22 17:03:26 2010 From: holger at merlinux.eu (holger krekel) Date: Thu, 22 Apr 2010 17:03:26 +0200 Subject: [py-dev] xdist sending info between master and slave - support for client code In-Reply-To: <y2pcf54a0ac1004220535obe97edabh7c5593b936d5b924@mail.gmail.com> References: <g2jcf54a0ac1004160209x3aebf4bak1c3acac2a3ba283d@mail.gmail.com> <20100416172143.GY26514@trillke.net> <w2wcf54a0ac1004162054jc0fd38a2zdc8f8f2b41f70928@mail.gmail.com> <20100419110454.GC26514@trillke.net> <m2kcf54a0ac1004210547pdcee4817ybd9e25fe96af09ee@mail.gmail.com> <20100421192724.GI26514@trillke.net> <20100422100525.GM26514@trillke.net> <y2pcf54a0ac1004220535obe97edabh7c5593b936d5b924@mail.gmail.com> Message-ID: <20100422150326.GA22675@trillke.net> Hi Meme, On Thu, Apr 22, 2010 at 22:35 +1000, meme dough wrote: > > Just did checkins to py-trunk and xdist implementing a first go at this, > > bumping internal version numbers to 1.2.2 and 1.2 respectively. > > (hudson testruns are ongoing at http://hudson.testrun.org/view/pytest/) > > > >> If you'd like to push ahead before i finish this you can just add the > >> hooks statically and i patch it out later. > > > > It should be easy to add new hooks now to xdist/newhooks.py > > I've pulled in changes and looks good. > > Another commit to xdist for putting slaveinput / slaveoutput on the > config object for slaves. For master remains on the node since many > nodes, this also allows checking if running as slave as you pointed > out. > > I renamed to slaveinput / slaveoutput as had in previous email. > > I also changed the test so that instead of asserts in the hooks on > slave or master it will output something at summary and the test > process can check. > > http://bitbucket.org/memedough/pytest-xdist-slaveready-config/changeset/5257145dbd4b I pulled it and did another patch to use a new configure-node hook. > An issue though: > > Last execnet commit is 1.0.6 version bump but not tagged or released > on pypi. Hence can't install or develop pytest-xdist. I changed > locally but didn't put in this changeset. Not sure if you want to > down xdist to point 1.0.5 or release execnet 1.0.6. Both - xdist now only requires 1.0.5 but i'd like to put out 1.0.6 soonish. A question - do i guess right that your distributed coverage plugin is to use the new slaveinput/slaveoutput mechanism? cheers, holger > :) > -- From memedough at gmail.com Sat Apr 24 03:20:41 2010 From: memedough at gmail.com (meme dough) Date: Sat, 24 Apr 2010 11:20:41 +1000 Subject: [py-dev] xdist sending info between master and slave - support for client code In-Reply-To: <20100422150326.GA22675@trillke.net> References: <g2jcf54a0ac1004160209x3aebf4bak1c3acac2a3ba283d@mail.gmail.com> <20100416172143.GY26514@trillke.net> <w2wcf54a0ac1004162054jc0fd38a2zdc8f8f2b41f70928@mail.gmail.com> <20100419110454.GC26514@trillke.net> <m2kcf54a0ac1004210547pdcee4817ybd9e25fe96af09ee@mail.gmail.com> <20100421192724.GI26514@trillke.net> <20100422100525.GM26514@trillke.net> <y2pcf54a0ac1004220535obe97edabh7c5593b936d5b924@mail.gmail.com> <20100422150326.GA22675@trillke.net> Message-ID: <o2zcf54a0ac1004231820u475049es80ba87073367b19e@mail.gmail.com> Hi, >> > Just did checkins to py-trunk and xdist implementing a first go at this, >> > bumping internal version numbers to 1.2.2 and 1.2 respectively. >> > (hudson testruns are ongoing at http://hudson.testrun.org/view/pytest/) >> > >> >> If you'd like to push ahead before i finish this you can just add the >> >> hooks statically and i patch it out later. >> > >> > It should be easy to add new hooks now to xdist/newhooks.py >> >> I've pulled in changes and looks good. >> >> Another commit to xdist for putting slaveinput / slaveoutput on the >> config object for slaves. ?For master remains on the node since many >> nodes, this also allows checking if running as slave as you pointed >> out. >> >> I renamed to slaveinput / slaveoutput as had in previous email. >> >> I also changed the test so that instead of asserts in the hooks on >> slave or master it will output something at summary and the test >> process can check. >> >> http://bitbucket.org/memedough/pytest-xdist-slaveready-config/changeset/5257145dbd4b > > I pulled it and did another patch to use a new configure-node hook. This new hook is better. I didn't like the second receive to get the slaveinput and having to check on each send. More clear to send slaveinput over at same time as config. >> An issue though: >> >> Last execnet commit is 1.0.6 version bump but not tagged or released >> on pypi. ?Hence can't install or develop pytest-xdist. ?I changed >> locally but didn't put in this changeset. ?Not sure if you want to >> down xdist to point 1.0.5 or release execnet 1.0.6. > > Both - xdist now only requires 1.0.5 but i'd like to put out 1.0.6 soonish. > > A question - do i guess right that your distributed coverage plugin > is to use the new slaveinput/slaveoutput mechanism? Yes, I will move it to the latest xdist now. Thanks for the release, this is really good. :) From memedough at gmail.com Sat Apr 24 16:01:02 2010 From: memedough at gmail.com (meme dough) Date: Sun, 25 Apr 2010 00:01:02 +1000 Subject: [py-dev] Testing and Coverage against different python interpreter versions In-Reply-To: <u2ve1a84d571004210123gb5a9349fhe760e981730d5312@mail.gmail.com> References: <m2te1a84d571004201943x1fd749eam5e7c4c316d7140bc@mail.gmail.com> <k2hcf54a0ac1004210115k6ef8cb05zd7bd557a36fe7ee4@mail.gmail.com> <u2ve1a84d571004210123gb5a9349fhe760e981730d5312@mail.gmail.com> Message-ID: <v2gcf54a0ac1004240701u4dfc94a1pcc3a5f160bb8828@mail.gmail.com> Hi, >> I have written a new pytest coverage plugin from the ground up to >> fully support distributed testing since both pytest and coverage >> provide good support for doing this. ?Plus it gives access to pretty >> much everything that coverage has that I know about. > > Do you have a link to this at all ? > Do you intend to maintain and support this new plugin ? With the latest py and pytest-xdist release the master and slave can exchange information. Until this release I had local mods and suggested patches. Thanks Holger! There are still a number of loose ends to tidy up but it's now up on pypi and bitbucket so you can throw it into a virtualenv and take it for a test run if you like. http://pypi.python.org/pypi/pytest-cover/ http://bitbucket.org/memedough/pytest-cover I will definitely be using and maintaining it. > I'm happy to drop the one I maintain on bitbucket in favor of yours if need be. I didn't know quite what to do about the pytest-coverage repo. I thought of contributing to it, however it's a complete rewrite so it's a bit much to ask to take a patch that probably only has pytest hook lines in common. Then I also kind of didn't like the repo having the figleaf history and the 1.0 tag when that latest tag was only at 0.3. Maybe I'm just being picky and shouldn't... Not sure it should be dropped though, it's there, stable, and probably being used by people. And since I didn't patch the existing one but wrote something new it's currently unproven. Maybe best to see how things stack up after a while, doesn't seem any harm having options - we have figleaf too. Lets just wait for a while - maybe we merge some things and get best of all. >> I had not thought of the case you suggest where it may be desirable to >> instruct the plugin to actually combine the results from each slave >> where each slave has run all tests. >> >> I will consider this further. > > Please do as this is an important use-case ihmo (and others agree). > (Maybe not for distributed testing, but certainly for testing a library > against more than one supported version of python). This is a good suggestion, I added it so you can do dist=each and use --cover-combine-each option to get single combined report from all slaves. Do note however that I haven't tested on all python versions so I probably have some work to do. Should be alright if you're only doing 2.5 / 2.6 as your email suggested though. :) From memedough at gmail.com Sun Apr 25 04:06:00 2010 From: memedough at gmail.com (meme dough) Date: Sun, 25 Apr 2010 12:06:00 +1000 Subject: [py-dev] Testing and Coverage against different python interpreter versions In-Reply-To: <v2gcf54a0ac1004240701u4dfc94a1pcc3a5f160bb8828@mail.gmail.com> References: <m2te1a84d571004201943x1fd749eam5e7c4c316d7140bc@mail.gmail.com> <k2hcf54a0ac1004210115k6ef8cb05zd7bd557a36fe7ee4@mail.gmail.com> <u2ve1a84d571004210123gb5a9349fhe760e981730d5312@mail.gmail.com> <v2gcf54a0ac1004240701u4dfc94a1pcc3a5f160bb8828@mail.gmail.com> Message-ID: <u2ycf54a0ac1004241906t57d5f488x87d8a421c4b446db@mail.gmail.com> Hi, > http://pypi.python.org/pypi/pytest-cover/ > > http://bitbucket.org/memedough/pytest-cover Renamed and moved to: http://pypi.python.org/pypi/pytest-cov http://bitbucket.org/memedough/pytest-cov It now coexists nicely with other plugins. :) From memedough at gmail.com Mon Apr 26 06:49:46 2010 From: memedough at gmail.com (meme dough) Date: Mon, 26 Apr 2010 14:49:46 +1000 Subject: [py-dev] distributed test slave activation of external plugin Message-ID: <v2rcf54a0ac1004252149x7af83806te1d3ce921ff7599e@mail.gmail.com> Hi, Is there any way to activate external plugins for distributed testing on the slave side? Currently pytest-cov requires that it is installed on the slave version of python. This differs from pytest-xdist that will rsync itself across, such that the only requirement on the slave side is python. If pytest-cov is not installed on the slave then the master has sent the slaveinput but the plugin is never activated on the slave hence no coverage. I can rsync pytest-cov over to the slave in same way as py / xdist already get done but unknown if I can activate the plugin. A separate but related issue: It would be good to have some way (cmd line / conftest / api) to tell xdist not to rsync py / xdist to the slave. The case is needed for the client code where it take care of all rsync so second rsync by xdist produce the warning about to two py and two xdist on path. :) From holger at merlinux.eu Mon Apr 26 11:29:39 2010 From: holger at merlinux.eu (holger krekel) Date: Mon, 26 Apr 2010 11:29:39 +0200 Subject: [py-dev] Testing and Coverage against different python interpreter versions In-Reply-To: <u2ycf54a0ac1004241906t57d5f488x87d8a421c4b446db@mail.gmail.com> References: <m2te1a84d571004201943x1fd749eam5e7c4c316d7140bc@mail.gmail.com> <k2hcf54a0ac1004210115k6ef8cb05zd7bd557a36fe7ee4@mail.gmail.com> <u2ve1a84d571004210123gb5a9349fhe760e981730d5312@mail.gmail.com> <v2gcf54a0ac1004240701u4dfc94a1pcc3a5f160bb8828@mail.gmail.com> <u2ycf54a0ac1004241906t57d5f488x87d8a421c4b446db@mail.gmail.com> Message-ID: <20100426092939.GF22675@trillke.net> Hi Meme, On Sun, Apr 25, 2010 at 12:06 +1000, meme dough wrote: > Hi, > > > http://pypi.python.org/pypi/pytest-cover/ > > > > http://bitbucket.org/memedough/pytest-cover > > Renamed and moved to: > > http://pypi.python.org/pypi/pytest-cov > > http://bitbucket.org/memedough/pytest-cov > > It now coexists nicely with other plugins. I looked into the new plugin and have some immediate issues/questions: - ETOOMANYCMDLINEOPTIONS ... could most of the command line options go away if we rather rely on configurability through the config file? - can you add basic (functional or unit) tests? - is there a need to have a pytest_cov package instead of just a single module? cheers, holger From holger at merlinux.eu Mon Apr 26 11:41:09 2010 From: holger at merlinux.eu (holger krekel) Date: Mon, 26 Apr 2010 11:41:09 +0200 Subject: [py-dev] distributed test slave activation of external plugin In-Reply-To: <v2rcf54a0ac1004252149x7af83806te1d3ce921ff7599e@mail.gmail.com> References: <v2rcf54a0ac1004252149x7af83806te1d3ce921ff7599e@mail.gmail.com> Message-ID: <20100426094109.GG22675@trillke.net> Hi Meme, On Mon, Apr 26, 2010 at 14:49 +1000, meme dough wrote: > Hi, > > Is there any way to activate external plugins for distributed testing > on the slave side? it is usually done through "proper" installation or command line specification. > Currently pytest-cov requires that it is installed on the slave > version of python. This differs from pytest-xdist that will rsync > itself across, such that the only requirement on the slave side is > python. execnet and pytest-xdist are pure-python libs and known to work from just being transfered on a directory-level. This is often not true for packages requiring C-extensions. > If pytest-cov is not installed on the slave then the master has sent > the slaveinput but the plugin is never activated on the slave hence no > coverage. Yes, but currently you need to have the 'coverage' package installed anyway. > I can rsync pytest-cov over to the slave in same way as py / xdist > already get done but unknown if I can activate the plugin. We need a way to setup "remote" environments in a more local more controlled manner. This is a major step i'd like to take with xdist: configure automatic 'virtualenv' environments remotely, installing dependencies as needed. It would then be just a matter of ensuring 'pytest-cov' is in the dep list. Installed plugins are automatically discovered through the "standard" setuptools entry points. > A separate but related issue: It would be good to have some way (cmd > line / conftest / api) to tell xdist not to rsync py / xdist to the > slave. The case is needed for the client code where it take care of > all rsync so second rsync by xdist produce the warning about to two py > and two xdist on path. I suggest we move to use an 'xdist.ini' file where this could be specified. There we would also specify the virtualenvironments and global and specific package dependencies. The code that reads this config file and sets up environments should probably be useable outside of pytest as well - it'd be a nice aid for automatic config/install/control of app installs. holger From memedough at gmail.com Mon Apr 26 16:00:17 2010 From: memedough at gmail.com (meme dough) Date: Tue, 27 Apr 2010 00:00:17 +1000 Subject: [py-dev] Testing and Coverage against different python interpreter versions In-Reply-To: <20100426092939.GF22675@trillke.net> References: <m2te1a84d571004201943x1fd749eam5e7c4c316d7140bc@mail.gmail.com> <k2hcf54a0ac1004210115k6ef8cb05zd7bd557a36fe7ee4@mail.gmail.com> <u2ve1a84d571004210123gb5a9349fhe760e981730d5312@mail.gmail.com> <v2gcf54a0ac1004240701u4dfc94a1pcc3a5f160bb8828@mail.gmail.com> <u2ycf54a0ac1004241906t57d5f488x87d8a421c4b446db@mail.gmail.com> <20100426092939.GF22675@trillke.net> Message-ID: <h2tcf54a0ac1004260700m2e7c342arbff84b15a5c6ec60@mail.gmail.com> Hi, > I looked into the new plugin and have some immediate issues/questions: > > - ETOOMANYCMDLINEOPTIONS ... > ?could most of the command line options go away > ?if we rather rely on configurability through the config file? Yes/No, while more cmd options with -h that maybe take up more space, it does give lots of control of coverage. I think this is good. It could move some of them to config file only but I don't think it good to remove completely. The good thing with pytest is the cmd options are auto done for conftest and env var which you can see with the --help-config with no work to do and consistent. Also all options have --cov prefix so clear and no clash. If only downside is more output on -h then seems small con. > - can you add basic (functional or unit) tests? Was early release with loose ends because people expressed an interest, so can test drive and have look. I have tests but are simple compared to real project testing with. Will commit when python versions issues are tidied up. > - is there a need to have a pytest_cov package instead of just a single module? Like this in preparation for rsync and activate plugin since better rsync dir I think, but your other email I like better. For now just require coverage / pytest-cov installed on slaves. But future have slave env setup through normal python way (distutils2 and virtualenv?). I will likely change to just module, but it seems more style thing so not sure if matters too much either way it done. :) From holger at merlinux.eu Mon Apr 26 18:03:31 2010 From: holger at merlinux.eu (holger krekel) Date: Mon, 26 Apr 2010 18:03:31 +0200 Subject: [py-dev] Testing and Coverage against different python interpreter versions In-Reply-To: <h2tcf54a0ac1004260700m2e7c342arbff84b15a5c6ec60@mail.gmail.com> References: <m2te1a84d571004201943x1fd749eam5e7c4c316d7140bc@mail.gmail.com> <k2hcf54a0ac1004210115k6ef8cb05zd7bd557a36fe7ee4@mail.gmail.com> <u2ve1a84d571004210123gb5a9349fhe760e981730d5312@mail.gmail.com> <v2gcf54a0ac1004240701u4dfc94a1pcc3a5f160bb8828@mail.gmail.com> <u2ycf54a0ac1004241906t57d5f488x87d8a421c4b446db@mail.gmail.com> <20100426092939.GF22675@trillke.net> <h2tcf54a0ac1004260700m2e7c342arbff84b15a5c6ec60@mail.gmail.com> Message-ID: <20100426160331.GH22675@trillke.net> On Tue, Apr 27, 2010 at 00:00 +1000, meme dough wrote: > > I looked into the new plugin and have some immediate issues/questions: > > > > - ETOOMANYCMDLINEOPTIONS ... > > ?could most of the command line options go away > > ?if we rather rely on configurability through the config file? > > Yes/No, while more cmd options with -h that maybe take up more space, > it does give lots of control of coverage. I think this is good. It > could move some of them to config file only but I don't think it good > to remove completely. > > The good thing with pytest is the cmd options are auto done for > conftest and env var which you can see with the --help-config with no > work to do and consistent. Also all options have --cov prefix so > clear and no clash. > > If only downside is more output on -h then seems small con. having around 10 options for one plugin makes it hard to know what the canonical usage is. And what happens if you have a mixed config-file/cmdline option situation? Also i doubt that e.g. being able to control output directories for html, xml, annotate needs to be that fine-grained, a '--cov-outputdir=...' is enough IMO. What about a generic --cov-option="...." that can be specified multiple times and can override config-ini values? This way you can (if you want to) still specify details on the command line but generally have a single place and way to specify config for coverage (in the ini file). > > - can you add basic (functional or unit) tests? > > Was early release with loose ends because people expressed an > interest, so can test drive and have look. > > I have tests but are simple compared to real project testing with. > Will commit when python versions issues are tidied up. > > > - is there a need to have a pytest_cov package instead of just a single module? > > Like this in preparation for rsync and activate plugin since better > rsync dir I think, but your other email I like better. For now just > require coverage / pytest-cov installed on slaves. But future have > slave env setup through normal python way (distutils2 and > virtualenv?). > > I will likely change to just module, but it seems more style thing so > not sure if matters too much either way it done. great, thanks for getting back on the issues. holger From holger at merlinux.eu Sun May 2 15:12:13 2010 From: holger at merlinux.eu (holger krekel) Date: Sun, 2 May 2010 15:12:13 +0200 Subject: [py-dev] execnet-1.0.6 released (jython/win32 and race condition fixes) Message-ID: <20100502131213.GD22675@trillke.net> execnet-1.0.6 is a backward compatible release fixing jython/win32 and general race condition issues. execnet is a small, stable and well documented pure-python library for automatically deploying and interacting with clusters of Python interpreters. It seamlessly instantiates remote interpreters through the 'ssh' command line tool or socket connections. It supports interactions between Python 2.4 through to 3.1, Jython-2.5.1 and pypy-c, therefore enabling generic Python-to-Python bridging. More info here: http://codespeak.net/execnet/ cheers, holger From holger at merlinux.eu Tue May 4 17:54:15 2010 From: holger at merlinux.eu (holger krekel) Date: Tue, 4 May 2010 17:54:15 +0200 Subject: [py-dev] extending "-p" / plugin pre-loading? Message-ID: <20100504155415.GC17693@trillke.net> Hi all, i am preparing a py/test-1.3.0 release and stumbled across the need and idea to extend the '-p' option - comprising these usages: py.test -p NAME # import plugin with the given name py.test -p path/to/conftest.py # pre-load this conftest.py file py.test -p path/to/NAME.py # load given file as a plugin The first line works with py-1.2.1 already. The other two would be new and useful if you have some special plugin/test running modes you only want to enable explicitely. The specified conftest/plugin code gets loaded before command line parsing so it can add new cmdline options already. However it turns out I don't currently have the need myself for this extension anymore and wonder if anybody is interested in this. If not i just keep my local patch in some attic. best, holger From holger at merlinux.eu Wed May 5 21:45:33 2010 From: holger at merlinux.eu (holger krekel) Date: Wed, 5 May 2010 21:45:33 +0200 Subject: [py-dev] py.test-1.3.0: new options, per-plugin hooks, better reporting Message-ID: <20100505194533.GP17693@trillke.net> py.test/pylib 1.3.0: new options, per-plugin hooks, fixes ... =========================================================================== The py.test/pylib 1.3.0 release introces new options, bug fixes and improved compatibility with Python3 and Jython-2.5.1 on Windows. If you already use py-1.2 chances are you can just use py-1.3.0. See the below CHANGELOG for more details and http://pylib.org/install.html for installation instructions. py.test is an advanced automated testing tool working with Python2, Python3, Jython and PyPy versions on all major operating systems. It offers a no-boilerplate testing approach and has inspired other testing tools and enhancements in the standard Python library for more than five years. It has a simple and extensive plugin architecture, configurable reporting and provides unique ways to make it fit to your testing process and needs. See http://pytest.org for more info. cheers and have fun, holger krekel Changes between 1.2.1 and 1.3.0 ================================================== - deprecate --report option in favour of a new shorter and easier to remember -r option: it takes a string argument consisting of any combination of 'xfsX' characters. They relate to the single chars you see during the dotted progress printing and will print an extra line per test at the end of the test run. This extra line indicates the exact position or test ID that you directly paste to the py.test cmdline in order to re-run a particular test. - allow external plugins to register new hooks via the new pytest_addhooks(pluginmanager) hook. The new release of the pytest-xdist plugin for distributed and looponfailing testing requires this feature. - add a new pytest_ignore_collect(path, config) hook to allow projects and plugins to define exclusion behaviour for their directory structure - for example you may define in a conftest.py this method:: def pytest_ignore_collect(path): return path.check(link=1) to prevent even collection of any tests in symlinked dirs. - new pytest_pycollect_makemodule(path, parent) hook for allowing customization of the Module collection object for a matching test module. - extend and refine xfail mechanism: ``@py.test.mark.xfail(run=False)`` do not run the decorated test ``@py.test.mark.xfail(reason="...")`` prints the reason string in xfail summaries specifiying ``--runxfail`` on command line virtually ignores xfail markers - expose (previously internal) commonly useful methods: py.io.get_terminal_with() -> return terminal width py.io.ansi_print(...) -> print colored/bold text on linux/win32 py.io.saferepr(obj) -> return limited representation string - expose test outcome related exceptions as py.test.skip.Exception, py.test.raises.Exception etc., useful mostly for plugins doing special outcome interpretation/tweaking - (issue85) fix junitxml plugin to handle tests with non-ascii output - fix/refine python3 compatibility (thanks Benjamin Peterson) - fixes for making the jython/win32 combination work, note however: jython2.5.1/win32 does not provide a command line launcher, see http://bugs.jython.org/issue1491 . See pylib install documentation for how to work around. - fixes for handling of unicode exception values and unprintable objects - (issue87) fix unboundlocal error in assertionold code - (issue86) improve documentation for looponfailing - refine IO capturing: stdin-redirect pseudo-file now has a NOP close() method - ship distribute_setup.py version 0.6.10 - added links to the new capturelog and coverage plugins From holger at merlinux.eu Wed May 5 21:53:18 2010 From: holger at merlinux.eu (holger krekel) Date: Wed, 5 May 2010 21:53:18 +0200 Subject: [py-dev] pytest-xdist 1.2: distributed testing getting stable Message-ID: <20100505195318.GQ17693@trillke.net> pytest-xdist 1.2 brings bug fixes and improved stability for distributed testing with py.test. This plugin allows to ad-hoc distribute test runs to multiple CPUs, different Python interpreters or remote machines. It requires setuptools or distribute which help to pull in the neccessary execnet and pytest-core dependencies. See here for extensive docs: http://pytest.org/plugin/xdist.html best, holger 1.2 ------------------------- - fix issue79: sessionfinish/teardown hooks are now called systematically on the slave side - introduce a new data input/output mechanism to allow the master side to send and receive data from a slave. - fix race condition in underlying pickling/unpickling handling - use and require new register hooks facility of py.test>=1.3.0 - require improved execnet>=1.0.6 because of various race conditions that can arise in xdist testing modes. - fix some python3 related pickling related race conditions - fix PyPI description From adrian.dziubek at gmail.com Fri May 7 11:51:30 2010 From: adrian.dziubek at gmail.com (Adrian Dziubek) Date: Fri, 7 May 2010 11:51:30 +0200 Subject: [py-dev] [py.test] import context problems (causes Django unit test failure) Message-ID: <r2p6345f8901005070251sc6b410bez431d9674d71835c0@mail.gmail.com> I posted the following on StackOverflow: http://stackoverflow.com/questions/2783090/py-test-import-context-problems-causes-django-unit-test-failure Please, take a look. Thanks in advance! -- Adrian From holger at merlinux.eu Sun May 9 11:25:18 2010 From: holger at merlinux.eu (holger krekel) Date: Sun, 9 May 2010 11:25:18 +0200 Subject: [py-dev] [py.test] import context problems (causes Django unit test failure) In-Reply-To: <r2p6345f8901005070251sc6b410bez431d9674d71835c0@mail.gmail.com> References: <r2p6345f8901005070251sc6b410bez431d9674d71835c0@mail.gmail.com> Message-ID: <20100509092518.GT17693@trillke.net> Hi Adrian, For me on both py-1.2.1 and py-1.3.0 things seems to work fine when i type in your example (on a a Mac machine). could you zip up the exact directory? best & greetings from Pycon Italy :) holger On Fri, May 07, 2010 at 11:51 +0200, Adrian Dziubek wrote: > I posted the following on StackOverflow: > > http://stackoverflow.com/questions/2783090/py-test-import-context-problems-causes-django-unit-test-failure > > Please, take a look. Thanks in advance! > -- > Adrian > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > -- From adrian.dziubek at gmail.com Sun May 9 14:49:32 2010 From: adrian.dziubek at gmail.com (Adrian Dziubek) Date: Sun, 9 May 2010 14:49:32 +0200 Subject: [py-dev] [py.test] import context problems (causes Django unit test failure) In-Reply-To: <20100509092518.GT17693@trillke.net> References: <r2p6345f8901005070251sc6b410bez431d9674d71835c0@mail.gmail.com> <20100509092518.GT17693@trillke.net> Message-ID: <y2h6345f8901005090549x442e9413lcae4eabaf0fb2b40@mail.gmail.com> 2010/5/9 holger krekel <holger at merlinux.eu>: > Hi Adrian, > > For me on both py-1.2.1 and py-1.3.0 things seems to work fine when > i type in your example (on a a Mac machine). > could you zip up the exact directory? I did the test on my Linux laptop and it works fine too, strange... I'll try again tomorrow at work, where I left the original example. It's probably some setup problem then. Thanks for answering. -- Adrian From adrian.dziubek at gmail.com Mon May 10 13:19:07 2010 From: adrian.dziubek at gmail.com (Adrian Dziubek) Date: Mon, 10 May 2010 13:19:07 +0200 Subject: [py-dev] [py.test] import context problems (causes Django unit test failure) In-Reply-To: <y2h6345f8901005090549x442e9413lcae4eabaf0fb2b40@mail.gmail.com> References: <r2p6345f8901005070251sc6b410bez431d9674d71835c0@mail.gmail.com> <20100509092518.GT17693@trillke.net> <y2h6345f8901005090549x442e9413lcae4eabaf0fb2b40@mail.gmail.com> Message-ID: <l2y6345f8901005100419g4979e0cbu4920c11943e42870@mail.gmail.com> 2010/5/9 Adrian Dziubek <adrian.dziubek at gmail.com>: > 2010/5/9 holger krekel <holger at merlinux.eu>: >> Hi Adrian, >> >> For me on both py-1.2.1 and py-1.3.0 things seems to work fine when >> i type in your example (on a a Mac machine). >> could you zip up the exact directory? > I did the test on my Linux laptop and it works fine too, strange... > I'll try again tomorrow at work, where I left the original example. > It's probably some setup problem then. Thanks for answering. I cannot replicate it any more at work. I found a __init__.py file in moduletest directory, but I can remember explicitly caring about not putting it there... I must have been tired on Friday. I also realised that Django puts __init__.py in the project directory, so it's no wonder, that py.test treats the whole project as a package. I'm still having problems with reverse() inside Django, but it seems irrelevant to py.test. Sorry for bothering, I'll remove the StackOverflow question too. -- Adrian From holger at merlinux.eu Tue May 11 11:19:24 2010 From: holger at merlinux.eu (holger krekel) Date: Tue, 11 May 2010 11:19:24 +0200 Subject: [py-dev] [py.test] import context problems (causes Django unit test failure) In-Reply-To: <l2y6345f8901005100419g4979e0cbu4920c11943e42870@mail.gmail.com> References: <r2p6345f8901005070251sc6b410bez431d9674d71835c0@mail.gmail.com> <20100509092518.GT17693@trillke.net> <y2h6345f8901005090549x442e9413lcae4eabaf0fb2b40@mail.gmail.com> <l2y6345f8901005100419g4979e0cbu4920c11943e42870@mail.gmail.com> Message-ID: <20100511091924.GW17693@trillke.net> On Mon, May 10, 2010 at 13:19 +0200, Adrian Dziubek wrote: > 2010/5/9 Adrian Dziubek <adrian.dziubek at gmail.com>: > > 2010/5/9 holger krekel <holger at merlinux.eu>: > >> Hi Adrian, > >> > >> For me on both py-1.2.1 and py-1.3.0 things seems to work fine when > >> i type in your example (on a a Mac machine). > >> could you zip up the exact directory? > > I did the test on my Linux laptop and it works fine too, strange... > > I'll try again tomorrow at work, where I left the original example. > > It's probably some setup problem then. Thanks for answering. > > I cannot replicate it any more at work. I found a __init__.py file in > moduletest directory, but I can remember explicitly caring about not > putting it there... I must have been tired on Friday. I also realised > that Django puts __init__.py in the project directory, so it's no > wonder, that py.test treats the whole project as a package. > > I'm still having problems with reverse() inside Django, but it seems > irrelevant to py.test. > > Sorry for bothering, I'll remove the StackOverflow question too. thanks for getting back on this. Maybe it would help if py.test would be more verbose about "test package root" detection or even let projects define this explicitely. best, holger From simon.callan at infoshare-is.com Tue May 11 11:44:27 2010 From: simon.callan at infoshare-is.com (Simon Callan) Date: Tue, 11 May 2010 10:44:27 +0100 Subject: [py-dev] Setup and teardown functions Message-ID: <001e01caf0ee$8be1bc70$a3a53550$@callan@infoshare-is.com> Hi, I'm trying to understand how to implement setup() and teardown() functions, but finding it a bit difficult. What I want is the following: 1) A setup() teardown() executed when the test process starts and stops. 2) A setup() teardown() executed when a group of tests are started /finished. 3) A setup() / teardown() executed when a test is run. I.e., something like this: global_setup() group1_setup() setup() test_1() teardown() setup() test_2() teardown() setup() test_3() teardown() group1_teardown() group2_setup() setup() test_4() teardown() setup() test_5() teardown() setup() test_6() teardown() group2_teardown() global_teardown() Is this possible? The documentation implies that it should be possible, but I cannot figure it out. Simon Infoshare Ltd Millennium House 21 Eden Street Kingston upon Thames Surrey KT1 1BL United Kingdom Phone: + 44 (0) 20 8541 0111 Support: + 44 (0) 20 8481 4760 Web: www.infoshare-is.com E-mail: info at infoshare-is.com Infoshare Ltd is registered in England and Wales. Registered Office as above. Registered Number 2877612 VAT Number GB 678 1443 10 The content of this e-mail (and any attachment to it) is confidential. Any views or opinions do not represent the views or opinions of Infoshare Ltd. If you have received this e-mail in error please notify the sender and delete it. You may not use, copy or disclose the information in any way. Infoshare Ltd monitors incoming and outgoing e-mails. Please consider the environment. Do you really need to print this email? From holger at merlinux.eu Tue May 11 12:02:24 2010 From: holger at merlinux.eu (holger krekel) Date: Tue, 11 May 2010 12:02:24 +0200 Subject: [py-dev] Setup and teardown functions In-Reply-To: <001e01caf0ee$8be1bc70$a3a53550$@callan@infoshare-is.com> References: <001e01caf0ee$8be1bc70$a3a53550$@callan@infoshare-is.com> Message-ID: <20100511100224.GZ17693@trillke.net> Hi Simon, i think i can whip up an example. One question though: Do you already have a view on how you want to to group tests in your source code? In principle you can do this by classes/modules or by decorators - or by having custom ways. cheers, holger On Tue, May 11, 2010 at 10:44 +0100, Simon Callan wrote: > Hi, > > I'm trying to understand how to implement setup() and teardown() functions, > but finding it a bit difficult. > > What I want is the following: > 1) A setup() teardown() executed when the test process starts and stops. > 2) A setup() teardown() executed when a group of tests are started > /finished. > 3) A setup() / teardown() executed when a test is run. > > I.e., something like this: > global_setup() > group1_setup() > setup() > test_1() > teardown() > setup() > test_2() > teardown() > setup() > test_3() > teardown() > group1_teardown() > group2_setup() > setup() > test_4() > teardown() > setup() > test_5() > teardown() > setup() > test_6() > teardown() > group2_teardown() > global_teardown() > > Is this possible? The documentation implies that it should be possible, but > I cannot figure it out. > > Simon > > Infoshare Ltd > Millennium House > 21 Eden Street > Kingston upon Thames > Surrey > KT1 1BL > United Kingdom > > Phone: + 44 (0) 20 8541 0111 > Support: + 44 (0) 20 8481 4760 > Web: www.infoshare-is.com > E-mail: info at infoshare-is.com > > Infoshare Ltd is registered in England and Wales. > Registered Office as above. > Registered Number 2877612 > VAT Number GB 678 1443 10 > > The content of this e-mail (and any attachment to it) is confidential. > Any views or opinions do not represent the views or opinions > of Infoshare Ltd. > If you have received this e-mail in error please notify the sender > and delete it. You may not use, copy or disclose the information > in any way. > > Infoshare Ltd monitors incoming and outgoing e-mails. > > Please consider the environment. Do you really need to print > this email? > > > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > -- From simon.callan at infoshare-is.com Tue May 11 12:32:30 2010 From: simon.callan at infoshare-is.com (Simon Callan) Date: Tue, 11 May 2010 11:32:30 +0100 Subject: [py-dev] Setup and teardown functions In-Reply-To: <20100511100224.GZ17693@trillke.net> References: <001e01caf0ee$8be1bc70$a3a53550$@callan@infoshare-is.com> <20100511100224.GZ17693@trillke.net> Message-ID: <001f01caf0f5$426bb850$c74328f0$@callan@infoshare-is.com> Hi, > i think i can whip up an example. One question though: Do you already have a > view on how you want to to group tests in your source code? In principle you > can do this by classes/modules or by decorators - or by having custom ways. We're still at the early stages of laying out our tests, so we're pretty free-form at the moment, however, I think the easiest way to do it would be to have all the test functions in a module to be considered part of a group. Simon Infoshare Ltd Millennium House 21 Eden Street Kingston upon Thames Surrey KT1 1BL United Kingdom Phone: + 44 (0) 20 8541 0111 Support: + 44 (0) 20 8481 4760 Web: www.infoshare-is.com E-mail: info at infoshare-is.com Infoshare Ltd is registered in England and Wales. Registered Office as above. Registered Number 2877612 VAT Number GB 678 1443 10 The content of this e-mail (and any attachment to it) is confidential. Any views or opinions do not represent the views or opinions of Infoshare Ltd. If you have received this e-mail in error please notify the sender and delete it. You may not use, copy or disclose the information in any way. Infoshare Ltd monitors incoming and outgoing e-mails. Please consider the environment. Do you really need to print this email? From holger at merlinux.eu Tue May 11 15:33:07 2010 From: holger at merlinux.eu (holger krekel) Date: Tue, 11 May 2010 15:33:07 +0200 Subject: [py-dev] Setup and teardown functions In-Reply-To: <001f01caf0f5$426bb850$c74328f0$@callan@infoshare-is.com> References: <20100511100224.GZ17693@trillke.net> <001f01caf0f5$426bb850$c74328f0$@callan@infoshare-is.com> Message-ID: <20100511133306.GA17693@trillke.net> Hi Simon, i attach an initial solution as a tar.gz file: * tests/conftest.py : the per-project file to put your configuration and test setup machinery in * tests/test_module1.py a test module with a test function and a test method * tests/test_module2.py another test module for demo purposes If you untar and run "py.test", a 'setup.log' file gets created which visualizes setup and teardown invocations: global_setup() group_setup('tests.test_module1') setup() executing function 'test_function' teardown() setup() executing method 'test_method' teardown() group_teardown('tests.test_module1') group_setup('tests.test_module2') setup() executing function 'test_function' teardown() setup() executing method 'test_method' teardown() group_teardown('tests.test_module2') global_teardown() A few notes which you might find helpful for understanding the solution: * each test functions needs to note down the 'mysetup' argument. It gives each test function to global/group/per-test function setup state in case you acually do anything in the setup/teardown functions other than logging and want the tests to access it :) If you don't specify the 'mysetup' argument your tests will still run but not trigger any setup/teardown functions - might be useful for pure unit-tests that don't depend on prepared state. * funcargs, the request object and the cached_setup() method are documented in http://pytest.org/funcargs.html * you may choose to optionally call "inlined" setup/teardown methods, i.e. ones that are defined in the test module; this can be done easily because the setup functions in the conftest.py file have a reference to the module object (request.module) or test functions (request.function). You may alternatively choose to declare test configuration options in a test module or per decorators to test functions and use this to drive the setup/teardown machinery, i.e. put into a test module a variable like: group_config = dict(key1=value1, ...) and interpret it from the hooks in the conftest.py. And maybe a final note: the mechanism i noted down requires conftest.py writers to understand py.test internals but test module writers can remain ignorant about it. Please let me know if this makes sense to you. best, holger On Tue, May 11, 2010 at 11:32 +0100, Simon Callan wrote: > Hi, > > > i think i can whip up an example. One question though: Do you already > have a > > view on how you want to to group tests in your source code? In principle > you > > can do this by classes/modules or by decorators - or by having custom > ways. > > We're still at the early stages of laying out our tests, so we're pretty > free-form at the moment, however, I think the easiest way to do it would be > to have all the test functions in a module to be considered part of a group. > > Simon > > Infoshare Ltd > Millennium House > 21 Eden Street > Kingston upon Thames > Surrey > KT1 1BL > United Kingdom > > Phone: + 44 (0) 20 8541 0111 > Support: + 44 (0) 20 8481 4760 > Web: www.infoshare-is.com > E-mail: info at infoshare-is.com > > Infoshare Ltd is registered in England and Wales. > Registered Office as above. > Registered Number 2877612 > VAT Number GB 678 1443 10 > > The content of this e-mail (and any attachment to it) is confidential. > Any views or opinions do not represent the views or opinions > of Infoshare Ltd. > If you have received this e-mail in error please notify the sender > and delete it. You may not use, copy or disclose the information > in any way. > > Infoshare Ltd monitors incoming and outgoing e-mails. > > Please consider the environment. Do you really need to print > this email? > > > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > -- -------------- next part -------------- A non-text attachment was scrubbed... Name: setup-example.tar.gz Type: application/octet-stream Size: 722 bytes Desc: not available Url : http://codespeak.net/pipermail/py-dev/attachments/20100511/23eb2a9b/attachment.obj From simon.callan at infoshare-is.com Tue May 11 18:38:32 2010 From: simon.callan at infoshare-is.com (Simon Callan) Date: Tue, 11 May 2010 17:38:32 +0100 Subject: [py-dev] Setup and teardown functions In-Reply-To: <20100511133306.GA17693@trillke.net> References: <20100511100224.GZ17693@trillke.net> <001f01caf0f5$426bb850$c74328f0$@callan@infoshare-is.com> <20100511133306.GA17693@trillke.net> Message-ID: <004801caf128$64b638d0$2e22aa70$@callan@infoshare-is.com> Hi, > i attach an initial solution as a tar.gz file: > > * tests/conftest.py : the per-project file to put your configuration > and test setup machinery in > * tests/test_module1.py a test module with a test function and a test method > * tests/test_module2.py another test module for demo purposes ... > Please let me know if this makes sense to you. This appears to be just what I wanted, and what I was slowly groping toward myself. I now have another question. Normally, py.test is started using the executable. Is it possible to start it from a python script? This would make it a lot easier to debug the test scripts, as I could set breakpoints, etc using PyDev. Simon Infoshare Ltd Millennium House 21 Eden Street Kingston upon Thames Surrey KT1 1BL United Kingdom Phone: + 44 (0) 20 8541 0111 Support: + 44 (0) 20 8481 4760 Web: www.infoshare-is.com E-mail: info at infoshare-is.com Infoshare Ltd is registered in England and Wales. Registered Office as above. Registered Number 2877612 VAT Number GB 678 1443 10 The content of this e-mail (and any attachment to it) is confidential. Any views or opinions do not represent the views or opinions of Infoshare Ltd. If you have received this e-mail in error please notify the sender and delete it. You may not use, copy or disclose the information in any way. Infoshare Ltd monitors incoming and outgoing e-mails. Please consider the environment. Do you really need to print this email? From sdoherty at avvasi.com Tue May 11 19:30:37 2010 From: sdoherty at avvasi.com (Stuart Doherty) Date: Tue, 11 May 2010 13:30:37 -0400 Subject: [py-dev] Setup and teardown functions In-Reply-To: <004801caf128$64b638d0$2e22aa70$@callan@infoshare-is.com> References: <20100511100224.GZ17693@trillke.net> <001f01caf0f5$426bb850$c74328f0$@callan@infoshare-is.com> <20100511133306.GA17693@trillke.net>, <004801caf128$64b638d0$2e22aa70$@callan@infoshare-is.com> Message-ID: <02AF59B4F3B6F242A5F08CCACBBF9BD4087C667FF2@air.MATTER.LOCAL> Hi Simon, I'm doing just that by importing the "py" package, and invoking py.test with: >>>py.test.cmdline.main([]]) The argument to the "main" function is a python list of the arguments you would pass at the shell prompt. I've been able to debug using pydev/eclipse. Stu ________________________________________ From: py-dev-bounces at codespeak.net [py-dev-bounces at codespeak.net] On Behalf Of Simon Callan [simon.callan at infoshare-is.com] Sent: May 11, 2010 12:38 PM To: py-dev at codespeak.net Subject: Re: [py-dev] Setup and teardown functions Hi, > i attach an initial solution as a tar.gz file: > > * tests/conftest.py : the per-project file to put your configuration > and test setup machinery in > * tests/test_module1.py a test module with a test function and a test method > * tests/test_module2.py another test module for demo purposes ... > Please let me know if this makes sense to you. This appears to be just what I wanted, and what I was slowly groping toward myself. I now have another question. Normally, py.test is started using the executable. Is it possible to start it from a python script? This would make it a lot easier to debug the test scripts, as I could set breakpoints, etc using PyDev. Simon Infoshare Ltd Millennium House 21 Eden Street Kingston upon Thames Surrey KT1 1BL United Kingdom Phone: + 44 (0) 20 8541 0111 Support: + 44 (0) 20 8481 4760 Web: www.infoshare-is.com E-mail: info at infoshare-is.com Infoshare Ltd is registered in England and Wales. Registered Office as above. Registered Number 2877612 VAT Number GB 678 1443 10 The content of this e-mail (and any attachment to it) is confidential. Any views or opinions do not represent the views or opinions of Infoshare Ltd. If you have received this e-mail in error please notify the sender and delete it. You may not use, copy or disclose the information in any way. Infoshare Ltd monitors incoming and outgoing e-mails. Please consider the environment. Do you really need to print this email? _______________________________________________ py-dev mailing list py-dev at codespeak.net http://codespeak.net/mailman/listinfo/py-dev From holger at merlinux.eu Tue May 11 19:56:29 2010 From: holger at merlinux.eu (holger krekel) Date: Tue, 11 May 2010 19:56:29 +0200 Subject: [py-dev] Setup and teardown functions In-Reply-To: <004801caf128$64b638d0$2e22aa70$@callan@infoshare-is.com> References: <20100511100224.GZ17693@trillke.net> <20100511133306.GA17693@trillke.net> <004801caf128$64b638d0$2e22aa70$@callan@infoshare-is.com> Message-ID: <20100511175629.GC17693@trillke.net> On Tue, May 11, 2010 at 17:38 +0100, Simon Callan wrote: > > i attach an initial solution as a tar.gz file: > > > > * tests/conftest.py : the per-project file to put your configuration > > and test setup machinery in > > * tests/test_module1.py a test module with a test function and a test > method > > * tests/test_module2.py another test module for demo purposes > ... > > Please let me know if this makes sense to you. > > This appears to be just what I wanted, and what I was slowly groping toward > myself. > > I now have another question. > > Normally, py.test is started using the executable. Is it possible to start > it from a python script? This would make it a lot easier to debug the test > scripts, as I could set breakpoints, etc using PyDev. In principle you can put this in a script: import py py.test.cmdline.main() or if you want to repeatedly run tests in the same process you may use this helper function: https://bitbucket.org/hpk42/py-trunk/src/tip/contrib/runtesthelper.py I also just committed a fix that makes the runtesthelper script unneccessary - so in >=1.3.1 you will just be able to call py.test.cmdline.main() multiple times in the same process. best, holger From issues-noreply at bitbucket.org Wed May 12 11:14:14 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Wed, 12 May 2010 09:14:14 -0000 Subject: [py-dev] New issue 93 in py-trunk: stdout should be captured while importing conftest.py Message-ID: <482ec0488b4e01c046cef853fb32f476@bitbucket.org> --- you can reply above this line --- New issue 93: stdout should be captured while importing conftest.py http://bitbucket.org/hpk42/py-trunk/issue/93/stdout-should-be-captured-while-importing amauryfa on Wed, 12 May 2010 11:14:14 +0200: Description: With py.test 1.1.1, conftest.py was imported during the collection phase, and stdout was in 'captured' mode. But with 1.3.0, stdout is not captured any more. This makes a difference in pypy, where importing some package have side-effects (configuration, compilation attempts, etc.) that prints several lines. These lines should be masked, and output captured, like it is in setup_module(), for example. Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Wed May 12 11:59:49 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Wed, 12 May 2010 09:59:49 -0000 Subject: [py-dev] New issue 94 in py-trunk: py.test can stop with no output Message-ID: <783856a912d1e21a209823faaad927b7@bitbucket.org> --- you can reply above this line --- New issue 94: py.test can stop with no output http://bitbucket.org/hpk42/py-trunk/issue/94/pytest-can-stop-with-no-output amauryfa on Wed, 12 May 2010 11:59:48 +0200: Description: To reproduce: in current pypy (r74497), add a new AppTest: {{{ #!python class AppTest: def test_raises(self): raises(ValueError, "x") }}} You may have to run the test twice to get the problem. This triggers three bugs in various places: * pypy uses __file__ to create traceback entries, which may end in '.pyc' (the normal way is to use co_filename, which is always a '.py') * then py.test attempts to get the source code, but opens the .pyc file, and calls syntax_checker(). This fails of course and leads to a pile of INTERNALERROR lines. * then pytest_capture is lost: pytest_runtest_makereport failed, suspendcapture() is not called, and the next call to resumecapture() will raise the error "cannot resume, already capturing with 'no'". I think all three places should be modified (and it's not only pypy's fault!) Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Fri May 14 12:05:44 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Fri, 14 May 2010 10:05:44 -0000 Subject: [py-dev] New issue 95 in py-trunk: py.test will not start if zlib is not available Message-ID: <cc5a95b87d0ea6b54004c672ea740e23@bitbucket.org> --- you can reply above this line --- New issue 95: py.test will not start if zlib is not available http://bitbucket.org/hpk42/py-trunk/issue/95/pytest-will-not-start-if-zlib-is-not amauryfa on Fri, 14 May 2010 12:05:43 +0200: Description: if the zlib module is not available (because it's a custom build of python or pypy), py.test will fail to import the pytest_genscript plugin, and nothing will work. if zlib is not available, the "--genscript" option cannot work, but this should not prevent py.test from running tests. I suggest a line like the following: {{{ zlib = py.test.importorskip("zlib") }}} Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Sun May 16 23:30:31 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Sun, 16 May 2010 21:30:31 -0000 Subject: [py-dev] New issue 96 in py-trunk: AssertionError when cancelling a test Message-ID: <bf4efbf17756e47df049f23f3fce2fd0@bitbucket.org> --- you can reply above this line --- New issue 96: AssertionError when cancelling a test http://bitbucket.org/hpk42/py-trunk/issue/96/assertionerror-when-cancelling-a-test amauryfa on Sun, 16 May 2010 23:30:30 +0200: Description: I got this traceback when pressing Ctrl-C. This happens from time to time, and there is not line "!!! KEYBOARD INTERRUPT !!!" {{{ module\cpyext\test\test_typeobject.py .......... module\cpyext\test\test_unicodeobject.py Traceback (most recent call last): File "C:\dev\pypy\trunk\pypy\test_all.py", line 6, in ? py.cmdline.pytest() File "C:\dev\pypy\trunk\py\_test\cmdline.py", line 17, in main exitstatus = session.main(colitems) File "C:\dev\pypy\trunk\py\_test\session.py", line 113, in main self.config.hook.pytest_keyboard_interrupt(excinfo=excinfo) File "C:\dev\pypy\trunk\py\_test\pluginmanager.py", line 347, in __call__ return self.hookrelay._performcall(self.name, mc) File "C:\dev\pypy\trunk\py\_test\pluginmanager.py", line 333, in _performcall return multicall.execute() File "C:\dev\pypy\trunk\py\_test\pluginmanager.py", line 244, in execute res = method(**kwargs) File "C:\dev\pypy\trunk\py\_plugin\pytest_capture.py", line 232, in pytest_keyboard_interrupt self.suspendcapture() File "C:\dev\pypy\trunk\py\_plugin\pytest_capture.py", line 170, in suspendcapture outerr = cap.suspend() File "C:\dev\pypy\trunk\py\_io\capture.py", line 166, in suspend assert not hasattr(self, '_suspended') py._code.assertion.AssertionError: assert not hasattr(<py._io.capture.StdCaptureFD object at 0x01AC9250>, '_suspended') }}} Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Mon May 17 11:59:40 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Mon, 17 May 2010 09:59:40 -0000 Subject: [py-dev] New issue 97 in py-trunk: Dynamic distribution of tests on different hosts Message-ID: <8edcd9abd0a7c415b0bcf9de364b295b@bitbucket.org> --- you can reply above this line --- New issue 97: Dynamic distribution of tests on different hosts http://bitbucket.org/hpk42/py-trunk/issue/97/dynamic-distribution-of-tests-on-different Noufal Ibrahim / nibrahim on Mon, 17 May 2010 11:59:40 +0200: Description: Right now, the host setup is done globally upfront. This means that an invocation of py.test can send tests only to a single set of hosts. It would be nice to be able to customise this in the `conftest.py` files so that I can indicate that all tests in (say) "test_foo" directory should be run on host X. The person who invokes the tests wouldn't need to care and can directly just run tests. The config options would take care of distributing the tests to the right machines. Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Mon May 17 12:02:29 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Mon, 17 May 2010 10:02:29 -0000 Subject: [py-dev] New issue 98 in py-trunk: Option errors in xdist documentation Message-ID: <7680533c0ce34e05b2a0e6e243321429@bitbucket.org> --- you can reply above this line --- New issue 98: Option errors in xdist documentation http://bitbucket.org/hpk42/py-trunk/issue/98/option-errors-in-xdist-documentation Noufal Ibrahim / nibrahim on Mon, 17 May 2010 12:02:29 +0200: Description: Using some pytest_option settings as described in http://codespeak.net/py/dist/test/plugin/xdist.html#specifying-test-exec-environments-in-a-conftest-py results in a py._plugin.pytest_helpconfig.PluginValidationError: being raised with a "found unknown hook: pytest_option_tx" message. Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From simon.callan at infoshare-is.com Mon May 17 17:27:24 2010 From: simon.callan at infoshare-is.com (Simon Callan) Date: Mon, 17 May 2010 16:27:24 +0100 Subject: [py-dev] Running the least run test Message-ID: <002501caf5d5$73181e00$59485a00$@callan@infoshare-is.com> Hi, Having got good answers to my previous questions, I'm back for more help, with a few related questions. 1) Is it possible to programmatically stop py.test halfway through running a set of tests? I.e., for an external program to say "complete the current test and then stop". 2) If I have a number of tests (let's say a1 to a5), and I aborted the test during the running of test a3, is it possible for the next invocation of py.test to remember where it got to, and start running again at test a4? Simon Infoshare Ltd Millennium House 21 Eden Street Kingston upon Thames Surrey KT1 1BL United Kingdom Phone: + 44 (0) 20 8541 0111 Support: + 44 (0) 20 8481 4760 Web: www.infoshare-is.com E-mail: info at infoshare-is.com Infoshare Ltd is registered in England and Wales. Registered Office as above. Registered Number 2877612 VAT Number GB 678 1443 10 The content of this e-mail (and any attachment to it) is confidential. Any views or opinions do not represent the views or opinions of Infoshare Ltd. If you have received this e-mail in error please notify the sender and delete it. You may not use, copy or disclose the information in any way. Infoshare Ltd monitors incoming and outgoing e-mails. Please consider the environment. Do you really need to print this email? From holger at merlinux.eu Mon May 17 19:18:20 2010 From: holger at merlinux.eu (holger krekel) Date: Mon, 17 May 2010 19:18:20 +0200 Subject: [py-dev] Running the least run test In-Reply-To: <002501caf5d5$73181e00$59485a00$@callan@infoshare-is.com> References: <002501caf5d5$73181e00$59485a00$@callan@infoshare-is.com> Message-ID: <20100517171820.GN17693@trillke.net> Hi Simon, On Mon, May 17, 2010 at 16:27 +0100, Simon Callan wrote: > Having got good answers to my previous questions, I'm back for more help, > with a few related questions. > > 1) Is it possible to programmatically stop py.test halfway through running a > set of tests? I.e., for an external program to say "complete the current > test and then stop". That's not easily possible although distributed testing internally this feature. Can you create an enhancement issue and maybe describe your use case / intention of using it a bit more? This helps me a lot to avoid coding features nobody needs. > 2) If I have a number of tests (let's say a1 to a5), and I aborted the test > during the running of test a3, is it possible for the next invocation of > py.test to remember where it got to, and start running again at test a4? Rerunning the last failure(s) and continuing on would probably be nice. It's not possible currently, requires adding some persistence between py.test invocations. Could you also create an issue? If you or somebody else has anything to add or to suggest, shoot :) holger From meine at informatik.uni-hamburg.de Thu May 20 12:43:13 2010 From: meine at informatik.uni-hamburg.de (Hans Meine) Date: Thu, 20 May 2010 12:43:13 +0200 Subject: [py-dev] Problems with test discovery (ImportError) Message-ID: <201005201243.13849.meine@informatik.uni-hamburg.de> Hi, I had successfully toyed with py.test in the past, and just wanted to use it again, instead of nosetests. However, py.test fails with E ImportError: No module named something.test_foo This seems to be caused by a package subdirectory called 'io'. Here's how to reproduce this (not necessarily minimal yet): mkdir -p test_ImportError/io/something touch test_ImportError/io/something/test_foo.py touch test_ImportError/io/something/__init__.py touch test_ImportError/io/__init__.py py.test test_ImportError/io/something Obviously, this is a bug that should be fixed, but I would already happy if you could tell me a workaround (i.e. different invocation of py.test, without changing the directory name of course). Have a nice day, Hans PS: Please CC me as I did not subscribe for this single report. Thanks. From holger at merlinux.eu Thu May 20 12:49:31 2010 From: holger at merlinux.eu (holger krekel) Date: Thu, 20 May 2010 12:49:31 +0200 Subject: [py-dev] Problems with test discovery (ImportError) In-Reply-To: <201005201243.13849.meine@informatik.uni-hamburg.de> References: <201005201243.13849.meine@informatik.uni-hamburg.de> Message-ID: <20100520104931.GV17693@trillke.net> Hi Hans, On Thu, May 20, 2010 at 12:43 +0200, Hans Meine wrote: > Hi, > > I had successfully toyed with py.test in the past, and just wanted to use it > again, instead of nosetests. However, py.test fails with > > E ImportError: No module named something.test_foo > > This seems to be caused by a package subdirectory called 'io'. Here's how to > reproduce this (not necessarily minimal yet): > > mkdir -p test_ImportError/io/something > touch test_ImportError/io/something/test_foo.py > touch test_ImportError/io/something/__init__.py > touch test_ImportError/io/__init__.py > py.test test_ImportError/io/something > > Obviously, this is a bug that should be fixed, but I would already happy if > you could tell me a workaround (i.e. different invocation of py.test, without > changing the directory name of course). The error message is a bit obscure but scrolling a bit up you see that py.test tries to import 'io.something.test_foo' - i.e. you have created a package 'io' and that overlaps with the standard library's 'io' package. Solution: rename 'io' to something else (and generally don't use a package name that overlaps with standard library ones). cheers, holger From meine at informatik.uni-hamburg.de Thu May 20 13:12:16 2010 From: meine at informatik.uni-hamburg.de (Hans Meine) Date: Thu, 20 May 2010 13:12:16 +0200 Subject: [py-dev] Problems with test discovery (ImportError) In-Reply-To: <20100520104931.GV17693@trillke.net> References: <201005201243.13849.meine@informatik.uni-hamburg.de> <20100520104931.GV17693@trillke.net> Message-ID: <201005201312.16344.meine@informatik.uni-hamburg.de> On Thursday 20 May 2010 12:49:31 holger krekel wrote: > On Thu, May 20, 2010 at 12:43 +0200, Hans Meine wrote: > > [...] py.test fails with > > > > E ImportError: No module named something.test_foo > > > > [...] > > > > Obviously, this is a bug that should be fixed, but I would already happy > > if you could tell me a workaround (i.e. different invocation of py.test, > > without changing the directory name of course). > > The error message is a bit obscure but scrolling a bit up you see > that py.test tries to import 'io.something.test_foo' Can I circumvent this? (IIRC, for instance nosetests differentiates between full filenames and importable module names.) > - i.e. you have created a package 'io' and that overlaps with the standard > library's 'io' package. Ah, bad. But I won't change that now, after using the name myself for several years already. Who needs stdlib's io anyway.. ;-p Another workaround is to prepend my main package directory to PYTHONPATH (tested, works -> fine). Thanks for the explanation, Hans PS: No need to CC me anymore, the list turned out to be subscribers-only. :-/ From holger at merlinux.eu Thu May 20 14:00:29 2010 From: holger at merlinux.eu (holger krekel) Date: Thu, 20 May 2010 14:00:29 +0200 Subject: [py-dev] Problems with test discovery (ImportError) In-Reply-To: <201005201312.16344.meine@informatik.uni-hamburg.de> References: <201005201243.13849.meine@informatik.uni-hamburg.de> <20100520104931.GV17693@trillke.net> <201005201312.16344.meine@informatik.uni-hamburg.de> Message-ID: <20100520120029.GW17693@trillke.net> On Thu, May 20, 2010 at 13:12 +0200, Hans Meine wrote: > On Thursday 20 May 2010 12:49:31 holger krekel wrote: > > On Thu, May 20, 2010 at 12:43 +0200, Hans Meine wrote: > > > [...] py.test fails with > > > > > > E ImportError: No module named something.test_foo > > > > > > [...] > > > > > > Obviously, this is a bug that should be fixed, but I would already happy > > > if you could tell me a workaround (i.e. different invocation of py.test, > > > without changing the directory name of course). > > > > The error message is a bit obscure but scrolling a bit up you see > > that py.test tries to import 'io.something.test_foo' > > Can I circumvent this? (IIRC, for instance nosetests differentiates between > full filenames and importable module names.) Could you elaborate on nose's logic a bit? > > - i.e. you have created a package 'io' and that overlaps with the standard > > library's 'io' package. > > Ah, bad. But I won't change that now, after using the name myself for several > years already. Who needs stdlib's io anyway.. ;-p Hehe :) However, it only works as long as no pluguin and no code you are using depends on the standard 'io' module. py.test itself uses a fallback in case it can't "from io import StringIO" but in general a user package named 'io' will lead to breakage, latest when using Python3. > Another workaround is to prepend my main package directory to PYTHONPATH > (tested, works -> fine). You could also try to not have __init__.py files in test directories - as long as all tests have unique names that's fine (and py.test should warn you if there is a conflict). > Thanks for the explanation, > Hans have fun, holger > PS: No need to CC me anymore, the list turned out to be subscribers-only. :-/ P.S.: yeah, sorry. otherwise a lot of spam-sorting would be needed to let the occassional post through. From anto.cuni at gmail.com Thu May 20 14:01:49 2010 From: anto.cuni at gmail.com (Antonio Cuni) Date: Thu, 20 May 2010 14:01:49 +0200 Subject: [py-dev] Running the least run test In-Reply-To: <20100517171820.GN17693@trillke.net> References: <002501caf5d5$73181e00$59485a00$@callan@infoshare-is.com> <20100517171820.GN17693@trillke.net> Message-ID: <4BF524AD.4080308@gmail.com> On 17/05/10 19:18, holger krekel wrote: >> 2) If I have a number of tests (let's say a1 to a5), and I aborted the test >> during the running of test a3, is it possible for the next invocation of >> py.test to remember where it got to, and start running again at test a4? > > Rerunning the last failure(s) and continuing on would probably be nice. > It's not possible currently, requires adding some persistence > between py.test invocations. Could you also create an issue? > > If you or somebody else has anything to add or to suggest, shoot :) some time ago I implemented such a feature without any need to add persistence: basically, I assigned a progressive number to each test (printed on failure) and added an option to skip the first N. In combination with -x it did exactly what Simon described, but unfortunately I lost the code during an svn up. I think it can be a good quick&dirty solution, while waiting for the proper one with persistence between runs. ciao, Anto From issues-noreply at bitbucket.org Fri May 21 04:20:55 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Fri, 21 May 2010 02:20:55 -0000 Subject: [py-dev] New issue 99 in py-trunk: unexpected pass crashes with junitxml plugin Message-ID: <2b257e921a78f655e5186375dd4a5404@bitbucket.org> --- you can reply above this line --- New issue 99: unexpected pass crashes with junitxml plugin http://bitbucket.org/hpk42/py-trunk/issue/99/unexpected-pass-crashes-with-junitxml Anonymous on Fri, 21 May 2010 04:20:55 +0200: Description: Unexpected pass should cause the tests to report failure, however with the junitxml plugin it crashes with the following: INTERNALERROR> def append_failure(self, report): INTERNALERROR> self._opentestcase(report) INTERNALERROR> #msg = str(report.longrepr.reprtraceback.extraline) INTERNALERROR> self.appendlog('<failure message="test failure">%s</failure>', INTERNALERROR> > report.longrepr) INTERNALERROR> E AttributeError: 'ItemTestReport' object has no attribute 'longrepr' -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Tue May 25 11:52:39 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Tue, 25 May 2010 09:52:39 -0000 Subject: [py-dev] New issue 100 in py-trunk: Please tag pytest-xdist 1.2 Message-ID: <028421ae674c296bdec738348c449554@bitbucket.org> --- you can reply above this line --- New issue 100: Please tag pytest-xdist 1.2 http://bitbucket.org/hpk42/py-trunk/issue/100/please-tag-pytest-xdist-12 memedough on Tue, 25 May 2010 11:52:39 +0200: Description: Hi, The pytest-xdist mercurial repo has no tag for release 1.2 which is on pypi. Please add tag. :) Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Tue May 25 12:16:59 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Tue, 25 May 2010 10:16:59 -0000 Subject: [py-dev] New issue 101 in py-trunk: Add pytest-cov to plugins page? Message-ID: <1b120fc54ab73ca097eae5314e4d9af3@bitbucket.org> --- you can reply above this line --- New issue 101: Add pytest-cov to plugins page? http://bitbucket.org/hpk42/py-trunk/issue/101/add-pytest-cov-to-plugins-page memedough on Tue, 25 May 2010 12:16:58 +0200: Description: Hi, Would you be agreeable to adding pytest-cov to the plugins page? I've been using it for a while now and it's performing well and stable for my usage pattern. It provides another option and may be useful for others. :) Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Tue May 25 12:29:30 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Tue, 25 May 2010 10:29:30 -0000 Subject: [py-dev] New issue 102 in py-trunk: Allow -x to take an argument Message-ID: <7a3955868b24eada98145157adb0ed8f@bitbucket.org> --- you can reply above this line --- New issue 102: Allow -x to take an argument http://bitbucket.org/hpk42/py-trunk/issue/102/allow-x-to-take-an-argument alex_gaynor on Tue, 25 May 2010 12:29:29 +0200: Description: Right now -x doesn't take an arguments, and it just stops the tests after a single failure. Sometimes I want to see a few failures, but not all 100 :) If -x had an argument it would basically run the tests up until that number of errors/failures. Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From holger at merlinux.eu Tue May 25 12:32:18 2010 From: holger at merlinux.eu (holger krekel) Date: Tue, 25 May 2010 12:32:18 +0200 Subject: [py-dev] New issue 101 in py-trunk: Add pytest-cov to plugins page? In-Reply-To: <1b120fc54ab73ca097eae5314e4d9af3@bitbucket.org> References: <1b120fc54ab73ca097eae5314e4d9af3@bitbucket.org> Message-ID: <20100525103218.GL17693@trillke.net> Hi Meme, well, i'd rather like to see the two coverage approaches / features merge into a single good coverage plugin with tests. For my use cases both don't work completely well ;) holger On Tue, May 25, 2010 at 10:16 -0000, issues-noreply at bitbucket.org wrote: > --- you can reply above this line --- > > New issue 101: Add pytest-cov to plugins page? > http://bitbucket.org/hpk42/py-trunk/issue/101/add-pytest-cov-to-plugins-page > > memedough on Tue, 25 May 2010 12:16:58 +0200: > > Description: > Hi, > > Would you be agreeable to adding pytest-cov to the plugins page? I've been using it for a while now and it's performing well and stable for my usage pattern. It provides another option and may be useful for others. > > :) > > > Responsible: > hpk42 > -- > This is an issue notification from bitbucket.org. > You are receiving this either because you are the > owner of the issue, or you are following the issue. > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > -- From holger at merlinux.eu Tue May 25 21:15:28 2010 From: holger at merlinux.eu (holger krekel) Date: Tue, 25 May 2010 21:15:28 +0200 Subject: [py-dev] py.test-1.3.1: new py.test.xfail, --maxfail, better reporting, fixes Message-ID: <20100525191528.GM17693@trillke.net> The pylib/py.test 1.3.1 release brings: - the new imperative ``py.test.xfail()`` helper in order to have a test or setup function result in an "expected failure" - a new option ``--maxfail=NUM`` to stop the test run after some failures - markers/decorators are now applicable to test classes (>=Python2.6) - improved reporting, shorter tracebacks in several cases - some simplified internals, more compatibility with Jython and PyPy - bug fixes and various refinements See the below CHANGELOG entry below for more details and http://pylib.org/install.html for installation instructions. If you used older versions of py.test you should be able to upgrade to 1.3.1 without changes to your test source code. py.test is an automated testing tool working with Python2, Python3, Jython and PyPy versions on all major operating systems. It offers a no-boilerplate testing approach and has inspired other testing tools and enhancements in the standard Python library for more than five years. It has a simple and extensive plugin architecture, configurable reporting and provides unique ways to make it fit to your testing process and needs. See http://pytest.org for more info. cheers and have fun, holger krekel Changes between 1.3.0 and 1.3.1 ================================================== New features ++++++++++++++++++ - issue91: introduce new py.test.xfail(reason) helper to imperatively mark a test as expected to fail. Can be used from within setup and test functions. This is useful especially for parametrized tests when certain configurations are expected-to-fail. In this case the declarative approach with the @py.test.mark.xfail cannot be used as it would mark all configurations as xfail. - issue102: introduce new --maxfail=NUM option to stop test runs after NUM failures. This is a generalization of the '-x' or '--exitfirst' option which is now equivalent to '--maxfail=1'. Both '-x' and '--maxfail' will now also print a line near the end indicating the Interruption. - issue89: allow py.test.mark decorators to be used on classes (class decorators were introduced with python2.6) and also allow to have multiple markers applied at class/module level by specifying a list. - improve and refine letter reporting in the progress bar: . pass f failed test s skipped tests (reminder: use for dependency/platform mismatch only) x xfailed test (test that was expected to fail) X xpassed test (test that was expected to fail but passed) You can use any combination of 'fsxX' with the '-r' extended reporting option. The xfail/xpass results will show up as skipped tests in the junitxml output - which also fixes issue99. - make py.test.cmdline.main() return the exitstatus instead of raising SystemExit and also allow it to be called multiple times. This of course requires that your application and tests are properly teared down and don't have global state. Fixes / Maintenance ++++++++++++++++++++++ - improved traceback presentation: - improved and unified reporting for "--tb=short" option - Errors during test module imports are much shorter, (using --tb=short style) - raises shows shorter more relevant tracebacks - --fulltrace now more systematically makes traces longer / inhibits cutting - improve support for raises and other dynamically compiled code by manipulating python's linecache.cache instead of the previous rather hacky way of creating custom code objects. This makes it seemlessly work on Jython and PyPy where it previously didn't. - fix issue96: make capturing more resilient against Control-C interruptions (involved somewhat substantial refactoring to the underlying capturing functionality to avoid race conditions). - fix chaining of conditional skipif/xfail decorators - so it works now as expected to use multiple @py.test.mark.skipif(condition) decorators, including specific reporting which of the conditions lead to skipping. - fix issue95: late-import zlib so that it's not required for general py.test startup. - fix issue94: make reporting more robust against bogus source code (and internally be more careful when presenting unexpected byte sequences) From holger at merlinux.eu Tue May 25 21:35:43 2010 From: holger at merlinux.eu (holger krekel) Date: Tue, 25 May 2010 21:35:43 +0200 Subject: [py-dev] pytest-xdist-1.3 fixes and a sidenote/tip Message-ID: <20100525193543.GN17693@trillke.net> Hi again, i also uploaded a new pytest-xdist-1.3 plugin to the package index which requires the new py-1.3.1. Main thing: If you experienced the strange phenomenon of -f|--looponfail to show tracebacks with _old_ source code, this release should fix that. It also adjusts to the new --maxfail=NUM option of py.test so that in a distributed test run this option works properly: aborting the test run if more then the maximum number of failures are encountered, bringing all nodes down. Btw, did you know you can set this as well as other options through your shell environment or a conftest.py file? For example, export PYTEST_OPTION_MAXFAIL=3 # in bash or option_maxfail = 3 # in conftest.py to always abort test runs after three failures. To find out about available ENV or conftest.py names type py.test --help-config best, holger From holger at merlinux.eu Wed May 26 14:57:01 2010 From: holger at merlinux.eu (holger krekel) Date: Wed, 26 May 2010 14:57:01 +0200 Subject: [py-dev] coverage (was issue101) In-Reply-To: <20100525103218.GL17693@trillke.net> References: <1b120fc54ab73ca097eae5314e4d9af3@bitbucket.org> <20100525103218.GL17693@trillke.net> Message-ID: <20100526125701.GQ17693@trillke.net> Hi Meme, i actually now added your pytest-cov plugin to the plugin index page: http://pytest.org/plugin/index.html :) We now advertise two coverage plugins, not a too fortunate situation. seems like your plugin now works much better for me than some versions ago. I still find your command line option design a bit obnoxious - i usually try to keep it much more minimal. Are you open to advise/suggestions there? Also, is it possible to start a Python script/subprocess and get its coverage included as well? pylib/.test itself has a lot of functional tests and those start subprocesses which is a usual scenario. best, holger On Tue, May 25, 2010 at 12:32 +0200, holger krekel wrote: > Hi Meme, > > well, i'd rather like to see the two coverage approaches / features merge > into a single good coverage plugin with tests. For my use cases > both don't work completely well ;) > > holger > > On Tue, May 25, 2010 at 10:16 -0000, issues-noreply at bitbucket.org wrote: > > > --- you can reply above this line --- > > > > New issue 101: Add pytest-cov to plugins page? > > http://bitbucket.org/hpk42/py-trunk/issue/101/add-pytest-cov-to-plugins-page > > > > memedough on Tue, 25 May 2010 12:16:58 +0200: > > > > Description: > > Hi, > > > > Would you be agreeable to adding pytest-cov to the plugins page? I've been using it for a while now and it's performing well and stable for my usage pattern. It provides another option and may be useful for others. > > > > :) > > > > > > Responsible: > > hpk42 > > -- > > This is an issue notification from bitbucket.org. > > You are receiving this either because you are the > > owner of the issue, or you are following the issue. > > _______________________________________________ > > py-dev mailing list > > py-dev at codespeak.net > > http://codespeak.net/mailman/listinfo/py-dev > > > > -- > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > -- From memedough at gmail.com Thu May 27 13:24:55 2010 From: memedough at gmail.com (meme dough) Date: Thu, 27 May 2010 21:24:55 +1000 Subject: [py-dev] coverage (was issue101) In-Reply-To: <20100526125701.GQ17693@trillke.net> References: <1b120fc54ab73ca097eae5314e4d9af3@bitbucket.org> <20100525103218.GL17693@trillke.net> <20100526125701.GQ17693@trillke.net> Message-ID: <AANLkTinPP-4fJQVZXZVLkgrVKxD-ij_iUxK1B5Z872mY@mail.gmail.com> Hi, > i actually now added your pytest-cov plugin to the plugin index page: > > http://pytest.org/plugin/index.html > > :) We now advertise two coverage plugins, not a too fortunate situation. Yay, thx! > seems like your plugin now works much better for me than some versions ago. > I still find your command line option design a bit obnoxious - i usually > try to keep it much more minimal. Are you open to advise/suggestions there? With the early release I noticed that it got out of sync a couple of times since it depended on specific dev versions and I was chasing, but I think that has stabilised more now. Yes open to discussion. I have already thought about the options issue and played with some alternatives but not yet good replacement happy with, but definitely on the todo. Initially I wanted to support all of coverage since part of reason to write was that I couldn't do what I needed. I'm sure the options can be refactored down to a better set while still giving good control. > Also, is it possible to start a Python script/subprocess and get its coverage > included as well? pylib/.test itself has a lot of functional tests and > those start subprocesses which is a usual scenario. This is a good idea and I think that it is possible, I will have to look into the details. :) From issues-noreply at bitbucket.org Sat May 29 15:47:47 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Sat, 29 May 2010 13:47:47 -0000 Subject: [py-dev] New issue 103 in py-trunk: Introduce "with py.test.raises(...)" Message-ID: <08e74fa16004514f2525d9e0d595ae39@bitbucket.org> --- you can reply above this line --- New issue 103: Introduce "with py.test.raises(...)" http://bitbucket.org/hpk42/py-trunk/issue/103/introduce-with-pytestraises holger krekel / hpk42 on Sat, 29 May 2010 15:47:47 +0200: Description: For those who want to keep their code compatible to >=Python2.6 it'd be nice to extend py.test.raises with a third form: {{{ with py.test.raises(ValueError) as excinfo: #codeblock }}} which would be equivalent to using the previous string-notation: {{{ excinfo = py.test.raises(ValueError, "#codeblock") }}} I believe this could be implemented in a manner that preserves py.test's single-source nature for Python2.4-Python3.1. And i wonder if there are other potential extensions/uses for newer syntax. Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Wed Jun 9 03:55:24 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Wed, 09 Jun 2010 01:55:24 -0000 Subject: [py-dev] New issue 104 in py-trunk: xml output does not escape "<" ? Message-ID: <635183120d30af5c76c6e37d7fe0aa63@bitbucket.org> --- you can reply above this line --- New issue 104: xml output does not escape "<" ? http://bitbucket.org/hpk42/py-trunk/issue/104/xml-output-does-not-escape Anonymous on Wed, 9 Jun 2010 03:55:24 +0200: Description: Hello, The xml plugin doesn't escape "<" chars in the name attrib so hudson fails to parse the file. I renamed the generated test as a workaround. The plugin should be escaping since it's xml should it not? -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From memedough at gmail.com Wed Jun 9 12:13:09 2010 From: memedough at gmail.com (meme dough) Date: Wed, 9 Jun 2010 20:13:09 +1000 Subject: [py-dev] pytest-cov options Message-ID: <AANLkTilqRrrdKgnk1x4HMRc7brc1rIVaNUUtz6P42C81@mail.gmail.com> Hi, The next version of coverage will be 3.4 and looks like it will contain a (nice) backward incompatible change. It has include and omit which are fnmatch patterns. Ned also mentioned that in addition there may be able to specify the roots of source code (but I haven't seen any code for this one yet). For pytest-cov I am considering refining to the options below. coverage reporting with distributed testing support: --cov=pat include coverage for filename pattern (multi-allowed) --cov-omit=pat exclude coverage for filename pattern (multi-allowed) --cov-report=type type of report to generate: term, term-missing, annotate, html, xml (multi-allowed) --cov-combine-each for dist=each mode produce a single combined report --cov-config-file=path config file for coverage, default: .coveragerc This seems to work nicely, though it means you always have to specify like "my/source/root/*" for both --cov and --cov-omit. If the other option added then I will probably add one more so that it is in line with coverage. I haven't committed yet and still playing but wanted to see if anyone had any suggestions. :) From holger at merlinux.eu Wed Jun 9 14:55:23 2010 From: holger at merlinux.eu (holger krekel) Date: Wed, 9 Jun 2010 14:55:23 +0200 Subject: [py-dev] pytest-cov options In-Reply-To: <AANLkTilqRrrdKgnk1x4HMRc7brc1rIVaNUUtz6P42C81@mail.gmail.com> References: <AANLkTilqRrrdKgnk1x4HMRc7brc1rIVaNUUtz6P42C81@mail.gmail.com> Message-ID: <20100609125522.GC17693@trillke.net> Hi Meme, On Wed, Jun 09, 2010 at 20:13 +1000, meme dough wrote: > The next version of coverage will be 3.4 and looks like it will > contain a (nice) backward incompatible change. It has include and > omit which are fnmatch patterns. Ned also mentioned that in addition > there may be able to specify the roots of source code (but I haven't > seen any code for this one yet). do you know, is there going to be pytest-coverage plugin included by default? > For pytest-cov I am considering refining to the options below. > > coverage reporting with distributed testing support: > --cov=pat include coverage for filename pattern (multi-allowed) > --cov-omit=pat exclude coverage for filename pattern (multi-allowed) > ... > This seems to work nicely, though it means you always have to specify > like "my/source/root/*" for both --cov and --cov-omit. If the other > option added then I will probably add one more so that it is in line > with coverage. my comments: a) I guess you could also specify path relative to the CWD and they could be expanded before passing it on. Makes them shorter. b) is the "exclude" pattern applied after the include-pattern? c) is it possible to specify python module/package names? d) can the patterns above not be specified in the coverage rc file? > --cov-report=type type of report to generate: term, term-missing, > annotate, html, xml (multi-allowed) > --cov-combine-each for dist=each mode produce a single combined report why not make this the default if --dist=each is specified? I don't imagine i'd be interested in separate reports if running --dist-each. And if so, i can still run with just one target. > --cov-config-file=path > config file for coverage, default: .coveragerc --cov-config=path is enough i think. > I haven't committed yet and still playing but wanted to see if anyone > had any suggestions. let me/py-dev when you do so i can play around with it a bit after which i can hopefully make more substantiated comments :) holger From issues-noreply at bitbucket.org Wed Jun 9 15:19:05 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Wed, 09 Jun 2010 13:19:05 -0000 Subject: [py-dev] New issue 105 in py-trunk: Internal TypeError when assert is on a line with multiple commands and the assertion fails Message-ID: <0f0041650667ca85c0518344f7e4e3e7@bitbucket.org> --- you can reply above this line --- New issue 105: Internal TypeError when assert is on a line with multiple commands and the assertion fails http://bitbucket.org/hpk42/py-trunk/issue/105/internal-typeerror-when-assert-is-on-a-line-with-multiple-commands-and-the-assertion David Horat / davidhorat on Wed, 9 Jun 2010 15:19:05 +0200: Description: When I run py.test over the attached file test_py_error.py I got an Exception TypeError from py code. This seems to be with some parsing problem because it only appears when we have an assert on the a line with multiple commands and the assertion fails. The code and output files attached are self explanatory. Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Sat Jun 12 19:35:46 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Sat, 12 Jun 2010 17:35:46 -0000 Subject: [py-dev] New issue 106 in py-trunk: multiline statement in import error looks bogus Message-ID: <d4f764ac1932339224b20d3918556921@bitbucket.org> --- you can reply above this line --- New issue 106: multiline statement in import error looks bogus http://bitbucket.org/hpk42/py-trunk/issue/106/multiline-statement-in-import-error-looks fijal on Sat, 12 Jun 2010 19:35:46 +0200: Description: Attached simple example. Basically I think it's important to have the whole statement presented in case it's a multiline statement. This applies only when there is an error during import Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Sat Jun 12 19:39:51 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Sat, 12 Jun 2010 17:39:51 -0000 Subject: [py-dev] New issue 107 in py-trunk: --pdb does not work with import error Message-ID: <f3946ff3cefde9c7009c179b91baa79f@bitbucket.org> --- you can reply above this line --- New issue 107: --pdb does not work with import error http://bitbucket.org/hpk42/py-trunk/issue/107/pdb-does-not-work-with-import-error fijal on Sat, 12 Jun 2010 19:39:51 +0200: Description: When import fails, --pdb does not enter pdb (and instead exits py.test) Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From holger at merlinux.eu Mon Jun 14 15:18:40 2010 From: holger at merlinux.eu (holger krekel) Date: Mon, 14 Jun 2010 15:18:40 +0200 Subject: [py-dev] quick poll: collection errors and --pdb In-Reply-To: <f3946ff3cefde9c7009c179b91baa79f@bitbucket.org> References: <f3946ff3cefde9c7009c179b91baa79f@bitbucket.org> Message-ID: <20100614131840.GJ17693@trillke.net> Hi all, quick polling ... related to the below issue: did you miss being able to say "--pdb" on collection errors (import errors, syntax errors, errors during code-execution-at-import time)? Or the other way round: would you mind if it did also enter the PDB prompt in that case? best, holger On Sat, Jun 12, 2010 at 17:39 -0000, issues-noreply at bitbucket.org wrote: > --- you can reply above this line --- > > New issue 107: --pdb does not work with import error > http://bitbucket.org/hpk42/py-trunk/issue/107/pdb-does-not-work-with-import-error > > fijal on Sat, 12 Jun 2010 19:39:51 +0200: > > Description: > When import fails, --pdb does not enter pdb (and instead exits py.test) > > Responsible: > hpk42 > -- > This is an issue notification from bitbucket.org. > You are receiving this either because you are the > owner of the issue, or you are following the issue. > _______________________________________________ > py-dev mailing list > py-dev at codespeak.net > http://codespeak.net/mailman/listinfo/py-dev > -- From cfbolz at gmx.de Mon Jun 14 15:21:39 2010 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Mon, 14 Jun 2010 15:21:39 +0200 Subject: [py-dev] quick poll: collection errors and --pdb In-Reply-To: <20100614131840.GJ17693@trillke.net> References: <f3946ff3cefde9c7009c179b91baa79f@bitbucket.org> <20100614131840.GJ17693@trillke.net> Message-ID: <4C162CE3.201@gmx.de> Hi Holger, On 06/14/2010 03:18 PM, holger krekel wrote: > related to the below issue: did you miss being able to say "--pdb" > on collection errors (import errors, syntax errors, errors during > code-execution-at-import time)? Or the other way round: would you > mind if it did also enter the PDB prompt in that case? I probably wouldn't mind, but I don't remember ever needing this. While we are talking about --pdb: what kind of bugs me is that xfailing tests drop into a debugger when they fail. I already *know* about this failure, so usually I don't want that. Cheers, Carl Friedrich From holger at merlinux.eu Tue Jun 15 12:20:34 2010 From: holger at merlinux.eu (holger krekel) Date: Tue, 15 Jun 2010 12:20:34 +0200 Subject: [py-dev] py.test talk at DZUG/Dresden? Message-ID: <20100615102034.GN17693@trillke.net> Hi all, i have been asked to talk at http://new.zope.de/tagung/Dresden_2010 about py.test. But i can't make it there for personal reasons. Anybody interested in giving a talk about py.test status? If so, drop me a note - i am happy to help prepare. best, holger -- From holger at merlinux.eu Wed Jun 16 13:20:49 2010 From: holger at merlinux.eu (holger krekel) Date: Wed, 16 Jun 2010 13:20:49 +0200 Subject: [py-dev] quick poll: collection errors and --pdb In-Reply-To: <4C162CE3.201@gmx.de> References: <f3946ff3cefde9c7009c179b91baa79f@bitbucket.org> <20100614131840.GJ17693@trillke.net> <4C162CE3.201@gmx.de> Message-ID: <20100616112049.GR17693@trillke.net> Hi Carl Friedrich, On Mon, Jun 14, 2010 at 15:21 +0200, Carl Friedrich Bolz wrote: > Hi Holger, > > On 06/14/2010 03:18 PM, holger krekel wrote: > > related to the below issue: did you miss being able to say "--pdb" > > on collection errors (import errors, syntax errors, errors during > > code-execution-at-import time)? Or the other way round: would you > > mind if it did also enter the PDB prompt in that case? > > I probably wouldn't mind, but I don't remember ever needing this. Actually i just tried entering PDB on an import error and it seems that locals/globals don't have meaningful values if the import fails. So --pdb would not be of much use anyway. > While we are talking about --pdb: what kind of bugs me is that xfailing > tests drop into a debugger when they fail. I already *know* about this > failure, so usually I don't want that. sure, thanks for reporting. i just fixed this in py-trunk, also unified traceback reporting between --pdb and non-pdb mode. best, holger From red_socks at gmx.de Thu Jun 24 16:35:59 2010 From: red_socks at gmx.de (Frank Hempel) Date: Thu, 24 Jun 2010 16:35:59 +0200 Subject: [py-dev] funcarg name clash / conftest load behaviour Message-ID: <20100624143559.234190@gmx.net> Hello, I experienced the possibility of funcarg name clashes when defining the funcarg factories in conftest.py. Lets say there are two project directories side by side and each has it "own" conftest.py. If both conftest.py's define a funcarg factory for the the same funcarg-name only the factory of the last loaded conftest.py is the one used for both directories. And this holds also when I start the py.test-command from within a project directory. If i'm in bad luck I get the funcargs from the project' conftest.py sitting next to me (and there is actually the chance of other side-effects from the configure/unconfigure methods of the others project conftest.py). I encountered this problem when i integrated third party projects into mine via svn:externals. Maybe i overlooked it, is there the possibility to change the behaviour of loading the conftest.py's? Thanks you, best regards, Frank -- GMX DSL: Internet-, Telefon- und Handy-Flat ab 19,99 EUR/mtl. Bis zu 150 EUR Startguthaben inklusive! http://portal.gmx.net/de/go/dsl From holger at merlinux.eu Thu Jun 24 16:40:58 2010 From: holger at merlinux.eu (holger krekel) Date: Thu, 24 Jun 2010 16:40:58 +0200 Subject: [py-dev] funcarg name clash / conftest load behaviour In-Reply-To: <20100624143559.234190@gmx.net> References: <20100624143559.234190@gmx.net> Message-ID: <20100624144057.GD14601@trillke.net> Hello Frank, On Thu, Jun 24, 2010 at 16:35 +0200, Frank Hempel wrote: > Hello, > > I experienced the possibility of funcarg name clashes when defining the funcarg factories in conftest.py. Lets say there are two project directories side by side and each has it "own" conftest.py. If both conftest.py's define a funcarg factory for the the same funcarg-name only the factory of the last loaded conftest.py is the one used for both directories. > And this holds also when I start the py.test-command from within a project directory. If i'm in bad luck I get the funcargs from the project' conftest.py sitting next to me (and there is actually the chance of other side-effects from the configure/unconfigure methods of the others project conftest.py). > > I encountered this problem when i integrated third party projects into mine via svn:externals. > > Maybe i overlooked it, is there the possibility to change the behaviour of loading the conftest.py's? Are you using the latest 1.3.1 version? It is meant to call the funcarg factory defined in the closest conftest.py file as seen from the test file. IOW, if you have a/conftest.py b/conftest.py then a/test_1.py and b/test_1.py should use the respective factories in the conftest files. If you experience different behaviour i'd consider it a bug and would kindly ask you to submit a more detailed example. best, holger From holger at merlinux.eu Fri Jun 25 10:55:03 2010 From: holger at merlinux.eu (holger krekel) Date: Fri, 25 Jun 2010 10:55:03 +0200 Subject: [py-dev] funcarg name clash / conftest load behaviour In-Reply-To: <4C246640.8030508@gmx.de> References: <20100624143559.234190@gmx.net> <20100624144057.GD14601@trillke.net> <4C246640.8030508@gmx.de> Message-ID: <20100625085503.GG14601@trillke.net> Hi Frank, On Fri, Jun 25, 2010 at 10:18 +0200, Frank Hempel wrote: > Hi Holger, > > thank you for the immediate reply. > > Am 24.06.2010 16:40, schrieb holger krekel: > > Are you using the latest 1.3.1 version? It is meant to call the funcarg > > Shame on me. I used 1.0.2. > Indeed with 1.3.1 the nearest funcarg factories are called. Perfect. good. > There is just one little thing left where i'm not sure what the benefit > is in it. Consider the dir structure as meant in the last mails: > > a/conftest.py > b/conftest.py > > If py.test is started from within the "a"-directory not only the > conftest.py's of parent directories of "a" are loaded (as stated in the > docs), also the conftest.py of the "b"-directory is loaded. I > intuitively wouldn't consider "b" a parent of "a". Also the tests from > "b" are not run of course. Does it makes sense to load "b"'s conftest.py > too in this situation? I agree it's a bit suprising. I'll explain why also to clarify it for myself :) * py.test sees the command line arg b/test_hello.py * it performs collection starting from the directory where 'b' is contained, call it ROOTDIR * collecting ROOTDIR will collect all matching subdirs and files, so 'a' and 'b' * [HERE] collecting the 'b' subdir will consult b/conftest.py to allow it to influence the directory collection objects (it will call the pytest_collect_directory(...) hook to be precise) * after all subdirs/files have been collected the one matching the name 'b' is dived into * ... I think [HERE] is the issue: we could say that a conftest.py cannot determine the directory collection object of its containing directory (only of the subdirs) so for collecting 'b' the conftest.py would not need to be loaded. It's an edge case and so far i have been wary to change it for backward-compat reasons. However, i'd like to try to get rid of the py.test.collect.Directory subclass customization possibility alltogether and rather provide a simple hook which directly implements directory collection, it would simplify internal code and as a side effect remove the surprise you encountered. (because no attempt to collect anything under 'b' would be made and thus no a/conftest.py consulted when calling py.test on 'a') best, holger From holger at merlinux.eu Fri Jun 25 10:59:00 2010 From: holger at merlinux.eu (holger krekel) Date: Fri, 25 Jun 2010 10:59:00 +0200 Subject: [py-dev] funcarg name clash / conftest load behaviour In-Reply-To: <20100625085503.GG14601@trillke.net> References: <20100624143559.234190@gmx.net> <20100624144057.GD14601@trillke.net> <4C246640.8030508@gmx.de> <20100625085503.GG14601@trillke.net> Message-ID: <20100625085900.GH14601@trillke.net> On Fri, Jun 25, 2010 at 10:55 +0200, holger krekel wrote: > However, i'd like to try to get rid of the py.test.collect.Directory subclass customization > possibility alltogether and rather provide a simple hook which directly implements directory > collection, it would simplify internal code and as a side effect remove the surprise you encountered. > (because no attempt to collect anything under 'b' would be made and thus no a/conftest.py consulted > when calling py.test on 'a') um, ... fixed: because no attempt to collect anything under 'b' would be made and thus no b/conftest.py consulted when calling py.test on 'a' all those little a's and b's :) holger From red_socks at gmx.de Fri Jun 25 11:44:18 2010 From: red_socks at gmx.de (Frank Hempel) Date: Fri, 25 Jun 2010 11:44:18 +0200 Subject: [py-dev] funcarg name clash / conftest load behaviour In-Reply-To: <20100625085503.GG14601@trillke.net> References: <20100624143559.234190@gmx.net> <20100624144057.GD14601@trillke.net> <4C246640.8030508@gmx.de> <20100625085503.GG14601@trillke.net> Message-ID: <4C247A72.3060903@gmx.de> Am 25.06.2010 10:55, schrieb holger krekel: >> There is just one little thing left where i'm not sure what the benefit >> is in it. Consider the dir structure as meant in the last mails: >> >> a/conftest.py >> b/conftest.py >> >> If py.test is started from within the "a"-directory not only the >> conftest.py's of parent directories of "a" are loaded (as stated in the >> docs), also the conftest.py of the "b"-directory is loaded. I >> intuitively wouldn't consider "b" a parent of "a". Also the tests from >> "b" are not run of course. Does it makes sense to load "b"'s conftest.py >> too in this situation? > > I agree it's a bit suprising. I'll explain why also to clarify it for myself :) > > * py.test sees the command line arg b/test_hello.py > * it performs collection starting from the directory where 'b' is contained, call it ROOTDIR > * collecting ROOTDIR will collect all matching subdirs and files, so 'a' and 'b' > * [HERE] collecting the 'b' subdir will consult b/conftest.py to allow it to influence the > directory collection objects (it will call the pytest_collect_directory(...) hook to be precise) > * after all subdirs/files have been collected the one matching the name 'b' is dived into > * ... This process I was expecting when calling "py.test" (without arguments or with 'b/test_hello.py') from the ROOTDIR, as i think in your explaination you meant having py.test being called from ROOTDIR because the argument 'b/test_hello.py' you mentioned has 'b/' in it, right?. When calling py.test (without arguments or 'test_hello.py') from within the 'b'-directory i would expect that the collecting will only be done for that 'b'-directory and its child directories. To make sure we are talking about the same a's and b's ;) the dir structure is: ~/ROOTDIR/ a/ conftest.py test_hello.py b/ contest.py test_hello.py on the cmd line i do: ~$ cd ROOTDIR/b ~/ROOTDIR/b$ py.test ================ test session starts ================ platform linux2 -- Python 2.5.5 -- pytest-1.3.1 test object 1: ~/ROOTDIR/b test_hello.py . ============= 1 passed in 0.01 seconds ============== if both conftest.py's have a pytest_configure and pytest_unconfigure, they will be called in this order: configure of b configure of a unconfigure of a unconfigure of b I agree that both 'a' and 'b' should be inspected when calling py.test from ROOTDIR, but when calling it within 'a' or 'b'...? ...i hope i don't bother you if i misunderstood your explaination (sorry in that case). but that you wrote "* py.test sees the command line arg b/test_hello.py" made me wonder whether you misunderstood me in the way that you thought i meant calling py.test from ROOTDIR. Thank you, Best regards, Frank From holger at merlinux.eu Fri Jun 25 12:05:36 2010 From: holger at merlinux.eu (holger krekel) Date: Fri, 25 Jun 2010 12:05:36 +0200 Subject: [py-dev] funcarg name clash / conftest load behaviour In-Reply-To: <4C247A72.3060903@gmx.de> References: <20100624143559.234190@gmx.net> <20100624144057.GD14601@trillke.net> <4C246640.8030508@gmx.de> <20100625085503.GG14601@trillke.net> <4C247A72.3060903@gmx.de> Message-ID: <20100625100535.GI14601@trillke.net> On Fri, Jun 25, 2010 at 11:44 +0200, Frank Hempel wrote: > Am 25.06.2010 10:55, schrieb holger krekel: > >> There is just one little thing left where i'm not sure what the benefit > >> is in it. Consider the dir structure as meant in the last mails: > >> > >> a/conftest.py > >> b/conftest.py > >> > >> If py.test is started from within the "a"-directory not only the > >> conftest.py's of parent directories of "a" are loaded (as stated in the > >> docs), also the conftest.py of the "b"-directory is loaded. I > >> intuitively wouldn't consider "b" a parent of "a". Also the tests from > >> "b" are not run of course. Does it makes sense to load "b"'s conftest.py > >> too in this situation? > > > > I agree it's a bit suprising. I'll explain why also to clarify it for myself :) > > > > * py.test sees the command line arg b/test_hello.py > > * it performs collection starting from the directory where 'b' is contained, call it ROOTDIR > > * collecting ROOTDIR will collect all matching subdirs and files, so 'a' and 'b' > > * [HERE] collecting the 'b' subdir will consult b/conftest.py to allow it to influence the > > directory collection objects (it will call the pytest_collect_directory(...) hook to be precise) > > * after all subdirs/files have been collected the one matching the name 'b' is dived into > > * ... > > This process I was expecting when calling "py.test" (without arguments > or with 'b/test_hello.py') from the ROOTDIR, as i think in your > explaination you meant having py.test being called from ROOTDIR because > the argument 'b/test_hello.py' you mentioned has 'b/' in it, right?. I understand the confusion but indeed, i really meant py.test b/test_hello.py will cause a collection of the ROOTDIR (containing 'b' and 'a'). py.test always collects from the root now matter how you call it. Otherwise collection customizations would not work uniformly. If you call py.test --collectonly you can see the collection result, btw. > ... > if both conftest.py's have a pytest_configure and pytest_unconfigure, > they will be called in this order: > > configure of b > configure of a > unconfigure of a > unconfigure of b > > I agree that both 'a' and 'b' should be inspected when calling py.test > from ROOTDIR, but when calling it within 'a' or 'b'...? I understand your surprise which is why i told the background story and how i'd like to fix it. > ...i hope i don't bother you if i misunderstood your explaination (sorry > in that case). but that you wrote "* py.test sees the command line arg > b/test_hello.py" made me wonder whether you misunderstood me in the way > that you thought i meant calling py.test from ROOTDIR. you are welcome and thanks for trying to clarify. best, holger From red_socks at gmx.de Fri Jun 25 13:05:34 2010 From: red_socks at gmx.de (Frank Hempel) Date: Fri, 25 Jun 2010 13:05:34 +0200 Subject: [py-dev] funcarg name clash / conftest load behaviour In-Reply-To: <20100625100535.GI14601@trillke.net> References: <20100624143559.234190@gmx.net> <20100624144057.GD14601@trillke.net> <4C246640.8030508@gmx.de> <20100625085503.GG14601@trillke.net> <4C247A72.3060903@gmx.de> <20100625100535.GI14601@trillke.net> Message-ID: <4C248D7E.70104@gmx.de> Am 25.06.2010 12:05, schrieb holger krekel: > > I understand the confusion but indeed, i really meant > > py.test b/test_hello.py > > will cause a collection of the ROOTDIR (containing 'b' and 'a'). > py.test always collects from the root now matter how you call it. This backs with my expectations. This means for our play-case: No matter whether I execute ~/ROOTDIR$ py.test or ~/ROOTDIR$ py.test b/test_hello.py it always inspects 'a' and 'b'. That's ok. But my concern is not about how i call it (from ROOTDIR) but _from_ _where_ i call it. Because when i enter the 'b' directory and doing ~/ROOTDIR/b$ py.test it will still examine ~/ROOTDIR/a. By the way ROOTDIR does not contain a __init__.py in this example, what at first I guessed would be the cause to let py.test take all neighbor directories into account. What i found out is, that py.test walks up one directory from where it was started and then examines the directories found regardless whether if found __init__.py on that upper level or not. Only the walking up stops when there is no __init__.py found, but the last walk-up step (or the first step where no __init__.py was found) is still used for collecting. Another example: ~/ROOTDIR/ a/ __init__.py c/ test_hello.py b/ conftest.py and I change dir to ~/ROOTDIR/a/c and call py.test from there. Then it walks from 'c' up to 'a', looking for a __init__.py. Because it found one then it walks up to ROOTDIR, does not find __init__.py there, but nevertheless takes 'b' into consideration. If we remove a/__init__.py and call py.test from ~ROOTDIR/a/c again then it only walks from 'c' up to 'a', does not find __init__.py and stops walking up. So the point what makes me surprise is, that the collection process takes directories from an upper dir-level into account even when there was no __init__.py found on that level. I hope I could make my concern clearer now. Thanks, Regards, Frank From red_socks at gmx.de Fri Jun 25 13:43:33 2010 From: red_socks at gmx.de (Frank Hempel) Date: Fri, 25 Jun 2010 13:43:33 +0200 Subject: [py-dev] funcarg name clash / conftest load behaviour In-Reply-To: <4C248D7E.70104@gmx.de> References: <20100624143559.234190@gmx.net> <20100624144057.GD14601@trillke.net> <4C246640.8030508@gmx.de> <20100625085503.GG14601@trillke.net> <4C247A72.3060903@gmx.de> <20100625100535.GI14601@trillke.net> <4C248D7E.70104@gmx.de> Message-ID: <4C249665.7070700@gmx.de> Am 25.06.2010 13:05, schrieb Frank Hempel: > ... > there is no __init__.py found, but the last walk-up step (or the first > step where no __init__.py was found) is still used for collecting. ...'collecting' was the wrong word. I meant examination which is loading of conftest.py. Tests are not collected of course. Regards, Frank From jgustak at gmail.com Fri Jun 25 16:10:08 2010 From: jgustak at gmail.com (Jakub Gustak) Date: Fri, 25 Jun 2010 15:10:08 +0100 Subject: [py-dev] adding env option to xspec (initialy posted to [execnet-dev]) Message-ID: <AANLkTilXSGVaaRZPYmAiW6H2KiPKuhd8KV2xP8B1DNfe@mail.gmail.com> > Hi Jakub, > > not sure it makes sense to add this to execnet, maybe better > to xdist code? ?Things is it's easy to set environment variables > when initializing gateways yourself. Just when using pytest-xdist > you cannot easily do this because that API level is not exposed. > So xdist could grow a configuration option to interpret > some additional xspec settings. > > I suggest to use > > ? ?popen//env:NAME=value > > as syntax. ? That being said i am not totally opposed to adding > it directly to execnet. ?Maybe other people here on the list also > have comments ... > > best, > holger > > On Thu, Jun 24, 2010 at 13:30 +0100, Jakub Gustak wrote: >> Hello, >> >> I need to specify some ENV variable when running a sub-process (this >> case for xdist). AFAIK there is no support for such feature. >> >> I do a workaround by popen//python=./python_with_env where >> ./python_with_env does specify required ENV. >> >> It would be nice to have xspec env option (e.g. popen//env='VAR1=value >> VAR2=value' ). >> >> I can fry up a patch for this functionality if you consider it a >> feature worth inclusion. >> >> Cheers, >> Jakub Gustak Moving this topic to py-dev list in this case. I prepared a patch, please review. What I don't like about it is using XSspec.__dict__ because env:NAME=VALUE requires a iteration through XSpec attributes: + def env(self): + return [(env.lstrip('env:'), value) + for env, value in self.gateway.spec.__dict__.iteritems() + if env.startswith('env:')] I believe a small extension to XSpec API would be nice. Any other ideas? regards, jakub -------------- next part -------------- A non-text attachment was scrubbed... Name: env_xspec.patch Type: application/octet-stream Size: 3570 bytes Desc: not available Url : http://codespeak.net/pipermail/py-dev/attachments/20100625/1f6d535e/attachment-0001.obj From holger at merlinux.eu Fri Jun 25 16:19:08 2010 From: holger at merlinux.eu (holger krekel) Date: Fri, 25 Jun 2010 16:19:08 +0200 Subject: [py-dev] funcarg name clash / conftest load behaviour In-Reply-To: <4C248D7E.70104@gmx.de> References: <20100624143559.234190@gmx.net> <20100624144057.GD14601@trillke.net> <4C246640.8030508@gmx.de> <20100625085503.GG14601@trillke.net> <4C247A72.3060903@gmx.de> <20100625100535.GI14601@trillke.net> <4C248D7E.70104@gmx.de> Message-ID: <20100625141908.GJ14601@trillke.net> On Fri, Jun 25, 2010 at 13:05 +0200, Frank Hempel wrote: > Am 25.06.2010 12:05, schrieb holger krekel: > > > > I understand the confusion but indeed, i really meant > > > > py.test b/test_hello.py > > > > will cause a collection of the ROOTDIR (containing 'b' and 'a'). > > py.test always collects from the root now matter how you call it. > > This backs with my expectations. This means for our play-case: No matter > whether I execute > ~/ROOTDIR$ py.test > or > ~/ROOTDIR$ py.test b/test_hello.py > it always inspects 'a' and 'b'. That's ok. > > But my concern is not about how i call it (from ROOTDIR) but _from_ > _where_ i call it. It is basically equivalent if you have a CWD of 'b' and call "py.test" or if you are one dir up and call "py.test b". This is what i meant with "no matter how you call it". > Because when i enter the 'b' directory and doing > ~/ROOTDIR/b$ py.test > it will still examine ~/ROOTDIR/a. By the way ROOTDIR does not contain a > __init__.py in this example, what at first I guessed would be the cause > to let py.test take all neighbor directories into account. What i found > out is, that py.test walks up one directory from where it was started > and then examines the directories found regardless whether if found > __init__.py on that upper level or not. Only the walking up stops when > there is no __init__.py found, but the last walk-up step (or the first > step where no __init__.py was found) is still used for collecting. Yes, it's true and documented here (only indirectly though): http://codespeak.net/py/dist/test/customize.html#id22 > Another example: > > ~/ROOTDIR/ > a/ > __init__.py > c/ > test_hello.py > b/ > conftest.py > > and I change dir to ~/ROOTDIR/a/c and call py.test from there. Then it > walks from 'c' up to 'a', looking for a __init__.py. Because it found > one then it walks up to ROOTDIR, does not find __init__.py there, but > nevertheless takes 'b' into consideration. If we remove a/__init__.py > and call py.test from ~ROOTDIR/a/c again then it only walks from 'c' up > to 'a', does not find __init__.py and stops walking up. Yes, what you are seeing is the algorithm behind finding the ROOTDIR. > So the point what makes me surprise is, that the collection process > takes directories from an upper dir-level into account even when there > was no __init__.py found on that level. It internally needs to perform a collection on the ROOTDIR in order to "select" 'a'. In fact, the reason behind all this is that py.test tries to construct a *canonical* collection tree up to the command line specified argument and this three starts from the ROOTDIR / root node which is indeed determined as the last non-init-containing directory. > I hope I could make my concern clearer now. I think so - your analysis is right - i guess i didn't succeed yet in explaining the logic/reasoning behind the current behaviour, hope it's clearer now. best, holger From holger at merlinux.eu Fri Jun 25 16:41:06 2010 From: holger at merlinux.eu (holger krekel) Date: Fri, 25 Jun 2010 16:41:06 +0200 Subject: [py-dev] adding env option to xspec (initialy posted to [execnet-dev]) In-Reply-To: <AANLkTilXSGVaaRZPYmAiW6H2KiPKuhd8KV2xP8B1DNfe@mail.gmail.com> References: <AANLkTilXSGVaaRZPYmAiW6H2KiPKuhd8KV2xP8B1DNfe@mail.gmail.com> Message-ID: <20100625144105.GK14601@trillke.net> Hi Jakub, On Fri, Jun 25, 2010 at 15:10 +0100, Jakub Gustak wrote: > > I suggest to use > > > > ? ?popen//env:NAME=value > > > > as syntax. ? That being said i am not totally opposed to adding > > it directly to execnet. ?Maybe other people here on the list also > > have comments ... > ... > Moving this topic to py-dev list in this case. > > I prepared a patch, please review. > > What I don't like about it is using XSspec.__dict__ because > env:NAME=VALUE requires a iteration through XSpec attributes: > > + def env(self): > + return [(env.lstrip('env:'), value) > + for env, value in self.gateway.spec.__dict__.iteritems() > + if env.startswith('env:')] > > I believe a small extension to XSpec API would be nice. Any other ideas? I think this is bearable as xspecs have very few key-value pairs. You could use env[4:] instead of lstrip() to optimize though :) One other bit: could you try to rework your patch so that it transfers the environment variables in the "slaveinput" dictionary? This is also processable from pytest_configure and produces a smaller patch i guess. best, holger From jgustak at gmail.com Fri Jun 25 17:05:01 2010 From: jgustak at gmail.com (Jakub Gustak) Date: Fri, 25 Jun 2010 16:05:01 +0100 Subject: [py-dev] adding env option to xspec (initialy posted to [execnet-dev]) In-Reply-To: <20100625144105.GK14601@trillke.net> References: <AANLkTilXSGVaaRZPYmAiW6H2KiPKuhd8KV2xP8B1DNfe@mail.gmail.com> <20100625144105.GK14601@trillke.net> Message-ID: <AANLkTilDcshwCdCuth16SipXbv3Q67I6KEu6v1vHQvNB@mail.gmail.com> On Fri, Jun 25, 2010 at 3:41 PM, holger krekel <holger at merlinux.eu> wrote: > Hi Jakub, > > On Fri, Jun 25, 2010 at 15:10 +0100, Jakub Gustak wrote: >> > I suggest to use >> > >> > ? ?popen//env:NAME=value >> > >> > as syntax. ? That being said i am not totally opposed to adding >> > it directly to execnet. ?Maybe other people here on the list also >> > have comments ... >> ... >> Moving this topic to py-dev list in this case. >> >> I prepared a patch, please review. >> >> What I don't like about it is using XSspec.__dict__ because >> env:NAME=VALUE requires a iteration through XSpec attributes: >> >> + ? ?def env(self): >> + ? ? ? ?return [(env.lstrip('env:'), value) >> + ? ? ? ? ? ? ? ? ? ?for env, value in ?self.gateway.spec.__dict__.iteritems() >> + ? ? ? ? ? ? ? ? ? ?if env.startswith('env:')] >> >> I believe a small extension to XSpec API would be nice. Any other ideas? > > I think this is bearable as xspecs have very few key-value pairs. > You could use env[4:] instead of lstrip() to optimize though :) > > One other bit: could you try to rework your patch so that it transfers > the environment variables in the "slaveinput" dictionary? > This is also processable from pytest_configure and produces a > smaller patch i guess. Can you explain in more details? Passing in the slaveinput is not a problem, but where you would like the environ be updated exactly? jakub From holger at merlinux.eu Fri Jun 25 17:56:19 2010 From: holger at merlinux.eu (holger krekel) Date: Fri, 25 Jun 2010 17:56:19 +0200 Subject: [py-dev] adding env option to xspec (initialy posted to [execnet-dev]) In-Reply-To: <AANLkTilDcshwCdCuth16SipXbv3Q67I6KEu6v1vHQvNB@mail.gmail.com> References: <AANLkTilXSGVaaRZPYmAiW6H2KiPKuhd8KV2xP8B1DNfe@mail.gmail.com> <20100625144105.GK14601@trillke.net> <AANLkTilDcshwCdCuth16SipXbv3Q67I6KEu6v1vHQvNB@mail.gmail.com> Message-ID: <20100625155619.GL14601@trillke.net> On Fri, Jun 25, 2010 at 16:05 +0100, Jakub Gustak wrote: > > One other bit: could you try to rework your patch so that it transfers > > the environment variables in the "slaveinput" dictionary? > > This is also processable from pytest_configure and produces a > > smaller patch i guess. > > Can you explain in more details? > > Passing in the slaveinput is not a problem, but where you would like > the environ be updated exactly? Just like now but taking it from slaveinput['env']. holger From anto.cuni at gmail.com Fri Jun 25 17:58:25 2010 From: anto.cuni at gmail.com (Antonio Cuni) Date: Fri, 25 Jun 2010 17:58:25 +0200 Subject: [py-dev] adding env option to xspec (initialy posted to [execnet-dev]) In-Reply-To: <20100625144105.GK14601@trillke.net> References: <AANLkTilXSGVaaRZPYmAiW6H2KiPKuhd8KV2xP8B1DNfe@mail.gmail.com> <20100625144105.GK14601@trillke.net> Message-ID: <4C24D221.60803@gmail.com> On 25/06/10 16:41, holger krekel wrote: >> > + def env(self): >> > + return [(env.lstrip('env:'), value) >> > + for env, value in self.gateway.spec.__dict__.iteritems() >> > + if env.startswith('env:')] >> > > You could use env[4:] instead of lstrip() to optimize though :) using [:4] instead of lstrip is not much about performance, but about correctness: >>> 'env:env'.lstrip('env:') '' ciao, Anto From red_socks at gmx.de Fri Jun 25 18:02:41 2010 From: red_socks at gmx.de (Frank Hempel) Date: Fri, 25 Jun 2010 18:02:41 +0200 Subject: [py-dev] funcarg name clash / conftest load behaviour In-Reply-To: <20100625141908.GJ14601@trillke.net> References: <20100624143559.234190@gmx.net> <20100624144057.GD14601@trillke.net> <4C246640.8030508@gmx.de> <20100625085503.GG14601@trillke.net> <4C247A72.3060903@gmx.de> <20100625100535.GI14601@trillke.net> <4C248D7E.70104@gmx.de> <20100625141908.GJ14601@trillke.net> Message-ID: <4C24D321.7090804@gmx.de> Am 25.06.2010 16:19, schrieb holger krekel: > > It internally needs to perform a collection on the ROOTDIR > in order to "select" 'a'. In fact, the reason behind all this is that > py.test tries to construct a *canonical* collection tree up to the command > line specified argument and this three starts from the ROOTDIR / root node > which is indeed determined as the last non-init-containing directory. > >> I hope I could make my concern clearer now. > > I think so - your analysis is right - i guess i didn't succeed yet in > explaining the logic/reasoning behind the current behaviour, hope it's > clearer now. Ok. Thanks so far for taking the time and explaining this. Nevertheless i'm not absolutely happy with the situation that all directories of the "first non-init-containing directory" (should be called first, right?) are considered for conftest-loading. Because without changing anything to one's project 'a' it can make a difference in the sense of tests passing or failing for the project 'a' in dependency of what other project folders one puts side-by-side to 'a'. One not too unlikely side effect could be: the pytest_configure function of each conftest.py (the one of folder 'a' and of 'b') sets something to sys.path with a commonly used statement like "sys.path.insert(*0*, ...)". When testing project 'a' and with the fact that the order of the conftest-loadings is that "b"'s will be loaded after "a"'s, the sys.path.insertion of 'b' will take precedence. Regardless of the fact the the project 'b' might have have nothing to do with project 'a', it just sits side-by-side in the filesystem. And i'm actually not complaining th order. For my expectaion "b"'s conftest should not be loaded at all if i'm testing 'a' and the containing directory of 'a' and 'b' does not contain an __init__.py. ...do you agree with that :) ? Thanks again, Best regards, Frank > > best, > holger From jgustak at gmail.com Fri Jun 25 18:36:45 2010 From: jgustak at gmail.com (Jakub Gustak) Date: Fri, 25 Jun 2010 17:36:45 +0100 Subject: [py-dev] adding env option to xspec (initialy posted to [execnet-dev]) In-Reply-To: <4C24D221.60803@gmail.com> References: <AANLkTilXSGVaaRZPYmAiW6H2KiPKuhd8KV2xP8B1DNfe@mail.gmail.com> <20100625144105.GK14601@trillke.net> <4C24D221.60803@gmail.com> Message-ID: <AANLkTimHWECiuPUrdRpQ38LQ4WNacMz88Il8PMzC6mOq@mail.gmail.com> >>> > ?+ ? ?def env(self): >>> > ?+ ? ? ? ?return [(env.lstrip('env:'), value) >>> > ?+ ? ? ? ? ? ? ? ? ? ?for env, value in >>> > ?self.gateway.spec.__dict__.iteritems() >>> > ?+ ? ? ? ? ? ? ? ? ? ?if env.startswith('env:')] >>> > > >> You could use env[4:] instead of lstrip() to optimize though :) > > > using [:4] instead of lstrip is not much about performance, but about > correctness: > >>>> 'env:env'.lstrip('env:') > '' True, I somehow forgot strip removes chars not whole string. _ jakub From jgustak at gmail.com Fri Jun 25 18:29:58 2010 From: jgustak at gmail.com (Jakub Gustak) Date: Fri, 25 Jun 2010 17:29:58 +0100 Subject: [py-dev] adding env option to xspec (initialy posted to [execnet-dev]) In-Reply-To: <20100625155619.GL14601@trillke.net> References: <AANLkTilXSGVaaRZPYmAiW6H2KiPKuhd8KV2xP8B1DNfe@mail.gmail.com> <20100625144105.GK14601@trillke.net> <AANLkTilDcshwCdCuth16SipXbv3Q67I6KEu6v1vHQvNB@mail.gmail.com> <20100625155619.GL14601@trillke.net> Message-ID: <AANLkTilX1jPqkpngaPhiBOxPlTJJWijhsaR-TbaxFV9g@mail.gmail.com> On Fri, Jun 25, 2010 at 4:56 PM, holger krekel <holger at merlinux.eu> wrote: > On Fri, Jun 25, 2010 at 16:05 +0100, Jakub Gustak wrote: >> > One other bit: could you try to rework your patch so that it transfers >> > the environment variables in the "slaveinput" dictionary? >> > This is also processable from pytest_configure and produces a >> > smaller patch i guess. >> >> Can you explain in more details? >> >> Passing in the slaveinput is not a problem, but where you would like >> the environ be updated exactly? > > Just like now but taking it from slaveinput['env']. > > holger > There we go! _ jakub -------------- next part -------------- A non-text attachment was scrubbed... Name: env_xspec.patch Type: application/octet-stream Size: 3044 bytes Desc: not available Url : http://codespeak.net/pipermail/py-dev/attachments/20100625/80fca0ef/attachment.obj From holger at merlinux.eu Fri Jun 25 19:08:36 2010 From: holger at merlinux.eu (holger krekel) Date: Fri, 25 Jun 2010 19:08:36 +0200 Subject: [py-dev] adding env option to xspec (initialy posted to [execnet-dev]) In-Reply-To: <AANLkTilX1jPqkpngaPhiBOxPlTJJWijhsaR-TbaxFV9g@mail.gmail.com> References: <AANLkTilXSGVaaRZPYmAiW6H2KiPKuhd8KV2xP8B1DNfe@mail.gmail.com> <20100625144105.GK14601@trillke.net> <AANLkTilDcshwCdCuth16SipXbv3Q67I6KEu6v1vHQvNB@mail.gmail.com> <20100625155619.GL14601@trillke.net> <AANLkTilX1jPqkpngaPhiBOxPlTJJWijhsaR-TbaxFV9g@mail.gmail.com> Message-ID: <20100625170836.GM14601@trillke.net> On Fri, Jun 25, 2010 at 17:29 +0100, Jakub Gustak wrote: > On Fri, Jun 25, 2010 at 4:56 PM, holger krekel <holger at merlinux.eu> wrote: > > On Fri, Jun 25, 2010 at 16:05 +0100, Jakub Gustak wrote: > >> > One other bit: could you try to rework your patch so that it transfers > >> > the environment variables in the "slaveinput" dictionary? > >> > This is also processable from pytest_configure and produces a > >> > smaller patch i guess. > >> > >> Can you explain in more details? > >> > >> Passing in the slaveinput is not a problem, but where you would like > >> the environ be updated exactly? > > > > Just like now but taking it from slaveinput['env']. > > > > holger > > > > There we go! did you run the tests? they fail for me. holger From holger at merlinux.eu Fri Jun 25 19:11:15 2010 From: holger at merlinux.eu (holger krekel) Date: Fri, 25 Jun 2010 19:11:15 +0200 Subject: [py-dev] funcarg name clash / conftest load behaviour In-Reply-To: <4C24D321.7090804@gmx.de> References: <20100624143559.234190@gmx.net> <20100624144057.GD14601@trillke.net> <4C246640.8030508@gmx.de> <20100625085503.GG14601@trillke.net> <4C247A72.3060903@gmx.de> <20100625100535.GI14601@trillke.net> <4C248D7E.70104@gmx.de> <20100625141908.GJ14601@trillke.net> <4C24D321.7090804@gmx.de> Message-ID: <20100625171115.GN14601@trillke.net> On Fri, Jun 25, 2010 at 18:02 +0200, Frank Hempel wrote: > Am 25.06.2010 16:19, schrieb holger krekel: > > > > It internally needs to perform a collection on the ROOTDIR > > in order to "select" 'a'. In fact, the reason behind all this is that > > py.test tries to construct a *canonical* collection tree up to the command > > line specified argument and this three starts from the ROOTDIR / root node > > which is indeed determined as the last non-init-containing directory. > > > >> I hope I could make my concern clearer now. > > > > I think so - your analysis is right - i guess i didn't succeed yet in > > explaining the logic/reasoning behind the current behaviour, hope it's > > clearer now. > > Ok. Thanks so far for taking the time and explaining this. > > Nevertheless i'm not absolutely happy with the situation that all > directories of the "first non-init-containing directory" (should be > called first, right?) are considered for conftest-loading. > Because without changing anything to one's project 'a' it can make a > difference in the sense of tests passing or failing for the project 'a' > in dependency of what other project folders one puts side-by-side to 'a'. > One not too unlikely side effect could be: the pytest_configure function > of each conftest.py (the one of folder 'a' and of 'b') sets something to > sys.path with a commonly used statement like "sys.path.insert(*0*, > ...)". When testing project 'a' and with the fact that the order of the > conftest-loadings is that "b"'s will be loaded after "a"'s, the > sys.path.insertion of 'b' will take precedence. Regardless of the fact > the the project 'b' might have have nothing to do with project 'a', it > just sits side-by-side in the filesystem. And i'm actually not > complaining th order. For my expectaion "b"'s conftest should not be > loaded at all if i'm testing 'a' and the containing directory of 'a' and > 'b' does not contain an __init__.py. > > ...do you agree with that :) ? Somewhat although i consider it bad practise to do sys.path manipulations in pytest_configure. It probably also means that you can get trouble when you run things from the root dir and both the 'a' and 'b' tests are run. Anyway, i agree it'd be nicer if py.test would not implicitely load the sibling directory's conftest.py. Could you care to file an "enhancement" issue in the bitbucket tracker? If you add a test case that you'd like to work that'd be good. I am not sure i fix it for the soon upcoming minor release but i definitely give it a try for 1.4. best, holger From red_socks at gmx.de Fri Jun 25 19:28:15 2010 From: red_socks at gmx.de (Frank Hempel) Date: Fri, 25 Jun 2010 19:28:15 +0200 Subject: [py-dev] funcarg name clash / conftest load behaviour In-Reply-To: <20100625171115.GN14601@trillke.net> References: <20100624143559.234190@gmx.net> <20100624144057.GD14601@trillke.net> <4C246640.8030508@gmx.de> <20100625085503.GG14601@trillke.net> <4C247A72.3060903@gmx.de> <20100625100535.GI14601@trillke.net> <4C248D7E.70104@gmx.de> <20100625141908.GJ14601@trillke.net> <4C24D321.7090804@gmx.de> <20100625171115.GN14601@trillke.net> Message-ID: <4C24E72F.3080802@gmx.de> Am 25.06.2010 19:11, schrieb holger krekel: > On Fri, Jun 25, 2010 at 18:02 +0200, Frank Hempel wrote: >> Am 25.06.2010 16:19, schrieb holger krekel: > > Somewhat although i consider it bad practise to do sys.path manipulations > in pytest_configure. It probably also means that you can get trouble when > you run things from the root dir and both the 'a' and 'b' tests are run. I agree with your opinion. I just could not get something more stupid into mind... ;) > Anyway, i agree it'd be nicer if py.test would not implicitely load the > sibling directory's conftest.py. Could you care to file an "enhancement" > issue in the bitbucket tracker? If you add a test case that you'd like to > work that'd be good. I am not sure i fix it for the soon upcoming minor > release but i definitely give it a try for 1.4. I will do that. Thank's for your agreement :) Best Regards, Frank From jgustak at gmail.com Sat Jun 26 09:55:52 2010 From: jgustak at gmail.com (Jakub Gustak) Date: Sat, 26 Jun 2010 08:55:52 +0100 Subject: [py-dev] adding env option to xspec (initialy posted to [execnet-dev]) In-Reply-To: <20100625170836.GM14601@trillke.net> References: <AANLkTilXSGVaaRZPYmAiW6H2KiPKuhd8KV2xP8B1DNfe@mail.gmail.com> <20100625144105.GK14601@trillke.net> <AANLkTilDcshwCdCuth16SipXbv3Q67I6KEu6v1vHQvNB@mail.gmail.com> <20100625155619.GL14601@trillke.net> <AANLkTilX1jPqkpngaPhiBOxPlTJJWijhsaR-TbaxFV9g@mail.gmail.com> <20100625170836.GM14601@trillke.net> Message-ID: <AANLkTik9l6qTc6243Gp9XYoaS-u8Kc86Ls2VSgGItKZF@mail.gmail.com> > did you run the tests? they fail for me. With this patch the tests are passing. Also I found out nice interesting thing. To test those changes I have to use the same xdist version (e.g installed setup.py develope) for testing. This is interesting paradox. How do you setup your development environment properly? _ jakub -------------- next part -------------- A non-text attachment was scrubbed... Name: env_xspec.patch Type: text/x-patch Size: 3049 bytes Desc: not available Url : http://codespeak.net/pipermail/py-dev/attachments/20100626/9d456de1/attachment.bin From issues-noreply at bitbucket.org Sat Jun 26 19:44:58 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Sat, 26 Jun 2010 17:44:58 -0000 Subject: [py-dev] New issue 108 in py-trunk: Warnings should be acknowledged somehow Message-ID: <8499fc8451598dc5c125f691fcb97ee0@bitbucket.org> --- you can reply above this line --- New issue 108: Warnings should be acknowledged somehow http://bitbucket.org/hpk42/py-trunk/issue/108/warnings-should-be-acknowledged-somehow inducer on Sat, 26 Jun 2010 19:44:58 +0200: Description: Sometimes one detects a condition that should be fixed, but is not severe enough to warrant failing a test altogether. One such example are warnings reported through Python's warnings module. So far, these get swallowed up by py.test. I would argue that they should be reported somehow. Responsible: hpk42 -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue. From issues-noreply at bitbucket.org Sun Jun 27 14:03:15 2010 From: issues-noreply at bitbucket.org (issues-noreply at bitbucket.org) Date: Sun, 27 Jun 2010 12:03:15 -0000 Subject: [py-dev] New issue 109 in py-trunk: implicit loading of sibling directory's conftest.py Message-ID: <977e5b7304223f4592389013670f0331@bitbucket.org> --- you can reply above this line --- New issue 109: implicit loading of sibling directory's conftest.py http://bitbucket.org/hpk42/py-trunk/issue/109/implicit-loading-of-sibling-directorys Anonymous on Sun, 27 Jun 2010 14:03:15 +0200: Description: As discussed on the mailing list (subject: funcarg name clash / conftest load behaviour) it would be nice if py.test does not implicitely load the sibling directory's conftest.py in case those directories are not packaged with an __init__.py. Consider the directory structure: ROOTDIR/ a/ conftest.py test_hello.py b/ conftest.py test_hello.py Calling py.test from ROOTDIR with argument 'a' (or from ROOTDIR/a without args) should not load b/conftest.py. The applied patch against py-1.3.1 adds a test case for this situation. I have to say that I'm not sure whether the location is best and things in it might be done better, because this was the fist time I'm in touch with the code. Best regards, Frank -- This is an issue notification from bitbucket.org. You are receiving this either because you are the owner of the issue, or you are following the issue.