[pypy-dev] Re: unittest-module

[holger krekel]

[Michael Hudson Wed, Jul 02, 2003 at 11:34:31AM +0100]
> holger krekel <hpk at trillke.net> writes:
> 
> > [Michael Hudson Tue, Jul 01, 2003 at 02:29:00PM +0100]
> >> hpk at codespeak.net writes:
> >> > Log:
> >> > - improved the hack to perform specific objspace tests only when
> >> >   actually running the tests with the respective objspace. 
> >> >   (i.e. objspace.ann* tests are skipped for StdObjSpace and
> >> >    objspace.std* for AnnObjSpace)
> >> 
> >> I think *a* way to do this nicely is to have test.objspace() raise a
> >> "TestSkip" exception when asked for a specific object space that is
> >> not the one currently being tested.
> >
> > I agree this is a good way. Also the "shouldStop" stuff would probably
> > be better implemented by a Control-Flow Exception. 
> 
> ?  Not sure I follow.  The same sort of control flow exceptions the
> annotation object space uses?

No, sorry, that was unclear.  I just meant *a* control-flow exception
(not the interpreter ones) and more specifically, that the attribute 
"shouldStop" which is set on a TestResult instance and checked for at 
upper layers.  I think the generally cleaner way is to define an exception 
hieararchy like

    TestFlowException
    |
    |-- TestSkipException
    |
    |-- TestCaseAbortException 
    |
    |-- TestFailureException (aka failureException)
    |
    |-- ...

and use them to bail out of lower layers without calling methods and
settings attributes you have to check at all "calling" places. 
 
> >> This probably means rewriting
> >> more of unittest.py, though (we want four possible outcomes: pass,
> >> fail, error, skip).
> >
> > yep, looking at the unittest.py source this probably means
> > overriding even more methods. To be honest IMO we should make our
> > life easier and just drag unittest.py from python-2.3, add "skips",
> > remove the leading "__" from some names plus probably do some other
> > cleanups/niceties.
> 
> Would this make life easier?  As much as we bitch about unittest.py,
> it has been and will continue to be a "getting us from here to there"
> tool -- if we'd had to write a unittest module from scratch when we
> started the project, we either wouldn't have unittests for pypy or we
> wouldn't have pypy.  It may be that one day we'll be able to remove
> "import unittest" from the top of tool/test.py, but that won't
> diminish the usefulness unittest.py has given us.

Fully agreed.  I didn't mean to say (at all) that unittest.py hasn't 
been useful to us.  But i doubt that the current 

    tool/test.py + 
    std/unittest.py + 
    interpreter/unittest_w.py 

is that easy and nice to understand any more. And if we want to add more 
features then i think that having our own unittest2.py (or whatever) forked
from the standard unittest.py makes some sense. After all, we have to 
care for different objectspaces and different "levels" (interpreter-level and
application-level) and want different modes of operation (CTS).
The current unittest.py doesn't make this easy, does it? 

> > Doesn't seem like incredibly hard work and we may offer it back to
> > cpython, later :-)
> 
> This suffers from the distutils problem: we've no way of knowing what
> other people have done in the way of subclassing unittest.Foo, so even
> the slightest change carries the risk of breaking other people's test
> rigs.  There might be a way to provide new behaviour by default whilst
> presenting old behaviour to subclassers, but not trivially (it would
> have been easier if both distutils and unittest had been written with
> this in mind...).

Agreed. So contributing it back isn't so interesting for the time beeing. 
If we integrated a test-coverage mechanism at some point then the
situation might possibly change. I talked to some Zope3 and other people and
nobody knows of such a tool, btw. 

cheers,

    holger