[z3-five] Integration tests with Five and ZCML

[Philipp von Weitershausen]

Martin Aspeli wrote:
> Hi,
> 
> We've known for some time that it's Bad™ that PloneTestCase does:
> 
> ZopeTestCase.installProduct('Five')
> 
> I discovered that if I mix any tests using PloneTestCase (and this 
> statement, at module level) with any pure-Zope3 test that uses 
> zope.app.testing.placelesssetup.setUp (with or without a corresponding 
> .tearDown) I get strange and confusing errors even though the tests in 
> question run fine if the PloneTestCase tests are separated from the 
> Zope3-style ones.

Right. This is EXACTLY why the installProduct('Five') thing is bad. It
makes tests that care about (and expect) proper test clean up break,
because installProduct('Five') itself knows nothing about proper cleanup
and cannot do it.

> Well, you told me so. But my question is, how *should* PloneTestCase be 
> behaving.

Well, IIRC, PTC is pretty much used for any test case in Plone-related
tests? I mean, how can you even *call* these things unit tests...

Anyways, if PTC is meant to handle integration tests only, and you want
PTC to load up Five's ZCML and all the ZCML of all the products there
are, then here's what you do:

  def afterSetUp(self):
      from Products.Five import zcml
      zcml.load_site() # this loads site.zcml

  def beforeTearDown(self):
      zope.testing.cleanup.cleanUp()

Now, of course, this will sloooooow. It will load and tear down all ZCML
stuff for *each* test. A better way would be to use a layer:

  class FiveIntegrationTestLayer(object):

      @classmethod
      def setUp(cls):
          from Products.Five import zcml
          zcml.load_site() # this loads site.zcml

      @classmethod
      def tearDown(cls):
          zope.testing.cleanup.cleanUp()

And PTC would simply do:

  class PloneTestCase(ZopeTestCase):
      layer = FiveIntegrationTestLayer
      ...

Note that neither PTC nor any test that derives from PTC should register
new components or even tear down anything (e.g. using
zope.testing.cleanup). THe idea with layers is that the layer does the
setup and tear down of test fixtures ONCE.

Perhaps we should add FiveIntegrationTestLayer to Five. Even better, we
should convert ZopeTestCase to use layers as well,
FiveIntegrationTestLayer just being one of those layers. I've been
meaning to do that for some time now
(http://www.z3lab.org/sections/blogs/philipp-weitershausen/2006_03_10_death-to-zopetestcase),
just haven't gotten around to do it... too busy dealing with Zope bugs
at EuroPython...

Philipp