Reducing the amount of ZCML directives
=======================================

:Author:   Philipp von Weitershausen, philikon@philikon.de
:Status:   IsProposal
:Version:  3
:Date:     $Date$
:Original: $URL$


Current status
--------------

We have roughly 80 ZCML directives in Zope 3.2.  Two of them,
``adapter`` and ``utility``, can be considered elementary.  Another
two not so elementary but still commonly used directives are ``view``
and ``browser:page``.  Many of the other ZCML directives perform
simple tasks revolving around those four directives.  Sometimes they
only save a couple of lines at the expense of having an extra
directive.


Problems
--------

* It is hard for people to remember all of those numerous directives
  and remember their usage.  This is a main part of criticism towards
  ZCML.

* Another common criticism is that certain things shouldn't be done in
  ZCML but in Python.  In general, ZCML should only be responsible for
  registering a component, but not for defining its behaviour.

* Finally, it seems very confusing that factories and vocabularies,
  for example, are looked up as utilities but they have their own ZCML
  directive for registration.


Goals
-----

Apart from the "elementary" directives (``adapter`` and ``utility``),
ZCML directives are good for...

* registering something that isn't a utility or adapter.  E.g. setting
  a security policy, registering a TALES namespace adapter, global
  principals, registering a meta type/portal type in Zope 2/CMF.

* registering an adapter or utility with added policy information that
  belongs in configuration, not in code. E.g.:

  - security information (e.g. ``view``, ``browser:page`` in addition
    to ``adapter``, the ``content``/``class`` directive)

  - configuration information for the component (e.g. SMTP host for
    the mailer utility, locales directory for the gettext message
    catalogs, text file for the help topic)

* additional ways of defining application policy.  E.g. granting
  permissions on roles, etc.

Other tasks, **especially automation**, is better done in Python as it
is not a matter of defining or changing application policy.  In short,
ZCML directives should be more like on/off switches that turn
behaviour defined in Python on or off.


Proposal
--------

The following eight ZCML directives automate certain processes
associated with the registration of certain components or are not
associated with registering components at all:

* ``factory``

* ``vocabulary``

* ``modulealias``

* ``renderer:renderer``

* ``rdb:provideConnection``

* ``dav:provideInterface``

* ``browser:addview``

* ``browser:localUtility``

The proposal calls for deprecating these in Zope 3.3/2.10 and removing
them after 12 months (which would correspond to Zope 3.5/2.12).  In
addition, it is proposed to deprecate and remove the alias ``content``
of the ``class`` directive and the ``factory`` and ``implements``
subdirectives of this directive.

All of this will also get rid of two ZCML namespaces, ``renderer`` and
``dav``.  In addition, it might be a good opportunity to get rid of
``rdb`` as well (there'll only be ``rdb:gadflyRoot`` left).

Details
~~~~~~~

The following lists outlines solutions for avoiding the deprecated
directives in the future.

``factory``
  This is just a shortcut to the ``utility`` directive and saves one
  line in ZCML (for specifying the additional ``IFactory`` interface)
  and two lines in Python (for the title and description of the
  factory).  For example::

    <factory
        id="alias.SydneyBristow"
        component=".alias.sydneyFactory"
        title="Agent Sydney Bristow"
        description="Sydney is an undercover agent working for the CIA"
        />

  would become::
        
    <utility
        provides="zope.component.interfaces.IFactory"
        component=".alias.sydneyFactory"
        name="alias.SydneyBristow"
        />

  whereas ``title`` and ``description`` now have to be specified as
  regular attributes on the factory component.  Note that these values
  should generally be i18n messages because they might appear in
  browser menus.

``vocabulary``
  Despite its name, this directive doesn't register vocabularies but
  vocabulary factories (because vocabularies are content dependent and
  might need to be initialized with more than just the context).  It
  is probably one of the most magical ZCML directives of all.

  First, it is the only directive to take arbitrary arguments and by
  this it defies one of the initial aspects of ZCML (which is being
  restricted to a certain set of directives and parameters).  Second,
  it creates a magical wrapper around vocabularies so that we can
  register them as vocabulary factories.  All of this isn't needed at
  all, people can deal with these things much better in Python code,
  e.g.::

    class SD6Vocabulary(object):
        zope.interface.implements(IVocabulary)
        zope.interface.classProvides(IVocabularyFactory)

        # the existence of this method satisfies IVocabularyFactory
        def __init__(self, context):
            ...

  Now, this class simply gets registered as a utility providing
  ``IVocabularyFactory``::

    <utility
       component="alias.SD6Vocabulary"
       provides="zope.app.schema.vocabulary.IVocabularyFactory"
       name="alias.SD6"
       />

  instead of::

    <vocabulary
       factory="alias.SD6Vocabulary"
       name="alias.SD6"
       />

  Note that this will get rid of a (rather confusing) feature that
  involves the reuse of vocabulary implementations.  A common type of
  vocabulary is the utility vocabuluary.  A custom vocabulary listing
  a certain type of utility is easily registered with::

    <vocabulary
        name="SD6 Agents"
        factory="zope.app.component.vocabulary.UtilityVocabulary"
        interface="alias.interfaces.ISD6Agent"
        />

  Note that ``interface`` is an arbitrary argument here and is passed
  as a unicode string to the ``UtilityVocabulary`` constructor.  To
  achieve the same thing without the ``vocabulary`` directive, one
  would have to create a class in Python::

    class SD6AgentsVocabulary(UtilityVocabulary):
        classImplements(IVocabularyFactory)
        interface = alias.interfaces.ISD6Agent

  and register this in ZCML with the ``utility`` directive as shown
  above.  This proposal argues that this is much more straightforward
  and better documentable than the current behaviour.

``modulealias``
  This directive can put Python modules into ``sys.modules`` under a
  new alias, typically to preserve backward compatability for old
  pickles in ZODB.  It is strongly questionable whether this has
  anything to do with registering and/or configuring components at
  all, let alone the fact that people like the author of this proposal
  feel a bit uncomfortable with putting something like that in ZCML in
  the first place.  If a ``sys.modules`` hack is needed, it should be
  done in Python with a proper BBB comment.

  Since the ``zope.modulealias`` package provides nothing else than
  this ZCML directive, the proposal calls for deprecating and removing
  the ``zope.modulealias`` package altogether.

``renderer:renderer``
  Since the major simplification of the Component Architecture, this
  is probably one of the most useless directives of all times.  It
  simply registers an adapter and does nothing else.  Using the
  ``adapter`` directive here would not even cost you any extra lines.
  A search through the Zope 3 source code also indicates that it isn't
  used anymore.

``rdb:provideConnection``
  This is just a shortcut to the ``utility`` directive and saves you
  exactly one line of ZCML.  E.g.::

    <rdb:provideConnection
        name="alias.UplinkToMarshal"
        connection="alias.marshal.uplinkConnection"
        />

  would become::

    <utility
        component="alias.marshal.uplinkConnection"
        provides="zope.app.rdb.interfaces import IZopeDatabaseAdapter"
        name="alias.UplinkToMarshal"
        />

``dav:provideInterface``
  This is just a shortcut to a call to the ``interface`` directive
  (with an appropriate ``type`` argument) and saves only a few lines
  in ZCML.  For example::

    <dav:provideInterface
        for="http://purl.org/dc/1.1"
        interface="zope.app.dublincore.interfaces.IZopeDublinCore"
        />

  would become::

    <interface
        interface="zope.app.dublincore.interfaces.IZopeDublinCore"
        type="zope.app.dav.interfaces.IDAVNamespace"
        />

    <utility
        provides="zope.app.dav.interfaces.IDAVNamespace"
        component="zope.app.dublincore.interfaces.IZopeDublinCore"
        name="http://purl.org/dc/1.1"
        />

  While this might seem like a lot of extra lines that are needed now,
  one should remember that ``dav:provideInterface`` is rarely used.
  Zope 3 itself uses it twice and it is imaginable that custom DAV
  apps might use it once or twice as well.  That is due to the fact
  that the number of DAV namespaces is rather limited, at least
  limited enough that a custom directive is hardly justified.

  The WebDAVInterfacesWidgetsAndAdapters_ proposal also proposed to
  get rid of this directive in the context of a refactoring of the DAV
  support.

``browser:addview``
  This is a shortcut to the ``browser:view`` directive and would save
  one line in ZCML.  Another problem with it is that it seems broken
  (by looking at its code).  Plus, it is also untested and untested
  code is broken by definition.

  This proposal therefore calls for its immediate removal (no
  deprecation period).

``browser:localUtility``
  This is just a short-cut around the ``content`` directive and
  letting classes implement ``IAttributeAnnotatable`` and
  ``ILocalUtility``.  So it saves two lines of Python code.

  Note that another proposal, `Local Component Management
  Simplification`_ is proposing to get rid of ``ILocalUtility``.

``content`` and ``class``
  These are two names for the same directive which is confusing.
  Since this directive is mainly about making security assertions of
  any class, not only content components, it is proposed to remove the
  ``content`` alias and only leave ``class``.

  The following two items propose the removal of the two subdirectives
  to ``class`` that are not about security declarations.  That way,
  the ``class`` complex-directive will be greatly simplified, only
  having two possible subdirectives dealing with security (``require``
  and ``allow``).

``class/factory``
  For the same reasons the ``factory`` directive is suggested to be
  removed, this subdirective of ``class`` is also proposed to be
  removed.  It would better be served with appropriate statements in
  Python.  E.g.::

    <content class=".alias.Sydney">
      <factory
          id="alias.SydneyBristow"
          title="Agent Sydney Bristow"
          description="Sydney is an undercover agent working for the CIA"
          />
      ...
    </content>

  would become::

    <class class=".alias.Sydney">
      ...
    </class>

    <utility
        component=".alias.sydneyFactory"
        provides="zope.component.interfaces.IFactory"
        name="alias.SydneyBristow"
        />

  with ``.alias.SydneyFactory`` being defined in Python::

    from zope.component.factory import Factory
    sydneyFactory = Factory(
        SydneyBristow,                     # a callable, e.g. a class
        title=_(u'Agent Sydney Bristow'),
        description=_(u'Sydney is an undercover agent working for the CIA')
	)

``class/implements``
  This directive is often used to define policy by applying a marker
  interface to a class.  The first problem with it is that you might
  want to use it independently from a ``class`` directive.  Sometimes
  even (e.g. when in a 3rd party package), you would be conflicting
  with the ``class`` directive from the original package.  Second, you
  might want to make factories other than a class implement something
  (e.g. a function).

  This proposal therefore calls for the ``class/implements``
  subdirective to be moved to its own top-level ``implements``
  directive.  E.g.::

    <content class=".alias.Sydney">
      <implements
          interface="alias.interfaces.IHighestSecurityClearance"
          />
      ...
    </content>

  would become::

    <class class=".alias.Sydney">
      ...
    </class>

    <implements
        factory=".alias.Sydney"
        interface="alias.interfaces.IHighestSecurityClearance"
        />

  ``factory`` could be a class (in which case ``classImplements``
  would be used) or any other object, e.g. a function (in which case
  ``implementer`` would be used).

.. _WebDAVInterfacesWidgetsAndAdapters: http://www.zope.org/Members/mkerrin/WebDAVInterfacesWidgetsAndAdapters
.. _Local Component Management Simplification: http://dev.zope.org/Zope3/LocalComponentManagementSimplification


Risks
-----

* The majority of the above mentioned directives are documented,
  especially in printed books.  Removing them would eventually render
  these books outdated.

  This shouldn't stop us from simplifying ZCML, though.  Software
  books will always become outdated eventually.

* The usage of less specialized directives might increase the usage of
  dotted names referring to interfaces and such.  A valid argument is
  that short directive names are sometimes easier to remember than
  long dotted names.

  On the other side, ZCML's usabililty benefits from the usage of more
  general directives (because there are less directives to work with
  and less to memorize), so the net effect might still be positive.
  In addition to that, the usage of dotted names has been reduced in
  the past by making the ``required`` and ``provides`` arguments of
  the ``utility`` and ``adapter`` directives optional, essentially
  turning them into on/off switches.
  

Implementation status
---------------------

This proposal has been implemented on the Zope 3 trunk, except for:

* ``rdb:provideConnection`` wasn't removed.  On a second thought, this
  directive also contains deployment information relevant to the
  system administrator (the DSN) that should not reside in Python code.

* ``dav:provideInterface`` wasn't removed.  Michael Kerrin's work on
  DAV is also calling for its removal so this is left for him to
  remove.

* ``browser:localUtility`` wasn't removed.  Its removal might collide
  with Jim's on-going work on local component simplifications.  The
  removal is therefore postponed, if it won't be done by Jim anyways.

* ``class/implements`` and ``class/factory`` weren't removed yet.
  Since this might be controversial, it should be brought up again on
  the mailinglist and then implemented on a new branch.

Zope 2/Five has not been updated yet.


Potential follow-ups
--------------------

This proposal isn't addressing many directives that might also have a
questionable status.  Here are a few **unsorted** ideas that
originated in a `related blog post`_ together with the rest of this
proposal (these ideas are **not** up for discussion as part of this
proposal, they merely set the mood for potential upcoming proposals):

.. _related blog post: http://www.z3lab.org/sections/blogs/philipp-weitershausen/2005_12_14_zcml-needs-to-do-less

``xmlrpc:view``
  It is probable that given the correct the usage of decent base
  classes, this directive could be replaced with the simple ``view``
  directive.

``browser:page``, ``browser:view``
  These two directives involve a lot of magic.  Their feature span is
  great, they might be doing too much at a time.  Their behaviour
  regarding the on-the-fly creation of classes is definitely
  cumbersome.  Also, their distinction seems outdated nowadays.  What
  should be done with them could fit into a separate proposal (which
  might also tackle ``xmlrpc:view`` at the same time as it involves
  similar issues.)

``browser:containerViews``
  This is a shortcut to a bunch of things, so it actually saves more
  than just a few lines.  It might perhaps not be worth removing it,
  though it is inflexible and questionable how many people with the
  need for custom containers actually use it.

``browser:addform``, ``browser:editform``, ``browser:schemadisplay``
  If the form views these directives register were defined in Python
  (which would make the overriding of widgets much easier, too), a
  plain boring ``browser:page`` directive could take over.  When using
  subclassing sensibly, a few lines of Python code should be enough
  for most use-cases.

  This is how ``zope.formlib`` does it, actually, so perhaps we should
  simply not worry about ``zope.app.form`` and deprecate it all
  together (after moving out the widgets and other useful parts).  In
  fact, ``zope.formlib`` is a good example of how things should be
  done in many respects.

  A separate proposal should outline what happens to
  ``zope.app.form``.

``browser:viewlet``, ``browser:viewletManager``
  The whole viewlet concept is new and needs to be field-tested before
  we know more about its preferred usage, etc.  It is certain that the
  similarities between the directive handlers for these directives and
  others (e.g. ``browser:page``) should allow us to simplify the
  architecture eventually.

  Perhaps someone familiar with viewlets and their implementation will
  come up with a proposal that consolidates ZCML in this respect.

Many directives of the ``browser`` namespace support the registration
of menu items in addition to registering the component in question.
Though this can be considered useful because it might save some
typing, keeping directives as simple as possible (on/off switches!)
might be weighed higher.  People are certainly intimidated by the
length of some of the ``browser`` directives (such as
``browser:editform``); by taking the menu functionality out, we could
reduce many directives by two lines making them easier to understand
by themselves (of course, we'll have to add another menu directive,
but it'll only be 3 or so lines).