Clean up Zope 2 packages ======================== :Author: Philipp von Weitershausen, philikon@philikon.de :Status: IsDraftProposal :Date: $Date$ :Original: $URL$ Current situation ----------------- Zope 2 and 3 have the same functionality implemented in similar (or sometimes even the same) packages: ========================== ===================================== Zope 2 Zope 3 ========================== ===================================== ``StructuredText`` ``zope.structuredtext`` ``DocumentTemplate`` ``zope.documenttemplate`` ``Interface`` ``zope.interface`` ``Persistence`` ``persistent`` (ZODB) ``RestrictedPython`` ``RestrictedPython`` ``zExceptions`` ``zope.exception`` et.al. ``OFS.content_types`` ``zope.app.content_types`` ``TAL`` ``zope.tal`` ``Products.PageTemplates`` ``zope.pagetemplate``, ``zope.tales`` ``docutils`` ``docutils`` ``zLOG`` ``logging`` (standard library) ``DateTime`` ``datetime`` (standard library) ========================== ===================================== Note that this list is incomplete if you also count ``ZPublisher``, ``ZServer``, ``AccessControl``. They do have rough equivalents in Zope 3 but those are quite far advanced compared to their Zope 2 ancestors and should be the subject of separate proposals. Problem ------- Preventing an old code base (e.g. ``DocumentTemplate``, ``StructuredText``) from bit rotting is already hard enough. Having to maintain two codebases of the essentially the same thing -- not to mention having to maintain forks of the same thing -- leads to a lot of confusion, errors and nasty workarounds. Proposal -------- Use as much as possible from the Zope 3 code base and deprecate the Zope 2 packages. Incidentally, this will also reduce the amount of packages in Zope 2 in the long term, making the Zope 2 distribution slimmer in terms of clutter. ``StructuredText`` vs. ``zope.structuredtext`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``zope.structuredtext`` is a fork of the Zope 2 equivalent which I made on 19 February 2004 when the repository was still in CVS. In the process, the Zope 3 package naming scheme was adopted and a few class and function names were renamed to resembled their functionality. Other than that the package resembles its original which has received a couple of bugfixes since the fork was made, though. **Proposal:** Make ``StructuredText`` a facade of ``zope.structuredtext``, deprecate it in Zope 2.10 and remove it in Zope 2.12. **Status:** This was implemented by myself on the Zope 2 trunk in r41225. **Yet to be resolved:** There is the ``StructuredText.Zwiki`` module whose purpose is not clear to me. I propose that the Zwiki product will incorporate this code if it still depends on it. ``DocumentTemplate`` vs. ``zope.documenttemplate`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``zope.documenttemplate`` is a copy of ``DocumentTemplate`` from the time when the Zope 3 rewrite started. The exact date and time of the fork is lost in CVS history due to the several renamings. ``zope.documenttemplate`` has seen only one big change since the fork, which is the introduction of untrusted DTML code (in r26737, r26826). Other changes only involved cleanup such as exception raising style, new-style classes, etc. Modules in ``DocumentTemplate`` that need to be ported as a whole: - cDocumentTemplate (optimized version of pDocumentTemplate) NOT DONE: can't simply be copied because it depends on ExtensionClass; either rewrite to use new-style classes or don't bother with it at all. - ustr, tests/testustr (DONE) - sequence (DONE) Bugfixes in ``DocumentTemplate`` that most probably need to be ported to ``zope.documenttemplate`` individually: * r21156 is the last revision before the fork * dt_util.ValidationError should be zExceptions.Unauthorized / zope.exception.Unauthorized - r21192: Accelerations (DT_If, DT_Var) NOT DONE: can't be merged, uses accelerations only possible in C (see above) - r21204: Enable STX inner links in DTML (DT_Var) NOT DONE: not applicable anymore * r21647: Line ending problems (tests/testDTML) * r21932: new env. var "STX_DEFAULT_LEVEL" (DT_Var) * r21949: toby-stiff-cache-branch, toby-unicode-branch (DT_Util, DT_Var, pDocumentTemplate, tests/testDTMLUnicode) * r22052: Fix Collector #358: new dtml-in parameter no_push_item (DT_In, tests/testDTML) * r22107: Join sequences unicode-awarely (DT_In) * r22303: Fix Collector #396/#397: better XHTML compatiblity (DT_Var) * r22331: Fix tests (tests/testDTML) * r22634: Make DTML automatically html quote data indirectly taken from REQUEST (DT_String, DT_Util, DT_Var) * r22667: Added 'url_unquote' and 'url_unquote_plus' modifiers to DTML r22669: Add url_unquote and url_unquote_plus. (DT_Var) * r22814: Fix for Collector #565: Fix for a regression. (DT_Var, tests/testDTMLUnicode) * r22841: Exception clause syntax. r22842: Whitespace and docstring normalization. r22843: Missing import. r22846: Avoid circular references. (pDocumentTemplate) * r23493: reStructuredText support. (DT_Var) This requires that the ``reStructuredText`` package is properly dealt with. * r23579: Get rid of apply(). (DT_Let, DT_Util) * r24169: Get rid of string exceptions. (DT_In, DT_Var) * r24200: New-style class dictionaries are immutable. (DT_Util) * r40317: Fix for Collector #1954. (DT_String) **Proposal:** Try to bring ``zope.documenttemplate`` up-to-date with ``DocumentTemplate``, then deprecate the latter. ``Interface`` vs. ``zope.interface`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The Zope 3 interface implementation is not compatible with the old one (they also don't conflict). The Zope 3 interface functionality is a superset of the Zope 2 interface functionality, so except for backward compatability, there's no reason it is still around. Since version 2.8.1b1 Zope 2 has the functionality of bridging interfaces in either direction and most if not all Zope 2-style interfaces have been replaced by Zope 3 equivalents in the Zope 2 codebase; other projects have done so already as well. **Proposal:** Deprecate Zope 2-style interfaces in Zope 2.10 and remove them in Zope 2.12. ``Persistence`` vs. ``persistent`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ XXX deprecate Persistence ``RestrictedPython`` vs. ``RestrictedPython`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Fred Drake made a copy of Zope 2's ``RestrictedPython`` in r26688 to be used in Zope 3. Since then the Zope 3 copy has only been modified to not generate further deprecation warnings on ``whrandom`` where as the Zope 2 original has seen several bugfixes, including Python 2.4 compatability. **Proposal:** Delete the Zope 3 copy of ``RestrictedPython`` and use an ``svn:external`` to the Zope 2 version instead. Using an external the other way around is also fine (provided that the Zope 3 version was deleted and the Zope 2 version was moved to the Zope 3 repository); however, Zope 2 makes much heavier use of ``RestrictedPython`` than Zope 3 does, thus it's likely to receive more attention in the Zope 2 codebase. ``zExceptions`` vs. ``zope.exception`` et.al. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ XXX zExceptions would be a facade to various other zope.* packages ``OFS.content_types`` vs. ``zope.app.content_types`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``zope.app.content_types`` is an older copy of ``OFS.content_types`` and lacks a few bugfixes and newer features. **Proposal:** Port bugfixes and features from ``OFS.content_types`` to ``zope.app.content_types`` and deprecate ``OFS.content_types``. **Status:** Andreas Jung converted ``zope.app.content_types`` to a package and ported the bugfixes and features from Zope 2 on the Zope 3.2 branch in r40761 and deprecated ``OFS.content_types`` on the Zope 2.9 branch in 41015. Brian Sutherland also renamed ``zope.app.content_types`` to ``zope.app.contenttypes`` on the Zope 3 trunk. ``TAL`` vs. ``zope.tal`` ~~~~~~~~~~~~~~~~~~~~~~~~ XXX ``Products.PageTemplates`` vs. ``zope.pagetemplate`` & ``zope.tales`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ XXX ``docutils`` vs. ``docutils`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ There exist two copies of ``docutils``, one in Zope 2, one in Zope 3. They are both vendor imports, thus should be identical. Having two essentially equal vendor imports is clutter. **Proposal:** Remove the Zope 2 copy and sitch the Zope 3 copy in via an ``svn:external``. **Status:** This was done on the Zope 2 trunk by Andreas Jung in r41136. ``zLOG`` vs. ``logging`` ~~~~~~~~~~~~~~~~~~~~~~~~ ``zLOG/__init__.py`` says:: Note: This module exists only for backward compatibility. Any new code for Zope 2.8 and newer should use the logging module from Python's standard library directly. zLOG is only an API shim to map existing use of zLOG onto the standard logging API. That means ``zLOG`` already is a facade to ``logging`` and just needs to be formally deprecated (IOW, generate deprecation warnings). I'm not sure if the differences in logging levels will be important to backward compatability. ``ZODB/loglevels.py`` explains the differences:: # In the days of zLOG, there were 7 standard log levels, and ZODB/ZEO used # all of them. Here's how they map to the logging package's 5 standard # levels: # # zLOG logging # ------------- --------------- # PANIC (300) FATAL, CRITICAL (50) # ERROR (200) ERROR (40) # WARNING, PROBLEM (100) WARN (30) # INFO (0) INFO (20) # BLATHER (-100) none -- defined here as BLATHER (15) # DEBUG (-200) DEBUG (10) # TRACE (-300) none -- defined here as TRACE (5) # # TRACE is used by ZEO for extremely verbose trace output, enabled only # when chasing bottom-level communications bugs. It really should be at # a lower level than DEBUG. # # BLATHER is a harder call, and various instances could probably be folded # into INFO or DEBUG without real harm. **Proposal:** Deprecate the whole ``zLOG`` package in favour of the standard ``logging`` module from Python. **Status:** zLOG has been deprecated by Andreas Jung on the 2.9 branch in r41222 (after the 2.9.0 release), though so far only the trunk has been changed to use the ``logging`` module. See also a relevant thread on zope-dev@zope.org in which Chris Withers has offered help: http://mail.zope.org/pipermail/zope-dev/2005-December/026031.html ``DateTime`` vs. ``datetime`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ XXX Create frankendatetime whose API satisfies IDateTime (the old API) but whose pickles are datetime.datetime compatible; the IDateTime API would be deprecated frm the beginning through. Deprecate DateTime.DateTime, use frankendatetime everywhere possible Questions to be resolved: timezone naive or aware? XXX On creating facades ~~~~~~~~~~~~~~~~~~~ When facades are created, original code is removed in favour of facade calls. In that process, however, the original tests should not be removed. Instead they serve as an indicator whether the facade actually works according to the old semantics, in other words, if backward compatability is granted by the facade. Things still in discussion ~~~~~~~~~~~~~~~~~~~~~~~~~~ What's the purose of the following packages? * ``Lifetime`` * ``MethodObject`` * ``Missing`` * ``MultiMapping`` * ``Record`` * ``Signals`` * ``ThreadLock`` * ``TreeDisplay`` * ``initgroups`` * ``nt_svcutils`` XXX * HelpSys -> use Zope 3 help system perhaps? * ComputedAttribute -> use property() instead? Risks ------ * The French-speaking Zope community could kill me for not writing facade with a "c cedille". * Zope 2.10 will generate *a lot* of deprecation warnings with code that was perfectly fine on Zope 2.9. To avoid too much confusion on the Zope 2 developer's end, perhaps a document explaining every deprecation warning and its fix in detail would help. Maybe this proposal can be said document if it was extended accordingly. * For cases where forks with parallel bugfixes have occurred, we need to be very careful not to lose a bugfix from one of the branches. Implementation progress ----------------------- None so far, still in editing mode.