[z3-five] Proposal: use $INSTANCE_HOME/package-includes

[Martijn Faassen]

Philipp von Weitershausen wrote:
> Martijn,
> 
>> Take the case of a Plone or Silva. For Silva to follow a 
>> package-includes strategy we'd need to place the views and adapters
>>  in the package-includes direcory, and everything else remains in 
>> the Silva directory.
> 
> 
> I don't understand that. Everything would remain in the Silva 
> directory. The package-includes directory only contains a small
> 
> <include package="Products.Silva" />
> 
> snippet.

Yes, I understand this. :)

But it means that the whole idea of having the directories be anywhere
it not useful for projects which need to evolve from Zope 2 to Zope 3.
It's only useful if you're writing a fresh Five project.

[snip]

>> Another advantage of the 'Products' way is that people are already 
>> familiar with it;
> 
> Nothing prevents them from continuing to do so. Using 
> package-includes will be an option, not a requirement. I think the 
> proposal explicitly stated that.

Yes, but we should be very careful in making the option being only an
option, not the "preferred way to do it". I think we have override
requirements for evolution that aren't entirely solvable with
package-includes, or anything that doesn't have a singly rooted ZCML 
include tree (for at least a shared root between what is overridden and 
the stuff that does the overriding).

>> So, can we really go the Zope 3 way here the whole hog? It looks to
>>  me we'll have to support the Product way for Five as well at the 
>> very least.
> 
> 
> Right now, support the "Product way" is a simple boiler plate in 
> whatever products:
> 
> def initialize(context): zcml.process('configure.zcml', 
> package=Products.FiveViewsDemo)
> 
> People will still be able to do it. I personally would like to have 
> the option for an automation. That's why I wrote the proposal.

My point is that the product way needs more consideration by itself. We
haven't solved all issues yet.

>>> Zope 3, on the other hand, does not enforce packages to be in a 
>>> certain place as long as they can be lookuped in PYTHONPATH.  It
>>>  allows packages that want to be configured to drop in a small 
>>> ZCML file into a directory called 'package-includes'.

>> Do these get dropped in automatically? How does that mechanism 
>> work? Or does an actual sysadmin need to drop these in, instead? 
>> From what you say later, the latter; that's an extra burden on the 
>> sysadmin.
> 
> But it's pretty annoying that everything in Products automagically 
> gets picked up. And yes, IIRC, zpkg drops in the file automatically.

It's very convenient that everything in Products gets automatically
picked up. :) Like it or not, sysadmins are used to the Zope 2 way, and
having a new class of Zope 2 products that works differently isn't going
to charm them off their feet...

>>> Five allows us to almost write Zope3-compatible code which works 
>>> in Zope 2 (using Five).
>> 
>> 
>> As I argued before, the main goal of Five is to allow *existing* 
>> products to use Zope 3 code. The main goal is *not* to write Zope 3
>>  compatible code in Zope 2. That's only useful if you write new 
>> packages.
> 
> Your goal is the former, my goal is the latter. The idea of this 
> proposal does not involve a conflict of the two interests.

I'm still happily going to defend the former if I think the latter is
causing us to lose sights of that goal.

Being able to write Zope 3 compatible code in Zope 2 by itself is of
limited utility; you could as well use Zope 3 and be done with it. Being 
able to add Zope 3 abilities to Zope 2 code is of tremendous utility. 
It'll allow existing products to become *closer* to compatibility to 
Zope 3. Of course the one doesn't preclude the other, but we shouldn't 
lose sight of the option with the tremendous utility. :)

>> I highly doubt a developer will actually need to turn off 
>> particular features in Five. I think the flexibility to disable 
>> parts of Five is a YAGNI.
> 
> I beg to differ. I see use cases where people don't need this or that
>  or where they want to *customize* parts.

This sounds very dubious to me. If I install one Product that builds on
Five and then another, I don't want conflicts between what I need to
turn off and on, or override in Five. I'll call YAGNI until I see actual
use cases that also describe how they work around this problem.

>>> I propose to simulate the Zope3 package experience in Five. There
>>> will be a 'package-includes' directory in a Zope 2 instance, just
>>> like in a Zope 3 instance.  It will contain a few default include
>>> files for the packages that Five itself depends on.  A developer
>>> will then be able to install an addon package by dropping a ZCML
>>> file in there, just like in Zope3.
>>  
>> This replaces one action by a sysadmin (install Product) by two 
>> (install Python package, drop in ZCML). The main audience for 
>> installation isn't developers, but sysadmins.
> 
> zpkg generates a setup.py that should do this automatically.

How does the setup.py know where the package-includes directory is? This
sounds like an impossible thing to solve unless there's only a single
package-includes per system? At the very least you'll need some sysadmin 
input about where to install.

> Anyway, my goal is to also support the Z3 way; and if the Z3 way is a
> bit longer (for now), then that's ok with me. Again, you won't have
> to do it.

Sure, as long as we have a clear sense of direction. Five is about Zope 
2 as much as about Zope 3.

>> * Supporting the Products way should not be in any way be 
>> deprecated and in fact requires our special attention. We need 
>> something that is similar to package-includes but actually doesn't 
>> require the moving around of zcml snippets into directories by a 
>> sysadmin. Perhaps we can make it so that installing a Product is 
>> equivalent to the dropping in of ZCML in package-includes, without 
>> actually doing so. After all, in Zope 2, people install Products 
>> expecting to be able to use them. 
> 
> Right. That's why I think the convention should be that everything 
> that goes into Products doesn't require a package-includes snippet. 
> But I would like to be able to use it.

Sounds good, as long as we don't deprecate Products. It needs special 
attention and I don't want the message to the developers about all this 
to become confused. That issue is now firmly on the table so next time 
we won't need as long a set of emails. :)

Regards,

Martijn