[z3] Zemantic in z3base

[Michel Pelletier]

Phillip contacted me recently about introducing my project to the z3base list 
for inclusion.  Right now I have a private svn at https://zemantic.rdflib.net/ 
but I would not want to impose too much longer on Dan to keep it running, so 
z3 would be perfect for me and I apreciate Phillip's offer.  There is a 
smattering of info on http://zemantic.org including links to the repo.

Zemantic is a semantic web catalog for Zope 3.  Zemantic takes information 
about resources, internal to Zope or external, and catalogs that information 
in a way that can be queried from python or from a TALES expression syntax.  
Zemantic stores the information acording to the RDF data model, more of which 
can be read about in the RDF primer: http://www.w3.org/TR/rdf-primer/

In short, RDF is the underlying data model of the semantic web initiative.  
There is nothing technically new about RDF's data model, relational data 
models and finite concept analysis have been around for decades, the semantic 
web is just a standardization of one form of these models.  The W3C is 
heavily endorsing this standard as a way to improve not only the way human 
interact with web services, but drastically improve the interoperability of 
the services themselves.

In terms of scalability, Zemantic is performing quite well.  Zemantic stores 
RDF relations in the ZODB using three-dimensionally nested BTrees, one 
dimension for each unique subject, predicate, and object in an rdf statement, 
and there is some optimization that is done with integers and IOBTrees.  On 
average, Zemantic uses about 700-800 bytes of ZODB storage per triple.  I 
intend future versions to also allow rdf relations to be put into (surprise!) 
a relational database, or I imagine one could use APE to store the data 
elegantly enough, but the ZODB is performing very well so far in my 
prototypes, and expect other storages like DirectoryStorage on the right kind 
of filesystem to fair even better.

If anyone has any questions about Zemantic, please feel free to discuss them 
with me on or off list.  If you have any confusions about RDF, please read 
the primer link above, just the first few pages of which will familiarize you 
with the general problem domain and use cases for Zemantic.  Thanks,

-Michel