[Z3-zemantic] thaughts about zodb backend storage

[Michel Pelletier]

On Tuesday 29 March 2005 12:23 pm, you wrote:
> Michel Pelletier wrote:
> > I think Tres is right, your performance issue isn't related to big
> > triples, but because you are adding all of this RDF in one database
> > transaction.  As he suggested, commiting sub-transactions per batch of
> > mails will help keep the speed up and the RAM consumption down.
>
> I'd be willing to bet that the whole thing would go faster even without
> batching if Tarek just dropped the "body" statement.

I agree.

> C'mon, you have to admit that the "reverse" index for a 'body' predicate
> is absurd.  Having Zemantic store the bodies as "keys" in a BTree is
> ridiculous.

Yeah I suppose that is a good argument, forward index just the MD5 hash?

I suppose if zemantic is only storing the "shape" of the graph, that the MD5 
hash is sufficient to hold that shape, i'll keep thinking about it.

-Michel