[Z3-zemantic] thaughts about zodb backend storage

[Tarek Ziadé]

Tres Seaver wrote:

>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>Michel Pelletier wrote:
>
>  
>
>>I think Tres is right, your performance issue isn't related to big triples, 
>>but because you are adding all of this RDF in one database transaction.  As 
>>he suggested, commiting sub-transactions per batch of mails will help keep 
>>the speed up and the RAM consumption down.
>>    
>>
>
>I'd be willing to bet that the whole thing would go faster even without
>batching if Tarek just dropped the "body" statement.
>  
>
ok thank to both of you for the tips. I'll will try asap when i am back 
in Paris
 and give feedback

>[snip]
>  
>
>>In the near future when 3.1 is released, this problem will go away, you'll put 
>>your text indexeable content in a catalog and your RDF graph in a Zemantic 
>>and probably not stuff then entire message body into the RDF graph.he
>>    
>>
>
>Then we will still need a way to join the results of a query against
>Zemantic's own triples with a query against the external text index.
>  
>
yes, that would probably be some kind of (id, content) like in the IOBTree

Regards

Tarek

-- 
Tarek ZIADE, Nuxeo SARL: Zope Service Provider.
Mail: tz at nuxeo.com - Tel: +33 (0)6 30 37 02 63
Nuxeo Collaborative Portal Server: http://www.nuxeo.com/cps
Gestion de contenu web / portail collaboratif / groupware / open source