[Z3-zemantic] Scalability, part 2

[Michel Pelletier]

On Wednesday 09 March 2005 07:28 am, Paul Everitt wrote:
> On Mar 9, 2005, at 4:10 PM, Michel Pelletier wrote:
> > On Wednesday 09 March 2005 02:50 am, Paul Everitt wrote:
> >> Sorry for following up to my own note, but this is an interesting and
> >> academically-rigorous article on the mathematical techniques in RDF
> >> indexing and querying.
> >
> > Zemantic is implemented just like they describe here, it uses a
> > forward/reverse index that maps nodes to integers, and then stores
> > those
> > integer mappings into a four dimensional btree index.  Essentially (and
> > coincidentally) exactly the same as described in this paper.
>
> Well, that's freaking good news. :^)  How bout the whole "context"
> thing?

An InformationStore is a TripleStore with contexts.

so yes. ;)  you get to choose between triple graphs and "quads", essentially.

-Michel

>
> I'll be interested in what your AMD64 box does on their 3M triple
> benchmark. :^)  I like the part where they said Kowasi (or whatever it
> is called) couldn't be tested because it died when the disk filled up.

Yeah i wonder about that whole benchmark thing, they say 10, 50 and 250 
million statements, but what does that mean, what hardware?  what kinds of 
schemas?  lots of different predicates?  all the same predicates?

and the bypassing the database thing can be good, and bad.  what about 
transactions?  what if you *want* your triples in a relational table?   I 
take their benchmark statements with a grain of salt, sure, give me a 4-way 
64 bit Itanium with 8 GB RAM and a terrabyte of storage, a reiserFS 
filesystem using ZODB directory storage and I bet I could pump 250 million 
statements into zemantic, but what does that prove?  neither what they say or 
what I propose is ar eal benchmark w/o more information.

-Michel

>
> :^)
>
> --Paul