DERI Galway announced recent performance of their YARS2 federated RDF repository:

The Semantic Web Search Engine developed at DERI is able to answer queries with more than 7 billion RDF statements in fractions of a second - the largest number reported so far anywhere in the world.

The detailed description is here. The YARS (older version project page is here. The YARS (olders version) wiki is here. The subversion repository is here.

Below are YARS2 implementation techniques that validate the texai approach.

  • YARS2 extends RDF triples to include context - calling this a quad. The common binary proposition stored in the texai KB also has a predicate (RDF property), first argument (RDF subject), second argument (RDF object) and context.
  • YARS2 allocates portions of the quad store to separate servers in a federation.

Here are YARS2 implementation techniques that could be adapted for use in texai:

  • YARS2 has a sophisticated secondary index scheme that minimizes index joins.

    A naive index structure for RDF graph data with context would require four indices: one on subject, predicate, object and context. For a single quad pattern lookup containing more than one constant, such a naive index structure needs to execute a join over up to four indices to derive the answer … Instead, we implement a complete index on quads which allows for direct lookups on multiple dimensions without requiring joins

    Texai has a need for example to lookup arg2 given the predicate, arg1 and context. An optimal composite secondary key, that also could be used for other query patterns is: <arg1, predicate, arg2, context>.

  • YARS2 performs Huffman coding to achieve up to 90% space compression for on-disk indicies.