SPARQL 1.1 conformance

Claim boundary — read this before quoting any number on this page. Quipu passes all Working Group–approved W3C SPARQL 1.1 Query, Update, Protocol and Results tests at rdf-tests 369a90d: query syntax 86/86, query evaluation 168/168, update 93/93, protocol 34/34, result format 10/10. Exceptions, each named below: federated query (SERVICE) passes 6/7, with 1 refused by policy (variable endpoints); entailment regimes are scored separately (35/70 passed, 0 failing, 35 declared non-goals); SHACL-SPARQL, OWL, RIF and D entailment are declared non-goals. What these counts are. Working Group–approved tests only. The query-evaluation manifests list 225 tests, and the 168 approved ones are scored; the 57 Proposed or unclassified are not run. The update-syntax suites are not run yet. The SPARQL 1.0 suite is not run. These manifests hold what SPARQL 1.1 added; the SPARQL 1.0 tests (rdf-tests sparql/sparql10) also bear on SPARQL 1.1 Query conformance, and this harness does not score them yet. A trial run found real failures there, including sameTerm inside FILTER, so read the counts above as the 1.1 additions only (aegis-soqv1r). This score is fitted to this suite. Quipu’s failures here were found by running this suite and fixed against it, case by case, so a perfect score is partly a record of that work rather than an independent sample. Other stores measured with the same harness were not tuned to it. Every class below is scored separately and is never combined into a single compliance percentage, because a blended figure would hide exactly the classes that are not implemented at all.

These results come from the W3C RDF Tests suite at a pinned revision, run against throwaway stores by a checked-in runner. You can re-derive every number on this page yourself — the commands are below.

For what Quipu does with a graph once it is correct — handing it to another store, and composing another store’s without trusting it — see Sharing & Federation. That page states its own claim boundary for SERVICE, including the configured-endpoint policy deviation scored below.

What was measured

Results by class

unsupported means the harness cannot execute the test at all — the capability is missing, not merely wrong. Those cases stay in the denominator and each one carries a named reason further down this page.

The final row is an arithmetic total, not a score. It is here so the class rows can be checked against the ledgers, not so it can be quoted as a percentage.

Other stores, same harness

The same discovery, test selection and result comparison, run against other stores at the same rdf-tests revision (369a90d1). Scores use RDF term equality, the rule quipu is held to. “Same value” counts failures whose answer had the right values in a different lexical form; they stay failures and are shown separately, so a design choice is not presented as a wrong answer.

The quipu row is this page’s own ledger. Quipu’s runner compares exact labels and has no same-value tag, so that cell is empty rather than zero.

Disclosure. Quipu parses SPARQL with spargebra and models RDF with oxrdf, both from the Oxigraph project. Where the two agree, part of that agreement is shared code.

Quipu’s score is fitted to this suite. Its failures were found by running this suite and fixed against it, case by case. The other stores were not tuned to this harness.

Pinned versions, the fairness rules, every competitor deviation checked by hand, and the per-case ledgers are in benchmark/competitors.

RDF syntax

The W3C RDF 1.1 and RDF 1.2 syntax suites at the same rdf-tests revision (369a90d1). Every manifest case is counted, including cases the manifests have not marked approved.

RDF 1.2 is measured and not supported. Quipu is built without RDF 1.2, so it cannot parse a triple term. The RDF 1.2 cases are enumerated from the pinned manifests and not run: a loader that rejects all RDF 1.2 input would “pass” every negative-syntax case, and those passes would read as partial support. No RDF 1.2 case is scored as a pass until the support exists.

Ledgers: rdf11-syntax.json and rdf12-syntax.json.

Query evaluation, by feature family

The family is the pinned suite’s own directory for each manifest, so this grouping is re-derivable by anyone holding the same suite revision. It is the map of where the work is: the largest failing families are where conformance moves the most per fix.

The not-yet-passing tests, named

Every test that does not pass is listed here with its W3C identifier, so a claim of progress can be checked against a specific case rather than a count.

Federated query (SERVICE)

Quipu passes 6/7 approved W3C Basic Federated Query cases. SERVICE is a query-planned subquery path using the same declarations and labels as RemoteProvider; it is separate from GraphProvider whole-query fanout and is not open federation. The variable-endpoint case is a deliberate policy deviation because query data cannot widen the configured remote allowlist.

Why a class is unsupported

Grouped by the reason the runner recorded.

Corrections

Kept on the page so a changed number never changes silently.

• 2026-09-24: update was 37 of 93. Until this date the page reported update 37/37. The runner discovered only 37 of the 93 approved update tests: the delete, delete-data, delete-insert, delete-where, clear and drop manifests declare their tests with Turtle’s a rather than rdf:type, and the parser matched only the latter. On the full 93, quipu passes 93/93. The runner now pins the approved count per class at the pinned suite revision and refuses a run that discovers a different number.

Re-derive these numbers

git clone https://github.com/w3c/rdf-tests /tmp/rdf-tests
git -C /tmp/rdf-tests checkout 369a90d1a60c021b746df2e411da0ff36258a758
cargo build --release --bin quipu --bin quipu-server --features shacl,onnx,server
QUIPU_BIN="$(cargo metadata --no-deps --format-version 1 | python3 -c 'import json,sys; print(json.load(sys.stdin)["target_directory"])')/release/quipu"
SUITE="/tmp/rdf-tests/sparql/sparql11"
python3 benchmark/public/sparql11_syntax.py --suite "$SUITE/syntax-query" \
  --quipu "$QUIPU_BIN" --output /tmp/sparql11-syntax.json
python3 benchmark/public/sparql11_evaluation.py --suite "$SUITE" \
  --quipu "$QUIPU_BIN" --output /tmp/sparql11-evaluation.json
python3 benchmark/public/sparql11_federated.py --suite "$SUITE" \
  --quipu "$QUIPU_BIN" --output /tmp/sparql11-federated-query.json

A nonzero exit from an incomplete runner is expected while any test fails: it writes the complete ledger first, then reports that not everything passed. Use the regression gate below to tell worse than committed apart from not yet perfect.

Per-class reproduction commands are embedded in the ledger itself, under reproduce.per_class.

The regression gate

python3 benchmark/public/check_regression.py \
  --baseline benchmark/public/results/sparql11-evaluation.json \
  --candidate /tmp/sparql11-evaluation.json

It exits nonzero when a class’s pass count drops or when a test that passed in the committed ledger stops passing, and it names the tests. Improvements exit zero and print the tests that newly pass — a prompt to refresh the baseline. This is what runs in CI on every release, so a published number cannot silently get worse.

Full ledgers

• benchmark/public/results/sparql11-syntax.json
• benchmark/public/results/sparql11-evaluation.json
• benchmark/public/results/sparql11-federated-query.json

Each row records its class, test identifier, manifest, query and result paths, status, and diagnostic or unsupported reason.

W3C SHACL conformance

Quipu passes 98/98 manifest-reachable SHACL Core cases. SHACL-SPARQL remains a deliberate non-goal: 0/22 pass and 22 are unsupported. Quipu does not advertise SHACL-SPARQL support; the upstream SPARQL validator is incomplete, and support requires the full 22-case manifest gate rather than a partial claim. This score uses the context-free native validator; write-gate transaction behavior is tested separately.

Pinned W3C Data Shapes revision: 9c863967bceaef1a87c24e4dd761eda763823120. The pinned manifest exposes 120 approved cases (98 Core + 22 SHACL-SPARQL). nodeValidator-001.ttl exists in the checkout but is not manifest-reachable and is not scored.

Entailment-regime commitments

2 of 6 regimes are goals (RDF, RDFS): 35/35 of their cases pass. The remaining 4 are deliberate non-goals. Ledger re-derived 2026-09-26T03:25:18Z by CI run, from quipu 60f4e9b6b5a2. Local RDFS and OWL extensions beyond a goal regime are not standards-regime claims.

Do not read the goal-regime fraction as “nearly done”. The two numbers have different characters. Most RDF-regime cases are bind* tests answerable under simple entailment, so they pass without any additional inference — a high RDF score is not evidence of an entailment engine. The RDFS score DOES reflect one: an RDFS closure (rdfs2/3/5/7/9/11) is materialised into the graph’s companion inferred graph and composed into the default graph when the regime is in force, which is what a query like SELECT ?x WHERE { ex:a ?x ex:c } needs — its predicate is a variable, so the entailed triple has to EXIST and cannot be produced by rewriting the pattern. What remains failing is not more of the same closure: it is container and axiomatic shapes beyond those six rules, and OWL-flavoured cases filed under RDFS.

Machine ledgers: shacl-core.json and sparql11-entailment.json.