Every number is reproducible

At a Glance

How Attest compares to conventional approaches on the metrics that matter for knowledge systems.

Graph/Vector/Relational figures are based on public documentation. Attest numbers come from automated benchmarks in this repo.

Engine Performance

Attest ships a custom Rust storage engine — append-only claim log with maintained indexes, file locking, and CRC32 crash recovery.

How we measured

• Rust engine: Criterion benchmarks in rust/attest-store/benches/store_bench.rs. 1,000 pre-built claims, 100 entities, in-memory store. black_box() prevents compiler optimization.
• ContextFrame assembly bottleneck is Python-side, not storage lookup — Python overhead for query result assembly.

Reproduce it

# Rust microbenchmarks
$ cd rust && cargo bench

# Python performance tests
$ uv run pytest tests/integration/test_performance.py -v

Curator Accuracy

The curator triages incoming claims: store, skip, or flag for review. We test this against a set of 250 expert-labeled claims.

This is the heuristic curator (no LLM). It runs offline, with zero API calls. LLM-backed curators can achieve higher accuracy on nuanced claims but require an API key.

Reproduce it

$ uv run pytest tests/eval/test_curator_accuracy.py -v

Edge Recovery (Link Prediction)

Given 80% of a real biomedical knowledge graph (Hetionet), can the system predict the withheld 20%? This tests whether structural embeddings capture real biomedical relationships — not just text similarity.

Reproduce it

$ uv run pytest tests/eval/test_hetionet_holdout.py -m slow -s

Structural Embeddings vs. Vector DB

Attest computes embeddings from graph topology (SVD on normalized adjacency), not from text. "Aspirin" is near "inflammation" because they’re connected in the graph, not because the words appear together in a corpus. This means:

• No embedding model needed — zero API cost, no GPU, deterministic results
• Embeddings encode relationships, not words — entities connected to similar neighbors are close in embedding space
• Auto-update — when claims change, embeddings recompute from the new graph. No separate vector DB to sync
• Link prediction built-in — 17.35% edge recovery on Hetionet proves the embeddings capture real biomedical structure

Comparison based on public documentation review:

Ingestion Pipeline

Attest’s connectors aren’t just data loaders — they run a full extraction pipeline:

Three lines to ingest from any source with full provenance:

db = AttestDB("knowledge.attest")
conn = db.connect("slack", token="xoxb-...", channels=["#research"])
result = conn.run(db)  # extracts, validates, ingests with provenance

Without Attest, you’d build each of these yourself:

Cross-Language Verification

The Python and Rust layers must produce identical results — same entity IDs, same claim hashes, same content IDs. We verify this with 118 golden test vectors covering entity normalization, hashing, chain hashes, and confidence scoring.

Reproduce it

# Generate vectors from Python
$ uv run python scripts/generate_golden_vectors.py

# Verify in Rust
$ cd rust && cargo test

What Only Claim-Native Can Benchmark

Traditional graph database benchmarks (LDBC SNB, etc.) measure query throughput and traversal latency. Those benchmarks don't test the things that make Attest different, because no other database does them.

Test Suite

# Run everything except slow eval tests
$ uv run pytest tests/unit/ tests/integration/ -q

# Run Rust tests
$ cd rust && cargo test

# Run full eval suite (slow, downloads data)
$ uv run pytest tests/eval/ -m slow -s

Capability Comparison

Most tools store facts. Attest stores claims — with provenance, confidence, and contradiction handling built into the engine. Here's how that changes what's possible.

This comparison is based on public documentation review. Where we've tested a system directly, we note it. Capabilities marked as "possible with custom code" mean the core engine doesn't provide it out of the box.

The Core Difference

Attest is not a general-purpose database, a vector store, or an LLM memory layer. It's a claim-native database — purpose-built for the case where knowledge comes from multiple sources, contradicts itself, and needs to be retracted or corrected over time.

If your use case is "store text and retrieve it by similarity," a vector database is simpler. If your use case is "model a fixed graph schema," Neo4j is battle-tested. If your use case is "conversational memory for a chatbot," Mem0 or Zep may be a better fit.

But if you need to know who said what, when, and how confident they were — and you need the system to handle the case where a source turns out to be wrong — that's what Attest was built for.