Run 35190d5 · Own gold set (private corpus)

dirty tree · not cited
determinism ok
private corpus · not reproducible

Benchmark metrics
Metric	Value	Definition
Recall@5	1.0000	Mean over questions of \|evidence ∩ top-k\| / \|evidence\|.
nDCG@5	0.9513	With binary relevance.
MRR	0.9426	Mean of 1/rank of the first evidence session.
Precision@5	0.2250	Mean over questions of \|evidence ∩ top-k\| / k. It can't reach 1 when a question has fewer than k evidence sessions, so it is shown on receipts only.

Conditions
Queries	56
k	5
Strategy	bm25
Gold path	benchmarks/recall-gold.json
Tests ran	255
Tests failed	0
Environment	Hardware, OS and Python version: not recorded for this run.
Ledger row	59

Raw row

{"ablation": {"baseline": "bm25", "delta": {"context_tokens_mean": -35.553571428571104, "latency_ms_mean": 0.04012278519179979, "latency_ms_p50": 0.046999492042232305, "mrr": 0.0, "ndcg_at_k": 0.0, "precision_at_k": 0.0, "recall_at_k": 0.0}, "treatment": "bm25+acr"}, "acr_gate": {"checks": {"age_stratum": {"old_age_days": 90, "old_count": 0, "old_ndcg_baseline": 0.0, "old_ndcg_treatment": 0.0, "passed": true, "recent_count": 56}, "no_harm": {"determinism_ok": true, "latency_p50": {"baseline": 4.787604011653457, "budget_ms": 10.0, "passed": true, "treatment": 4.834603503695689}, "mrr": {"baseline": 0.9425595238095238, "passed": true, "treatment": 0.9425595238095238}, "passed": true, "recall": {"baseline": 1.0, "passed": true, "treatment": 1.0}}, "primary": {"delta_ndcg": 0.0, "margin": 0.01, "passed": false}, "significance": {"alpha": 0.05, "df": 55, "mean_delta": 0.0, "n": 56, "passed": false, "t": 0.0, "t_crit": 2.009}, "win_rate": {"improved": 0, "passed": false, "regressed": 0, "ties": 56}}, "decided_strategy": "bm25", "params": {"alpha": 0.05, "latency_budget_ms": 10.0, "margin": 0.01, "mrr_eps": 0.0, "old_age_days": 90, "recall_eps": 0.0, "stratum_eps": 1e-09}, "passed": false}, "as_of": "2026-06-25", "checks": {"determinism_ok": true, "index_validate_ok": true, "test_pass_rate": 1.0, "tests_failed": 0, "tests_ran": 255}, "commit": "35190d5", "committed": "2026-06-25T04:17:03Z", "dirty": true, "gold_path": "benchmarks/recall-gold.json", "metrics": {"context_tokens_mean": 3654.8035714285716, "latency_ms_mean": 4.894943446613199, "latency_ms_p50": 4.834603503695689, "mrr": 0.9425595238095238, "ndcg_at_k": 0.9512578784218547, "precision_at_k": 0.22500000000000003, "recall_at_k": 1.0}, "mode": "acr-gate", "num_queries": 56, "reliability": 100.0, "run": "2026-06-25T04:50:44Z", "schema": "agentware-bench/v1", "strategy": "bm25", "subject": "feat(doc): onboarding to include knowledge base autocommit ON recommended with opt out", "top_k": 5}

reliability is an internal composite health score (0–100), not a benchmark metric.

Run 35190d5 Own gold set (private corpus), recorded 25 June 2026, 04:50:44 UTC.