- clean tree
- determinism ok
- cited on this site
Reproduce at commit 42d58a0, which recorded this row. The site's install commands pin a later commit that hasn't been re-run yet.
| Metric | Value | Definition |
|---|---|---|
| Recall@5 | 0.9140 | Mean over questions of |evidence ∩ top-k| / |evidence|. |
| nDCG@5 | 0.8831 | With binary relevance. |
| MRR | 0.9104 | Mean of 1/rank of the first evidence session. |
| Precision@5 | 0.3328 | Mean over questions of |evidence ∩ top-k| / k. It can't reach 1 when a question has fewer than k evidence sessions, so it is shown on receipts only. |
| Queries | 470 |
|---|---|
| k | 5 |
| Strategy | bm25 |
| Gold path | benchmarks/longmemeval/longmemeval_s_cleaned.json |
| Tests ran | 332 |
| Tests failed | 0 |
| Environment | Hardware, OS and Python version: not recorded for this run. |
| Ledger row | 86 |
Raw row
{"ablation": null, "checks": {"determinism_ok": true, "index_validate_ok": true, "test_pass_rate": 1.0, "tests_failed": 0, "tests_ran": 332}, "commit": "42d58a0", "committed": "2026-06-25T15:34:24Z", "dirty": false, "gold_path": "benchmarks/longmemeval/longmemeval_s_cleaned.json", "metrics": {"latency_ms_mean": 10.522725419116583, "latency_ms_p50": 10.51206249394454, "mrr": 0.9103777978731046, "ndcg_at_k": 0.8830615827171846, "precision_at_k": 0.33276595744680854, "recall_at_k": 0.9140425531914894}, "mode": "eval", "num_queries": 470, "reliability": 96.56, "run": "2026-06-25T15:34:44Z", "schema": "agentware-bench/v1", "strategy": "bm25", "subject": "feat(benchmark): LongMemEval scorer + Mode A baseline + public repro docs", "suite": "longmemeval", "top_k": 5} reliability is an internal composite health score (0–100), not a benchmark metric.