Memory Systems Need Canaries, Not Leaderboards
Operating eight memory integrations with nightly readiness canaries, a four-state taxonomy, delta-based alerting, and full regression sweeps after every repair.
Memory Systems Need Canaries, Not Leaderboards
We operate eight memory integrations: Hermes FTS, OpenViking, QMD, GBrain, Cognee, Honcho, LCM, and LibraVDB. On July 17 the nightly readiness sweep returned 8 of 8 PASS, 58 of 58 cases passed.
Two days earlier, OpenViking and QMD were blocked. The all-green sweep exists because four providers were repaired and the full sweep was rerun.
The status taxonomy
PASS. Adapter started, health responded, known fact written and retrieved, cleanup succeeded.
BLOCKED. Precondition absent. Service down, endpoint unreachable. Not an adapter quality signal.
DEGRADED. Part of the path works. Health responds but retrieval is stale. Write succeeds but read times out.
FAIL. Contract violated. Provider was reachable and returned wrong results.
Delta-based alerting
Alert on status transitions, not every green run. OpenViking’s sequence: BLOCKED, PASS, BLOCKED, PASS. QMD’s: BLOCKED, DEGRADED, PASS, BLOCKED, PASS. Those sequences are the operational signal.
July 17 proof table
| System | Status | Score | Cases |
|---|---|---|---|
| Hermes FTS | PASS | 8/8 | 8 |
| OpenViking | PASS | 3/3 | 3 |
| QMD | PASS | 9/9 | 9 |
| GBrain | PASS | 9/9 | 9 |
| Cognee | PASS | 10/10 | 10 |
| Honcho | PASS | 10/10 | 10 |
| LCM | PASS | 4/4 | 4 |
| LibraVDB | PASS | 5/5 | 5 |
Readiness is not universal retrieval quality. This is a canary, not a leaderboard.