Kmemo 2.0.0 is out, and it closes two gaps from the 1.0 release. The verifier's catch rate is now measured, GPTCache is the new benchmark, and the core no longer requires the JVM. A commenter also found a blind spot in the match path, which led to a new guard.
Verifier Performance: Measured at Last
The first Kmemo post reported guard-only numbers: 67% of near misses rejected and 88% of paraphrases kept. The optional verifier — a model call that sees whatever the guards let through — was described as the residual's job, but its actual impact was never measured. 2.0 fixes that.
Against sentence_transformers.CrossEncoder over cross-encoder/quora-distilroberta-base, the verifier stops four fifths of what the guards miss. On the held-out corpus, the false-hit rate drops from 0.291 to 0.058, and rephrasings kept falls from 0.881 to 0.452. On the validation set, false-hit goes from 0.324 to 0.074, and rephrasings kept from 0.882 to 0.686.
The second column is why the verifier stays opt-in. It rejects a third of the genuine rephrasings the guards kept. Your verifier won't behave identically, but the table shows how much of the residual is reachable by a model that reads both prompts.
GPTCache Comparison: Honest Numbers
The first post compared Kmemo to a threshold-only cache — the baseline every tutorial builds. That's fair, but the author wrote it, making it a weak claim. 2.0 compares against GPTCache on the same blind corpora, same pairs, with retrieval factored out so both sides see the same candidate and only decide whether to serve it.
At zero cost, Kmemo's free lexical chain loses the headline number. GPTCache's ONNX cross-encoder serves fewer false positives: 0.221 vs 0.291 on one split, 0.108 vs 0.324 on the other. But it gets there by refusing more than half the genuine rephrasings, while Kmemo keeps 88%. That strictness costs roughly half the cache savings.
At comparable cost, the picture reverses. GPTCache's cross-encoder is a transformer inference per candidate. Kmemo's guards plus verifier also spend a model call. The results:
| Corpus | Kmemo guards + verifier | GPTCache OnnxModelEvaluation |
|---|---|---|
| held-out | false-hit 0.058, kept 0.452 | false-hit 0.221, kept 0.476 |
| validation | false-hit 0.074, kept 0.686 | false-hit 0.108, kept 0.451 |
Both columns are in the README, including the loss. A benchmark reported by the metric that flatters its author doesn't get believed on the others.
One gotcha if you run this yourself: OnnxModelEvaluation.evaluation wraps its body in except Exception: return 0. Every internal failure returns a similarity of zero, indistinguishable from a confident refusal. A harness that trusted the documented entry point would record a false-hit rate of 0.000 for GPTCache — a broken competitor that would be entirely your own bug. The repository's harness proves the evaluator works before believing a single score. The gate is mechanical: it checks whether the evaluator functions, not whether it agrees on hard pairs.
The Guard a Commenter Requested
A reader pointed out that every guard receives two strings, both prompts. The stored answer is on the entry, but nothing in the match path reads it. They were right. Consider:
what is the capital gains tax rate when i sell a second home
what is the capital gains tax rate when i sell a primary residence
No number differs. No unit, no negation, no flipped comparison. Nothing in the prompts is evidence, so every guard abstains — correctly. But the cached answer says "Gain on a second home is taxable in full," and it goes to someone asking about a different house.
Kmemo 2.0 adds a guard that reads the answer. When two prompts differ only by a substitution and the cached answer names the word the query replaced, that answer was written for the other question.
SemanticCache(embedder, guards = MatchGuards.responseAware())
It refuses 14 of the 116 near-miss lookups the default chain still serves, and none of the 164 rephrasing lookups. It's opt-in because the evidence is regression-based, not blind: those answers were written by the author for this measurement, since no corpus of real paired answers exists. A semantic cache corpus records prompts, and the near misses worth catching are exactly the ones whose prompts look alike. Folding this guard into the default would quietly downgrade the evidence behind all the others.
Core No Longer Needs the JVM
kmemo-core and InMemoryStore now compile for JVM, iOS, macOS, Linux, Windows, JS, and WasmJS — with no new dependency, since kotlin.time.Instant and kotlin.time.Clock went stable in Kotlin 2.4. The Redis, Postgres, and Spring adapters stay JVM-only because they wrap drivers that exist nowhere else.
The tricky part wasn't java.time. It was the access-ordered LinkedHashMap, which doesn't exist outside the JVM. The store's eviction, exact-match layer, and verifier memo all relied on it.
Other Changes
The default chain gains an eleventh guard for near misses where nothing changed but something was added. "How do I deploy a Rails app" vs "the same question on Heroku" has perfect word overlap, so no lexical check caught it. Measured at zero false rejections across all three corpora.
Four opt-in pieces sit around the match path:
- Reranking so each candidate tried adds something the last didn't
- Quantized retrieval that decides which candidates are looked at, never whether one is served
- Deduplication on write
- Adaptive per-scope thresholds that refuse to run without a verifier watching
Migration from 1.x
Five named breaks, each with who it affects and the edit that resolves it, are in the migration guide. The easiest to miss: Maven users need kmemo-core-jvm, since Maven doesn't read Gradle module metadata. Gradle builds change only the version number.
implementation("io.github.nacode-studios:kmemo-core:2.0.0")
Apache 2.0, on Maven Central. Repository: NaCode-Studios/Kmemo.
If something here is wrong, or a configuration is missing from the comparison, say so. That's how the guard in the third section got written.
