August 20, 2026. I am testing whether VALESKA can find the evidence a person actually needs, rather than whether I can produce an attractive search demonstration. VALESKA is the document and organizational-memory system I am building. Before an AI assistant answers from that memory, retrieval has to bring back the right material. A fluent answer cannot repair evidence that never reached it.
Yesterday I added a retrieval scorer. Today I make it usable through different clients and annotate 33 questions against the Guyana document collection. I establish the expected documents by reading the collection, not by accepting whatever the search returns. Otherwise I would be letting the system write its own answer key. These are internal research results, not a customer acceptance certificate.
What the numbers ask
Precision at one asks whether the first result is relevant. Recall at twenty asks whether the expected evidence appears within the first twenty results. Mean reciprocal rank, or MRR, rewards finding it nearer the top and gives a miss zero. Latency measures the wait. For a company deciding whether an assistant is useful, these describe different costs: checking the wrong first document, searching down a list, missing evidence entirely, and waiting.
I also correct the scorer itself. It had averaged reciprocal rank only over successful queries, so recovering a missed answer low in the list could make the average look worse. It now includes every annotated question. Earlier MRR figures are not comparable. Getting the arithmetic right matters before I use the score to choose an engineering change.
More evidence, more waiting
The baseline finds the right first result for 22 of 33 questions and the expected evidence within twenty results for 29 of 33. Broadening the approximate search's candidate pool improves the latter to 32 of 33. The first-result score stays at 22 of 33. MRR moves from 0.727 to 0.741. Median latency rises from 151 to 291 milliseconds; the 95th percentile rises from 186 to 546 milliseconds. That is a coverage improvement with a waiting-time cost, not an across-the-board win.
By this morning the change is applied, and both direct and service-based tests reproduce 32 of 33. One miss remains. Search can still miss relevant evidence in smaller collections. An empty result therefore does not establish that the answer is absent. The broader search mitigates this defect; it does not resolve the underlying design problem.
A separate correction to the clock
Yesterday's cost-model work exposes another measurement failure, this time in document classification rather than retrieval. Earlier timings of 345 and 397 seconds per document were contaminated by my own background jobs competing for the graphics processor. The recorded clean measurement is 8.2 seconds per document, with a much slower first call. The repository records the discrepancy as roughly 48-fold. This is removal of a benchmarking error, not a newly achieved model speedup.
I keep that correction beside the retrieval results because both affect a business decision. Bad timing can distort a capacity estimate; a selectively quoted search score can hide the review work left to staff. Neither experiment establishes production throughput, performance on another company's documents, or complete permission isolation.
What I need next
I need to test a correction to the collection-filtering problem against the same questions, preserving the misses and latency figures in the report. I also need repeatable classification timings with resource contention and cold starts stated. Today's evidence is enough to reject a flattering summary. It is not enough to declare retrieval accepted.
Historical basis: VALESKA commits fc2f77c, eaa00c1, 45c1449, 1643d91, 6607d62, August 19-20, 2026 (Pacific time).