Why the Trails Did Not Converge

We set a convergence criterion for trail learning, measured it, and missed it: hop counts fell by roughly half the threshold we had set. The report says so in plain text. This is the engineering post-mortem: what trails were supposed to buy, why this forest left them almost nothing to buy, and where they should start paying.

What trails were supposed to do

In a MonkeyLLM forest, an agent answers questions by navigating. It enters through search, reads a node's passport (its curated card: title, summary, tags), follows a link, and opens only what it needs. Pheromone is the third move: successful hunts deposit traces along the path they took, and repeated traces mint shortcut links. In plain English, paths that work get paved.

The hypothesis was simple. Run the same class of hunts repeatedly over the same territory, and average hops per correct answer should trend down, settling below a threshold we fixed before running anything. Ant colonies do this. We expected the forest to do it too.

What we measured instead

Hops did fall. They fell by roughly half the threshold, and the criterion was not met. We wrote that down and shipped the report with the failure in it, because a benchmark you can only pass is not a benchmark. The companion piece, why the report keeps a failure section, covers the policy decision. This post covers the mechanics.

There was almost nothing to compress

Entry search on this corpus is BM25 over SQLite: recall@5 = 1.00 at 1.3 ms p95. No embeddings, no vector database, no GPU. Weighting the curated passport fields (title, aliases) closed the gap to hybrid search entirely; the embedder stays supported and optional, it simply was not needed here.

With entry this sharp and passports this disciplined, the navigator lands next to its target cold. On this benchmark it averages about one and a half hops per answer before any trail exists. Trail learning works by compressing paths, and a walk of one and a half hops has almost no slack: the floor is one hop, so the mechanism was asked to squeeze savings out of a corridor that was already nearly straight. It squeezed out less than the criterion demanded.

Curation and trails are economic substitutes

The conclusion is not that trails do not work. It is that curation and trail learning purchase the same good, shorter future walks, from two different budgets. Curation pays at write time: ingest spends 1.71 s per document building passports whose summaries pass a sixty-token contract. Trails pay at read time, out of hunts that already succeeded. On a corpus where the write-time spend already delivers near-floor walks, there is nothing left for read-time learning to buy.

That framing makes a testable prediction. Trails should pay in deeper corpora, where entry search drops you in the right region rather than on the exact node, and walks stretch to many hops. There, a paved shortcut saves real tokens on every future hunt. It also lines up with the Forest Principle: spend intelligence on the environment so you can spend less on the model. We spent it on curation first, and on an 11-question benchmark of shallow walks, curation ate trail learning's lunch. 11 questions is small, and we say so.

What changes, what stays

Pheromone stays in the engine. The unmet criterion stays in the report exactly as measured. The next corpora get deeper on purpose, because if the substitute framing is right, trail savings should appear roughly where curated entry stops being enough on its own.

Reproduce the failure

The corpus, question sets and harness are committed, and every table regenerates with one command, including the one we failed. Clone MonkeyLLM on GitHub, rerun the trail experiment on your own corpus, and tell us where convergence starts for you. The preprint and everything else live at monkeyllm.com.