The Usual Patches, and Where They Stop

Nobody ships naive top-k RAG and leaves it alone; the multi-hop failures show up in the first week, and the patches arrive in the second. Rerankers, query rewriting, HyDE, iterative loops, agentic retrieval: each one fixes something real, which is exactly why they are everywhere. The useful question is what each one costs, and where each one stops.

Rerankers

What they fix: ordering. A cross-encoder reads the query and each candidate together and catches relevance the first-stage retriever scored badly. On single-hop questions with a deep candidate list, this is often the cheapest quality win available.

What they cost: an extra model pass over every candidate, with latency growing in the depth of the list.

Where they stop: a reranker can only reorder what retrieval fetched. Multi-hop failure is an absence failure, not an ordering failure. The document that names the final answer typically shares no vocabulary with the question, scores near the bottom of the whole corpus, and is not in the top 50 to begin with. No permutation of the retrieved set contains it.

Query rewriting

What it fixes: vocabulary mismatch and underspecified queries. It is cheap, and often worth doing.

Where it stops: a better query is still one query. Rewriting cuts a better key for the same single lock. A three-hop question is three locks, and you only learn what the second key looks like after opening the first door.

HyDE

What it fixes: the query-document gap. Generate a hypothetical answer document, embed that instead of the query, and one-hop paraphrase retrieval genuinely improves.

Where it stops: to write text that resembles the hop-three document, the model would need to already know the answers to hops one and two. HyDE hallucinates a destination; the multi-hop problem is not knowing the route.

Iterative loops

What they fix: the actual structure of the problem. Decompose the question, retrieve, read, retrieve again. This is the honest baseline, because it takes real hops.

What they cost, measured rather than asserted: against budgeted forest navigation, on the same questions with the same local model, the navigator spent 0.58x tokens per correct answer vs the iterative-RAG baseline, at 8.4 s p95 vs the baseline's 17.5 s.

Per correct answer is the denominator that matters, because a loop that fails fast looks cheap per attempt while being expensive per answer. Failing cheaply is not economy. The full measurement is in agentic RAG vs budgeted navigation.

Where they stop: every iteration searches the same flat chunk pool. There is no structure to steer by, no cheap label to read before paying for a full chunk, and nothing learned in one run survives to the next.

Agentic retrieval

What it fixes: planning. Give the model search as a tool and let it decide when to call it again. This is the closest relative of what we build, and it does help.

Where it stops: it is agency over chunk soup. The agent can reason, but the environment gives it nothing to reason over: no links to follow from a document to its named dependency, no passports (curated cards of title, summary and tags) to read before opening anything, no budget shaping the walk, and no pheromone (usage traces from past successful hunts) accumulating into shortcuts. The same agency over structured terrain is where the numbers above came from.

The pattern in the patches

Every patch adds intelligence at read time to compensate for structure the store never had. Stack all five and you are still approximating, at inference prices, what a linked and labeled corpus provides directly. The alternative is to pay once at write time and structure the corpus itself. For why top-k dead-ends in the first place, see why multi-hop breaks RAG.

Read the numbers, including the failed one

The preprint also reports a convergence criterion we did not meet, and keeps it in the report (here is why). Read the preprint via monkeyllm.com, and if the framing holds up, star the repo.