Tokens per Correct Answer, Not per Query
Per-query cost makes failure look like a discount. Measured per correct answer, the forest navigator spends 0.58x the tokens of an iterative-RAG baseline, same 12B model. Here is the arithmetic.
Every cost dashboard prices LLM systems per query, and per query, failure looks like a discount. In our benchmark the forest navigator spends 0.58x the tokens of an iterative-RAG baseline per correct answer, on the same 12B model. The denominator is the whole story.
The wrong denominator
Tokens per query answers the question "what does it cost to try?" Nobody ships a product that tries. The thing you actually buy from a question-answering system is correct answers, so the honest unit is tokens per correct answer: total spend across the workload, divided by the answers you could actually use.
The moment you switch denominators, cheap-but-wrong pipelines stop looking cheap. Every failed query still burned its retrievals and its context stuffing, and then it bought you something with negative value: a retry, an escalation to a bigger model, or a confident wrong answer somebody acts on. Failing cheaply is not economy.
An honest worked example
These numbers are invented for the arithmetic and labeled as such; our measured ratio comes after. Take two systems over the same 100 questions:
- System A spends 1,000 tokens per query and gets 40 right. Total 100,000 tokens, which is 2,500 tokens per correct answer.
- System B spends 1,500 tokens per query and gets 75 right. Total 150,000 tokens, which is 2,000 tokens per correct answer.
B is 50 percent more expensive per query and 20 percent cheaper per correct answer, while also failing 35 fewer times. A per-query dashboard ranks A first. A per-correct-answer dashboard ranks B first, and it is the only one of the two that matches your invoice to the value you received.
What we measured
On the 11 strictly multi-hop questions (each needing at least three chained hops), the same 12B model ran as an iterative-RAG loop and as a MonkeyLLM forest navigator. Counting every token spent and dividing by correct answers, the navigator lands at 0.58x the baseline's cost.
It is also faster where it hurts: 8.4 s p95 against the baseline's 17.5 s. The full head-to-head, including how the baseline works, is in agentic RAG vs budgeted navigation.
Where the 42 percent comes from
Iterative RAG pays rent on the same real estate every round: each reformulation re-retrieves overlapping chunks and the model re-reads them. The navigator reads passports instead (a passport is the node's curated title, summary and tags), follows links, and opens a full node only when it needs one, with every hop token-budgeted over MCP.
That thrift is purchased at write time. Ingest spends 1.71 s per document building the curation, with 100% of summaries passing a sixty-token contract. Spend intelligence on the environment so you can spend less on the model.
Correctness stays auditable too. In the Studio's Ask screen every answer arrives standing on the nodes that support it, so "correct" is something you check, not something you feel.
Check the arithmetic
The corpus, question sets and harness are committed, and every table regenerates with one command. Read the preprint (we call it a preprint because that is what it is), rerun the numbers from the repo, and find the rest at monkeyllm.com.