Every Number Here Ran on One RTX 3060

The hardware section of most benchmark reports reads like a datacenter invoice. Ours reads like a mid-range gaming PC: one RTX 3060 with 12 GB of VRAM. Every number we publish, 0/11 to 11/11, 0.58x tokens per correct answer, 8.4 s p95, came off that single card.

The whole rig

A quantized 12B model fits the 3060 whole: no sharding, no offload, no second card. That one model did all the reasoning in the benchmark, reading passports (the curated title, summary and tags on each node), choosing links, and writing answers. Token budgets are what make a model this size sufficient: every hop the agent takes over MCP is budgeted, so each reasoning pass sees a small, curated context instead of a sprawling one.

The retrieval side asks for even less. Entry search is BM25 over SQLite, hitting recall@5 = 1.00 at 1.3 ms p95 with no embeddings, no vector database, and no GPU at all. Weighting curated fields like title and aliases closed the gap to hybrid search entirely. The embedder is optional and supported, not removed; the point is that you are never forced to fund one.

What fits where

Your card does not need to be our card. The paths, from most local to least:

• 12 GB (RTX 3060 class): a quantized 12B runs whole. This is the exact benchmark rig.
• 8 GB: tighter quantization, or CPU offload via llama.cpp, trading some speed for fit.
• No GPU: point the chat binding at any OpenAI-compatible endpoint, such as OpenRouter, a llama.cpp server, or Ollama running on whatever box has the memory.

Bindings are configuration, not code. The Studio's Models screen holds the endpoint, the token budgets, and the answer store's running score, so swapping a local Ollama for a remote endpoint is an edit, not a migration. The forest itself does not care where the tokens come from; it only cares that each hop stays inside its budget.

Mid-range was the point

It was also the thesis. The benchmark argues that multi-hop performance comes from structure, not scale: the forest carries curated passports and typed links, so a 12B model navigating it outperforms the very same model reading top-k chunks, 11/11 against 0/11. Spend intelligence on the environment so you can spend less on the model. That claim would ring hollow demonstrated on eight datacenter GPUs. Demonstrated on a card you can buy used, it is checkable by almost anyone, which is what a claim like this needs to be. The longer version of the argument is the Forest Principle.

The usual honesty applies here too. Eleven questions is a small set and we say so, the paper is a preprint and we say so, and the corpus, question sets and harness are committed so that every table regenerates with one command, on hardware most readers already own.

Deploy on yours

Self-hosting is four commands. The engine is Apache-2.0, the apps are AGPL, and your data never leaves your infra. First boot drops you into the setup screen and asks for exactly what it needs, nothing more.

The end-to-end walkthrough lives in the Docker guide and the deploy docs. Bring a 3060, an 8 GB card, or no GPU at all, deploy from the repo, and grow a forest on your own metal tonight. Details at monkeyllm.com.