Callosum: Bridging quick decisions and deep work

TL;DR

Callosum on GitHub (pip install callo for the CLI) is a llama.cpp sidecar server that gives your generative text model the ability to generate tokenless decision probabilities, and additionally introduces a novel hybrid decision mode where class probabilities are measured after calibrated short reasoning converges the probability or reaches a time or token limit. Callo gives self-hosted generative models System One capabilities that eliminate the RAM cost of loading an additional model, thus enabling System One when you otherwise wouldn’t get it, or enables you to run a bigger model where you would otherwise load two models side by side.

• System One API support
• One model for both your agent and its quick judgements: System One and System Two running at the same time
• Use any text model that runs on llama.cpp
• Most llama.cpp forks are supported
• Calibrate your model, point the agent at Callo, and that’s it
• Optional short reasoning improves System One probability accuracy, especially on logic-dependent decisions.
• System One short reasoning can be budgeted by time, tokens or both, and stops early at probability convergence.
• Calibrate and compare speed and accuracy of several models at different settings against each other, and pick from the optimal configurations.

Why

An agent spends most of its time on slow work: long prompts, long answers. But it, and the tools around it, also need quick judgements all the time. Is this message positive or negative? Which of these four tools fits? Which way should the character move? The usual answer is to call a cloud model like “Jev”, or load a second, dedicated System One model next to your agent’s own, in order to get probabilities per option. If you stay local it takes more memory, which you may not have.

It turns out the model you already run can produce zero-token classifications with probabilities itself, nearly as well, if you read its answer through the llama.cpp logprobs API. The catch on stock llama.cpp settings is speed: a quick question that arrives while the agent is reading a long prompt waits in line behind it, about 40 seconds in my 32k token tests with a 35B-A3B model. And llama.cpp does not expose a System One API natively for generative models, only log-probabilities for next tokens. Callo solves that for you by calling llama.cpp with the right parameters and repackaging probabilities in a System One API response.

Additionally, Callo lets you extract the same zero-token class probabilities after a (short) reasoning trace (150-512 tokens goes a long way with the Sharp system prompt). Since dedicated System One models are not capable of reasoning, this is a unique additional capability of Callo.

How

callo sits in front of your local model server (llama.cpp) as a small front door that also exposes the System One API. Your agent talks to it exactly as before, and its traffic passes straight through. Quick questions get a lane of their own on the backend, so they never queue behind the slow work: the same question that waited 40 seconds on stock llama.cpp comes back in about two. And the probabilities previously hidden behind the logprobs API are now exposed as System One.

For each question, Callo asks the model to pick one of your options and reads how likely it finds each one, in a single step. It bins all next-token probabilities corresponding to each lettered option (“a”, “A”, “A.” etc.) into a single bin. You get a probability for every option, not just a single best-guess token, and images work too.

The name comes from the corpus callosum, the bridge between the brain’s two halves: here, between fast thinking and slow.

Use cases

With Callo’s system one API your model of choice can quickly

• Flag and block dangerous terminal commands in a coding agent harness
• Triage tickets
• Play games in real time
• Use the browser
• Classify documents, images, audio transcripts and video

…and more, all while simultaneously operating as an agent or generating text

What now?

Callosum opens the door for a new class of hybrids that are finetuned on both System One classification probabilities
and System Two agentic problem solving and text generation, where the user gets the best of both worlds. These kinds of finetunes do not exist yet, but in the meantime the generative model you already run will do a good job.

FAQ

Do native System One decision models have an architectural speed advantage over Callo?

TL;DR: No.

Assuming no concurrent work, neither has an architectural speed advantage. A native decision model and Callo’s zero-token System One request do the same core work: one forward pass over the prompt, with no generated tokens. Per token of prompt and per parameter, the compute is the same.

The speed differences come from what surrounds that forward pass.

1. How the answer is read out (this is the one real structural difference).
   • Llama.cpp’s native System One API reads every option’s score from fixed positions inside the single prompt batch.
   • Callo reads the top-k next-token logprobs at one answer slot. With single-token labels like A/B/C that is also one pass, so the cost is equal. If a label spans several tokens, or isn’t in the top-k, Callo’s marginalizer generates one extra token per level of its search. Each of those is a short extra request to llama-server.
2. Full-vocabulary logits. Callo needs logits over the whole vocabulary (150–260k entries) at the answer slot, plus a top-k sort. That’s small next to the prompt pass, but it isn’t free on short prompts.
3. Prompt length. Callo wraps the question in the model’s chat template and instructions. Native models use a compact format they were trained on, and fewer tokens means a faster prompt pass.
   • This cuts the other way too: native requests ignore id_slot, so they can’t pin a slot and reuse a cached prefix. Callo can. On a long, repeated context, Callo only processes the new tokens.
4. Position-bias correction. Some native decision models show a choice question twice, with the options in reverse order, to cancel the bias toward the first option, so it does roughly double the work. Callo’s correction uses a prior fitted ahead of time, which costs nothing per request. It also accounts for models where bias leans towards middle-options rather than first or last.
5. The proxy hop. Callo adds a loopback HTTP round trip and some JSON handling, a few milliseconds.
6. Model size. In practice this matters more than everything above: prompt-processing cost scales with parameters times tokens.

So a native model wins when its labels are multi-token or its format is much shorter. Callo wins on repeated long contexts thanks to prefix reuse, and on bias correction. Otherwise they’re even.

Is “any model as system one” unique to Callo?

TL;DR: Yes on llama.cpp, otherwise “no, but”!

No:

• SGLang 0.5.21 serves System One on any chat model with a Jinja template, when the answer labels are single tokens.
• vLLM: Only on main, only for three Qwen architectures, and only for choice questions.
• llama-server wrappers: a couple of third-party wrappers over plain llama-server exist (llamacpp-jev, llmman).

But, Callo has these additional features and advantages:

• Summing an option’s spelling variants, including labels that span several tokens
• Fitted temperature calibration
• Measured position-bias correction
• Think budgets, with a measured speed/accuracy trade-off
• Callo compare, that puts several models and configurations on one chart and picks according to your accuracy/speed preference
• The private lane, that keeps quick questions from waiting behind the agent’s long prompts
• Audio and video input
• Audio transcriptions that don’t wipe your agent’s cache: speech-to-text requests go to their own slot instead of any free one

Callosum on GitHub, with the install, the API and the measurements.