Think in Latent Space, Act in Text: How HybridAgents Changes the Cost Structure

To use a tool, an agent currently has to verbalize almost every intermediate thought. It reads a search result, picks the next query, interprets a calculation, and writes all of it as tokens. That makes the trace easy to audit and easy to connect to tools, but cost and latency grow with the number of reasoning steps.

Latent reasoning points the other way. It performs intermediate computation in continuous vector space and avoids token decoding. The trade-off is that latent models have no native path to tool calls and no human-readable trace. The two approaches have required different media.

HybridAgents, published on October 13 by researchers at Stanford, UIUC, and Together AI, tries to move between them inside one generation process. Internal reasoning runs in latent space; tool calls and external observations stay in text. The paper reports an average 10.4% performance improvement across 11 benchmarks and a 35.5–54.7% reduction in text-token usage.

On the surface, that sounds like a cheaper and smarter agent. But you cannot bolt this onto an API model. It requires post-training and a modified serving stack, and the comparison conditions need careful reading. This article explains the architecture and numbers, then separates what builders can borrow from what they cannot.

The problem: thinking and acting share one medium

The common language of modern LLM agents is text. The model samples the next token; that token may be reasoning or a tool call. The environment responds in text, and the response is appended to the context. That simple protocol lets an agent combine search, Python execution, and file operations in one loop.

The cost is equally clear. Longer intermediate reasoning means more decoding, more latency, and more context. Latent reasoning can reduce that cost, but today's latent reasoners are sealed inside the model boundary. They have no way to call a tool and no way for a human to inspect the reasoning.

HybridAgents asks: can an agent think beyond human language while still acting through it? The answer is to switch between two modes within one generation.

Architecture: HybridLink and working memory

Two components carry the design: a lightweight adapter called HybridLink, which bridges text and latent modes, and a continuous working memory that preserves the hybrid context across turns.

[One generation process]

Text mode
  "I need to search"  -->  <latent> anchor token
                               |
                               v
Latent mode (no token decoding)
  h_t --> HybridLink --> e_{t+1} --> next hidden state
  repeat for several steps in continuous space
                               |
                               v
  return to text when </latent> wins the alignment check
                               |
                               v
Text mode
  tool call --> environment observation --> append to working memory
                               |
                               v
  next turn resumes from the text + latent + observation KV memory

HybridLink maps the last-layer hidden state h_t back into the input embedding space:

e_{t+1} = R(h_t) = h_t + W2 · σ(W1 · h_t)

The residual connection preserves the original semantics, so the adapter only needs to align the distributional difference rather than learn a full projection. The parameter count is roughly 2d_h², about 10 million for a 4B model. That is much smaller than retraining the backbone.

In text mode, the model samples tokens as usual. When it emits the <latent> token, it enters latent mode. Each step feeds the last hidden state through HybridLink as the next input. The mode ends when the token with the highest cosine similarity to </latent> is selected. In other words, the model decides when to return to text.

Tool calls happen only in text mode. Search or Python results come back as text, and their KV entries accumulate in working memory. The next turn resumes directly from that memory without reconstructing latent thoughts as text. The paper argues that this preserves the full multi-turn context losslessly.

There is a practical hint here. Latent mode is used only for spans that do not need tools. The moment a search or code execution is required, the agent returns to text. That division is the basis for auditability. It does not hide every thought; it exposes the parts that tools and humans need to see.

Training: SFT connects the modes, RL learns when to think

HybridAgents does not retrain a backbone from scratch. It keeps the backbone and uses post-training to connect HybridLink and the two modes.

The first stage is HybridSFT, which uses two passes. In the first pass, the model is teacher-forced on a ground-truth hybrid sequence, and HybridLink learns to predict the next input at latent positions. In the second pass, the ground-truth latent embeddings are replaced with the model's own latent inputs. The backbone is trained with cross-entropy at text positions and a regression loss at latent positions. Without this self-conditioning, the backbone never learns to reason over the latent thoughts produced by its own HybridLink. The paper reports that self-conditioning improves over HybridLink activation alone by an average of 30.5%.

The training data is built by converting existing text-based agentic SFT data into hybrid trajectories. The paper uses 54K hybrid trajectories. An LLM expert judgment and the target model's prediction entropy decide which reasoning steps go latent, while a small set of tool-interaction and debugging steps stay in text.

The second stage is HybridRL, which has a structural problem. Standard RL methods such as PPO or GRPO assume every generated position has a token likelihood. Latent positions are continuous vectors, not sampled tokens, so the token-level importance ratio is undefined.

The paper's solution has two parts. First, hybrid policy factorization: latent thoughts are produced deterministically by HybridLink, tool responses come from the environment, and the policy samples only text tokens. The paper shows that likelihood-ratio updates over text positions are sufficient to optimize the full hybrid trajectory. Second, exact rollout replay: the latent inputs recorded during rollout are held fixed, and the text-token likelihoods are recomputed for the policy update.

The reward is simple: answer correctness, a format penalty, and an incentive for multi-tool use. There is no explicit reward for mode usage. The agent learns when to think in latent space and when to act in text on its own.

The numbers: the cost structure is more interesting than the score

The paper uses Qwen3-Base backbones at 4B, 8B, and 14B. The 11 benchmarks cover mathematics (AIME25, AIME26, MATH500), knowledge reasoning (GPQA-Diamond, MedQA, MuSiQue), and agentic reasoning (GAIA, HLE, WebWalker, SealQA, DeepSearchQA).

Representative agentic results are below. The table shows 8B and 14B HybridAgent alongside ARPO, one of the strongest baselines.

On math and knowledge benchmarks, the 14B HybridAgent scores 43.3 on AIME25, 45.8 on AIME26, 88.2 on MATH500, 55.1 on GPQA-Diamond, 81.7 on MedQA, and 34.5 on MuSiQue.

The reported improvement depends on the baseline. The abstract says an average 10.4% improvement over frontier LLM agents and agentic frameworks spanning text-based and latent methods. The body also says 10.4% over matched text and latent baselines at the RL stage, and 17.4% for the 14B model over the strongest agentic baseline. The same 10.4% refers to different comparison sets in different places, so quote it with the condition attached.

The cost side is more practical. The paper reports 35.5–54.7% fewer text tokens, 1.8–2.6x faster inference, and 2.7–3.7x lower cost. Estimated cost per one million questions:

These are decoding-compute estimates assuming $2.50 per H100-hour and 10% of peak BF16 throughput. They exclude prompt prefill, context-dependent attention, and external tool execution. Read them as a signal of which direction the architecture pushes cost, not as a cloud bill.

Auditability: latent reasoning is not automatically a black box

The main worry about latent reasoning is hidden thought. The paper offers an interesting counterpoint. Latent steps can be read out through HybridLink by mapping each step to its nearest token embedding, and the result often looks like pseudo-text that aligns with the surrounding reasoning.

More important is the density of the text mode. Table 7 reports that HybridAgent uses 43–49% fewer text tokens while exposing 26–28% more key reasoning steps per 100 visible tokens. At 8B, the weak-monitor AUROC is 0.840 versus 0.772 for a text agent; at 14B, it is 0.860 versus 0.818. First-error localization is 99.2% versus 94.2% at 8B and 98.1% versus 100.0% at 14B.

Intervention experiments support the same point. On MATH500, replacing latent vectors with random token embeddings drops the 8B model to 41.2 and the 14B model to 44.6. Replacing them with the nearest token embedding yields 82.6 and 87.2, only 1–2 points below the original 84.6 and 88.2. The latent thoughts are causally used, and the nearest-token readout preserves much of their meaning.

The interpretation still has limits. Nearest-token readout is an approximate projection, not a faithful self-explanation. A 1–2 point drop is a sign that the projection is imperfect. The accurate claim is not "latent reasoning is fully auditable" but "the visible text can be denser, and the latent part can be approximately read."

Limits: you cannot bolt this onto an API model

The biggest constraint is structural. The method post-trains an open backbone such as Qwen3-Base and extends the serving stack to support multi-turn hybrid generation and persistent KV working memory. The paper notes that current serving engines do not support multi-turn hybrid generation, so the authors extended the training and inference infrastructure themselves. This is not a prompt change you can apply to an OpenAI or Anthropic API.

The experimental conditions are also narrow. The backbones are 4B, 8B, and 14B Qwen3-Base. Tool use is capped at 10 calls per trajectory, and search uses Tavily with the top 10 snippets. Training uses 54K SFT trajectories, 12K deep-reasoning examples, and 1K agentic search examples. Evaluation runs on H100 GPUs with five seeds, rollout temperature 1.0, evaluation temperature 0.6, top-p 0.95, and top-k 20. The numbers should not be assumed to transfer outside those conditions.

There is no independent reproduction yet. Every number cited here is author-reported. Comparisons against frontier models cross different backbones and harnesses, so "14B beats GPT-5.6-Luna" is not a valid generalization. Look at the matched-size, matched-budget comparisons.

There is also a governance trade-off. The paper argues that latent reasoning is readable only through the provider-owned HybridLink, which reduces the risk of leaking a text chain of thought. At the same time, the provider can see the full latent trace while the customer sees only part of it. That asymmetry is both a technical advantage and a policy question.

Four things builders can borrow

Even if you cannot reproduce the paper, the design principles are useful.

First, separate the thinking medium from the acting medium. Not every intermediate thought needs to be text. Tool calls, approvals, and final answers should stay in text. Where you draw that line determines both auditability and cost.

Second, measure cost per successful task. Token reduction alone is half the picture. Track total tokens, latency, and tool calls per successful outcome. This is the same lens as AI agent KPIs: from tokens to outcomes.

Third, design a text audit channel. Even with latent reasoning, tool calls and key decisions should remain in text. Table 7 suggests that fewer text tokens can still carry more key steps per token. That is a prompt and logging design problem as much as a model problem.

Fourth, if you claim interpretability, run an intervention test. "Readable" and "faithful" are different. Check how performance changes when you perturb the latent representation, and how much the nearest-token readout preserves. Comparing random replacement with nearest-token replacement, as in Table 5, is a minimal check.

Conclusion: it is not about fewer tokens, but about redesigning the unit of work

The real contribution of HybridAgents is not "35% fewer tokens." It is splitting the generation process into two media and training each to do what it is good at. Latent space handles fast, dense internal computation. Text handles tool calls and human review.

If this architecture becomes common, agent evaluation has to change too. Accuracy alone is not enough. Cost per success, latency, auditability, and the faithfulness of latent reasoning all matter. That is why raising reasoning effort and comparing cost against benefit are no longer separate questions.

You cannot apply this to an API model today. But the principle, "do not make the agent verbalize every thought; keep text for action and verification," is usable in agent design right now.

References

• Hybrid Reasoning Agents (arXiv:2610.18861, Oct 13, 2026)
• HybridAgents paper HTML
• togethercomputer/HybridAgent (GitHub, MIT)

Go deeper with a course

Designing an agent that separates its reasoning medium from tool execution means deciding which steps stay in text and which run as internal computation. Building a loop where execution and verification repeat makes the paper's division of labor concrete.

• Explore the Harness · Loop · Graph Engineering course