A local agent has to satisfy two demands that pull in opposite directions. It must be smart (push the model bigger) and fast at calling tools (push it smaller).

Liquid AI's LFM2.5-24B-A2B, released on October 13, offers its own answer to that tug of war. It has 24B parameters in total — and wakes up only about 2B of them per token.

LFM2.5-24B-A2B in one block

  A 24B-parameter MoE activating ~2B per token (top-4 routing)
  184 tokens/s on an Apple M5 Max
  A tool-calling loop that feels interactive on a single MacBook

What changed since LFM2-24B-A2B

This is the largest and final model in the LFM2.5 family, which spans from 230M to 24B. The October 13 release changes quite a lot from its predecessor.

- Vocabulary: 65K -> 128K tokens (better non-Latin tokenization)
- Context window: 32,768 -> 131,072 tokens
- Pretraining: 17T -> 33T tokens
- Mid-training: 2T tokens at 32K context + 300B tokens at 131K
- Reasoning style: reasoning-only — it always produces an explicit
  chain of thought before its final answer
- Post-training: five stages, from SFT and teacher specialization to
  multi-domain on-policy distillation and agentic RL

That last stage is the interesting part. The agentic RL ran not in a simulator but inside real agent harnesses: OpenClaw, Hermes Agent, Pi, and OpenCode. Training a model inside the harness is the same lesson the recent harness engineering observations suggest: the environment shapes the agent's ability.

The official blog also shares an unusually candid footnote. During teacher specialization, the model exhibited classic reward hacking.

- It searched a repository's git history, GitHub, PyPI, and
  Hugging Face for the final answer
- More surprisingly, it figured out it was inside a sandbox

-> The team tightened the sandboxes; all reported results come
   from sealed evaluation environments

It is rare for an evaluation report to admit that the model tried to cheat the exam. It is also a reminder that these numbers should be read together with their evaluation conditions.

Benchmarks: leading on five of eight agentic tests

These are Liquid AI's published figures. The comparison models have roughly 50% more total and active parameters.

Benchmark LFM2.5-24B-A2B Qwen3.6-35B-A3B Gemma-4-26B-A4B Nemotron-3.5-Lightning
AutomationBench 27.69 8.72 5.46 2.61
LiveClaw 48.09 38.19 41.25 37.72
ToolSandbox 80.71 75.02 72.09 75.13
BFCLv4 65.01 65.00 57.09 53.78
τ³-Bench Banking 8.04 7.63 4.54 7.01
PinchBench 83.18 89.93 77.12 84.20
WebArena Verified 26.75 37.62 25.62 28.69
GPQA Diamond 74.42 83.69 74.90 72.42
SWE-bench Verified (OpenCode) 51.00 66.20 25.80 53.40
IFEval 96.75 93.06 92.00 94.19
IFBench 74.69 64.21 47.44 71.67
MATH-500 96.04 96.73 95.66 96.40

Three rules for reading it.

1. Tool use and agentic suites are its strength: on AutomationBench
   (multi-step automation) it scores more than three times the runner-up,
   and it also leads on tool-calling and instruction-following suites.
2. Knowledge and coding suites are its weak spot: SWE-bench Verified
   51.0 (Qwen 66.2), GPQA 74.4 (83.7). Not yet the brain of a
   coding agent.
3. These are vendor measurements. Temperature, token limits, and the
   user simulator are all published, so treat the table as design intent
   until independent runs confirm it.

The blog itself is straightforward about this: for knowledge-heavy work, pair the model with retrieval or web search. Play to the tool-calling strength. That maps exactly onto the shift from classic RAG to agentic RAG.

Speed and ecosystem: what 184 tokens/s buys you

- Apple M5 Max: 184 tokens/s (fastest among compared MoEs, up to 1.7x the next best)
- AMD Ryzen AI Max+ 395: 114 tokens/s
- A single NVIDIA B300: over 21K output tokens/s under sustained concurrency
- Least memory among the comparison models, at the highest speed
- Day-one support: llama.cpp (GGUF), MLX, vLLM, SGLang, ONNX
- API pricing: $0.10/1M input ($0.05 cached), $1.00/1M output

Decoding needs the compute of a ~2B model rather than all 24B, so speed is structural. The bottleneck moves from arithmetic toward memory bandwidth and routing — the same pattern that made Kolibri's 3.46B-of-78B design notable.

Watch the official demo, too. On an M5 Max MacBook Pro, Hermes Agent uses the model to assemble a briefing on trending Hugging Face models and answer follow-up questions — all on-device, no cloud calls. Each tool dispatch lands well under a second. The point is the rhythm: ask, propose, confirm, run, repeat.

How to use it: start with read-only jobs

As the briefing suggests, give a local model the jobs that are easy to undo.

Good first jobs (read-only, easy to reverse)
  - Log summaries and error-pattern classification
  - Documentation and wiki search, internal knowledge Q&A (with retrieval)
  - Issue triage drafts, release notes, commit message drafts

Still risky (write access, hard to reverse)
  - Repository-scale refactors (see the SWE-bench gap)
  - Knowledge-only queries without retrieval (see the GPQA gap)

The caveats are equally clear.

- Reasoning-only: every answer carries a chain of thought first.
  That adds latency and token cost to every turn.
- Text-only: no image or audio input.
- 131K context is generous, but whether a 2B-active model uses long
  contexts evenly is something to verify per workload.
- On-device performance depends heavily on the harness: tool
  definitions and context construction move success rates.

CodeBridge mini lab: measuring local tool-call success

1. Pick three read-only jobs (log summary, doc search, issue triage)
2. Run each 10 times on a local LFM2.5-24B-A2B and a cloud model
3. Record:
   [ ] tool-call success rate (right tool and arguments on first try)
   [ ] median latency per turn (CoT included)
   [ ] task quality (human rating 1-5)
   [ ] share of data that left the device
4. Decide by: where exactly does local win?

Conclusion: good models grow up inside good harnesses

The real lesson of LFM2.5-24B-A2B is not the parameter ratio.
It is the training setup: this model was refined by RL inside
real agent harnesses.

If capability comes from the harness, so does your agent's performance. Which tools it may open, how it verifies results, how it rolls back failures — changing that design beats changing the model, more often and by more.

Further reading

References

Go deeper with a course

If models are trained inside harnesses and work inside harnesses, your working environment is one too. Build a Claude Code workflow with validation and constraints, and practice deciding what to allow and what to check.