From 92% to 100% Schema Accuracy in Tool Calls: Reading Ubicloud's vLLM Constrained-Decoding Case

When a model calls search_web instead of web_search, or passes city_name where the schema says location, the agent stops right there. Ubicloud, serving Kimi K2.6 on vLLM, reports seeing roughly 8% of such format errors and removing them with a server setting.

The point here is less the headline number than two things behind it: what exactly is measured, and why accuracy slid back to 91% when the same team moved to upstream vLLM. The post is dated October 20; the briefing surfaced it on October 26.

What the 92% and 100% measure

The conditions that can be confirmed from the post:

The 100% is format accuracy. It says the call fits the schema, not that the right tool was chosen or the right value supplied. The evaluation was run by Ubicloud in its own environment, and I found no independent reproduction.

Why it goes wrong, and how it is prevented

Unconstrained generation is probabilistic. When the tool schema seen in training drifts from the one in the current request, the model produces near-miss names or keys. Constrained decoding removes grammar-violating tokens from the sampling candidates, so a format error cannot be produced. The post says Moonshot recommends turning it on.

This is where meanings split. Forcing the format eliminates format errors, but a wrong choice by the model survives as a well-formed wrong call. vLLM's own docs say that required or named tool calls give "a validly-parsable function call - not a high-quality one."

Moving to upstream dropped it to 91%

This is the most practical part. Ubicloud had been using constrained decoding in its own fork. When it landed in vLLM 0.25, the team moved to upstream, and accuracy fell to 91%. The cause was not a bug but a changed default: requests without an explicit setting no longer used constrained decoding. vLLM turned it off by default because it can hurt some models, and the post cites reports for gpt-oss-20b, Qwen 3.6, and GLM 5.1. It adds that grammar-translation bugs can break calls or silently drop the constraint, and that interaction with speculative decoding can wrongly reject valid tokens.

The lesson is plain: upgrading the inference engine can make the same model behave like a different one. The weights stay put and the results move.

What vLLM's docs say now

The post says the feature shipped as a server-side flag in vLLM 0.31 without naming it. Here is what I confirmed in the current vLLM docs.

• --tool-strict-level takes auto, function, or parameter. auto follows the request's tool choice and per-tool strict; function constrains the call envelope (markup and function name) on every request; parameter also pins argument schemas as if every tool were strict: true.
• If unset, the parser default applies: function for glm47, auto for others.
• VLLM_ENFORCE_STRICT_TOOL_CALLING defaults to true; setting it to false disables structural tags and takes precedence over --tool-strict-level.
• Using strict: true on a tool requires additionalProperties: false, all properties required, and optional fields expressed as nullable.

The docs' kimi_k2 parser entry does not say whether it supports structural tags or strict mode. I did not run these settings on Kimi K2.6, so check your own model and version combination.

A harness that measures three things separately

To avoid reading "100% schema" as "the agent works," split the metrics. Below is an example computed on five synthetic cases I wrote. It is not Ubicloud's data or K2VV.

cases = [  # (expected call, model output)
 ({"name":"web_search","arguments":{"query":"vllm"}},      {"name":"web_search","arguments":{"query":"vllm"}}),
 ({"name":"web_search","arguments":{"query":"vllm"}},      {"name":"search_web","arguments":{"query":"vllm"}}),          # wrong name
 ({"name":"get_weather","arguments":{"location":"Seoul"}}, {"name":"get_weather","arguments":{"city_name":"Seoul"}}),    # wrong key
 ({"name":"get_weather","arguments":{"location":"Seoul"}}, {"name":"get_weather","arguments":{"location":"Busan"}}),     # well-formed, wrong value
 ({"name":"web_search","arguments":{"query":"rust pdf"}},  {"name":"get_weather","arguments":{"location":"Seoul"}}),    # well-formed, wrong tool
]
schema = sum(schema_ok(o) for _, o in cases)               # schema adherence
tool_sel = sum(e["name"] == o["name"] for e, o in cases)   # tool selection
task = sum(e == o for e, o in cases)                       # exact match

schema validity : 3/5
tool selection  : 3/5
exact task match: 1/5

Constrained decoding would fix the first two errors, but the last two stay well-formed. Schema accuracy can reach 5 of 5 while exact match stays at one to three. This is a toy that shows the structure; a real evaluation needs your tool definitions and ground-truth labels.

Checklist

1. Check your vLLM version and whether your parser supports strict mode, in the docs.
2. Run the same request set with and without the constraint and record schema adherence, tool-selection accuracy, and real task success separately.
3. Re-run that evaluation on every engine upgrade. Defaults can change while the model name stays the same.
4. Record side effects too: latency, request failures, speculative-decoding conflicts.

Takeaway

Removing format errors with one setting is practical. But 100% is a format number, and agent success needs its own measurement. The real lesson of this case came at the moment accuracy went back to 91%: treat the inference engine as part of the model, and keep its version and settings in your evaluation records.

The paid promotion at the end covers designing agents through harnesses, loops, and graphs. It does not teach vLLM serving or constrained decoding.

Related reading

• Ask Again When MCP Wants More: Claude Code 2.1.288
• A Rejected Request Read Someone Else's KV Cache: Checks from Cloudflare's GLM-5.3 Incident

Sources

• Ubicloud: Fixing Tool-Call Accuracy in vLLM (2026-10-20)
• vLLM docs: Tool Calling