When a client calls a local model through an OpenAI-compatible API, it often leaves out temperature and top_p. In that case the server decides which sampling values are used. The Ollama v0.40.14 release notes describe a change to exactly that default. Same model, same prompt, different default: the output can change. Anyone comparing local models should care.

This article relies on the release notes and the release page only. We did not run the two versions side by side.

What changed

The release notes say the following.

  • OpenAI-compatible endpoints no longer override a model's packaged sampler defaults.
  • A client that sends no temperature or top_p used to get 1.0. Now it gets the model's own values.
  • The stated goal is to match the behavior of /api/chat.

The key word is omitted. The change concerns what happens when the client leaves the fields out. The notes do not describe a change to values a client sets explicitly.

Request state Before v0.40.14 (per release notes) v0.40.14
temperature and top_p omitted 1.0 applied The model's packaged defaults
Set explicitly The client's value Not the subject of this change

The same release has other changes. ollama launch claude applies the model's output token limit, so cloud models no longer hit the 128k default cap. /v1/responses keeps readable input content, including earlier assistant refusals. On MLX, top-k sampling is faster for large vocabularies. The notes give no number for that speedup.

Why sampling defaults change the output

For each token, the model produces a probability distribution, and the sampler picks one token from it.

  • temperature sharpens or flattens that distribution. Higher values give lower-probability tokens more chances to be picked.
  • top_p keeps the smallest set of highest-probability tokens whose cumulative mass reaches p.

So whether a request runs at 1.0 or at the model's packaged value can change how varied and how stable the answers are, even with an identical prompt. Ollama typically defines a model's packaged values with PARAMETER entries in its Modelfile. Which values are "better" depends on the model and the task. This article does not claim that any particular value is correct.

What to record in a comparison

If you compare local models, you need to separate model differences from sampler-default differences. Record:

  • The Ollama version and the model tag
  • The temperature and top_p values the client actually sent, including whether they were omitted
  • The model's packaged sampler values
  • Results from the same prompt set run on both versions

We did not run this procedure. A reasonable check is to run the same prompt set on v0.40.13 and v0.40.14, once with sampling values set explicitly and once with them omitted, and compare the outputs.

Limits

  • The analysis depends on the release notes and the release page. We did not check the source diff or run the behavior.
  • The release page shows no year. The notes date the release to November 3.
  • The MLX speedup has no figures, so its size is unknown.
  • This article does not evaluate whether the new defaults improve answer quality.

Conclusion

This is a correction to a default, not a new feature. That is exactly why it can quietly change experiment results. If you compare models, record the sampler settings and the Ollama version together. A number obtained without those settings may not reproduce after an upgrade.

Reference