Primary Source Verified

Simon Willison's blog (simonwillison.net) was confirmed with a November 3, 2026 post. The experiment record of game identification using Qwen 3.8 27B local inference is included.

Verified sources:

Identifying Games from Weights with a Local Model

According to Simon Willison's experiment record published on November 3, the Qwen3.8 27B model identified 3 of 4 games using weights alone. Successful runs took approximately 46-65 minutes, and Freeway failed.

This is a personal experiment conducted separately from the research team's announcement, but the limited number of repetitions means it cannot be interpreted as a general success rate. Nevertheless, this experiment provides valuable information.

Why Did It Take 46-65 Minutes?

The fact that a local 27B model inference took 46-65 minutes carries several implications:

  • Model size — A 27B parameter model is large to run locally. Without quantization, GPU memory of 50GB+ is required.
  • Token count — The time depends on how many tokens the model generates to identify the game.
  • GPU performance — Inference speed varies significantly depending on the GPU used.

Whether this time is "practically suitable" depends on the use case. For a one-time analysis, 1 hour may be acceptable, but it is not suitable for repeated model evaluation.

Why Did Freeway Fail?

The fact that only Freeway failed among four games provides an interesting clue.

Freeway is one of the more complex Atari 2600 games, with multiple dynamic objects moving simultaneously. Structural differences between the other three games and Freeway:

  • Visual complexity — Freeway has many characters and obstacles moving on screen simultaneously.
  • Game state transitions — The state space may be wider than other games.
  • Weight distribution — The weights of models trained on Freeway may have different patterns than other games.

This difference shows that the claim "games can be identified from weights alone" does not apply to all games.

Local Inference Cost: Data for Practical Adoption

The biggest lesson this experiment offers for practical work is that "local models can verify research ideas, but the inference cost is high."

Specific cost factors:

Factor Estimated Range Notes
GPU memory 50GB+ (FP16) Base requirement for 27B model
Inference time 46-65 min/success Excluding Freeway
Power cost Varies by GPU Local environment baseline
Repetition cost Additional time on failure Including Freeway failure

How to Use This Information Practically

1. Record model version, quantization, context

When experimenting with local models, always record:

  • Model version (exact build number of Qwen3.8 27B)
  • Quantization method (FP16, INT8, INT4, etc.)
  • Context length setting
  • Inference time (success/failure distinction)
  • Failure cases (games that failed, like Freeway)

2. Understand the limitation of success rate

This experiment has limited repetitions. 3 out of 4 games succeeding cannot be interpreted as a "general success rate." More games and more repetitions are needed.

3. Possibility of local model research

The fact that individuals can conduct the same experiment with local models, separate from the research team's announcement, opens new possibilities for research accessibility. However, cost and time must be accepted.

Limitations

  1. Limited repetitions — 3/4 success cannot be interpreted as a general success rate.
  2. Hardware conditions undisclosed — GPU model, memory, and quantization method are not specified.
  3. Freeway failure reason unclear — Whether it is a game structural difference or model limitation is not distinguished.
  4. Personal experiment — Conducted separately from the research team's announcement; must be distinguished from official results.

Conclusion

The experiment showing that games can be identified from weights alone using a local Qwen 27B demonstrates that local models can verify research ideas. However, the 46-65 minute inference time and Freeway failure are important data for making practical adoption decisions.

Actions you can take now:

  1. Confirm Simon Willison's blog: https://simonwillison.net/
  2. Review Qwenputer GitHub: https://github.com/divagr18/qwenputer
  3. Review nesformer GitHub: https://github.com/dbreunig/nesformer
  4. Reproduce the same experiment on your hardware environment (verify GPU memory, inference time)
  5. Run repeated experiments on more games to estimate success rate

This article was updated to published status after primary source verification. Source URLs are current as of November 4, 2026.