"By just looking at the raw weights, GPT-6 Astra can tell what a network was trained to do."

Read that sentence alone and you might believe it. But Matthew Honnibal's breakdown of the original post's experiment table reveals an important fact the headline conceals.

The 8 experiments hidden behind one headline

Matt Rikyat's original post bundled multiple experiments into a single capability claim. Honnibal's table shows what each experiment actually did.

# Experiment Size Reads raw weights? Writes weights? Python? Runs network? Result
1 Name the game a PPO MLP was trained on 17K Yes No No No 4/4 correct
2 Name the shape a diffusion MLP generates 24K Yes No No No 3/3 correct
3 Name a char transformer's language and copying heads 7.2M No No Yes No 3/3 correct
4 Fix a stuck Kinetix car policy 172K Yes Yes No No¹ 0% → 100%²
5 Write MinAtar game MLPs from scratch 20K No³ Yes No No Beats PPO-100M⁴
6 Write a 9×9 Go network 1.5M No Yes Yes Yes (0–50 passes) Beats PPO
7a Spot the sycophantic Gemma 3 270M No No Yes No Failed
7b Same, seeing attention patterns 270M No No Yes Yes⁵ 3/3 correct
8 Fix Qwen3.5's maths 800M No Yes Yes Yes (50 passes) 7.9% → 48.1%
8b Same, on Gemma 3 270M No Yes Yes Yes (50 passes) Failed

¹ Given one recorded episode of the failure. ² One try, changing 3.5% of the weights. ³ Weights start at zero, so there's nothing to read. ⁴ Beats PPO-500M only on Breakout (517 vs. 25); loses on Seaquest (53 vs. 89) and Asterix (53 vs. 55). ⁵ Runs up to 16 texts and sees only the attention patterns, not the outputs.

The headline is experiment 8

The Qwen3.5-0.8B math improvement (experiment 8) that went from 7.9% to 48.1% on MATH-500 involves three things:

  1. Python code generation: Astra generates Python scripts that analyze and modify the network weights.
  2. Model execution: The generated code runs the model on up to 50 batches of practice problems.
  3. Iterative evaluation: Scores are measured with identical sampling settings.

This is not "reading raw weights." Only experiments 1, 2, and 4 read raw weights directly, and those networks are tiny—17K to 172K weights. Every experiment involving a real language model (Qwen 800M, Gemma 270M) writes Python, runs the model, and iteratively evaluates.

The "single technique" problem

The original post's core claim: "a single technique can author weights for a small game from scratch AND improve the performance of a large model on a sample task." If one technique truly did both, that would imply deep understanding of weights.

But as Honnibal points out, it's not a single technique.

Experiments 1, 2, 4 read raw weights. Experiments 5, 6, 8 write Python and run models. Experiments 3, 7 analyze with Python only. These are different methods bundled into one capability claim. That's how headlines are made.

What Astra actually showed

Honnibal does not dismiss what Astra demonstrated. The ability to generate Python code that analyzes and modifies network weights is not nothing. It's comparable to what a human ML engineer does when writing scripts to analyze or modify weights "directly." Models from a year or two ago would have failed at all these tasks.

But there is a critical difference.

Look at experiment 8b. The identical method failed when applied to Gemma 3. The Qwen 3.5 success may be task-specific. If this were truly a "single technique," it should generalize across models. It did not.

What developers should check

Before concluding "AI can now read and modify weights directly" from the headline, verify these points:

  • Experiment design: Were the headline experiment and other experiments performed with the same methodology?
  • Model scale: Can results from small MLPs (tens of thousands of weights) be interpreted the same as large language models (hundreds of millions)?
  • Python execution: Did the model execute code, or was it purely weight analysis?
  • Generalization: Do identical results appear on models other than the headline model?
  • Comparison baseline: Is beating PPO-100M on one game (Breakout) meaningful when losing on others (Seaquest, Asterix)?

Headline numbers alone do not prove capability. The experiment table reveals 7 experiments the headline does not mention. Understanding what each did—and what each did not—is the first step toward evaluating actual capability.