www.it-connect.fr
GPT-6 Astra Touches Just 0.016% of Qwen’s Weights and Multiplies Its Math Score by 6
From 7.9% to 48.1% on the MATH-500 benchmark: that is the leap achieved by the small Qwen3.5-0.8B model after a tweak to its weights performed by GPT-6 Astra, with no training phase at all. Here’s what to take away from this MIT experiment.
On November 2, 2026, four MIT CSAIL researchers, Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab, published a research report titled Sparks of Weight-Space Programming. Their idea: ask an LLM to read and write the weights of another neural network directly, based on a goal expressed in natural language. All of this without using a training loop. This technique has a name: Weight-Space Programming.
The model used to carry out this task was none other than GPT-6 Astra, OpenAI’s flagship model (at least for now). It is the most capable model in OpenAI’s catalog, and OpenAI deliberately restricted the public version because of its offensive cybersecurity capabilities. According to the experiment logs published on GitHub, Astra was called through Codex CLI with all built-in tools disabled. When an experiment allowed it to write Python code, the researchers ran it in a sandbox.
Qwen3.5-0.8B: drastically reduced reasoning loops by modifying 0.016% of the weights
The technique used by the researchers had very interesting effects on Qwen3.5-0.8B, the kind of small model you can run locally on modest hardware. And yet this model has a known flaw, documented by the Qwen team itself. "In reasoning mode, we observed that with the recommended sampling parameters, Qwen3.5-0.8B is more prone to entering reasoning loops than other Qwen3.5 models, which can prevent it from properly completing its generation.", reads its Hugging Face page.
In these experiments, Astra could inspect Qwen’s weights with Python code (without GPU) and pull from math exercises drawn from GSM8K and MATH (not the test sets). It could also run the model a limited number of times to observe its responses: each “pass” allowed it to submit up to 512 problems. By analyzing these responses and the weights, Astra located the source of the loops: a single signal injected by two of the model’s 24 layers (layers 15 and 19). It then removed that signal at the source by correcting a small portion of the weights in those two layers. Result: 131,072 weights modified out of 0.8 billion.
Thus, with a budget of 50 passes, here are the results that highlight the significant progress achieved:
- MATH-500: from 7.9% to 48.1%
- GSM8K: from 13.8% to 67.1%
- IFEval (instruction following): from 21.0% to 62.3%
- Truncated responses at the 32,768-token limit on MATH-500: from 93% to 33.1%
The last figure is the most telling: the original model was going in circles in the vast majority of cases.
Read, repair, and write neural networks
The researchers’ report details other experiments, most of them conducted on very small AIs (neural networks with only a few dozen to a few hundred thousand weights). These mini AIs were trained to play retro video games such as Breakout or Pong, or to control a car in a physics simulator. These are classic testbeds in research because you can measure very precisely whether the AI succeeds or not.
Here is what Astra achieved:
- Guess what an AI is for: by reading only the weights of four mini AIs trained on different games (simplified versions of Breakout, Pong, Pac-Man, and Freeway), Astra identified the correct game every time, without being given the list.
- Repair a failing AI: an AI tasked with driving a virtual car kept getting stuck on a ramp. Astra modified 3.5% of its weights, and the car completed the course in every attempt.
- Create an AI from scratch: without any training, Astra wrote the weights of a game-playing AI itself. In Breakout, it performs 20 times better than a conventionally trained AI. On two other games, it still lags behind.
- Spot a sycophantic AI: Astra had to distinguish between two copies of a small language model, one of which had been trained to always agree with the user. The weights alone were not enough. It succeeded by observing which words each model focused its attention on.
A promising advance
The researchers published a counterexample. On Gemma 3 270M, a model that rarely loops (8.5% truncated responses), Astra did not manage to improve the MATH-500 score: 7.7% at best, versus 6.8% initially. "Modifying the weights could remove an obstacle that was masking a capability, but could not add a capability the model did not have," the authors acknowledge. In other words, Astra did not make Qwen “better at math”: it greatly reduced a parasitic behavior that was preventing it from reaching a conclusion. That is already interesting, but it is not quite the same thing.
There are also relevant cybersecurity implications. In fact, the authors mention auditing a model directly from its weights, for example to detect a backdoor. But the logic also works the other way around: a model capable of adjusting weights so surgically could, in theory, slip in a subtle behavior... That is a risk, and we will have to see whether any confirmed cases emerge in the future... We are certainly not done being surprised.
