Editing 0.016% of the Weights Multiplied a Small Model's Math Score Sixfold
The MIT CSAIL team published a weight-space programming experiment on November 2: GPT-6 Astra edited Qwen3.5-0.8B's weights directly, changing only about 0.016% of the parameters and lifting its MATH-500 score from 7.9% to 48.1%, by eliminating repetitive reasoning loops.
- 01THE SETUP — The MIT CSAIL team had GPT-6 Astra edit Qwen's weights directly.
- 02THE DIAGNOSIS — Astra located the repetitive reasoning loop caused by layers 15 and 19.
- 03THE PATCH — After subtracting that numerical direction, MATH-500 rose from 7.9% to 48.1%.
WHAT HAPPENEDWhat Happened
A research team at MIT's Computer Science and Artificial Intelligence Laboratory published an experiment on November 2. They had OpenAI's GPT-6 Astra edit the weights of a small open-source model, Qwen3.5-0.8B, directly, without any conventional training process.
The result is striking. Astra changed only about 0.016% of the weights, 131,072 parameters, and Qwen's MATH-500 score rose from 7.9% to 48.1%, GSM8K from 13.8% to 67.1%, and instruction following on IFEval from 21.0% to 62.3%.
The mechanism is the key. Astra located a numerical direction written by the multilayer perceptrons in layers 15 and 19, a direction that drives the model into repetitive reasoning. After subtracting it, the share of MATH-500 answers hitting the 32,768-token output limit fell from 93.0% to 33.1%. The model did not get smarter; it can finally finish saying its answers.
WEIGHTS AS CODEWeights Can Be Read Like Code
The team calls the work weight-space programming. The idea is to treat a neural network's parameters as code that can be read, diagnosed and patched; the frontier model reads out which segment is broken and then fixes it by hand.
The conventional route is training, adjusting parameters indirectly through gradient descent. This route is surgery, changing the numbers directly. The difference is not in the score figures but in methodology: a frontier model has been shown, for the first time, to directly understand and operate on another model's parameters.
REPAIR, NOT KNOWLEDGEThe Sixfold Gain Fixes a Bug, Not Knowledge
The numbers are easily misread. Sixfold sounds like the small model suddenly learned mathematics; in fact it could always compute, it just started spinning in place halfway through and its answers never reached the end.
The control experiment says it plainly. On Gemma 3 270M, which rarely falls into loops, the same method improved MATH-500 by less than one percentage point. Weight editing can expose latent potential; it cannot implant abilities that were never learned.
EARLY AND EXPENSIVEThe Cost Is High and the Method Is Still Early
Finding this patch was not cheap. The strongest attempt used 50 evaluation requests, 158 conversation turns and 80 Python executions, taking 20.3 hours in total, of which 4.7 hours were stuck on service outages. The scalpel is precise; finding the lesion is still clumsy.
The team's next step is an automated benchmark and testing framework to systematically evaluate models' ability to edit models. Weight-space programming is now a proof of concept with evidence, still some distance from a general-purpose tool.
THE SHIFTWhat Changed
SIGNAL JUDGMENT — ANALYSISModels Repairing Models: a New Path Beyond Training
Tokenta Signal's judgment: the significance of weight-space programming is not the sixfold figure, but what it proves. A frontier model can read another model's parameters and repair it the way one repairs code.
WHAT REMAINS UNCERTAINWhat Remains Uncertain
EVIDENCEEvidence
The original confirms the MIT CSAIL team published the weight-space programming experiment on November 2: GPT-6 Astra changed only about 0.016% of the weights and lifted Qwen3.5-0.8B's MATH-500 score from 7.9% to 48.1%, by eliminating repetitive reasoning loops.
PRIMARY SOURCESPrimary Sources
- VentureBeat: 'Like gene editing for neural networks': MIT researchers used GPT-6 Astra to edit Qwen's open weights directly
https://venturebeat.com/technology/like-gene-editing-for-neural-networks-mit-researchers-used-gpt-6-astra-to-edit-qwens-open-weights-directly-boosting-math-performance - The Register: Forget retraining - AI can now tweak another AI's weights
https://www.theregister.com/ai-and-ml/2026/11/03/forget-retraining-ai-can-now-tweak-another-ais-weights/5307105