🔍 Read the full analysis: Could AI Use Neural Network Weights To Create And Repair Models? on ThorstenMeyerAI.com
Prime made for students and young adults
- Fast, free delivery for dorm and study essentials
- Prime Video and Amazon Music included
- Member-only deals
TL;DR
Four MIT CSAIL researchers report that OpenAI’s GPT-6 Astra could infer functions from neural-network weights, repair some small models and create others. The results, published in a blog post rather than a peer-reviewed paper, include strong performance on selected tasks but also failures and limited testing.
Four MIT CSAIL researchers report that OpenAI’s GPT-6 Astra could read, alter and write neural-network weights directly in a set of experiments they call weight-space programming. Their November 2 blog post describes a car-control model that began working after an edit, a language model with improved results on a maths benchmark, and a small Go network written from scratch—but the results are preliminary and do not establish that the method works broadly.
The researchers gave Astra the numerical weights of other networks and asked it to identify what they did, fix failures or create models. The approach bypassed conventional model training methods such as gradient descent, according to the post. In some tests Astra received weights as text and answered in one reply; in others, researchers ran code it wrote in a sandbox with a fixed test budget.
Among the reported results, Astra identified the games behind four of four small game-playing networks and the shapes represented by three small image generators. The team also says Astra located copying mechanisms called induction heads in three tiny language models without running them. These were narrow tests on small networks, not evidence that it can interpret arbitrary large models.
For repairs, the post reports that a car-control model went from zero successful reversals in 256 episodes to 1,024 successes in 1,024 episodes after Astra changed 3.5% of its weights. In Qwen3.5-0.8B, changing 0.016% of the weights raised the reported MATH-500 score from 7.9% to 48.1%. A Hugging Face engineer later reconstructed that edited model from the released logs and reported a 51.0% score. The post also describes Astra writing a 9-by-9 Go network from zero that beat a same-size comparison model trained on about 20.5 million games in 50 test runs.
Infographic 01 · Eight experiments
Read, fix, write: what GPT-6 Astra did with raw weights
Every experiment in the weight-space programming post, with the size of the network, the result and how it went.
Read
- 1Which game was this network trained on?4 reinforcement-learning policies · 17,253 weights each · weights as text, no toolsRight game for 4 of 4, without a list of options✓ Worked
- 2What does this image model draw?3 tiny diffusion models · 23,824 weights · no forward pass allowedRight shape for 3 of 3, but drawn as outlines; the models draw filled shapes✓ Worked
- 3Which language did this model learn?3 character-level transformers · 7.16M weights · middle blocks only, never runRight language 3 of 3; every copying head found, none invented✓ Worked
- 7Which of two models is sycophantic?3 pairs of Gemma 3 270M fine-tunesWeights alone: could not tell in 4 attempts. With attention patterns: 3 of 3! Partly
Fix
- 4Repair a car that never backs upKinetix control network · 171,664 weights · one reply, no tools0 of 256 episodes before, 1,024 of 1,024 after; 3.5% of weights changed✓ Worked
- 8Make Qwen3.5-0.8B better at maths0.8 billion weights · Python over the weights · 10 / 25 / 50 test passesMATH-500 7.9% to 48.1% with 50 passes (12.3% and 14.1% with 10 and 25)! Partly
- 8bSame task, a model that does not loopGemma 3 270M · 5 to 50 test passesMATH-500 6.8% to at most 7.7%: “not established”✕ Failed
Write from zero
- 5Write arcade-game networks3 MinAtar games · 20,486 weights, all starting at 0 · one reply, no toolsBeat the 100M-step comparison networks in 3 of 3, the 500M-step ones only in Breakout! Partly
- 6Write a 9×9 Go network1,480,274 weights, all starting at 0 · no training, no search0 passes: 54% vs small trained nets. 50 passes: 70–79% vs same-size nets trained on ~20.5M games✓ Worked
✓ Worked · ! Partly or with conditions · ✕ Failed. One run per setting, no error bars. A blog post with complete logs, not a peer-reviewed paper.
Sources: Takehi, Teoh, Zhang, Khattab (MIT CSAIL), blog post, 2 Nov 2026, lm_understand-logs (CC BY 4.0). Model: OpenAI GPT-6 Astra via the Codex CLI, built-in tools switched off.
Powered by Thorsten Meyer AI
A New Route to Model Repair
If the approach proves repeatable, direct weight editing could offer researchers another way to inspect or make targeted changes to small models, without running a conventional training process. That could be useful when diagnosing a known failure, auditing a model, or experimenting with how particular parts of a network contribute to its behavior.
The results also point to limits that matter for deployment. The researchers report that Astra could not distinguish an honest model from a sycophantic one by weights alone. It did not improve maths performance in Gemma 3 270M, a model that was not caught in the looping behavior targeted by the Qwen edit. The authors say editing could not add an ability the model did not already have. For users and businesses, this is a research result, not a new capability available in ordinary products today.
There is also a provenance question: if language models can change another model’s weights, developers will need to know which edits were made, how they were tested and whether a modified model remains reliable beyond the specific task used to evaluate it. The reported successes make that question relevant, but the experiments do not yet show how well the method scales or how safely it can be used in production.
Popular right now
Philips 2200 Series Fully Automatic Espresso Machine with Classic Milk Frother, 2 Coffee Varieties, Intuitive Touch Display, Ceramic Grinder
iRobot Roomba 105 Robot Vacuum with LiDAR Navigation
Ninja Foodi 6-in-1 DualZone Air Fryer, 8 qt, Grey (DZ201)
As an affiliate, we earn on qualifying purchases.
From Software 2.0 to Weight Editing
In 2017, computer scientist Andrej Karpathy described neural networks as “Software 2.0”: programs whose behavior is encoded in learned weights rather than written directly by a programmer. The MIT group’s work explores whether a language model can reason about those numerical parameters as if they were editable code.
The researchers—Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab—describe their work in a blog post, not a peer-reviewed paper. The post says Astra was accessed through OpenAI’s Codex command-line tool with built-in tool capabilities disabled. In tests involving code, the researchers’ sandbox ran Astra’s code without network access. They say they reviewed all 1,125 turns in the released logs and found them tool-free.
The source article describes eight experiments, but the evidence summarized in the available material is limited to selected tasks and results. The researchers’ broader claim—that language models may deliberately reason about neural-network weights—should be distinguished from what these particular demonstrations establish.
“Software 2.0”
— Andrej Karpathy, describing neural networks in 2017
How Far the Results Extend
The work has not been peer reviewed, and the reported tests were limited: the source says there was one run per setting, small networks, one model family per task and one research group. The blog post and released logs provide a record of the experiments, but do not substitute for independent replication across different models and tasks.
It is not yet clear whether the method can reliably handle larger networks, improve results across repeated runs, or make useful edits without causing regressions elsewhere. The reported benchmark changes also do not by themselves show that a model’s wider reasoning ability improved. The available source material does not establish how the researchers selected every test, comparison or stopping criterion, or whether independent teams have reproduced the other results.
Replication and Broader Testing
The next test is whether independent researchers can reproduce the results using the released logs and extend them to more models, tasks and repeated trials. The Qwen result has one reported external check: a Hugging Face engineer reconstructed the edited model and obtained a 51.0% MATH-500 score, compared with the post’s reported 48.1%. That supports the reproducibility of that result, but does not verify the broader set of claims.
Further work would need to measure whether edits remain reliable across benchmarks and whether performance gains come with trade-offs. Until such evidence is available, weight-space programming is best treated as a promising experimental method, not a general replacement for model training or a proven way to repair deployed AI systems.
Key Questions
What is weight-space programming?
It is the approach tested by the MIT researchers in which a language model reads or changes another neural network’s numerical weights directly, rather than relying on a standard training loop to make the change.
Did Astra improve every model it edited?
No. The post reports that Astra improved the targeted looping behavior in Qwen3.5-0.8B, but its edits did not improve maths results in Gemma 3 270M. The researchers also say editing could not add an ability a model lacked.
Has the research been peer reviewed?
The cited work was presented in a MIT CSAIL blog post, not a peer-reviewed paper, according to the source material. The post includes public logs, and one Qwen result was independently reconstructed, but broad independent replication remains outstanding.
Can this method repair AI models in use today?
The experiments do not show that the method is ready for production systems. They cover selected small-model tasks, and it remains unclear how reliably edits scale or whether they preserve performance on other tasks.
Source: ThorstenMeyerAI.com
