🔍 Read the full analysis: GPT-6 Astra Changed Qwen’s Weights And Raised Its Score From 49 To 70 Of 120 on ThorstenMeyerAI.com
Prime made for students and young adults
- Fast, free delivery for dorm and study essentials
- Prime Video and Amazon Music included
- Member-only deals
TL;DR
An internal experiment reported that GPT-6 Astra, acting as a Codex agent, changed one matrix in Qwen3-4B-Instruct-2507 and raised its exact score on a frozen text-to-JSON test from 49 of 120 to 70 of 120. The synthetic result is narrow and has not had an independent review; 50 final-test answers remained wrong, and a separate table-extraction task stayed at zero of 30.
An internal experiment reported on November 5, 2026 found that a targeted change made by OpenAI’s GPT-6 Astra, operating as a Codex agent, raised Qwen3-4B-Instruct-2507’s exact score from 49 to 70 of 120 cases on a test of copying special characters into JSON. The result is limited to a synthetic task and has not yet been checked by an independent reviewer.
The experiment targeted a specific failure: returning short text containing tabs, line breaks, quotation marks or backslashes inside JSON without changing any characters. A response counted as correct only if a JSON parser recovered text that matched the original exactly. The report says that a real tab had to be represented as the escaped sequence \t, rather than inserted raw into the JSON string.
According to the experiment report, GPT-6 Astra inspected the model’s weights and made a calculated rank-one update to the output projection matrix in the final multilayer perceptron layer. The work did not use gradient training. The report says the other 397 stored tensors were bit-identical to the original model.
On the frozen final test, the edited model answered 21 additional cases correctly, with no cases that had been correct becoming incorrect, according to the report. The original model scored 49 of 120 and the edited version 70 of 120, a gain of 17.5 percentage points. The report also says a clearer prompt alone did not improve the model on an earlier 32-case screening set: both the original prompt and a stronger escaping prompt scored 16 of 32.
Infographic 01 · The measured result
Where the edit helped, and where it did not
Qwen3-4B-Instruct-2507, original versus one edited matrix, on synthetic texts that must survive the trip into JSON without losing a character.
Final test · 120 cases by group
- Texts with tabsthe target failureOriginal19/30Edit30/30
- Line breaksalready perfectOriginal30/30Edit30/30
- Mixed special characterstabs, quotes, backslashesOriginal0/30Edit10/30
- Second table rowstill unsolvedOriginal0/30Edit0/30
Development, validation and controls
- Developmentused to fit the edit · not independentOriginal24/40Edit40/40
- Validationnew cases · 9 wins, 0 lossesOriginal10/40Edit19/40
- Random directioncontrol on validation · same strengthOriginal10/40Edit10/40
- Plain-text tabscontrol · real tabs must surviveOriginal10/10Edit10/10
Exact answer = after normal JSON parsing, the text field matches the source text character for character. Same prompt, temperature 0, seed and output limit for both models; saved and reloaded edit, no hooks, no grammar constraints, no repair. Final test, validation and controls frozen before any independent output of the edited model. Fifty of 120 final answers are still wrong.
Sources: our experiment records, 5 Nov 2026 (Lexassay research, internal): experiment 3 result report and final result by GPT-6 Astra working as a Codex agent (OpenAI). Synthetic test cases, not published.
Powered by Thorsten Meyer AI
A Narrow Gain in JSON Escaping
The reported result offers a small, task-specific example of changing a model through its weights rather than relying only on prompt wording or conventional gradient-based training. If independently reproduced, such edits could help researchers test whether a model’s particular failure can be corrected with a limited intervention. This experiment does not establish that the method generalizes to other tasks or models.
The remaining errors matter as much as the gain. The edited Qwen model still missed 50 of 120 final cases, and the report says table-row extraction remained at 0 of 30. The evidence therefore supports a bounded claim: performance improved on one targeted synthetic test, not that the model became generally more reliable at handling structured data.
Exact copying is relevant to software that passes user text through JSON-based services, forms and logs. A lost tab or an invalid escape can alter data or break parsing. But the test described here used constructed cases, and the report does not establish how the edit would behave on real-world inputs or in deployed systems.
Popular right now
Shark HydroVac MessMaster Wet/Dry Vacuum & Mop Combo (AW261)
iRobot Roomba 105 Combo Robot Vacuum and Mop
Ninja Foodi DZ201 8-Quart DualZone 6-in-1 Air Fryer
As an affiliate, we earn on qualifying purchases.
From a Failed Prompt to a Targeted Edit
This was described as the third experiment in a week focused on changing neural-network weights. In an earlier attempt on the same Qwen model, GPT-6 Astra tested extraction of five fields from synthetic business messages. The two versions both scored 97 of 200 on that experiment’s frozen final test. The report says both also produced 47 invalid answers, often by putting raw tabs into JSON strings. That shared error informed the later, more targeted test.
The earlier work also included a separate experiment in which Claude edited a local model called Ornith. GPT-6 Astra subsequently reviewed that work and identified problems, according to the source article. Claude did not take part in the Qwen edit described here; the source says GPT-6 Astra’s Codex session carried out the experiment, while Claude was used to draft and check the article against the experiment records.
The model is identified in the report as Qwen3-4B-Instruct-2507, described there as a small open model under the Apache-2.0 license. The test’s frozen status is intended to separate final evaluation from the work used to choose an edit. The source material does not provide enough detail here to independently assess how the cases were generated or whether the test fully avoids overlap with development examples.
“The starting hint for experiment 3 came from the previous attempt.”
— The experiment report
Independent Checks Still Pending
The result comes from an internal experiment and, according to the source article, has not received an independent review. The article says an earlier experiment was checked by a second AI that found faults; no equivalent check had yet been completed for this weight edit. The report’s scores and tensor comparison are presented by the experiment team, but the source material does not include an external replication.
The available account also leaves open how well the improvement holds beyond the frozen synthetic test. It does not establish performance on naturally occurring text, a larger or different benchmark, other model versions, or live applications. Nor does it report whether the edit creates side effects on tasks that were not tested. The exact test-generation and validation procedures are not fully described in the supplied material, limiting outside assessment of possible overlap or test-selection effects.
Replication and Broader Testing
The next clear step is an independent review of the experiment records, the test construction and the saved weight change. Further testing would need to check whether the gain persists on new cases and real-world text, while measuring performance on other tasks to detect unintended effects. The source article does not give a schedule for that review or describe a planned broader benchmark.
Until such checks are reported, the result should be treated as an internal, task-specific finding: a one-matrix edit was associated with a higher score on a frozen test of exact text preservation in JSON. Whether the technique can reliably improve other behaviors remains unanswered.
Key Questions
What changed in the Qwen model?
The experiment report says GPT-6 Astra made a rank-one update to one output-projection matrix in the final MLP layer of Qwen3-4B-Instruct-2507. It reports that the model’s other 397 stored tensors remained bit-identical.
What does the score increase measure?
It measures exact answers on a 120-case synthetic test in which the model had to preserve text containing special characters when placing it in JSON. The score rose from 49 correct cases to 70.
Did the edit fix JSON handling generally?
No such broad conclusion is established. The edited model still got 50 of 120 cases wrong, and the source says a separate table-row extraction test remained at zero of 30.
Has another team verified the result?
Not according to the source article, which says the experiment had not yet been checked by a second AI. The supplied material does not report an external replication.
Why did the experiment focus on tabs and escapes?
An earlier experiment on the same model reportedly produced invalid JSON answers when raw tab characters were copied into strings. That observed failure prompted a targeted test of whether the model could preserve such text using valid escape sequences.
Source: ThorstenMeyerAI.com
