Can LLMs redesign themselves as they train?

However long you train a neural network, only its weights change. Every other component, from the optimiser to the data mix and the architecture, remains static. What if, instead, we made everything learnable, including the architecture?

In this post, we find early signs that coding agents can redesign neural networks while they train. The resulting architectures can look rather alien. Below, we report early results on four experiments, each aimed at a basic limitation of today's transformers.

Nothing is frozen

Recent work has let coding agents run research projects end to end, tune training code overnight, rewrite the update rules of networks as they train [Bai et al., 2026] and even edit their weights directly [Takehi et al., 2026]. We simply ask: why not let them take over every part of training at once, including even the architecture itself?

Transformers have well-known limitations. They often fail to generalise to inputs longer than those seen in training, and scale alone does not fix this [Anil et al., 2022] [Delétang et al., 2023]. They struggle with compositional tasks that take many steps [Dziri et al., 2023]. Even simple logic (like inferring "B is A" from "A is B") is unreliable [Berglund et al., 2023].

Some of these limitations trace back to gradient descent itself. Gradient descent fits statistics extremely well, but it settles for whatever lowers the loss fastest, often shortcuts that hold only on the training data [Geirhos et al., 2020]. The representations it finds can be fractured and entangled [Kumar et al., 2025]. Code behaves differently. Programs apply the same rule to every input, seen or unseen, so they generalise by construction. Programs are also short. The shorter the description of a hypothesis, the fewer examples it needs to generalise [Blumer et al., 1987], an idea formalised in the minimum description length principle [Rissanen, 1978] [Shalev-Shwartz and Ben-David, 2014].

Coding agents can cover for gradient descent. When placed inside the training loop, the agents can find what the network fails to learn, work out why and write exact programs from a handful of failing examples. These programs go into the network itself, as new layers, which we call code layers. Such a change is beyond gradient descent, which moves in small, continuous steps. Rules can be written as code, but the countless statistical patterns hidden in massive data cannot. The agents can still use gradient descent to train everything else, including how much to rely on each code layer.

To decide where code layers are needed and what they should compute, the agents need an entirely new toolkit, which we call mechanistic intervention. Like mechanistic interpretability, these tools read the network's internals, but they do so during training, while the network can still adapt to what the agents change. These tools are still at an early stage. We call on the community to help develop them.

The four experiments are:

• Finding demand (nanochat). We introduce the phantom gradient, the first mechanistic intervention tool discovered by the agent, and show how the agent uses it to find where a language model needs help. We then look at the layers it wrote, including one that lets the network correct itself using its own recent errors.
• Evolution is mandatory (SWA). Comparing layers grown during training with the same layers planted at the start shows that timing matters, a new kind of training dynamic.
• Jaggedness (CLRS-Text). We train a transformer on CLRS-Text, a benchmark of classical algorithms. Trained normally, the model solves problems only at the sizes it saw in training, failing at the sizes in between. Code layers grown during training close most of these gaps.
• Closing the loop (DeepSeek). DeepSeek-V4.1-Flash acts as the agent on itself. Using a code layer it designed, which reads its own hidden states, it raises its accuracy on AIME 2026 and HMMT 2026, without thinking, from 64% to 71%.

To avoid overfitting, we keep the test set hidden from the agent in every experiment. Beyond that, the agent has free rein to come up with ideas, design its own experiments and mechanistic intervention tools, and insert the code layers. We describe each experiment in detail below.

Nanochat: where does a language model need help?

We start by training nanochat [Karpathy, 2025], a small ChatGPT-style language model. We train it as usual: next-token prediction, with plain gradient descent. The network is a twelve-layer transformer of about 286 million parameters.¹

Can a coding agent, brought into the training loop, help the network learn more from the same data? In other words, can we improve sample efficiency?

We train the same network again, this time with a coding agent inside the training loop. The setup is identical (same data, schedule, steps and gradient-based updates) except that, at checkpoints of its choosing, the agent is allowed to insert new layers written as code, editing the architecture itself. Once a layer is inserted, ordinary training resumes from that point until the next insertion.

Over the span of this experiment, the agent inserted three code layers, one at a time. After each insertion, the loss dropped further below ordinary training, and the gap persisted to the end of training.

How did the agent decide what to insert, and where? We describe its process below.

From mechanistic interpretability to mechanistic intervention

Mechanistic interpretability is a young field that tries to reverse-engineer a trained and frozen neural network. It asks which internal components carry which computations, and how they combine to produce behaviour. Its tools, such as probes [Alain and Bengio, 2016], ablations and attribution graphs [Ameisen et al., 2025], are often used to audit models for safety [Marks et al., 2025], steer behaviour and understand what their neurons learned.

In our case, the agent turns the question around: what is the network failing to learn, why, and what can be added to it while it still trains? To answer it, the same tools apply, and more. This task gives interpretability something it often lacks: an objective test. An intervention either improves the metric or it doesn't.

To answer this question, the agents have begun building a toolkit of their own, through trial and error on networks as they train. The tool the agent reaches for most often is a quantity computed on the live network, which we dub the phantom gradient.²

While ordinary gradients are taken with respect to the weights, the phantom gradient is taken with respect to the transformer’s internal representations, at every token and every layer, so it gives a far more detailed map of the change the network wants, which ordinary gradients average away.

We call it the phantom gradient because it is the gradient of weights that are not there. Imagine giving every token, in every document of the batch, its own zero-initialised weight at each layer, added to its representation. The phantom gradient is exactly the gradient those weights would receive.

More specifically, the residual stream at layer $\ell$ is:

$h_\ell$ holds everything the network has computed up to layer $\ell$, a tensor of shape [B, T, D], with B documents in the batch, T token positions in each, and an embedding of width D at every position.

The phantom gradient at layer $\ell$ is $\partial L/\partial h_\ell$, the gradient of the loss with respect to this representation. It has the same shape, [B, T, D], and at every position it points in the direction that would lower the loss there most.

Watching this demand map answers the agents’ first two questions: when to intervene, and where in the network to do so. Phantom gradients that persist in a layer across checkpoints mark demand that SGD does not satisfy, and trigger a code-layer insertion there.

Interestingly, when used raw, the phantom gradient tensor misleads. The agents discovered that one normalisation is required first, centring: for each layer, subtract from every phantom vector the mean over all documents whose correct next token is the same. The raw vectors are dominated by which token the answer is, and the subtraction leaves what the context asked for.

Using phantom gradients to find and score code

There are many ways to use phantom gradients. Below is one recipe, in the form the agents converged on during the CLRS-Text runs. The agents begin by grading the network on its own training data, every task and input size separately, at successive checkpoints. Cells whose errors stay high and flat are the targets. Cells that are still improving are left to gradient descent.

From the target cells, the agents collect the tokens the network gets wrong. For each such token, they backpropagate only that token's loss and read the phantom gradient at its position, at every layer. Backpropagating one loss at a time keeps other tokens' errors out of the measurement.

They then cluster the wrong tokens with spherical k-means, which groups vectors by direction rather than size. Each token's fingerprint gives equal weight to two directions, its phantom gradient at a layer near the demand peak and the network's hidden state at that token. Tokens in the same cluster therefore ask for a similar change from a similar state. Repeating the clustering on half the tokens gives the same clusters.

Each cluster yields examples. A member is the prefix of its document up to the wrong token, together with the token that should have come next. Reading them, the agent can name the failure. On CLRS-Text, one cluster held activity-selection entries the network wrongly marked 0. Another held tasks it wrongly included in a schedule. A third held the digits of the index returned by minimum. The phantom gradients also show at which layer each cluster's demand peaks.

From these examples, the agent names the failure mode and proposes candidate code layers. Each candidate reads what a code layer would read, the tokens of the prefix and the network's hidden states along it. Its learned projections are fitted so that its output at the wrong token points along that token's phantom gradient, centred as described above and normalised to unit length. The score is how closely the output points along it on documents held out of the fit, cluster by cluster.

Measured naively, this score misleads. The phantom gradient depends on the hidden state, so learned projections alone can partly reproduce it, whatever the code computes. A fit scored on its own examples also rewards memorisation. The agent therefore scores only on held-out documents and compares every candidate against a baseline, a plain linear map from the hidden state at the wrong token. A candidate counts only if it clearly beats this baseline, both overall and on the clusters where the baseline does worst.

Other recipes read the phantom gradient differently. In other runs, the agents compared each token's demand across all layers at once, or used it to reweight the training data instead of writing code.

Integrating a code layer

The phantom gradient narrows the field but does not predict reliably how a code candidate will perform once trained³, so the final decision is left to ordinary training: if deemed worthy by gradient descent, the code candidate is kept.

Each top-ranked candidate is planted at its layer behind a gate

with $g$ a learnable scalar initialised to zero.

At the moment of insertion the network is exactly identical to what it was. However, if the layer is useful, the gradient on the gate, $g$, will gradually indicate so, and the gate will open at whatever pace the rest of the network can absorb. This gate turns a discrete edit to the architecture into a continuous one for the gradient-based optimiser.

Ranking layers

Some methodology is needed to rank candidates and to separate signal from noise.

To rule out chance (a “lottery ticket”), each code layer candidate races against a twin, an identical run from the same checkpoint on the same batches, with the new layer switched off.

One also needs to rule out the effect size that an added layer has, by virtue of only adding parameters. For that, the agent also races shams. A sham keeps the layer's shape and parameters but scrambles the information it carries, for instance by feeding it errors from a different document. If a sham does almost as well as the real layer, the gain came from capacity alone. The agent drops such candidates.⁴

The agent then ranks the candidates that survive. It can rank them either by a predefined metric, such as validation loss or accuracy on a downstream task, or by its own finer-grained measurements, such as specific capabilities that loss and task accuracy do not fully capture.

Evolution is mandatory

If a code layer helps, why not build it in from step 0? Surprisingly, the timing matters. Inserting a layer mid-training can be more useful than training with it from scratch. This creates a new kind of training dynamic, that we're only beginning to study.

We set up a model with a blind spot. It is a 124M-parameter transformer trained on 8K-token documents, but its attention is limited to a sliding window of the last 500 tokens. Anything earlier cannot be attended to.

We then gave it a code layer that extends its reach: what we call an exact-match lookup. The layer retrieves the tokens that followed earlier occurrences of the last three tokens, and adds them to every attention head at every layer. They are treated like any position within a head’s window - they are scored with the head’s query-key product and their values are read.⁵ The lookup has no parameters of its own.

We then tested the lookup outside the window by measuring how well the model recalls the value of a variable defined 600 to 4,000 tokens earlier.

When the model is trained without the layer, recall is impossible: the variable is out of sight.

Inserting the layer raises recall. Yet despite near-identical loss curves, the network with the layer inserted halfway recalls far more than the one with the layer present from the start: 72% compared to 55%, a 17-point difference.

We believe it's a form of shortcut learning [Geirhos et al., 2020]. With the layer present from the start, the network leans on it instead of building circuits of its own. Inserted halfway, the layer arrives after the network has developed its own skills, and it learns to use the layer on top of them.

This points towards a different way of training: just as a network's weights evolve over training, so should its code and architecture.

Taking on jaggedness

Even frontier models suffer from jaggedness [Karpathy, 2024]. Often times they solve a hard problem and then fail on a trivially similar one. Neural networks are prone to shortcut learning [Geirhos et al., 2020]: gradient descent picks whatever cuts the loss fastest, starving the general solution that is harder to find [Pezeshki et al., 2021].

CLRS-Text [Markeeva et al., 2024] aims to train a network that carries out thirty classical algorithms entirely “in its head”: sorting, search, dynamic programming, string matching and more. We train a twelve-layer transformer of about 86 million parameters from scratch on it, the ordinary way. That is, with next-token prediction loss and plain gradient descent. After 20 thousand training steps, its accuracy plateaus at 55%.

insertion_sort:
key: [5 2 4 3 1], initial_trace: [5 2 4 3 1]
trace: [2 5 4 3 1], [2 4 5 3 1], [2 3 4 5 1]
pred:  [1 2 3 4 5]

In CLRS-Text, the inputs to the algorithmic tasks come in many lengths, but the training data covers only some. What would happen if we tested the network on the gaps in between? More often than not, the ordinary network fails. It solves problems at the lengths it saw in training, but only a third of the problems in between. Code layers grown during training raise that third to 79%.

On nanochat, the agent was in the position of anyone working on a difficult task: nobody fully knows what a language model should be computing. It had to combine mechanistic intervention tools, like phantom gradients and error analysis, with its own reasoning and judgement. CLRS-Text is different. Here the agent, powered by a frontier model, already knows how to solve the tasks, so it can lean on its reasoning more than on its tools.

We train the same network with the coding agent in the loop. The setup is identical (same data and number of steps), and this time the agent inserts ten code layers along the way. Overall, the code layer run reaches 84% accuracy, versus just 55% for ordinary training.

More training doesn't close the gap. Trained for twice as many steps, the ordinary network plateaus at similar accuracy.

Code complements what gradient descent learns: a program applies the same rule at every length, so it generalises by construction. Combining learned circuits with written code may be one way past jaggedness.

Neural in, neural out: probing for what the code should read

In principle, a code layer can be any program. It can read anything the network has, from the raw token IDs to the hidden representation at any layer, including the outputs of other code layers. Its result can go wherever the network can use it.

In some of our code layers, a learned projection turns the network's hidden representation into the program's inputs, for instance one number per position. The program then computes its result exactly, for example the running maximum of those numbers along the sequence. A second learned projection adds the result to the network's representation, through the gate described earlier. The two ends need not sit at the same depth. A layer can read where its inputs are easiest to find and write where the phantom gradient shows demand.

Some of the CLRS-Text layers need a specific quantity, such as the values of the numbers in a problem, for example the numbers to be sorted. No single token holds these values. To find where the network holds them, the agent reached for a new tool of mechanistic intervention, linear probes [Alain and Bengio, 2016], borrowed from interpretability. It fitted a linear predictor to each layer's hidden representation and checked how well it recovered the numbers' values. The new code layer then read from the layer where the values came out most cleanly. Gradient descent did the rest.

Closing the loop: DeepSeek improving itself

In the experiments so far, a frontier agent changed another, smaller network. Naturally we ask: can a model operate on itself?

In this experiment, the agent and the network are the same model. DeepSeek-V4.1-Flash, a 763B MoE model. We let it perform “open brain surgery” on itself, through mechanistic intervention. For its task we chose competition mathematics, answered without thinking mode, and the goal is simply to solve more problems correctly. We test it on AIME 2026 and HMMT February 2026, taken from MathArena and hidden from the model throughout.

Out of the box, DeepSeek answers 64% of these problems correctly. After operating on itself, it answers 71%.

A probe on its own residual stream

To improve itself, the model reached for the same tool as the agent on CLRS-Text, a linear probe. The probe operates on DeepSeek's own residual stream at the moment it finishes a solution, read at seven of its deepest blocks, and predicts whether the answer just written is right.

The model then wrapped the probe in a code layer of a different kind from those above. Instead of adding to the network's representation, this layer controls sampling. When a solution ends, the layer queries the probe. If the probe predicts a wrong answer, the solution is thrown away and a new one is sampled, up to three times.

def probe_and_resample(h, end):     # h[L]: residual stream entering block L; end: last token of the solution
    x_now  = concat(h[L][end] for L in (16, 20, 24, 28, 32, 36, 39))
    x_tail = concat(mean(h[L][end-255 : end+1]) for L in (24, 28, 32, 36, 39))
    right  = w_now @ standardise(x_now) + w_tail @ standardise(x_tail) + b   # the linear probe
    if right < threshold:           # the probe predicts a wrong answer
        resample()                  # up to three times

Knowing what it knows

DeepSeek built the probe from its own graded solutions to 243 problems from 2024 and 2025 contests, choosing the blocks, the form of the probe and the threshold itself. It found the signal only in its deep blocks, and a single linear readout did as well as a small neural network. The 2026 contests stayed hidden until the probe was frozen. From problems it had held out of the older contests, DeepSeek predicted that the probe would add about eight points. On the hidden contests, it added seven.

The probe rejects four in five of the model's wrong answers, and only one right answer in fifteen. It is not simply reading length, since restarting the longest solutions instead gains less than two points. The knowledge also appears only once the answer is written. Probing itself midway through a solution, DeepSeek found that its hidden states there mostly encode how hard the problem is. Between two attempts at the same problem, that probe could not tell the failing one apart. Language models (mostly) know what they know [Kadavath et al., 2022]. Here, the model finds where that knowledge lives in its own activations, and acts on it.

Qwen on Qwen: a small model writes its own policy

DeepSeek found the signal in its own activations. We then asked a narrower question of a much smaller model. Qwen3-8B, a dense 8B model, solves the same kind of competition problems with its thinking mode on. A frontier agent had already built it a set of gadgets: probes on Qwen's hidden state that report, at every paragraph break of its chain of thought, how the solution is going.

We let Qwen write a policy: when to stop, when to start over, and when to trust an answer. We let Qwen write that policy itself, in Python, with no gradient training.

Out of the box, when measured on the same contests (AIME and HMMT), Qwen3-8B answers 59% of these problems correctly. With the policy it wrote for itself, it answers 66%.

The gadgets and the actions

At every paragraph break of its thinking, Qwen's policy sees what the frontier agent built:

• A stop probe: has the answer settled, so that stopping now loses nothing?
• A doom probe: will this attempt end with a wrong answer?
• Three local probes: will the answer be right about a thousand tokens from now, will a wrong answer get fixed, and will a right one derail?
• The log probability of the answer it would give if forced to stop now, and how many times it has already stated it.

The policy can act in three ways - 1) stop and answer now, 2) abandon the attempt mid-thought and start over, or 3) reject the answer at the end of an attempt and solve the problem again.

Here’s what Qwen wrote:

def step(obs):
    if not obs.last and obs.zdoom[-1] >= 1.99 and obs.b[-1] >= 30000:
        return "restart"                          # doomed and about to hit the 32k cap
    settled = obs.zstop[-1] >= (80.97 if obs.last else 70.26) or (obs.soon[-1] >= 4.59 and obs.kmax[-1] >= 6)
    return "stop" if settled else "continue"      # answer now once the answer has settled

def end(obs):                                     # when an attempt finishes
    sure = obs.selfeval["full_q1"] < -12 and obs.selfeval["full_q2"] < -12
    return "accept" if sure else "reject"         # keep only answers it is near-certain of

Qwen keeps an answer only when its own self-evaluation is near-certain the answer is right. Otherwise it throws the attempt away and solves the problem again from scratch. Attempts that run past 30k tokens while the doom probe predicts a wrong answer are abandoned mid-thought. Thinking stops early once the stop probe says the answer has settled, or once the same answer has come up six times and the local probe expects it to stay right.

Evolving the policy

Qwen writes each policy as a short Python program, which we score on about 800 older contest problems. In each round, it sees a few earlier policies and their scores and writes a new one. Over the whole search it wrote about 15,000 policies.

In this experiment, one ingredient turned out to matter most. Before writing each policy, Qwen first writes analysis code over its own graded attempts: how each signal is distributed on right and wrong answers, how well it separates them, and a test of a candidate rule on the training problems. It reads the output, then writes the policy. Without this step, Qwen picked the right signals but guessed the thresholds.

Looking ahead

We believe it is time to let agents train themselves. We also think that architecture design no longer has to end where training begins. Here is what we think this makes possible.

• Alignment through code. Today, models are made of opaque weights. Since weights cannot be easily understood, we have to try to “reverse-engineer” them after training, via mechanistic interpretability. Code, on the other hand, is readable by humans. We can test and verify it like any other program. Whatever part of a network's computation moves into code becomes far easier to align.
• Mechanistic intervention (“mechinterv”). Mechanistic intervention takes the tools of interpretability into training, so the agent can act on what it finds while the network is still learning. The tools we have are still noisy. The phantom gradient is better at showing where a network needs help than what it needs. We hope the community will help build better ones.
• Performance engineering. Standard architectures use a few operations, like attention and matrix multiplication, that already have fast, hand-tuned GPU kernels. Code layers also require such kernels. An agent that adds layers during training has to make them fast as well as correct. At doubleAI, performance engineering and kernel optimisation are part of our expertise.
• Going beyond transformers. Today, nearly every model is a transformer whose architecture stays fixed throughout training. Our results challenge this paradigm. We welcome more research into what training with an evolving architecture entails.
• Recursive self-improvement. Models trained this way learn to learn, a setting known as self-referential meta-learning [Kirsch and Schmidhuber, 2022]. Besides getting better at the task, they can get better at training themselves. Every probe or code layer that is discovered during training could join an ever growing library of mechanistic intervention tools. A model that keeps improving both how it trains and the tools it trains with is on a direct path to recursive self-improvement.

Today, a transformer still transforms everything but itself. We think that’s about to change.