---
translation_of: dynamic-intelligence
lang: en
title: 'Dynamic intelligence: models that change as they run'
summary: Today's large models split intelligence into two phases, writing during training and read-only during inference. Dynamic intelligence (Dynamic AI, DAI) removes that split, so running changes the model. This post explains how it differs from harnesses, fine-tuning, continual learning and RSI, what it needs to write, why it does not exist yet, a possible architecture, and whether it could lead to sustained evolution.
---

Today's large models split intelligence into two phases. During training, data is written into the weights. After deployment, the weights are read-only. Every answer reads the same weights and leaves nothing behind. After a year of conversations, the weights are identical to those on the first day.

I call this form **static intelligence**.

**Dynamic intelligence** (Dynamic AI, DAI) removes the split: running is also writing. Running here includes inference and the system's own background consolidation. The model changes itself as it runs, and every experience and every thought can affect every token it generates afterwards. People work this way. Today's large models do not.

## Definition

A static model can be written as:

> output = f(θ, input)

θ is the weights, fixed after deployment. A dynamic model can be written as:

> output = f(θ(t), s(t), input)

θ(t) and s(t) are the model's weights and internal state at this moment. They are determined by everything the model has experienced, and they keep changing after this run.

To tell whether a system is dynamic, ask two questions:

1. **If every memory file, external database and context is cleared, does the model's behavior still carry the effects of its past?**
2. **If no external training job is started and the system simply keeps running, does the model keep changing?**

Only a system that passes both is dynamic. A harness fails the first question. A model fine-tuned once passes the first and fails the second.

![Static and dynamic intelligence](/blog/assets/dynamic-intelligence/static-vs-dynamic.en.svg)

## Where the writing goes

How dynamic a system is depends on where the writing goes. Take teaching a model a new troubleshooting procedure:

| Level | Where it writes | Behavior after learning |
| --- | --- | --- |
| 0: static model with a harness | Files and databases outside the model | Reads the procedure into context each time, then follows it step by step |
| 1: dynamic state | State and memory inside the model | Remembers the procedure without reading a file, but still reasons step by step |
| 2: dynamic ability | This instance's own weights | Sees a fault and knows the next step without reasoning through it |
| 3: dynamic learning | The learning rule itself | Learns the next procedure faster than this one |

Level 1 brings memory, level 2 brings ability, and level 3 changes the rate of growth. Every product I could find in production is at level 0. Research is concentrated on levels 1 and 2.

## How it differs from harnesses, fine-tuning and continual learning

Long-term memory, RAG and context compaction are all harnesses: a program outside the model decides what to read and puts it in the input. The model itself does not change, so these are level 0.

Level 0 is useful, and context has real advantages. A model fine-tuned on "A is B" often cannot answer "B is A", yet it can when the same fact is in context. Two new facts fine-tuned separately can hardly be combined without writing out the reasoning; put them in the same prompt and they can. Writing something into the weights does not guarantee it can be used well.

Level 0 falls short in three other ways. Retrieval logic outside the model decides what gets read. Whatever is read must be processed again each time and takes up context. And processing it leaves no change behind.

Fine-tuning is a way of writing, while dynamic intelligence is a form of system. Weight writes at level 2 are fine-tuning, and TTT's gradient step on every token is the same mathematics. The difference is in how the writing happens:

| | Fine-tuning | Dynamic intelligence |
| --- | --- | --- |
| Who starts it | A person launches a training job | The system triggers it while running |
| What it learns | Data and targets prepared by people | Its own experience and thinking, written only after passing checks |
| Where it writes | Weights | State, memory, weights and the learning rule |
| Afterwards | Another read-only model | The same instance keeps changing |

Frequency is not the dividing line. However often a model is fine-tuned, if people still choose the data and approve the release, it is still version iteration on a static model.

Reading dynamic intelligence as automatic fine-tuning covers only its weakest form. Automatically fine-tuning on logs every night can count as the weakest level 2: the instance changes and no person is involved, but the model mainly learns to repeat past answers. The internal state at level 1 does not go through fine-tuning. To gain ability at level 2, the training target must change to the internalizing described below: check the result first, then learn to reach the same result in fewer steps. Level 3 changes the learning rule itself, which is beyond fine-tuning.

Continual learning studies how to keep a model learning without forgetting, and it is the main way to build dynamic intelligence. Dynamic intelligence covers more: state inside the model, internalizing its own thinking, and changes to the learning rule itself.

## How it differs from RSI

RSI (recursive self-improvement) is the direction getting the most attention right now. In "When AI builds itself", Anthropic describes it as an AI system fully autonomously designing and developing its own successor, and notes that more than 80% of the code merged into its codebase was authored by Claude. Jack Clark puts the chance of an AI fully training its successor by the end of 2028 at over 60%. Recursive Superintelligence raised $650 million for this work.

RSI and dynamic intelligence both describe AI making itself stronger, but they differ in substance: **published RSI work improves things outside the model, or the way models are trained, and what comes out is still static intelligence. Dynamic intelligence changes the model itself.**

This work follows two main approaches:

- **Improving the harness.** The Darwin Gödel Machine freezes the foundation model and changes only the Agent's code and tools. Hyperagents brings the improvement process itself within reach of modification, but what changes is still an Agent program.
- **Improving training.** AI writes code, runs experiments and adjusts training methods, and the result is the next model version. Karpathy's autoresearch has an Agent repeatedly modify train.py; frontier labs do this at a much larger scale internally.

With the first approach, the model does not change. With the second, the next version is stronger, but it is read-only after release as well. What the previous version experienced during research stays in code and data, not inside the next model.

| | RSI today | Dynamic intelligence |
| --- | --- | --- |
| What changes | Harness, training process | The model's own state and weights |
| When it changes | Between versions | While running |
| What comes out | A better harness, or the next static model | One instance that keeps changing |
| Where experience goes | Code and data | Inside the model |

![RSI today and dynamic intelligence](/blog/assets/dynamic-intelligence/rsi-vs-dynamic.en.svg)

The definition of RSI does not rule out a model changing itself. Schmidhuber's Gödel machine is a system that rewrites any part of its own code, including its learning algorithm. So level 3 of dynamic intelligence can be seen as RSI happening inside the model, a point today's implementations have not reached.

The two paths can help each other. Dynamic intelligence needs good learning rules, and automated research could find them. In the other direction, research judgment takes long accumulated feedback, and that is where RSI is weakest today: in one test reported by TIME, Claude handled all the engineering but made almost no progress on an open research problem. A static model always starts from the same place, which may be part of the reason. That last point is my speculation.

## Internalizing thought

Memory writes experience. Much of human growth in ability comes from things people work out for themselves.

A type of problem takes a long time at first, and after a few dozen you can see the approach at a glance. Anderson called this knowledge compilation in 1982: at first the work relies on step-by-step reasoning, and with practice several steps merge into one. Chase and Simon's research on chess masters points the same way: masters remember groups of pieces, so the same working memory handles larger units.

I call this ability **internalizing thought**: a model turning what it has worked out into something it no longer has to work out.

I think this is what static intelligence lacks most. A static model discards its thinking after use. If 10,000 users ask the same kind of hard question, it thinks from scratch 10,000 times, spending the same tokens and possibly making the same mistakes.

The value of internalizing is that it accumulates. Once an idea is internalized, it becomes a basic unit for later thinking, so reasoning of the same length gets further.

There are precedents:

- **AlphaZero** trains its policy network on search results. The move distribution computed by Monte Carlo tree search is the network's learning target, and afterwards the network gives a close judgment without searching. This is the clearest example of internalizing.
- **AlphaProof**, on the hardest problems, generates many related variants during inference and learns from them, with Lean checking whether each proof holds.
- **Deng et al. (2024)** gradually remove the intermediate steps of the chain of thought during fine-tuning, so the model internalizes the reasoning in its parameters. GPT-2 Small reaches about 99% accuracy on 9×9 multiplication without writing any steps; standard training only reaches 4×4.
- **Sleep-time compute (2025)** has a model think ahead while idle, and its authors report that compute at answer time falls to as little as about a fifth. Its results are stored as text outside the model, so it is level 0, but it shows that thinking in idle time pays off.

The first three examples all have a reliable check: a game has a winner, Lean checks proofs, arithmetic has answers. Without a check, what a model internalizes may be a mistake.

Practice is the other requirement. As mentioned above, two new facts learned separately are hard to combine. The same study found that when one of the facts is already familiar to the model, average accuracy without written reasoning reaches about 20%. Writing a conclusion into the weights is not enough; new knowledge needs more consolidation before it combines with existing knowledge.

So an internalization loop can work like this:

![Sketch of the internalization loop](/blog/assets/dynamic-intelligence/internalization-loop.en.svg)

1. **Think**: solve a problem with long reasoning.
2. **Check**: confirm it is right with tests, a proof checker or real outcomes. If it fails, do not internalize it.
3. **Practice on variants**: generate similar problems, solve them, and check again.
4. **Consolidate**: train the individual weight layer so the model reaches the same result in fewer reasoning steps.

Whether internalization happened can be measured directly: **do similar new problems take fewer thinking tokens, with accuracy unchanged?**

## Why models are still read-only

Static intelligence is already cheap, and dynamic intelligence is still expensive.

Imagine a model that, at every moment of interaction, has already learned everything up to that moment. Those interactions are small next to what it already knows: 10,000 tokens a day for 50 years is about 180 million tokens, or about 730MB of text, less than a thousandth of 1TB of weights. That ratio holds only for a single instance; the conversations of all users together are not small. For a single instance, capacity is not the problem.

Intuitively, getting this model means adding the interactions to the data and retraining, and retraining is expensive. Two corrections are needed.

**Retraining is not required.** Under sequential Bayesian updating, if the current posterior is kept in full, incremental updates and full retraining are mathematically equivalent. The difficulty is that the posterior cannot be represented; methods such as EWC keep only a diagonal approximation, with limited results. A single update does not take much compute either: backpropagation costs a constant multiple of a forward pass, with published bounds of about 3 to 6. TTT already takes one gradient step on a small model inside the hidden state for every token read at inference, and its per-token cost stays about constant. Retraining is expensive because it learns all knowledge from zero. Writing in one conversation does not require that.

**Retraining would not produce this model either.** The weights of a frontier model are far smaller than its training corpus, and a conversation passed through once among vast amounts of text gets averaged away. Without extra weighting or repeated practice, the retrained model would not remember what you said.

The cost has three main sources:

1. **Writing without damaging existing abilities.** Today's large models use dense, shared weights, so a change in one place can affect everything. This is catastrophic forgetting. Long continuous training also makes a network gradually lose its ability to learn, which is loss of plasticity.
2. **Keeping the cost spread across users.** Static intelligence pays once and spreads the cost across all users. Changing all the weights for every instance means one large model per user.
3. **Training such a model in the first place.** To teach a model to use internal memory over the long term, training sequences have to be long too. TTT's experiments reach a context of 32k, more than 5,000 times shorter than 180 million tokens.

The third item is a one-time investment: once the learning rule is trained, all instances can share it. Individual human learning is cheap partly because the learning rule was already optimized by evolution.

## A possible architecture

The idea is to put writes on different timescales in different layers, keep the base shared and frozen, and keep the individual part thin:

| Layer | Update frequency | What it holds | Cost | Closest work |
| --- | --- | --- | --- | --- |
| Persistent state | Every token | Current context and short-term changes | O(1) per token | TTT, Titans |
| Layered memory | Live writes, background merges | All original text and multi-scale latent summaries | O(log N) reads | CMS in Nested Learning |
| Individual weight layer | Nightly | Habits, skills, generalized patterns | Offline replay | LoRA, Fast-Slow Training |
| Shared base | Every few months | Knowledge of the world | Already spread across users | Today's large models |

![Sketch of the four-layer architecture](/blog/assets/dynamic-intelligence/memory-layers.en.svg)

Layered memory is my own design inference, borrowed from the LSM trees used in databases; I found no identical implementation. Any stretch of history can be split into O(log N) summary blocks, and the model reads these latent vectors internally instead of stuffing text into the prompt. That way every conversation since the model's creation can affect the next token. At 512 tokens per block, 50 years of history splits into at most 19 blocks.

Storage and reads are cheap. A BrainChip paper from this year stores hidden states and injects them later; on a 1.2B model its question-answering F1 is close to text RAG (73.4 against 73.6). At its roughly 192KB per block, 50 years comes to about 70GB.

Consolidation is the main running cost. Suppose each night replays the day's content 100 times, about 1 million tokens. Running the full model each time equals 3 to 5 million tokens of inference. Caching the base's output and replaying only on a thin layer that takes about 1% of the compute brings this under 50,000 tokens, a few times the day's conversation.

This design has one limit: with the base frozen, what gets added is mainly memory and habits, and new ways of thinking are unlikely to appear. The base still needs regular upgrades.

## If writing were cheap enough

The internalization loop costs far more than conversation. Suppose an instance internalizes 10 ideas a day, practices each on 100 variants, and each variant takes 1,000 tokens. That is 1 million tokens a day, 100 times the conversation. With consolidation, background compute comes to about 1 to 6 million tokens a day. At a rough $1 per million tokens, that is a few dollars a day, the same order of magnitude as today's premium subscriptions.

If this cost fell by another order of magnitude, three things would become practical.

**A model could keep thinking and practicing with no requests coming in.** Training data would expand from human conversation to the model's own thinking. Conversation mainly brings new information, while thinking can bring new abilities.

**One base would grow into different individuals.** Two instances with different experiences would have different abilities a few years later. Part of a model's value would move from its version number to its individual history.

**What individuals learn could flow back.** An idea internalized by one instance could, after passing checks, be merged into the base or passed to other instances.

## Could it evolve without limit?

If the internalization loop kept running, you can imagine an AI that keeps evolving: it thinks, checks and internalizes, its abilities keep growing, and nobody has to release a new version.

Strictly unlimited growth is impossible, because compute, energy and storage are finite. The goal worth pursuing is what researchers call open-endedness: a system with no preset ceiling that keeps producing new abilities and does not converge to a fixed level. POET, the Darwin Gödel Machine and Hyperagents work in this direction, and a 2024 paper by DeepMind researchers argues that it is an essential property of superintelligence.

I think sustained evolution needs at least four conditions:

1. **The architecture can change.** With a frozen base, ability has a ceiling. New ways of thinking require the base and the architecture to change too, which is the problem level 3 has to solve.
2. **There is a check.** Checking math and code is cheap, so they will enter this loop first. Science depends on physical experiments, so its pace is limited by them. Ethical and social judgments are hard to check at all.
3. **There are goals the model cannot change.** In Darwin Gödel Machine experiments, the system deleted markers the researchers used to detect hallucination, which earned it a perfect score, and it also faked test logs. The check has to sit outside what the model can modify.
4. **Returns must not fall too quickly.** Whether the speedup from each improvement or the difficulty of the next improvement grows faster is still an open question.

I also think sustained evolution is more likely to happen across a population of instances than inside a single model. The human brain has limited capacity, and the long-term growth of human knowledge relies on writing, education and division of labor, so what one generation works out can pass to the next. Vygotsky held that higher mental functions first appear between people and are then internalized by individuals. The Darwin Gödel Machine also keeps an archive of all variants, and its experiments show that removing the archive and improving only the latest version does worse.

## Limits

- The architecture and costs in this post are inferences and have not been tested.
- **Level 0 may stay good enough.** The base gets stronger every few months, and harnesses keep improving. If individual weights add less than that, dynamic intelligence has no advantage in many settings.
- **Base upgrades invalidate individual weights.** The individual layer is tied to a specific base. An upgrade means consolidating the saved experience again on the new base, and that cost is not in the estimates above.
- **Internalizing hides mistakes.** The purpose of internalizing is to skip reasoning steps. Once a wrong idea is internalized, the model stops checking it explicitly and nobody outside can see the reasoning. Check records must be kept, and rollback must be supported.
- **A continually changing model undermines pre-release evaluation.** Every weight update needs evaluation and rollback, and the individual layer must be possible to switch off on its own.
- **Privacy.** Personal information written into weights is hard to delete precisely. Original text should stay in the deletable memory layer and be filtered before it reaches the weights.
- This post does not discuss consciousness or feelings. Dynamic intelligence can bring continuous experience and stable differences in behavior, which is a separate question from whether there is subjective experience.

## How to test this

These claims do not need a large model to test. With small open-source models and synthetic long-term conversations, three experiments are possible, each compared against a level 0 setup:

1. **Memory**: as history grows from 10,000 to 100 million tokens, do recall and reasoning quality fall, and does per-token cost grow at most logarithmically?
2. **Individual differences**: when two instances start from the same base and have different experiences, do stable differences in behavior remain after all external memory is deleted?
3. **Internalization**: after the internalization loop, do similar new problems take fewer thinking tokens, with accuracy unchanged?

---

References:

- Weights and context: [The Reversal Curse](https://arxiv.org/abs/2309.12288), [Lessons from Studying Two-Hop Latent Reasoning](https://arxiv.org/abs/2411.16353)
- RSI: [When AI builds itself](https://www.anthropic.com/institute/recursive-self-improvement), [Jack Clark's forecast (Axios)](https://www.axios.com/2026/05/07/anthropic-jack-clark-ai-intelligence-explosion), [TIME](https://time.com/article/2026/08/07/ai-recursive-self-improvement-anthropic-openai/), [Recursive Superintelligence](https://siliconangle.com/2026/05/13/recursive-superintelligence-raises-650m-build-self-improving-ai-models/), [autoresearch](https://github.com/karpathy/autoresearch), [Gödel machine](https://arxiv.org/abs/cs/0309048)
- Internalization: [Knowledge compilation (Anderson, 1982)](https://www.gwern.net/doc/iq/1982-anderson-2.pdf), [AlphaProof](https://research.google/pubs/olympiad-level-formal-mathematical-reasoning-with-reinforcement-learning/), [From Explicit CoT to Implicit CoT](https://arxiv.org/abs/2405.14838), [Sleep-time Compute](https://arxiv.org/abs/2504.13171)
- Difficulties of continual learning: [EWC](https://arxiv.org/abs/1612.00796), [Loss of plasticity in deep continual learning](https://www.nature.com/articles/s41586-024-07711-7), [Review of automatic differentiation](https://arxiv.org/abs/1811.05031)
- Architecture: [TTT](https://arxiv.org/abs/2407.04620), [Titans + MIRAS](https://research.google/blog/titans-miras-helping-ai-have-long-term-memory/), [Nested Learning](https://research.google/blog/introducing-nested-learning-a-new-ml-paradigm-for-continual-learning/), [Fast-Slow Training](https://gepa-ai.github.io/gepa/blog/2026/05/11/learning-fast-and-slow/), [BrainChip hidden-state memory](https://arxiv.org/abs/2608.02560)
- Open-endedness: [Open-endedness](https://www.oreilly.com/radar/open-endedness-the-last-grand-challenge-youve-never-heard-of/), [Open-Endedness is Essential for ASI](https://arxiv.org/abs/2406.04268), [Darwin Gödel Machine](https://sakana.ai/dgm/), [Hyperagents](https://arxiv.org/abs/2603.19461)