BriLLM-MetaPred Explores Language Modeling With a Unified Dynamical State

A Shanghai Jiao Tong University team compares its roughly 13.2M-parameter MetaPred language model with a same-size GPT baseline. The work offers initial engineering evidence, not validation of a complete AGI system.
Editorial correction (November 16, 2026): MetaPred's theoretical propositions depend on the paper's definitions and assumptions. The small-scale language-modeling experiments do not establish real-world AGI, a unique technical path or frontier-model performance.
When language models begin to grow out of mathematics: BriLLM-MetaPred puts the first core primitive derived from AGI theory into practice for the first time
Must language-model design rely only on humans continually stacking modules and adjusting structures? As the industry pursues increasingly expensive incremental improvements along extensions of the Transformer, Professor Hai Zhao's team at Shanghai Jiao Tong University offers a counterintuitive answer: first define AGI rigorously in mathematics, then let a machine-learning model grow out of that definition.
Their newly uploaded arXiv paper, BriLLM-MetaPred, is the first engineering realization of this path. It is not another better language model, but the first runnable instance, for language modeling, of the machine-learning primitive MetaPred derived from A Mathematical Theory of AGI.

- Paper: BriLLM-MetaPred: A Representation-Prediction Unity Language Model with Adaptive Self-Measurement
- arXiv: https://arxiv.org/abs/2610.34018
- Team: Institute of Artificial General Intelligence, Shanghai Jiao Tong University, Professor Hai Zhao's team
Convergence of two independent lines of evidence:
Isomorphism between mathematical derivation and principles of the brain
Looking only at the derivation of MetaPred from a definition of AGI, readers are entitled to ask whether it is merely a self-consistent formal system. A Mathematical Theory of AGI provides an easily overlooked but crucial addition here: at an abstract level, this mathematical path is highly isomorphic to the mainstream predictive-coding/free-energy principles in neuroscience.
The theoretical paper devotes its final chapter to this question. The natural balance principle in A Mathematical Theory of AGI—a ratio functional involving only energy (the system's cost) and entropy (the system's information gain)—also derives MetaPred as the optimum. This means MetaPred is obtained not only from the definition of the limit of infinite self-cognition, but also from the principle that any self-organizing system must strike an optimal balance between energy costs and information gains, which is closer to physical and biological constraints. When one mathematical primitive satisfies both the limit of cognition and nature's economy, it is unlikely to be a purely engineering coincidence.
BriLLM-MetaPred's Related Work section also explicitly lists Friston's free-energy principle and Bastos and colleagues' work on predictive-coding microcircuits as theoretical references. These are not decorative citations. Predictive coding's core proposition is that the brain maintains alignment between its internal model and the world by minimizing prediction error. MetaPred's core mechanism—a single dynamical state carrying both representation and prediction, while maintaining internal consistency at a fixed point through continuous self-measurement—is, at an abstract level, a mathematical restatement of predictive coding. Representation is not externally attached, and prediction is not read out from representation; the two are aspects of the same dynamical process. The self-supervision signal is not an externally imposed proxy task, but an intrinsic constraint driving the state toward its own fixed point.
This forms an indirect interdisciplinary argument, rather than circular reasoning. Neuroscience has not proved MetaPred correct, and the mathematical theory has not predicted that the brain must use MetaPred. But when a primitive derived from a purely mathematical definition meets a unifying principle inferred from experimental neuroscience in the same abstract structure, the credibility of this theoretical path no longer depends solely on its own claim of necessity and sufficiency. It gains an external anchor from an independent discipline. In other words, the theory behind BriLLM-MetaPred is not merely a formal system still wholly awaiting testing. At the level of what intelligence's core computation should look like, it has already converged with a principled description of brain mechanisms.
The starting point:
A self-contained, rigorous mathematical definition of AGI
Most AGI approaches either draw inspiration from biological brains or assemble engineering modules externally. A Mathematical Theory of AGI, on which BriLLM-MetaPred is based, takes a completely different path. Without presupposing any particular network topology or cognitive module, it starts from general cognition itself, gives a self-contained mathematical definition and then solves it.
The authors derive MetaPred under their definition of AGI and mathematical assumptions. The uniqueness here is a theoretical argument within that formal system, not a claim that real-world intelligence has only one possible implementation path.
The authors claim that MetaPred has necessity and sufficiency under their proposed mathematical definition. This is not experimental proof of general intelligence in the open real world, and cannot be used to exclude other machine-learning approaches.
The core of MetaPred:
One state, both representation and prediction
What does this primitive, grown from a mathematical optimum, actually look like?
MetaPred's defining feature is extremely simple but entirely different from today's mainstream architectures: the representation a model forms and the prediction it produces are two aspects of the same continuously evolving dynamical state.
Conventional language models separate representation and prediction into two structures: an encoder or hidden layers form representations, while an output head reads predictions from those representations. MetaPred removes that split. It maintains one continuously evolving dynamical state. Context is written into that state, the state evolves over time, and the next-token prediction is read directly from that same state.

Another key mechanism follows from this: continuous self-measurement. The model continually measures its own dynamical state, producing three effects: reducing inconsistency caused by finite depth, maintaining internal consistency, and providing an endogenous dynamical self-supervision signal in fixed-point form.
This deserves a pause for examination. Mainstream self-supervision relies on externally constructed proxy tasks—masked prediction, contrastive learning and next-token prediction—with signals originating outside the model's dynamics. MetaPred's self-supervision signal comes from the state's measurement of itself. Its training objective is formally equivalent to requiring internal representations to converge through iteration toward a self-consistent stable point. This is an endogenous signal, rather than externally imposed supervision.

Because training and inference share the same dynamical state, MetaPred exhibits symmetry across almost all key elements of machine learning. The paper describes this as an intuitive and observable property of the mathematical optimum in paradigm space.
BriLLM-MetaPred:
Putting the primitive into a runnable substrate
Theory provides the primitive, but a primitive needs a substrate to run on real hardware.
BriLLM-MetaPred's engineering path clearly connects two prior contributions. The AGI theory provides the construction and motivation of the MetaPred primitive. BriLLM's SiFu node–signal-flow paradigm provides the substrate that lets the primitive execute on existing software systems and hardware accelerators.
The paper explicitly defines its boundaries: BriLLM-MetaPred is not another recurrent language model or a simple extension of BriLLM. It is a bridge connecting a theoretically derived primitive to an engineering substrate that can run.
In implementation, it is a node-state language model. Token projection vectors are written into fixed interface rows, a recurrent operator updates the entire tensor through dynamical evolution, and the next-token distribution is read from designated output rows and a finite auxiliary pool. The same continuously evolving tensor carries both the input-conditioned representation and the state used for readout.
The key empirical result is that BriLLM-MetaPred's predictive performance is comparable to a GPT baseline of the same scale, at approximately 13.2 million parameters.
For a new paradigm derived from scratch, the significance is not that it surpasses GPT. It demonstrates that this mathematically derived primitive is feasible to implement for language modeling, a core cognitive task. The paper positions it as initial evidence for the engineering feasibility of the MetaPred framework.
The paper's core comparable result is a language-modeling comparison between a roughly 13.2-million-parameter model and a same-size GPT baseline. It cannot be converted into a universal performance multiple relative to GPT-1 or frontier models.
Larger-scale validation requires consulting the specific models, tasks and complete results. Being close to GPT-2 in size cannot be equated with matching GPT-2's performance across the board.

A distinctive position among parallel approaches:
An attempt to connect a theoretical primitive with language-modeling experiments
The paper provides an implementation that moves from theoretical motivation to a runnable language model. There is not yet sufficient cross-comparative evidence to establish it as the only approach to have completed research of this kind.
The central proposition of this comparison is simple. In assessing whether an AGI theoretical approach is serious, there is now an unavoidable engineering filter: can it be implemented in open-domain sequence tasks at the scale of language modeling? This is not to say that language is the only gateway to AGI. But if a theoretical approach cannot even achieve GPT-1-level sequence modeling, its empirical status remains one of not having confronted the strongest engineering reality.
FEP has deep neuroscientific foundations and an embodied closed loop, but has not provided a scalable implementation for open-domain language modeling. AIXI has a rigorous theory of optimality, but is proven uncomputable, with the barrier of approximation separating it from the engineering reality of LLMs. Approaches such as CAA-X have ambitious mathematical specifications, but have not yet produced comparable empirical results on LLM sequences. BriLLM-MetaPred fills this gap. What it demonstrates is not that MetaPred already outperforms GPT, but that a primitive derived from a mathematical theory of AGI can hold its own in the engineering arena of contemporary LLMs. This is a demonstration of eligibility: the theoretical approach can enter the same empirical conversation as mainstream paradigms, rather than remaining in philosophy or small toy tasks.
That is why the apparently restrained result of matching a same-size GPT baseline has significance for positioning this work in the broader competition among approaches. It does not announce that AGI has arrived. It brings a path that previously existed only in mathematical derivation to the engineering test bench of contemporary language models for the first time.
Alignment with pain points at the GPT frontier:
Not a larger encyclopedia, but a dynamical core that continually corrects itself
Understanding BriLLM-MetaPred merely as another language model misses the problem it really targets. The paper repeatedly emphasizes at the paradigm level that the difference between MetaPred and GPT is not a difference between architectural variants, but a difference in the machine-learning paradigm. GPT is built on the Transformer, with representation and prediction as two separate computational stages. MetaPred's defining feature, however, is that representation and prediction are two aspects of the same continuously evolving dynamical state, with training and inference sharing one state-evolution path. There are no separate training-forward and inference-forward passes. This paradigm-level difference lands precisely on several of the most closely watched structural pain points in the current GPT approach.
First, continual learning and catastrophic forgetting. Training and inference are separate in the mainstream GPT paradigm. After training, model parameters are frozen, and learning during deployment is achieved almost entirely indirectly through the context window or external retrieval. MetaPred's single dynamical state intrinsically couples training and inference: during training, self-measurement supplies gradients for parameter updates; during inference, the same self-measurement guides state correction. The paper's continuous-execution training regime retains node states and fast parameters between sequence segments, maintaining continuity of state within the sequence rather than reinitializing at every segment. This is not yet complete lifelong learning, but it supplies an architectural substrate in which training and inference are not split. Continual learning is no longer an extra module to be attached, but a natural implication of paradigm symmetry.
Second, the structural prerequisites for recursive self-improvement, or RSI. RSI's central requirement is that a system can measure its own performance and improve itself accordingly. MetaPred's continuous self-measurement—continually measuring its own dynamical state and generating an endogenous self-supervision signal in fixed-point form—already structurally contains the beginnings of a self-evaluation→self-correction loop. The paper describes it as an instantiation of Lambek's fixed-point lemma in machine learning: the difference between finite-step and deeper-step simulations forms a consistency signal that needs no external labels. There is still a long road to full RSI, but self-measurement moves from an external monitoring tool to an endogenous part of the model's dynamics. In a roadmap with RSI as its long-term direction, this is a more fundamental starting point than wrapping a self-evaluation module around GPT.
Third, scalability pain points, and the idea that AGI need not be an encyclopedia. Discussions of scaling in the GPT approach have long focused on jointly expanding parameter count, data and compute. MetaPred's theoretical derivation offers a different direction. Because the model substrate and dynamical state are weakly coupled, the state needs to align with only a subset of the substrate, rather than being strictly isomorphic to the entire architecture. The paper explicitly identifies this relaxation as the fundamental mechanism enabling BriLLM to support arbitrarily long inputs and outputs while keeping model size constant. In other words, longer contexts and more complex sequences need not come at the cost of endlessly expanding the model itself. This directly echoes the judgment that AGI need not be an encyclopedia or an infinitely large model. If intelligence's core is a dynamical process capable of continuous self-measurement and self-correction, knowledge need not all be compressed into parameters, and expanding capabilities need not mean linear or even superlinear growth in parameter count. MetaPred's theory even predicts that performance can be decoupled from model size. BriLLM-MetaPred matching a same-size GPT baseline at roughly one-tenth GPT-1's scale is the first testable engineering anchor in this direction.
It must be emphasized with restraint that these alignments are currently structural indications, not completed solutions. Continuous-execution training is not yet lifelong learning in the open world; the fixed-point self-measurement loop is not yet recursive self-improvement; and constant model size supporting arbitrary input lengths does not mean scalability problems have disappeared. Together, however, they point to a default assumption different from the GPT paradigm: intelligence's core is not storing as much knowledge as possible in frozen parameters, but a dynamical process that continually evolves, measures itself and maintains internal consistency through interaction with the environment. Under this assumption, a larger encyclopedia is not an unavoidable path to AGI. A sufficiently fundamental dynamical core may be closer to the definition of general cognition than a larger static knowledge base.
A core-component breakthrough, not the AGI endgame
The key to understanding BriLLM-MetaPred is seeing clearly what it has accomplished and what it has not claimed.
What is instantiated and tested here is a language model at a particular scale. Results against a same-size baseline support initial engineering feasibility, but do not establish overall capability comparable to contemporary frontier systems.
It does not claim that this language-model instance is itself a complete AGI system. The paper's title and abstract consistently position it as the first engineering realization in language-model form, rather than the complete realization of general cognition. System-level requirements in the theory, such as agency, an embodied closed loop and lifelong learning, belong to later parts of the theoretical blueprint, not this paper's engineering contribution.
BriLLM-MetaPred's value lies precisely in refusing to skip the hardest step. If an AGI theoretical path cannot be implemented on open-domain learning tasks of language modeling's scale and complexity, its credibility remains suspended. With the restrained but testable result of matching a same-size GPT baseline, BriLLM-MetaPred provides the first engineering anchor for deriving a machine-learning paradigm from mathematical theory.
BriLLM-MetaPred is a brain-like model, but differs from other brain-like models: its brain-like properties are the natural and necessary result of the theory and engineering implementation, rather than the design motivation.
It is not a faster engine, but the first engine prototype built to an entirely new, self-contained mathematical blueprint. And that prototype is already running on the test bench.
This article comes from the WeChat public account Synced and was written by Synced.
Related topics
After reading
Join the AINRK community
Read how to join the community and connect with people working in AI.




