AI news

Ted Xiao on AI in the physical world: how verification speed limits faster R&D

By Ted Xiao1 viewsSource: 海外独角兽
Ted Xiao’s talk on verification bottlenecks in AI, robotics and engineering

The talk examines verification bottlenecks in engineering, robot training and automated labs, arguing for testing verifiers in actual settings and using limited real-world tests to guide the next iteration.

Article illustration

If both AGI and general-purpose robots succeed, will a world of material abundance that benefits all humanity naturally follow?

In a talk at the Humanoid Hub Conference, Ted Xiao, a founding technical team member at Prometheus and former Google DeepMind researcher, began with this assumption to analyze the process from a new technology’s conception to mass production. Even if intelligence and labor become abundant, which stages will still slow research, development and production? And which research and business opportunities are worth investing in ahead of time?

In his view:

• According to Amdahl’s law, if half the time is spent on stages that cannot be accelerated, the total time can at most be halved, however fast the remaining work becomes.

• Verification determines the upper limit of model post-training.

• If a verifier has loopholes, a model may learn to exploit them for high scores without necessarily improving its actual performance.

• Verifiers themselves must also be tested. When they encounter new designs not covered by historical data, their judgments still need to be checked against real-world test results.

• Development teams need to go on-site to understand what counts as acceptable, design verifiers accordingly, and then use limited real-world testing to check and improve them.

We have compiled and translated this talk by Ted Xiao. The verification bottleneck he discusses is also one of the reasons we are optimistic about AI for Science, robotics and hardware manufacturing. As AI capabilities grow, models can propose solutions faster, but finding out whether those solutions work often still requires real experiments and engineering tests. Stronger models do not readily accelerate those tests directly.

01.

What is still missing once AGI and general-purpose robots are achieved?

Research and investment today broadly concentrate on two directions: bits at one end and atoms at the other. This trend is visible in Silicon Valley and even globally, and technical progress in both directions is rapid.

• On the bits side are AGI proponents who hope to build a “country of geniuses” inside data centers: create AGI first, then let AGI solve the other problems.

• The atoms side focuses on robot learning, dexterous manipulation involving contact with objects, and the widespread adoption of general-purpose robots.

Article illustration

Assuming both AGI and general-purpose robots succeed, preparing for a post-AGI society also requires considering what is missing between them: how can digital superintelligence genuinely improve quality of life and give people greater material abundance?

To answer that, we can first look at how material living conditions improve today.

Technological progress has always created wealth, driven social prosperity and improved people’s lives. An important part of this is inventing new physical technologies and actually making them. The comfortable lives we enjoy today are the result of generations of work by millions of engineers, scientists, designers, and white-collar and blue-collar workers around the world.

Specifically, turning an idea into a product that can be produced and used at scale requires design, engineering development, prototyping, preproduction testing and certification, production, gradually ramping up output, and expanding the scale of production.

Article illustration

To assess what is still missing between AGI and robots, we also need to place them within this concrete workflow. But this chain involves intellectual work such as design and analysis, physical labor in hands-on operations, and capital investment. Which parts can be handed to AGI, which to general-purpose embodied intelligence, and which still need other solutions?

In Ted Xiao’s classification, the blue parts of the diagram below are abstract, digital work. AI models operating only in digital environments have already demonstrated the ability to handle this kind of work. The pink parts involve contact with objects and dexterous manipulation: work that general-purpose robots could undertake.

Article illustration

Yet even if digital AGI and robotic dexterity reach very high levels, much work in the engineering chain still cannot be handed directly to them. This is apparent whether one looks at GDP, labor share, industry profit margins or the number of jobs in different fields. Even a rough list of this work reveals that obtaining human-level manipulation abilities does not mean the remaining engineering problems will solve themselves.

02.

Amdahl’s law: stages that have not been accelerated set the limit on overall speedup

To let everyone enjoy the material abundance that some people already have today, and to improve everyone’s quality of life, we must address the problems along this chain one by one. We cannot simply accelerate one end and assume the remaining stages will automatically keep up.

These unresolved stages affect how quickly a technology or product can be researched, developed, verified and brought into production at scale. As Amdahl’s law points out, even if a program has many operations that can run in parallel, its overall speed is still constrained by operations that must be completed sequentially and cannot run simultaneously. Innovation and invention face similar limits.

Article illustration

As shown above, suppose p is the fraction of the original total time taken by stages that can be accelerated, and s is the speedup factor for that work. We have already seen some stages take dramatically less time once s becomes very large. But however many times faster individual stages become, the upper limit on overall speedup still depends on p: how much time those stages originally occupied.

The gray, orange and blue curves in the diagram correspond to different fractions of work that can be accelerated. Suppose the work that can be accelerated accounts for 50% of the original total time—already broader coverage than was shown in the earlier engineering chain. Even if that work becomes infinitely fast, the total time for the whole process can at most be halved, making the overall speed twice what it was.

Only when p approaches 99%, 99.9% or even higher can the total time needed for research, repeated improvement and deployment potentially fall to one-tenth, one-hundredth or even less of its original duration.

The faster individual stages become, the more readily the parts that have not been accelerated become bottlenecks for the entire research, development and production process.

In fact, the time required for many kinds of work is hard to compress further. The underlying limitations mainly come from two sources:

One type of constraint is the time the testing itself requires.

• Testing fast-charging batteries requires load tests and charge–discharge cycle tests, a process that often takes more than a year and a half.

• Drug development, from the initial idea, research and reports through Phase III clinical trials, can sometimes take more than ten years. Because controlled human trials must assess a drug’s safety, efficacy and side effects, the time these experiments require cannot be dispensed with.

• Even after lengthy testing, experimental results can remain unstable and difficult to reproduce. A research team at US biotechnology company Amgen once tried to reproduce 53 influential preclinical cancer studies, succeeding in only six, or approximately 11%. Thus, even published research that appears highly valuable does not necessarily provide stable, reliable feedback.

The other problem is that engineering knowledge is distributed across different companies, systems and jobs. Accelerating more stages first requires understanding how that work is done. But some of this know-how is companies’ intellectual property, while some remains in practitioners’ experience and is hard to obtain directly.

It is worth recognizing that software development is actually rather unusual in this respect: code and version histories can be preserved in full, and clear rules and tools are often available to check whether a program runs as intended. The physical economy has also accumulated enormous engineering knowledge, but when we open GitHub, we do not find a complete body of material explaining how humanity built today’s world.

03.

Verification determines the upper limit of model post-training

To assess further which stages can still be accelerated, we can first look at how long verification takes. The earlier examples of batteries and drug development involve the same question: how long must we wait for trustworthy results that can guide the next round of experiments?

The answers vary enormously across tasks, from a few milliseconds to decades.

• Some results can be checked with models or simulations, and putting in more compute often provides faster feedback. Adding GPUs, CPUs and funding is therefore useful for such tasks. Although implementation may still be very difficult, we at least know where to begin accelerating them. Physics simulation, checking formal mathematical proofs with Lean, code unit tests and some design tasks all belong to this category of work that compute can accelerate.

• Other results, however, must be checked through real-world tests. At that stage, verification may take hours or even years.

Article illustration

The time verification takes matters because models need to continue learning from its results. Experience in frontier-model research suggests that as long as we can determine clearly and reliably whether results meet requirements, we can use those judgments as rewards to train models through RLVR—reinforcement learning with verifiable rewards—and improve their performance on the corresponding tasks.

Microsoft AI once described its model as a “hill-climbing machine.” This is an apt metaphor: as long as it is possible to judge reliably whether results meet the standard, a model can keep optimizing toward its target.

Verification therefore determines the level frontier models can reach during post-training, as demonstrated in fields such as mathematics and coding.

However, the rewards and verification results researchers use to train models may themselves be problematic. The program or model that checks results and supplies feedback—the verifier referred to here—may contain loopholes that a model can exploit to obtain higher scores. The more extensively a model is trained, the greater the impact of these defects may become.

04.

Robot training: rely on simulation or real-world data?

In the physical world, whether fast, reliable feedback is available also influences research methods.

Robot training is one example. Over the past decade, motion control, whole-body control and manipulation tasks have developed different research approaches, with very different ways of obtaining training feedback and evaluation results.

• One approach is sim-to-real, using physics simulation and model-based learning for rapid feedback.

• Another mainly uses human demonstration data collected in real environments: first train a model on these demonstrations, then evaluate the policy’s performance on real hardware, without relying on simulation or similar methods for feedback.

Article illustration

Take motion control. Today, you can take a Unitree G1, download code from GitHub, and see results after 15 minutes of training with one GPU and a very small MLP network. Training takes place entirely in simulation. A GPU-optimized physics simulation environment can simulate thousands or even millions of robot action executions in parallel. There is no need first to collect real humanoid-robot data or teach the robot to walk through teleoperation.

MLP (Multi-Layer Perceptron): a basic neural network made up of multiple layers of interconnected neurons. Here, it calculates the corresponding control actions from the robot’s state.

The resulting control policy—the model that determines how the robot acts—can be applied directly to control a real robot, enabling walking, backflips and even more agile movement and whole-body control. Such results would have been hard to imagine just a few years ago.

By contrast, robotic manipulation research over the past decade has mainly focused on collecting large-scale real-world data, almost all of which relies on human expertise and demonstrations. Whether using egocentric data, robot teleoperation, data gloves or devices such as UMI, researchers are expanding offline expert demonstration datasets through these approaches, so robots can learn the dexterous manipulation required in practical tasks.

After training their models, teams such as Physical Intelligence, Generalist, Dyna and Google DeepMind still need to run A/B evaluations on real hardware in the lab, often spending hundreds of hours before they can release a new model.

For example, when Google published the RT-1 and RT-2 papers, it put in dozens of contract workers, dozens of robots and hundreds of hours of human labor just to compare which of two checkpoints was better.

Article illustration

Because real-hardware feedback arrives slowly, research teams struggle to collect enough data with the new version after every model update. Ted Xiao recommends choosing data according to its training purpose. Pretraining, for example, can use demonstrations collected in the past, as long as they are close to ideal performance. When collecting correction data, however, teams should as far as possible let the current version perform the task first, recording the step at which intervention is needed and the actions after correction.

But many manipulation skills humans have already mastered have not yet been recorded as data that robots can learn from. The physical world and physical economy face a similar problem.

• In robotics, humans have acquired a wealth of physical skills through evolution. Researchers need to capture people using these skills so robots can learn from them. For now, the scale of this data is still nowhere near comparable to the internet. Collecting it and developing algorithms that can learn from limited data yet scale as more data becomes available have therefore remained central goals.

• The physical economy has also accumulated enormous amounts of such knowledge and data; otherwise, we could not have built today’s society. But this material has not yet been systematically recorded, stored and organized, so it cannot be taken up directly like content on Wikipedia or Reddit.

More importantly, in the physical economy, humans have not exhausted all the technologies that could possibly be invented. Engineering innovation still requires continual iteration. By contrast, for many dexterous manipulation skills we want robots to acquire, existing human abilities can serve as a benchmark—although there are also operations that need to exceed human abilities and some specialized industrial settings.

Thus, if someone builds a robot that reaches human-level performance across a variety of tasks, that is already a tremendous success. But an engineering system that can only help humanity reinvent technology from the mid-1950s would have very limited value.

Engineering innovation therefore requires both acquiring existing data and continuing to advance technology. We need to understand the current state of technology and how the best teams work. We also need to understand how a new idea becomes reality through design, experiments and engineering processes, and how those processes can be improved further.

05.

Engineering R&D: after faster computation, experiments still mean waiting

Looking further, engineering tests can also be arranged by feedback speed. Manual calculations, software analysis and automated checks can usually yield results relatively quickly. Tests involving real equipment, materials and environments, however, may take weeks, months, years or even decades.

Testing is also limited by resources and the number of attempts. For example, when F1 teams design a new car each year, rules restrict both the number of wind-tunnel tests and the compute used for CFD simulations. These testing and simulation allowances are also part of the teams’ R&D competition.

Article illustration

Beyond limited testing resources, the scope in which computational or simulation-based verification applies is another issue. The same design may pass the computational or simulation checks shown in blue below, yet reveal problems in the real-world tests shown in orange. Under what conditions these verification methods are reliable, and when they fail to reflect real performance, are questions engineering practice needs to clarify.

Article illustration

Ted Xiao says teams such as Anthropic, OpenAI and Gemini have begun training models for certain digital design and engineering tasks. Model performance has already improved markedly on these tasks, where results can be checked quickly and cheaply. The engineering design demos these teams have shown also impressed him deeply.

But when those designs are placed in engineering workflows for actual production, the results are not necessarily acceptable. Sometimes they cannot even pass engineers’ basic checks. At least for now, therefore, training solely on feedback from computational or simulation checks seems insufficient to make models directly meet practical engineering requirements.

This situation is not limited to CAD. The same is true in many fields: rapid progress is being made in the parts where computation can replace real-world tests, while the parts that still have to wait for real test results remain difficult to accelerate at the same pace.

Automated laboratories are trying to improve these real-world testing stages. Teams such as Periodic Labs, OpenAI and Anthropic are exploring how to use feedback from real experiments in areas including materials science and drug discovery. This includes wet-lab experiments involving the actual handling of samples and reagents. They are both improving experimental efficiency and trying to have models interpret experimental results or check experimental plans.

For example, Periodic Labs has announced a development: its automated laboratory can complete roughly 100 real experiments involving X-ray diffraction each day, and a new lab might increase that number to 1,000 a day. Even so, this scale is still far smaller than the millions of attempts possible in coding-model training and humanoid-robot simulation training.

Article illustration

Periodic Labs is therefore also trying to train an LLM judge to interpret X-ray diffraction patterns. Its agreement with PhD experts’ judgments reached 77%. The team hopes to improve the LLM judge using feedback from real experiments, then have the judge provide reward scores for training models.

However, the effectiveness of these methods differs across teams. Compared with coding, engineering computation and mathematics, progress in automated labs is more prone to stalls and reversals, with many stages still requiring a wait for real experiments.

Anthropic’s research on protein binders and CRISPR also demonstrates this limitation. Scientists worked with Anthropic’s internal model to repeatedly propose and check experimental designs on computers, then sent the selected experiments to a human-operated wet lab for execution. Collaboration between scientists and the model allowed more experimental design and preliminary checking to be done on computers. Work that originally took months or weeks now took weeks or days. But the manually executed wet-lab experiments still required weeks: faster design and analysis did not shorten the wait for the experiments themselves.

Article illustration

In general, experiments can run simultaneously, but if a later round is to be designed based on the preceding round’s results, it must wait for that round to finish. Even running more experiments in parallel at once only provides more information in that round. Researchers still need to see the results before using them to propose new hypotheses and arrange the next round of experiments. Parallel experiments can therefore increase exploration within each round, but cannot eliminate the wait between rounds.

06.

What are the limitations of replacing real-world tests with models?

To shorten verification time, many fields are trying to replace some real-world tests with computation. If measurements that were slow, cumbersome, vulnerable to interference and expensive can be performed reliably with models or simulations, more compute could accelerate verification. For real-world tests that remain necessary, we need to improve testing efficiency or find ways to avoid waiting for real test results at every design iteration.

Article illustration

Following Periodic Labs’ approach, we could design a value function: in other words, train a critic that evaluates proposals based on real-world feedback. If the model is sufficiently reliable, low-cost computation could replace some expensive real-world tests.

But there is a problem here: a verifier is trained on historical data, and it may be reliable only for the tasks and conditions covered by that data. What we often want to check, however, is a new design never seen before.

Models may indeed learn to handle situations not encountered during training, as in AlphaGo Zero. But Go outcomes can be determined by rules, without paying the high cost of real experiments for each verification. The physical world does not offer such conditions: how proteins fold and bind, and whether new materials and crystals can be discovered and synthesized, must ultimately stand the test of the laws of nature. Yet the real observations available for training verifiers, reward functions and value functions are very limited.

A model may be able to handle some new situations, but that ability has limits. The more a new design differs from the situations in historical data, the harder it becomes to obtain reliable, useful feedback.

Article illustration

Thus, a model may be much faster at reproducing an invention or design along an established path, but going beyond existing achievements remains very difficult. Learning a value function is worth trying, but it will not necessarily solve the problem of exploring the unknown directly.

07.

How should verification be improved next?

Drawing on his experience in robot learning and frontier-model research, as well as progress by his peers, Ted Xiao believes the most important next problem is to establish methods that are both trustworthy and able to scale verification, and to confirm that they remain reliable under actual operating conditions and in the next round of R&D tasks.

Article illustration

Article illustration

Returning to the initial assumption, first, the intelligence supplied by AGI and the labor capacity supplied by robots are both necessary conditions for material abundance. But even if both succeed in the future, the remaining problems in engineering and invention will not disappear automatically.

Second, areas where results can be verified reliably often make progress more readily. With trustworthy verification methods that can be used at scale, we can keep learning and developing technology from feedback, gradually approaching the capability limits in those tasks.

Finally, evaluation metrics must reflect the results we actually want. When scores improve, does the model’s performance in reality improve too? After AI models, robots and engineering workflows are combined, does the complete system do the work expected of it? All of this needs to be measured directly.

Article illustration

Three research directions

In terms of research, Ted Xiao believes three things are worth doing.

The first is to build more verifiers, including what people commonly call RL environments. Many teams now want to add training environments and then invest more compute in RL. But besides increasing their number and coverage, we also need to go deep into a field, understand how work is done there and what standards actual applications must meet. We cannot look only at academic benchmarks.

The second is to test whether verifiers are trustworthy. We need to enter the environments in which models actually work, understand the tasks and operating conditions they will encounter, and then check whether the verifiers’ judgments are reliable in those circumstances.

The third is to organize real-world testing well. Both verifiers and the complete system must ultimately be tested against reality. Such tests are expensive, take a long time and require substantial capital investment. The number that can be performed is limited, and some waiting times cannot be compressed. We can therefore supplement training with synthetic data for the parts that can be simulated reliably. When real testing is necessary, first clarify what needs to be verified, then prioritize experiments whose results will best guide the next round of design or training, making every result as helpful as possible in improving verifiers and technology.

Article illustration

What is needed to move forward in practice?

For the past year and a half, Ted Xiao has also been considering what specifically to do.

• For the expensive, limited real-world tests in the final stages, we can increase capital investment, secure more experimental resources and expand the scale of testing through broader collaboration.

• But offline data is scarce and scattered, making deployment especially important. Ted Xiao draws a distinction from the digital world: OpenAI could train models on internet data in the lab for years before releasing ChatGPT, whereas existing data in the physical world is scarce and scattered. We need to put systems to work in practice, continuously collect data, improve verifiers and models, and also learn through use exactly what problems need to be solved.

Taking metal cutting as an example, Ted Xiao suggests that software teams first go on-site to watch how operators carry out the process, listen to where they say problems tend to occur, and then articulate those problems in language both sides understand.

• This work also requires focus. There are many opportunities, and researchers and entrepreneurs can easily pursue several directions at once. But improving only one stage is not enough to substantially shorten the total time needed to develop a technology and put it into production. Local achievements may bring wealth, reputation and research influence without necessarily accelerating the entire process to the same degree.

• What ultimately matters is whether output in the physical economy has increased, whether the development and production launch of new technologies have accelerated, and whether industrial output has grown. High scores on a few benchmarks or attention on social media cannot substitute for these actual results.

This article comes from “Overseas Unicorn,” by Ted Xiao.

Related topics

After reading

Join the AINRK community

Read how to join the community and connect with people working in AI.

Community information (opens in a new tab)