RoboColiseum Updates Simulation Tracks; Fable 5.2 Leads Overall but Struggles With Dexterous Hands

The update covers dexterous manipulation, whole-body coordination, precision tasks and cross-embodiment transfer. Agents and policies are managed separately, with scores reflecting progress and execution efficiency.
Editorial correction (November 16, 2026): 96.86% is the correlation of scores on the tested tasks, not the rate of identical outcomes on individual runs or a guarantee of transfer to real robots for all new tasks and policies.

In the results announced for RoboColiseum's new season, Claude Fable 5.2 took first place overall with 18.30 points. It also led in precision manipulation and cross-embodiment transfer, scoring 33.04 and 17.99 respectively. Dexterous-hand manipulation, however, was the overall champion's lowest-scoring category.
First in precision manipulation, but why only 3.73 points on dexterous-hand tasks? Hanging clothes with a hand or holding an object steadily are everyday actions for humans. What makes them so difficult for large models?
RoboColiseum's new season brings the work robots actually need to do in the real world into the test arena. It adds dexterous-hand manipulation, whole-body coordination, precision manipulation and embodiment transfer as four tracks with 50 tasks. The evaluation considers both whether tasks can be completed from start to finish and whether execution is fast enough.

Four major new tracks
Focused on real long-horizon and complex tasks
Charging a phone, hanging clothes with a hanger, picking up rubbish and opening a door with a key—the new tracks contain jobs robots will need to do when they enter the real world. These seemingly everyday tasks require fingers, both arms and the body to work together.
RoboColiseum brings these practical needs into a simulated test arena, adding four tracks and 50 tasks to examine whether a model can finish a job from beginning to end.
Dexterous manipulation: from coordinating multiple fingers to using tools.
To hang a piece of clothing, a robot must first grasp and spread it open, put the hanger through it, then adjust its position and hang it on a clothes rail.
Clothing is soft and easily deformed. Multiple fingers need to cooperate in grasping, and both hands must coordinate their movements to perform this entire sequence well.
This track has ten tasks and also covers wok tossing, solving a Rubik's Cube, tying rubbish bags and carrying parcels with both hands.




Whole-body coordination: the base, torso and both arms have to work together.
Picking up rubbish requires first moving to a suitable position, then adjusting the body and reaching to grasp it. Wiping a blackboard requires coordinated base movement and torso adjustment so the arms can reach and clean different positions.
This track contains eight tasks, including picking up rubbish, placing trays, collecting goods from a conveyor belt, opening a door with a key and wiping a blackboard. It tests whether a robot can perform movement and manipulation in a continuous sequence.




Precision manipulation: you have found the connector, but can you plug it in?
To charge a phone, the two arms need to control the phone and connector separately, aligning their position and orientation at the same time. This track covers eight types of task, including charging phones, assembling building blocks, connecting Ethernet cables, installing bearings and packaging phones. It tests the alignment, contact and assembly of small parts. The closer the task is to completion, the more precise the movements need to be.




Embodiment transfer: change to another body and keep working.
The cross-embodiment track contains six types of task, including hanging cups, replacing keycaps, holding a pen to write and pouring water. They are tested separately on four robots—ARX, AgileX, G1 and G2—making 24 evaluation tasks in total.
When a robot's joint structure, range of motion and gripping tools change, the model must adjust its actions accordingly. The question here is whether the same model can adapt to different robots.




Scores for eight major models announced
Dexterous hands remain a shared weakness
The current leaderboard compiles results from eight agents and three policy models. Claude Fable 5.2 ranks first with an overall score of 18.30, with GPT-6 Astra and Claude Opus 5.5 close behind at 13.73 and 11.77 points, respectively.

Fable leads overall and also in precision manipulation and cross-embodiment transfer. Its scores in those two categories are 33.04 and 17.99 respectively. In whole-body coordination, Astra scored 18.96 and Opus 18.94, leaving the two almost tied.
On dexterous-hand tasks, however, all eight agents recorded their lowest scores. Overall leader Fable scored only 3.73, while Astra and GPT-6.1 Terra both scored 0.00; even the highest-scoring agent in this category, Gemini 4 Argon, managed only 5.19. Coordinating multiple fingers and steadily grasping and manipulating objects remain shared difficulties for these large models.
Policy models, meanwhile, achieved higher scores on dexterous-hand tasks. π₀.₅ led this track with 16.79 points, while LingBot-VLA-2.0 and GR00T N1.7 scored 10.66 and 5.53 respectively; all three outscored the eight agents on this leaderboard. π₀.₅ also ranked fourth overall with 11.22 points.
These results leave teams skilled in dexterous control with a clear direction for the challenge: policy models have already demonstrated advantages on operations that large models have yet to perform well, and have an opportunity to widen the gap further.
Three defenses against benchmark gaming
Agents and policies evaluated separately
With repeated practice on the same questions, models may also earn high scores by memorizing them. RoboColiseum's new season adds three defenses against benchmark gaming, reducing inflated scores from memorization and the use of privileged information so the results better reflect models' real abilities.
First, a hidden test set: practice scenarios are public, while official test questions are held back.
Developers can use the open-source scenarios for development and debugging. The official test scenarios on the server follow the same distribution but have different specific layouts, testing whether models can handle scenarios they have not practiced.
Second, dynamic spot checks: try a new set of tasks and reassess performance.
The platform keeps a fixed official test set for scoring and also has a dynamic test set that changes with every submission. The dynamic set does not count toward the total score; instead, it is compared with the official set to check whether a model has overfit the fixed test tasks. If the two sets of results differ too greatly, the relevant solution's eligibility for the leaderboard will be restricted.
Third, human review: check for inappropriate behavior that violates the evaluation rules.
In addition to automated checks, the platform adds human review to further identify behavior that violates evaluation rules and maliciously seeks to gain scores.
Agents and policy models are evaluated separately, with similar approaches compared on the same leaderboard.
Agents can repeatedly switch between observation, reasoning and action, while policy approaches such as VLA and WAM usually execute tasks through action sequences. Their interaction methods and execution rhythms differ. This update manages the two categories separately and gives agents an independent leaderboard, allowing participating teams to compare performance among approaches of the same kind.
Finally, execution efficiency also counts toward the score: how many executions did the model need to complete the task?
The new rules set a limit on agents' execution counts, with thresholds based on training data. As a model repeatedly observes, reasons and tries, execution beyond a reasonable count is penalized according to the extent of the excess.
Task completion is equally important. Scoring retains progress points for intermediate steps and gives an additional reward to approaches that finish within the prescribed step count. How far a task progresses, whether the entire job is completed, and how many steps are used all affect the final result.
96.86% consistency between simulation and real-robot results
Establishing "credibility" with measured data
An important question for simulation-based evaluation is how broadly it can predict performance on real robots. The platform uses a zero-shot real-to-sim comparison, comparing scores for selected robots and tasks. The two cannot be expected to be fully equivalent in every scenario.
Across four different robot embodiments and 22 complex tasks, the measurements found a 96.86% correlation between simulation and real-robot task scores.
The high score correlation supports simulation-based evaluation's reference value for this set of tasks. Task coverage, individual-task errors and validation on real robots still need to be checked, however; it does not guarantee that every policy effective in simulation can be transferred directly and safely.

Comparison of agreement between real-robot and simulation evaluation results on 22 complex tasks
Talk to AI
Bring your model into the competition
Prepare data, connect a model, submit an evaluation and view the results—the whole workflow can be completed by talking to AI.
RoboColiseum provides Skills that AI can call. After installation and the necessary configuration, developers can talk to AI to download training data, connect inference services and initiate evaluations, completing the workflow and obtaining evaluation scores more quickly.
After a task is created on the platform and inference is started, RoboColiseum automatically advances simulation runs, scenario scheduling and scoring, and offers queries for task progress, category scores and evaluation results. Every stage from submission to viewing scores is supported, leaving teams more time to analyze failures and improve models.
RoboColiseum's new season is open
Waiting for your model to take up the challenge
Eight major models have already entered RoboColiseum's embodied arena. This evaluation reveals substantial room for improvement in dexterous control, precision manipulation and cross-embodiment transfer. These unsolved challenges also leave opportunities for newcomers to surpass them.
The four new tracks and 50 tasks are open for challenges. We invite teams developing policy models and agent systems to bring their models into RoboColiseum and challenge the eight major models. Your model could be the next to surpass them.

The new season's challenges are open, and RoboColiseum's evaluation system will continue to expand.
Enter RoboColiseum's embodied arena, participate in evaluation and take up the challenge!
RoboColiseum: https://robocoliseum.ai/
Techreport: https://robocoliseum.ai/techreport
This article comes from the WeChat public account Xinzhiyuan and was written by Xinzhiyuan.
Related topics
After reading
Join the AINRK community
Read how to join the community and connect with people working in AI.




