AI news

XDOF reports GPT-6.2 Astra reaches 41.2% average success across Ember-0’s 16 tasks

By 量子位0 viewsSource: 量子位
Illustration accompanying XDOF’s Ember-0 evaluation of GPT-6.2 Astra on real dual-arm robots

XDOF reports GPT-6.2 Astra achieved a 41.2% equally weighted average success rate on 16 shared Ember-0 real dual-arm robot tasks, versus GPT-6 Astra’s 19.6%. First successes in knot tying and T-shirt unfolding apply to this benchmark’s record, not a universal ranking of embodied systems.

Editor's verification (November 22, 2026): The 41.2% and 19.6% figures are XDOF-reported, equally weighted averages of success rates on 16 shared tasks in Ember-0's specified real-robot environment, not a universal ranking across all embodied models or tasks. “No AI had previously succeeded” refers only to tested models recorded by this benchmark. A higher average does not imply significant improvement at every subtask or checkpoint. The methodology specifies 30 trials per task and excludes runs judged invalid by operators and failures of the evaluation stack, so actual valid denominators must be checked task by task. Multiple workstations and human annotation control and record errors; they do not eliminate randomness. XDOF's official evaluation methodology (opens in a new tab); Official leaderboard (opens in a new tab)

GPT-6.2 Astra has turned up the pressure on embodied AI again, unleashing a mighty Pegasus Meteor Fist.

In the latest hands-on tests by robotics infrastructure company XDOF, GPT-6.2 Astra controlled a dual-arm robot and its overall success rate across 16 manipulation tasks soared to 41.2%, from GPT-6 Astra's 19.6% two months ago—more than doubling.

Tying a knot and unfolding a T-shirt, two operations no AI had previously completed in the benchmark, were accomplished this time too.

Article illustration

After the results were announced, the official OpenAI Developers account reposted the news, and OpenAI co-founder and president Greg Brockman also proudly commented:

GPT-6.2 Astra is doing incredibly well at robotics!

Article illustration

To ensure rigorous evaluation, XDOF ran experiments in parallel on six YAM workstations, repeating each task thirty times and completing up to 480 real-world trials per day. Humans annotated substeps, grasping mistakes and collisions to filter out hardware discrepancies and random disturbances.

Even with such demanding standards, XDOF said in its latest post that model capabilities might soon saturate the Ember-0 benchmark.

Scores doubled in two months—how strong is it?

To understand the significance of this breakthrough, we first need to understand what makes the test tasks difficult.

GPT-6.2 Astra took XDOF's Ember-0 test.

This is a robotics evaluation benchmark for real physical environments, using dual-arm robots to perform 16 manipulation tasks, with each task following a protocol of 30 repeated trials.

The tests cover spatial reasoning, object grasping, dual-arm collaboration, tool use, fine manipulation and handling deformable objects.

Two months ago, GPT-6 Astra's overall task success rate on Ember-0 was 19.6%.

With GPT-6.2 Astra, that figure has surged to 41.2%, an increase of 21.6 percentage points and more than double the previous level. The 41.2% is the equally weighted average of success rates on 16 shared manipulation tasks, covering capabilities including object grasping, spatial reasoning, dual-arm collaboration and tool use.

So the progress did not come from a breakthrough on just one task: GPT-6.2 Astra's performance improved markedly across the test suite.

The most striking results were GPT-6.2 Astra easily completing “tying a knot in a rope” and “unfolding a T-shirt.”

Previously, Ember-0 had accumulated 143 valid T-shirt unfolding trials without any model successfully completing the task.

Beyond those two landmark breakthroughs, the model also improved markedly on several other tasks covered by Ember-0.

  • “Pull a towel to retrieve a distant plush toy” and “tilt a tray to roll a ball into a bowl,” requiring physical reasoning
  • “Find an occluded marker” and “follow hidden instructions on a red sticky note,” testing memory and perception
  • “Move a potato chip without crushing it,” “pick up bread with tongs” and “serve cereal onto a tray,” testing fine force control
  • ……

GPT-6.2 Astra showed significant improvements in both checkpoint completion and final success rates across the subtasks.

The clearest improvement was on picking up bread with tongs: GPT-6 Astra's success rate was only 10%, while GPT-6.2 Astra reached 90%.

Article illustration

Why are knot tying and T-shirt unfolding so difficult?

To understand the value of GPT-6.2 Astra's breakthrough, we first need to understand just how troublesome “tying a knot” and “unfolding a T-shirt” are for robots.

In robot learning, vision-language-action (VLA) models and specialized policy models, represented by Diffusion Policy, ACT and RT-2, typically rely on large amounts of human demonstration data to learn manipulation skills.

Humans demonstrate first; robots then learn how to grasp, move and place objects from those demonstrations.

This approach has already worked well for rigid objects and fixed procedures.

Objects such as cups and boxes have relatively stable geometries, and their grasping locations and movement trajectories are comparatively predictable. After training, models can often develop relatively stable manipulation policies.

But change the object or environment, or link several actions into a more complex task, and the original policy may no longer work. The model may need new demonstration data and additional training to adapt to the new operating conditions.

Deformable objects raise the difficulty another level.

When unfolding a T-shirt or tying a knot in a rope, the object's shape keeps changing with every touch, grasp and tug by the robotic arms, and its deformation is difficult to predict. The robot must understand a goal such as “unfold the garment,” continually assess its corners, folds and overall shape from wrist-mounted and overhead cameras, and then decide what to do next.

Article illustration

This places high demands on manipulation precision and coordination between the two arms.

Even a slight error in grasping position or action timing can cause a garment corner to slip or a rope to become entangled, potentially undoing all the work already completed.

In XDOF's evaluation framework, the core capability labels for “tying a knot” and “unfolding a T-shirt” cover deformation control, dual-arm coordination, dexterity and long-horizon planning.

With hardware that lacks tactile feedback, the large model also incurs inference delays of ten-plus to twenty-plus seconds between successive action chunks. While waiting for a new decision, the arms cannot make real-time fine adjustments, making each contact-control operation and recovery from failure extremely costly.

Consequently, knot tying and T-shirt unfolding test a whole set of combined capabilities: understanding three-dimensional space, coordinating both arms, precise contact control, dynamically tracking state from past records and planning long-horizon tasks.

Article illustration

Incidentally, speed has always been a challenge for System 2 models too. One criticism of these models has been their lengthy reasoning, which makes it difficult to meet the requirements of real-time robot operation.

GPT-6.2 Astra has also improved in speed this time.

During the model's release period, its average prediction latency fell to 10.1 seconds, from GPT-6 Astra's 15 seconds.

Compared with other model architectures, however, this reasoning approach remains relatively inefficient. For robots that require continuous perception and rapid responses, a wait of around ten seconds is still substantial.

Overall, embodied AI models often perform well on simple grasping and object placement, but their success rates fall sharply when they encounter complex contact, tool use and deformable objects.

Previously published Ember-0 results from XDOF showed similar issues.

For example, models can usually understand a task and plan seemingly reasonable actions, yet when those actions reach the robotic arms, problems readily arise during grasping, contact and sequences of operations.

GPT-6.2 Astra's breakthrough this time indicates precisely that general-purpose models have begun overcoming these difficulties.

Article illustration

△AI-generated image

From an industry perspective, general-purpose large models' impact on embodied AI is also continuing along this path.

From NVIDIA researchers testing GPT-6 Astra directly as a robot policy model, to Claude demonstrating complex manipulation in robotic simulations, to Google DeepMind extending Gemini's multimodal understanding to robot control, general-purpose models are pushing the boundaries of embodied AI's capabilities in successive rounds.

One More Thing

Jiafei Duan, a National University of Singapore Presidential Young Professor active in robotics and embodied AI, subsequently posted praise for GPT-6.2 Astra's impressive “incursion” into embodied AI.

In their comparative tests on real robots, GPT-6.2 Astra performed remarkably well, reaching first place in RoboColosseum's SO-101 Elo ranking within a day.

For the robot in action, see the clip below:

Article illustration

From the original model's helplessness with deformable manipulation to GPT-6.2 Astra doubling its success rate within two months and breaking through on several previously zero-success tasks, frontier multimodal large models' evolution in the physical world is continuing to accelerate.

And at a pace that is becoming rather hard to keep up with.

Exhausting. Truly exhausting.

I hereby declare every day in November a day to slump into a chair, dizzy from AI developments. Just kidding.

References:

[1]https://x.com/xdofai/status/2123822012066157011

[2]https://x.com/djiafei/status/2123925760281067567?s=46

This article is from the WeChat account “QbitAI,” by “QbitAI.”

Related topics

After reading

Join the AINRK community

Read how to join the community and connect with people working in AI.

Community information (opens in a new tab)