AI guides

Lightwheel launches RoboFinals RL Envs for long-horizon robotic simulation tasks

By 量子位3 viewsSource: 量子位
RoboFinals robot reinforcement-learning simulation environments

Lightwheel introduces robot reinforcement-learning environments during CoRL, demonstrating an over-eight-minute simulated car-repair task with dexterous hands, tools and deformable objects. The product links task construction, calibration, training and independent evaluation.

Editorial correction (November 17, 2026): The value of simulation and training/evaluation from a shared source is limited to the product's demonstrated approach. Absolute claims that it is the sole technical route have been removed. Company demonstrations are not independent production acceptance tests.

Reinforcement-learning environments, or RL Envs, are becoming a new business.

As large models move into tool use, software operation and multi-step tasks, training is changing. Models no longer learn only from existing data and answers. They also need to enter tasks, act, observe outcomes and continually adjust strategies through feedback.

For model teams, algorithms determine how models learn, while environments determine what they can practice, how many attempts they can make and how to judge whether an attempt works. Finding enough realistic tasks with assessable outcomes is becoming an important part of post-training.

The market has already shown how strong demand is. Mercor, Surge AI, Handshake and other new companies and businesses around model training and post-training have rapidly gained Silicon Valley's attention over the past year.

On November 10, Mercor announced a $750 million Series D at a $20 billion valuation. It operates a network of over five million experts, supplying evaluation and post-training data to frontier labs and enterprises. Earlier this year, it acquired Deeptune, whose team spent two years rebuilding hundreds of enterprise applications as training environments.

Surge AI demonstrates the connection between task environments and model training. It builds RL environments for long-horizon office work involving documents, spreadsheets, web research, planning and tool use. After post-training Qwen3.5-122B-A10B on them, the model improved 5.7 percentage points on the previously unseen SWE-Bench Pro.

Article illustration

Handshake AI focuses on experts and scenarios, supplying frontier labs with expert data, RL environments and graders across finance, life sciences, alignment and other areas. According to Time, citing its CEO, annualized revenue reached $1 billion about a year after launch.

These companies do not all operate in exactly the same way. Some focus on professional tasks, some provide training environments, and some combine expert experience, environment construction and evaluation.

But they point to one change: the core supply needed for model post-training has expanded beyond algorithms and compute to training data and environments: what tasks to do, where to practice and how to judge outcomes.

As models undertake long-horizon tasks, environments play an increasingly concrete role: determining what work can be practiced, what actions cause and whether training can keep iterating.

In digital environments, models operate software, modify files and run code. Their changes can usually be recorded and outcomes are comparatively easy to assess.

Moving from screens into the physical world makes environments more complex as digital tasks extend into physical interaction.

GPT-6 Astra, robotics' GPT-3 moment: large models enter physical AI

Advancing large-model capabilities are changing what robot development can envision.

On September 3, OpenAI released GPT-6 Astra, positioned as a frontier model for computer operation and programming.

Third-party evaluations appeared within a month. On RoboDojo's forty-two simulated tasks, Astra averaged 22.48% success without robot-specific fine-tuning, exceeding all forty trained robot policies on the leaderboard.

Article illustration

Investor Freda Duan called Astra robotics' GPT-3 moment on X. Former OpenAI researcher Will Depue was more direct: specialized models had received the bitter lesson.

OpenAI itself has not published robot benchmark scores, however, and real-robot tests showed clear strengths and weaknesses: nineteen successes in twenty attempts at placing a block in a bowl, but only two in twenty on precise peg insertion.

The underlying shift is real: progress in reasoning, tool use and multi-step tasks has turned foundation models' direct entry into the physical world from a vision into reality.

Moving from digital to physical worlds also exposes reinforcement learning to different feedback.

In digital environments, software agents' actions change files, webpages or program states, and the results are usually clear. Robots' actions, however, affect the position, orientation and contact states of objects; the location, force and outcome of a single contact also affect subsequent operations. Therefore, beyond understanding instructions and planning workflows, robots must continually adjust their actions during interaction.

This also means physical-AI RL environments cannot simply copy digital ones. They must support concrete work tasks, repeated action from different initial states and credible responses from key objects to policy actions.

Training options consequently narrow. Real-robot trial and error is constrained by equipment, resetting and site conditions, making scaling difficult. Simulation can provide repeatable conditions, provided the environment supports specific tasks and continually responds to policy actions.

Physically credible RL simulation is an important path for robot training. It can combine with real-robot data, demonstrations and other learning methods, but cannot be declared the sole necessary route for every foundation model entering the physical world.

How do RL environments teach robots complex, multi-stage, long-horizon work?

During CoRL2026, Lightwheel released RoboFinals RL Environments, or RoboFinals RL Envs, providing interactive reinforcement-learning simulations for robot models.

Unlike conventional training scenes built around one action, RoboFinals RL Envs focuses on real work tasks. Its launch demonstration showed a simulated car-repair workflow lasting more than eight minutes.

This task brings dexterous-hand manipulation, tool use, multi-stage workflows and interaction with deformable objects into the same work scenario. It involves different types of objects, including hoses, cables and gaskets.

The complexity it presents lies not only in the precision of an individual action itself, but even more in how multiple physical interactions become coupled and interrelated within the same task chain.

Article illustration

First, dexterous manipulation: many more degrees of freedom and contact points.

Most manipulation tasks use parallel grippers with only one opening-and-closing degree of freedom. Dexterous hands typically have a dozen to more than twenty degrees of freedom, with independently moving fingers.

For the policy, the dimensionality of the action space rises substantially. For simulation, multiple points of contact exist simultaneously between the fingers and objects, and the forces at each point need to be solved. If contact calculations are inaccurate, objects will slip out or interpenetrate, and the grasping method learned by the policy will not work in the real world.

Second, long-horizon tasks: errors accumulate over time.

Earlier robot RL environments commonly used relatively independent actions such as grasping and placing.

In pick-and-place, for example, a robot grasps an object and puts it at a designated location. Actual work combines multiple action sequences, so entering real work necessarily involves training on long-horizon tasks.

Longer tasks are harder to train. Early actions may have effects only much later, while small deviations accumulate at every step.

This is an important distinction between long-horizon work and simply joining independent tasks.

A common approach splits a long task into segments, each starting from a manually defined state. This lowers difficulty but removes a key condition of real work: the robot does not genuinely inherit changes caused by its preceding operation.

In the tasks in RoboFinals RL Envs, earlier and later stages are interrelated. Changes in object position, orientation and contact state caused by the previous step carry over into subsequent steps, and the policy must learn to continue and complete the task from imperfect intermediate states.

Third, deformable-object interaction: states are hard to describe and parameters hard to determine.

Hoses, cables and gaskets are deformable bodies that must be discretized into many nodes for simulation, imposing much greater computational and numerical-stability demands than rigid bodies.

Their behavior also depends on stiffness, damping, friction and other parameters. Small errors can make simulated bodies too soft, too stiff or subject to unrealistic jitter. Their shape response after interaction directly affects subsequent task stages.

The fingers of a dexterous hand need to make contact with objects; using tools involves precise control of position and orientation; hoses bend and sway when pulled; and gaskets deform under compression.

These objects have different physical properties and interaction modes. Showing one object's motion alone is not difficult. The challenge is placing them together in one task environment so a robot can interact with different objects through continuous operations.

Lightwheel's focus is not merely a car-repair simulation, but a robot RL environment combining fine manipulation, complex-object interaction and long-horizon tasks. Achieving all three requires building from the foundations.

Its in-house full-stack simulation foundation, SimFoundry, covers three stages: solving, measurement and generation.

The solver calculates interactions among rigid, articulated and deformable bodies correctly.

Measurement grounds objects' physical parameters in real data.

Generation is driven by AI agents, reconstructing assets, building physical models and defining tasks from real work. Agent-driven generation lets real scenes and tasks enter simulation in batches.

Article illustration

Lightwheel calls this standard RL Ready: task semantics provide clear completion conditions for every stage; physical interactions give reliable feedback during trial and error; and feedback signals can directly train models. Once connected, a model can begin experimentation and iteration.

Physical credibility: how accumulated simulation capabilities support long tasks

Physical-AI reinforcement-learning environments impose strict requirements on their builders:

Assets must be visually realistic and physically realistic. High-quality environments require quality assets, precise physics solvers and expert experience. Missing any one can stall long-horizon tasks.

The problem is that bringing all three capabilities into one team is difficult.

Asset builders do not work on solvers; simulation vendors lack real scenes; model teams know what counts as correct but have little incentive to build environments.

Anyone can assemble an isolated demo, only to find it cannot keep running. This has been the bottleneck delaying long-horizon RL environments.

Lightwheel's distinction is holding all three capabilities itself.

As the only Chinese company on the Newton Simulation Technical Committee, Lightwheel participates in defining SimReady standards and is a core contributor to the open-source SimReady Central Library.

Earlier demonstrations of plugging and unplugging cables and grasping plastic bags showed its ability to handle deformation, contact and state changes in flexible objects, which are much harder than rigid bodies.

The car-repair task involves different objects but the same class of problems. Assets, scenes and in-house solvers have been refined gradually over years.

Article illustration

△Lightwheel has achieved physically realistic joint interaction among granular materials, deformable bodies and rigid bodies.

Having the capabilities is not enough; they must fit together to produce trainable environments.

That is the role of its agentic calibration workflow. Agents propose directions, compare results and organize subsequent corrections, while Lightwheel's simulation experts decide at critical points, continuously refining the result.

Article illustration

One example in this release illustrates the workflow.

In the car-repair task, bending and swinging of an inflation hose under pulling affects robot operation. The initial simulation had abnormal jitter.

Through repeated automated calibration, the agent identified Young's modulus as the parameter most affecting hose response. Once candidate parameters and trajectories were produced, human experts compared simulated bending and swinging with the real hose to assist final calibration.

In the example, maximum positional deviation decreased by approximately three millimeters, while pronounced jitter diminished and hose response became closer to reality.

Article illustration

△From V1 to V3, errors measured against real data continually decreased.

Clear parameter-adjustment targets and comparable results provide a reproducible foundation for automated calibration.

Whether environment construction can scale determines how far the library can expand.

Lightwheel plans RL environments covering more robot jobs, difficulty levels and task horizons, with the library distribution following real tasks. Settings span homes, industry, retail and healthcare. Operations range from pick-and-place through alignment, tools and force control to precise closed-loop manipulation. Assets include rigid, articulated and deformable bodies and fluids.

Once the environment exists, model teams ask: how do we know it learned?

Training environments need aligned evaluation. RoboFinals RL Envs and RoboFinals Benchmark share task semantics, scene assets and capability definitions, so both refer to the same work requirements. Training tasks, reward-tuning processes and evaluation instances are strictly separated to preserve an independent test of learning.

Aligning training environments with the evaluation system helps maintain consistent task definitions. Alignment across teams or products can also be achieved through shared specifications. Whichever approach is used, training data, reward tuning and independent evaluation instances need to be kept separate to avoid evaluation leakage.

The final layer is delivery.

Open-source tools lower the barrier to starting simulations, but asset adaptation, physical calibration and operational checks remain necessary.

Lightwheel packages them into a deliverable product: model teams connect a policy and start training. Environment capabilities become purchasable, reusable standard components rather than something assembled inside one team.

Assets, open-source solvers and evaluation providers already exist across the industry; none is exclusive individually. Lightwheel has taken an early lead in connecting environment construction, calibration, evaluation and delivery into one complete process.

RL environments' value ultimately depends on whether policies can act continuously and adjust behavior from each interaction's results.

The release also shows the same task across policy-iteration stages:

Stage one, 5 steps: Hand movements remain jittery, and grasping is not completed.

Stage two, 150 steps: The policy begins attempting to grasp, but the screw slips during manipulation.

Stage three, 175 steps: The policy successfully grasps the screw and completes this task step.

This shows RoboFinals RL Env can support trial and error and behavioral change during policy training, making iterations observable and recordable within a repeatable simulated task.

From data and evaluation to RL environments: another link in continual learning

Large-model development has shown that improvements depend not only on algorithms, but also on training tasks, interaction conditions and feedback mechanisms.

For robots, the issue is more complicated.

As models progress from single actions to multi-stage workflows, environments must handle not just tasks but precise manipulation, physical interaction, continuous state change and long-horizon feedback. What robots can practice and how they receive feedback are becoming variables affecting training outcomes.

Lightwheel had already built capabilities across the full data, evaluation and deployment chain for physical AI. Its newly launched RoboFinals RL Envs further extends its product capabilities into interactive training environments.

Using SimFoundry, Lightwheel organizes complex tasks, object assets and interaction rules into runnable simulation environments, allowing models to control robots directly within those environments to execute tasks, observe changes in state, and explore different policies through repeated attempts.

For complex operations where post-training data is difficult to obtain in bulk, this offers another way to acquire learning experience.

After training, models need evaluation and diagnosis to identify the next improvement direction. RoboFinals RL Envs supports policy training, while RoboFinals Benchmark evaluates capabilities. Models interact and optimize in environments, then undergo independent evaluation to assess task performance and identify boundaries.

From data supply and simulation to large-scale evaluation and RL environments, Lightwheel's product development reflects changing physical-AI infrastructure needs: expanding from data and evaluation resources for model R&D to continuous interaction between models and physical environments.

As robots begin taking on more complex and more continuous work tasks, how to construct learning environments capable of supporting those tasks is also becoming an important question that physical AI infrastructure needs to answer.

This article was originally published by QbitAI. Author: QbitAI.

Related topics

After reading

Join the AINRK community

Read how to join the community and connect with people working in AI.

Community information (opens in a new tab)