Zhejiang University's VLA-Δ learns physical state changes, tested in simulation and on real robots

Using structured slots, optical flow and difference-text supervision, VLA-Δ reports 98.4% average success on LIBERO and 68% across 200 real-robot tests covering five task types.
A robotic arm reaches toward an apple, closes its gripper, and then moves toward a container... The motion is smooth, but the apple is still sitting on the table.
This is one of the most common shortcomings of current robots: their actions follow the procedure exactly, yet often fail to achieve the task objective.
Whether the gripper has made contact with the object, whether the object has been lifted, and how the support relationships have changed all affect whether the task is completed.
From the OmniAI team at Zhejiang University's ACES Laboratory, the latest work VLA-Δ focuses on precisely this layer of information in action demonstrations: in moving from the current state to a future state, what changes should occur in the robot and the objects?

The team turns task progress, spatial displacement, and contact changes into explicit pretraining targets, so that the internal representations on which action generation relies learn both task semantics and visual motion.
Built on a 2B-parameter vision-language backbone, VLA-Δ achieves an average success rate of 98.4% on LIBERO and an average success rate of 68.0% across five types of tasks on a real dual-arm robot.

△ An introduction to VLA-Δ, connecting the research motivation, three types of state-change supervision, model architecture, and experimental results.
Grasping, carrying, placing: what happens between the actions?
Today's mainstream vision-language-action models (VLAs) usually predict the next segment of actions based on camera images, task instructions, and the robot's own state. Vision-language pretraining provides rich understanding capabilities, and robot demonstrations then connect those representations to actions.
However, directly learning "what action to take based on what is seen" may also make a policy overly dependent on correspondences between observations and actions. How physical states change as a result of operations requires a more direct learning signal.
Take "put the apple into the container" as an example. During the approach, the distance between the gripper and apple decreases; during grasping, they go from being separate to being in contact; during transport, the apple leaves the tabletop and moves with the gripper; after release, the apple gains new support. The state formed at each stage changes the operating conditions for the next stage.
Existing research has introduced intermediate information into action generation through task planning, chains of thought, future images, or future latent representations. VLA-Δ goes further by targeting the learning objectives at task-relevant physical state changes between the present and the future: which positions will change, which contacts need to be established or maintained, and how these changes advance the task.

△ VLA-Δ learns representations of physical state changes for action generation through structured state changes, dense visual motion, and difference text.
Giving changes both semantics and motion details
VLA-Δ introduces a set of learnable Physical State Difference Queries. These Queries extract state-change representations from the current image and task instruction; the action expert then combines them with the robot's own state to generate continuous action segments. These Queries can be understood as a set of internal information carriers read by the action expert. To make them contain useful information about changes, the research team introduces three complementary types of supervision.
Structured transition slots: making key states and their changes explicit.
Structured Transition Slots organize information such as task progress and stage, the positions of the target and gripper, motion and relative displacement, interactable locations, gripper and contact changes, and obstacle risks into nine categories of supervision targets.
These targets give the change representations specific meanings: what stage of the operation is currently underway? Where is the target? Is the gripper about to close or release? How will the contact relationships change?
The corresponding supervision also encourages different Query groups to develop a semantic division of labor and jointly support action generation.
- Dense optical flow: preserving fine-grained visual motion.
Approaching an object, adjusting the gripper, and aligning precisely all involve small local movements.
VLA-Δ uses optical flow between the current frame and a subsequent frame from the same viewpoint, supplementing these details with dense two-dimensional displacements. Optical flow by itself cannot distinguish which movements are relevant to the task, so structured slots and difference text are also needed to provide semantic constraints.
- Future-state difference text: connecting local changes into task progress.
Future-State Difference Text uses coherent natural language to describe task progress, spatial changes, contact changes, and operational constraints between the present and the future. When learning this kind of text, information from the current image and instruction must first pass through the Queries before being used for text prediction. This encourages the internal representations to retain semantic content that can explain changes in the task.
The three types of signals jointly train this set of representations. The action expert then combines them with the robot's own state and uses Flow Matching to generate continuous action segments.

△ Queries bring together vision-language information, learn state changes through three types of supervision, and provide conditioning representations for the action expert based on Flow Matching.
Making robot demonstrations provide more explicit information about changes
To support the learning objectives above, the research team builds a pretraining corpus from robot demonstrations in DROID, MolmoAct, and RH20T. The paper reports a corpus of approximately 484 hours, 24 million frames, with an automated annotation pipeline designed to accompany it.

△ Using apple transport as an example to show operation stages, task progress, gripper and contact changes, and the process for constructing annotations at multiple levels.
How much can knowledge of these changes improve manipulation?

Across the 40 tasks in LIBERO, VLA-Δ achieves the highest average success rate among the methods compared in the paper. Across the 50 dual-arm tasks in RoboTwin 2.0, VLA-Δ also achieves the highest average success rate among the compared methods in both the Clean and Random settings, exceeding ZR-0 by 1.36 and 1.76 percentage points, respectively.
Applying change representations to real robots
In the real world, the research team uses the AgileX PiPER six-degree-of-freedom dual-arm platform and designs five types of tasks: obstacle avoidance, long-horizon execution, fine manipulation, spatial reasoning, and generalization to background changes. Each task is tested 40 times, for a total of 200 tests.

△ The average success rate on the real robot is 68.0%, compared with 62.5% for π₀.₅ and 64.5% for ZR-0.
VLA-Δ's average success rate across the five task types is 68.0%, which is 5.5 percentage points higher than π₀.₅. Among these, the success rates for obstacle avoidance, fine manipulation, and spatial reasoning are 42.5%, 72.5%, and 87.5%, respectively, all the highest among the methods compared in the figure. These tests show that state-change supervision during pretraining can also support real-world manipulation, covering tasks that require consideration of contact, spatial relationships, and operational constraints.
Changes in action demonstrations deserve to be learned directly
Every grasp or placement changes an object's position, orientation, or contact relationships. Demonstration trajectories record the actions as well as the results these actions produce in the environment. VLA-Δ turns the task-relevant changes within them into explicit supervision: it uses difference text to connect task semantics, structured slots to describe key states, and optical flow to retain motion details, then has a unified representation support continuous control. Treating physical state changes as trainable, testable learning objectives provides a concrete path for vision-language knowledge to better serve robotic manipulation.
About the authors
The OmniAl team at Zhejiang University conducts research on large models and agents, multimodal spatial reasoning, embodied intelligence, emotional interaction, and other directions. The author, Lu Qi, is a first-year master's student at Zhejiang University; Zhang Wenqi and Zhang Xuhong are co-corresponding authors, and Zhang Xuhong leads the OmniAl team.
Paper: https://arxiv.org/abs/2610.21306
Project homepage: https://zju-omniai.github.io/VLA-Delta/
Code: https://github.com/ZJU-OmniAI/VLA-Delta
This article comes from the WeChat official account "QbitAI," by "QbitAI."
Related topics
After reading
Join the AINRK community
Read how to join the community and connect with people working in AI.




