SLA adds spatial supervision to continual pretraining, testing robots under scene changes

HKUST (Guangzhou) and OLA Dimensions train spatial representations on diverse robot data, reporting gains on public benchmarks, perturbation tests, and few-shot adaptation to new robot bodies.
If spatial understanding is incorporated into a foundation model’s continual learning process, can robots better handle changes in the real world?
As a robot reaches out to grasp fruit or vegetables, someone moves the target away. An object that was already placed is thrown back onto the table. In another set of experiments, the tabletop keeps rising and falling, and the gripper must follow the target’s changing height. The task is still grasping and placing, but once spatial relationships change, can previously learned skills still be used?


Figure 1. Video of SLA operating on real robots
The Hong Kong University of Science and Technology (Guangzhou) and OLA Dimensions jointly address this question with SLA, the Spatial-Language-Action Model.
First, the results: in the public RoboDojo-Real evaluation on physical robots, SLA-Preview tops the leaderboard with an average score of 42.3 and a full-task success rate of 33.9%. Compared with π0.5, its success rate is 21.1 percentage points higher, or approximately 2.6 times as high.

Figure 2. Screenshot of the RoboDojo-Real physical-robot leaderboard, where SLA-Preview ranks first by average score
Across eight types of real-robot tests set up by the research team, SLA achieves an average object-level success rate of 83.8%, 24.0 percentage points higher than π0.5. From the public leaderboard to tests of dynamic grasping and changing tabletop heights, SLA demonstrates an ability to respond to changes in real-world scenes.
SLA’s core idea is to develop the spatial representations that support actions at the same time as robots learn those actions. Spatial supervision extends beyond task-specific post-training into larger-scale continual pretraining and is retained during downstream adaptation.
01. After topping the real-robot leaderboard
look at how the robot responds to change
RoboDojo-Real covers three robot embodiments, ARX X5, Piper, and Piper X, and a total of 18 tasks. SLA jointly uses demonstrations from all three embodiments for post-training, then completes the evaluations with a single model.
In the same public evaluation, π0.5 has an average score of 22.9 and a success rate of 12.8%, compared with SLA’s 42.3 and 33.9%. The score reflects partial progress on tasks, while the success rate measures whether the complete task was finished.
Leaderboard results provide a common reference. To see the role of spatial capabilities directly, one can also look at experiments with moving targets and rising and falling tabletops.
In the fruit and vegetable grasping experiment, the researchers first post-train with 100 demonstrations from the base setting, then use the same policy under three conditions: the base setting, positional perturbations, and dynamic interference. During testing, objects may be thrown back onto the table, and a person may move a target while it is being grasped.
No additional demonstrations of the corresponding disturbances were collected for post-training under the positional-perturbation and dynamic-interference conditions.

Figure 3. Basic fruit and vegetable manipulation, positional perturbations, dynamic interference, and few-shot cross-embodiment adaptation.
Under positional perturbations, SLA’s object-level success rate is 82%, compared with 52% for π0.5. When targets keep moving, the rates are 76% and 42%, respectively. In watch-grasping conditions with a rising and falling tabletop and constantly changing target height, the rates are 78% and 44%.
The cross-embodiment experiment also has an explicit data condition: ten Franka demonstrations are added to the original 100 Piper demonstrations for joint post-training, yielding a 78% object-level success rate on Franka. This result tests adaptation with a small amount of data from the target embodiment.
Each of the eight deployment conditions is evaluated on 50 object instances, and success requires completing the entire manipulation process for that object. Together, they address a practical question: when observed spatial relationships change, can the policy adjust its actions accordingly?
02. Spatial understanding:
how it enters robot foundation models
Vision-language-action models (VLAs) connect images, instructions, and actions and can use knowledge acquired through vision-language model pretraining. But robot manipulation also requires handling the relationships among distance, orientation, object occlusion, and gripper movement.
Recognizing a cup and accurately moving a gripper next to it do not require exactly the same abilities. If the object’s position changes, the approach path may need to change; if the grasp angle changes, wrist orientation must adjust accordingly.
One common approach introduces geometric information, such as depth maps and point clouds, during post-training. Real-world data, however, does not always have consistent 3D annotations: depth may be missing, camera positions and calibration states differ, and some public demonstrations contain only RGB images.
If spatial learning can rely only on orderly, complete geometric inputs, it becomes difficult to make full use of the heterogeneous robot experience already accumulated.
SLA instead aligns spatial information with the VLA’s own intermediate visual representations during training. It inherits π0.5’s pretrained architecture and weights, learns spatial representations through continual pretraining, and continues using spatial supervision when adapting to specific tasks in post-training.

Figure 4. Overall SLA framework: heterogeneous robot data is unified, and spatial alignment is performed during both continual pretraining and post-training.
Specifically, the frozen 3D perception model OmniVGGT acts as the spatial teacher. It reads multiview RGB images and uses depth or camera-calibration information already available in each sample to extract spatial features.
While learning to predict actions, the VLA also aligns its intermediate visual features with the teacher’s spatial features. Spatial supervision thus enters the representations on which action generation depends, allowing the model to acquire geometric information as it learns manipulation.
For different scenes, textures, and camera viewpoints, SLA also adaptively aggregates visual features at multiple levels according to the input. In the RoboTwin ablation, adaptive multilevel fusion achieves a Clean2Random success rate of 72.3%, 5.0 percentage points above the best fixed single-layer approach.
Another design is state dropout during training: robot-state inputs are removed from some samples with a certain probability, encouraging the policy to make greater use of current images and depend less on fixed state patterns in demonstration trajectories.
At deployment, the spatial teacher, alignment projector, and layer-selection gates are all removed. Depth and camera calibration are not required as additional policy inputs, and the spatial alignment branch adds no policy inference computation. RoboDojo long-horizon tasks additionally use a historical-image memory mechanism, so the leaderboard results reflect the performance of the complete system including that mechanism.
03. Data from six robot sources:
first unify the physical meaning of actions
For spatial supervision to work at a larger scale, a basic question must also be answered: how can data from different robots be learned together?
SLA uses six sources: InternData-A1, AgiBotWorld-Beta, RoboCasa365-Pretrain, RoboCOIN, DROID, and MolmoAct, covering real and simulated settings and single-arm and dual-arm robots. Before filtering and processing, these sources total approximately 1.71 million trajectories and 14,330 hours of demonstrations.

Figure 5. Data sources for continual pretraining. The figure shows sizes before filtering and processing; task counts are calculated separately for each dataset and then added together.
Beyond dataset size, control representations must also be consistent. Different robot arms have different numbers and arrangements of joints, so the same set of joint values does not have the same physical meaning. Mixing them directly can make learning difficult.
The team describes states and actions using end-effector poses and unifies robot coordinate conventions. For single-arm robots, the origin is the arm base; for dual-arm robots, it is the midpoint between the two arm bases. The axes consistently point forward, left, and up. Movement directions, rotations, and gripper actions thus acquire comparable physical meanings.
Rotation representations are also processed. Directly subtracting Euler angles can create outliers at angle-wraparound boundaries. SLA instead computes relative rotations through SO(3) composition, represents them as rotation vectors, and applies per-channel normalization.
For highly repetitive simulated trajectories, the team downsamples by source and task and filters out unsuitable or anomalous data. The continual pretraining stage also excludes post-training datasets to reduce the risk of downstream evaluation leakage.
This stage uses 48 NVIDIA H100 GPUs and takes approximately 110 hours. Data processing and spatial supervision together provide the foundation for continual learning.
04. As data keeps increasing,
can spatial capabilities grow with it?
If spatial understanding is to become a foundation model capability, it must also stand up to data-scaling experiments.
The researchers sample by task from the full mixed dataset, using four continual pretraining data fractions: 5%, 25%, 50%, and 100%. They separately control whether spatial alignment is used in continual pretraining and post-training, then compare performance under the RoboTwin Clean2Random setting.
When spatial alignment is used in both stages, success rises with dataset size from 49.2% to 60.5%, then 67.4%, and finally 72.3%. At every data fraction, this combination outperforms the other three settings.

Figure 6. Continual pretraining data-scaling experiment. Spatial alignment in both stages performs best at every dataset size.
With the full dataset, using spatial alignment only in continual pretraining yields a 63.2% success rate; using it only in post-training yields 57.1%; and using it in neither stage yields 48.2%.
These comparisons support SLA’s technical proposition: spatial representations can first be learned from broader robot experience and then strengthened through downstream adaptation. Within the scope of this experiment, the training objective and data coverage jointly affect final performance.
Other public evaluations further test different capabilities. Across RoboTwin 2.0’s 50 tasks, SLA jointly post-trains with 50 clean demonstrations per task, achieving success rates of 86.0% on Clean2Clean and 72.3% on Clean2Random, for an average of 79.2%. Clean2Random improves by 26.3 percentage points over π0.5, showing gains under randomized scenes.
Across RoboCasa-Panda’s 24 kitchen tasks, with 50 downstream demonstrations per task, SLA achieves a 68.4% average success rate. On LIBERO-Plus, after post-training on standard LIBERO demonstrations, the model undergoes perturbations to cameras, language, lighting, backgrounds, and other conditions, achieving a weighted success rate of 87.4%. These results correspond to different evaluation settings and cannot simply be combined into one universal success rate.
05. From learning actions
to adapting to change
The SLA study by HKUST (Guangzhou) and OLA Dimensions extends spatial supervision into foundation model continual pretraining and tests its effects through leaderboards, ablations, and real-robot manipulation. How spatial capabilities form and how they scale with robot experience thus become questions that can be tested individually.
The existing results still have a limited scope: there are clear differences across tasks and embodiments, and long-horizon manipulation, open-ended tasks, and larger-scale real-world deployment require further validation. SLA demonstrates a path for continually developing spatial representations while inheriting existing VLA capabilities.
Paper and project
- Paper: SLA: Scaling Spatial-Language-Action Model via Continual Pre-training
- arXiv: https://arxiv.org/abs/2611.08373
- Project homepage and dynamic experiments: https://embodiedsla.github.io/
- Code repository: https://github.com/OpenHelix-Team/Spatial-Forcing
- RoboDojo real-robot leaderboard: https://robodojo-benchmark.com/leaderboard?bench=realworld
This article comes from “Synced,” and was written by “Synced.”
Related topics
After reading
Join the AINRK community
Read how to join the community and connect with people working in AI.




