AgiBot LIFE links continuous perception and long-term memory to review a robot's day

The framework combines RoboBeats, Plastic Memory and tool exploration. The team reports perception latency on specified hardware, video question answering and limited real-robot memory experiments.

The opening clip is taken from the full demonstration at the end of the article.
Most existing embodied agents have a structural limitation: they are "task-driven." When an instruction arrives, they begin reasoning and calling tools; when the task ends, the system falls silent. This discrete interaction pattern leaves agents without continuous perception of their environment or long-horizon memory across periods of time.
As the real world—such as offices and homes—keeps changing dynamically, a model without continuous memory has nowhere to begin when asked to look back at a past state, such as "Who came by an hour ago?" or "Where was the object moved?"
To address this problem, AgiBot's Yuanzheng Research Team proposed the LIFE (Learning to Live) architecture. The framework aims to give robots continuous perception, long-term memory storage and adaptive exploration, moving embodied intelligence from "immediate response" toward "lifelong learning."

Figure | Figure 1 of the paper: six sets of RoboBeats benchmark results, from continuous perception to memory-based question answering. © Translation and compilation: Shenlan AI.
1
—
From a newcomer to knowing the place
When you first arrive at an office, you look around: who sits where, where things are kept and when it is appropriate to say hello. Some things need no explicit questions; spend a few days there, and you naturally find out.
We want robots to be like that too. When someone says goodbye at the door, a robot could wave back. If it has seen an object moved, it should not look in the old place next time. Over time, seeing the same face again should bring back what happened before, without starting from introductions every time.
At LIFE's core is an agent driven by a vision-language model (VLM). It combines what it sees now, what it remembers from the past and what the user needs, decides what to do next and calls tools to act.
But to become familiar with a place, simply waiting for tasks is not enough. The team connects it to RoboBeats for continuous perception of environmental changes. Experiences go into Plastic Memory for later review. When it encounters a problem it cannot solve, EDM helps explore new approaches and retain those that work.

Figure | Overall LIFE architecture: continuous perception and proactive exploration provide observations; memory stores experiences; and the agent reasons and acts using the current situation and historical evidence, continually accumulating reusable capabilities. © Translation and compilation: Shenlan AI.
2
—
RoboBeats: giving robots a "heartbeat"
When someone takes a phone call during a conversation, we pause. If someone nearby drops documents all over the floor, we may ask whether they need help. Such immediate reactions often need no prompting.
VLM agents, however, often proceed through "reason, call a tool, wait for the result." Before the tool has returned, the world has changed. While doing the task at hand, a robot also needs to know what is happening around it. RoboBeats bridges discrete tool calls and a continuously changing reality.
This "heart" is a lightweight decision model, following an approach similar to System One models such as Jev: rather than generating text, it directly judges the probability of each condition being true.
It reuses computational caches from the video stream to evaluate multiple conditions together. On a single RTX 5090, RoboBeats-2B takes about 75 milliseconds to process a new frame; simultaneously monitoring 16 conditions takes only about 77 milliseconds. Paying attention to more things barely changes latency.

Figure | Figure 4 of the paper: RoboBeats overview. Multiple conditions share a video stream, with a separate probability for each condition. © Translation and compilation: Shenlan AI.
When there is a task, the agent tells it what to watch for in natural language. RoboBeats continuously tracks when conditions begin and end, letting the robot adjust while acting, rather than waiting until a tool finishes to discover that circumstances have long since changed.

Figure | Real footage of RoboBeats proactive perception: the robot acts as soon as a condition is met. The large image is a third-person view, and the inset is the robot's first-person view; a green frame lighting up indicates that the condition is true. © Translation and compilation: Shenlan AI.
On StreamingBench's proactive-output task (PO), RoboBeats-4B's accuracy for its first trigger falling within two seconds before or after the reference time rose from the base model's 21.2% to 64.0%; on SPOT-Bench, action-boundary detection T-F1@5 rose from 26.4 to 45.8.
It watches even without a task: someone passes by, an object changes position or a new event appears in view. RoboBeats selects these observations for subsequent modules to understand and record, then the agent decides whether to respond.
With limited memory, it must remember what matters. The team built SSS-Bench: 171 videos and 618 questions, testing whether the system can retain useful observations without knowing the future questions.
Writing the same number of frames, using the same writing and question-answering models, and answering from text memory alone (Table 15 of the paper), RoboBeats-2B raised accuracy from the Qwen3.8-27B selection baseline's 40.9% to 60.5%, a gain of 19.6 percentage points.

Figure | Figure 10 of the paper: question-answering accuracy across SSS-Bench question types, including event timing, event order, repetition counts and where objects went. © Translation and compilation: Shenlan AI.
On OVO-Bench's retrospective question-answering subset, using text memory alone, the RoboBeats gating scheme improved the two question-answering models' average scores across episodic-memory and action-sequence-recognition tasks, compared with memory constructed by OneStreamer PHCM, by 6.2 and 6.1 percentage points, respectively.
Writing can also be more economical. In another OVO-Bench comparison, both used Qwen3.8-27B as the writing model. Compared with writing all sampled candidate frames, RoboBeats reduced writing-side input tokens by about 45%.
3
—
What was not understood then can be revisited later
When you drape a coat over the back of a chair, you probably do not specifically tell the robot to remember it. Only hours later, when you cannot find it, do you ask, "Where is my coat?"
The difficulty is that, at recording time, no one knows which detail will prove useful later. Plastic Memory stores the scene as it appeared then separately from its interpretation at the time: original images preserve the scene, while JSON records clues such as time, location, people and objects.
It uses a flat design, with two ways to use the same memory. An execution agent retrieves through tools to quickly find existing clues. A coding agent that can read files and write programs can revisit original images, extract new information and add fields and indexes.
For example, when passing a door, it may initially have recorded only the location. Later, when looking for a room, it can revisit that photo and read the door sign. Details not written into the text at the time remain in the image.

Figure | Plastic Memory: preserving the scene, time and location lets robots revisit experiences and extract new clues when encountering new questions. © Translation and compilation: Shenlan AI.
In LIFE's 24-hour real-robot memory experiment, the AgiBot A3 Ultra continuously captured first-person video for an entire day, then answered questions using the memory constructed from it.
Memory makes answers traceable: at 12:30 p.m., the user asked where the coat had last been. The robot found an image from 12:18 p.m. in that day's memory: the coat was draped over a chair in the pantry. It also returned the location from which the image had been captured. Determining who owned two cups in the meeting room required linking observations from different times. It traced records of people arriving and using cups, finding evidence for who owned the green cup. The black cup had appeared before the attendees arrived, so it was not hastily assigned to any of them.

Figure | A day of robot video is built into memory, supporting later questions about people, objects and activities. © Translation and compilation: Shenlan AI.
The team selected six questions about that day's experiences. Using the same GPT-6.1-sol question-answering model, LIFE scored 29.5/30 with an average time of 80.6 seconds; Codex, also given the constructed memory, scored 28.5/30 with an average time of 554.7 seconds.
Knowing where the coat last appeared provides a starting point for the next search. Once in the pantry, the robot still needs to check whether it remains on the chair. Memory leads the robot to the place, and new changes before its eyes are recorded in turn.
4
—
Can memory evolve on its own too?
"How many phone booths like this have I seen before?"
Given a photo, the robot must look through past images and recognize which show the same phone booth. If existing tools cannot do that, it first needs to find a way.
LIFE reuses the Explore–Distill–Mount (EDM) mechanism introduced in the earlier work ReSight. A coding agent examines historical evidence, writes programs and explores methods for retrieval and deduplication. Once it finds an effective approach, it distills and verifies it, then installs it in the system's toolbox.
The approach discovered this time need not be reinvented from scratch next time. The images in memory have not changed, but the robot has gained another way to find and use them.

Figure | Figure 12 of the paper: Explore–Distill–Mount. The starting question is "How many phone booths like this have I seen?" A coding agent explores photo-based retrieval, merges repeated observations of the same booth and ultimately mounts the result as a reusable tool. © Translation and compilation: Shenlan AI.
In the FindingDory experiment, after mounting EDM-built capabilities, the system's success rate on 597 queries not used to build those capabilities rose from 37.5% to 65.8%. These capabilities stayed frozen during testing, using previously explored approaches to solve new problems.
From remembering an experience to learning how to use it, memory has room to keep growing.
5
—
When it meets you again
"Please get my cup for me."
Between people who know each other, that sentence is often enough. You may have told the robot which cup is yours, or it may have seen you pick up that cup to drink. Remembering where it last saw it then tells the robot which cup to find and where to look. Past interactions have filled in details you left unsaid.

Sometimes the robot even speaks first. It still remembers the person who came for snacks a couple of days earlier. Meeting them again, it first asks whether they need help, then, after receiving consent, fetches the snacks and delivers them into the person's hands.

People develop an unspoken understanding after spending time together.
LIFE's exploration of lifelong embodied intelligence hopes to let robots gradually accumulate this familiarity too, so that the next time one answers, it may finally bring with it a past shared with you.
The heartbeat lets it keep pace with the world; memory lets the world leave its traces within it.
From then on, every tomorrow carries the self of yesterday.
Learning to Live.
Paper: Learning to Live: A Step Toward Lifelong Embodied Intelligence (https://arxiv.org/abs/2611.00180)
Project website: LIFE project website (https://agibotgeneral.github.io/life-site/)
Project and code: GitHub · AgibotTech/LIFE (https://github.com/AgibotTech/LIFE)
Model downloads: RoboBeats-0.8B (https://huggingface.co/superbigsaw/RoboBeats-0.8B) | 2B (https://huggingface.co/superbigsaw/RoboBeats-2B)| 4B (https://huggingface.co/superbigsaw/RoboBeats-4B)
This article comes from "Shenlan AI." Author: "Shenlan Academy."
Related topics
After reading
Join the AINRK community
Read how to join the community and connect with people working in AI.




