Weitu AI presents a streaming-video brain combining sparse perception and long-term memory

Weitu AI selects temporal segments and spatial features to control compute costs, reporting OVBench and S-EMBER results and exploring continuous video understanding for glasses, offices and industry. Outcomes depend on devices, deployment and task conditions.
Editorial correction (November 16, 2026)Devices, deployment conditions and tasks affect results. User counts and performance are team-reported. Two benchmark results do not guarantee continuous, accurate environmental understanding on every camera-equipped device.
Weitu AI develops models for perception, memory and reasoning in continuous video. Finding objects, tracking whiteboard changes and maintenance reminders are envisioned applications; actual results require validation against cameras, compute resources, deployment and task conditions.
Hong Kong multimodal-AI company Weitu AI currently serves dozens of enterprises worldwide and approximately six million individual users. The team wants to bring multimodal technology further into everyday life, offices and industry, enabling devices to perceive, reason and decide continuously.
According to the Weitu AI team, the central problem that a streaming-video brain aims to solve is how, as video keeps arriving, it can efficiently understand changes in a scene, continually generate memories, and use real-time understanding together with constantly updated memories to perform complex reasoning and make decisions. For users, this means that, as such streaming-video brains are put into use, the various multimodal devices that all need to continually perceive their surroundings and interact with them will be the first to experience the physical world's ChatGPT moment.
Efficient spatiotemporal compression enables continuous multimodal perception
Video in the real world is generated continuously, whereas a model has very limited computing resources and memory capacity. Keeping all the footage for the long term and repeatedly including it in computations would increase the processing burden; excessively subsampling frames or retaining only a very short time window, however, could miss crucial changes in the sequence of events. For example, the action of "putting the keys into the side pocket of a backpack" lasts only a moment. It is not until the user looks for the keys ten minutes later that this detail becomes particularly important.
Weitu AI is building a video-native multimodal architecture. Such an architecture enables efficient compression of streaming-video signals and perceptual learning. The design is based on organizing computation around the changes that currently need to be understood. In developing video-native representations and memory mechanisms, the team built a learning architecture called Dual Sparse Memory (two-layer sparse memory): it first automatically selects relevant segments along the time dimension, then extracts key spatiotemporal features from within those segments, preventing large amounts of irrelevant visual information from taking up the computing resources used for learning.
First comes temporal block-level sparse learning, or compressed perception over time. For retrospective questions such as what was written on the whiteboard at the meeting's start, the model focuses on relevant historical segments, sharply narrowing subsequent computation. Second is spatial sparse learning: within those segments, it extracts important visual details, such as subjects and changes, rather than processing every part of every frame equally intensively.
This mechanism brings both "when something was seen" and "which details are useful" into the network for learning together. Its aim is to retain key features while controlling the costs of attention computation and caching, making continuous video understanding better suited to the various kinds of hardware devices that need to run over long periods. For example, for different types of hardware systems, computing efficiency directly affects the responsiveness users experience, deployment costs, and the feasibility of solutions that coordinate processing on devices and in the cloud.
Continually updated memory supports complex reasoning
Efficient compressed perception and computation address the problem of continuous understanding, while a constantly updated memory module can help the model tackle more complex reasoning. In addition to understanding the current footage, Weitu AI's model includes a Long-term Memory Module, which compresses earlier visual and spatiotemporal information into compact representations and continually updates them as new footage arrives.
This kind of memory carries historical information that the model can use, including changes in the positions of objects, the order in which actions occur, and important states within a scene. The model needs both to remember that "the keys were once on the tabletop" and to retain the information that they "were later put into the side pocket of a bag," which involves changes across time and space. For example, in an industrial setting, this capability could be used to understand the steps in an earlier demonstration of an operation.

Memory architecture of the streaming-video brain.
The Weitu AI team explains that it views memory as an intelligent representation that has undergone selection and compression. What the memory module needs to learn encompasses not only compressed perception of current information, but also information that can support future understanding and reasoning. Similarly, when humans think, they often recall past experiences to find memories that can serve as information for reasoning. This provides inspiration for the design of memory in video models.
If a user asks whether the keys and books were put together, the system must connect locations and storage actions observed at different times before deciding whether they share a location. Memory thus supports complex judgment. Compression cannot preserve every raw-video detail, so the model must learn and continually update limited capacity to make memory useful for later reasoning.
Beyond this, the Weitu AI team is also exploring new scenarios for streaming-video brains, such as predicting and assessing states in a video, and making accurate predictions about the state of an event or action that is currently taking place. The Weitu AI team says: "For example, when an AI system is watching the process of brewing coffee, it needs to accurately distinguish between 'water is being poured' and 'this step has already been completed' before it can answer questions about the order of steps, whether an action has been completed, and so on. Including changes of state together with temporal relationships in memory helps it understand how an event or action takes place and when it moves into the next stage. Such questions are also very interesting new research problems: how can we predict the temporal boundary of an action that has not yet been completed? This is effectively a prediction of a future state."
Advantages in streaming-video and memory evaluations
Recent evaluations show architectural advantages in continuous understanding. OVBench [1] covers past memory, current perception and future prediction across sixteen subtasks. Weitu AI scores 69.9 overall, 11.1 above the established general multimodal baseline Gemini 3.1 Pro's 58.8. On Meta's S-EMBER [2], closely aligned with continuous wearable perception, it scores 58.23 versus the benchmark-aligned Gemini 3.1 Pro's 54.41, a 3.82-point advantage. S-EMBER uses first-person everyday video and requires answers based only on content seen before the question time.
These results show that Weitu AI's model demonstrates advantages both in streaming-video understanding and on benchmarks that evaluate memory. The product value this brings could enable more AI devices to better understand constantly changing scenes and to reason and make decisions by combining the current footage with memory.


The two illustrated examples show spatiotemporal-memory Q&A: the model responds at any point in a continuous stream, combining current visual input with relevant historical images to answer accurately.
Bringing continuous perception and memory to more applications
The scope of applications for a streaming multimodal brain covers all kinds of devices and products that need to continuously understand their surroundings and respond. From AI glasses and wearable cameras to other visual terminals, this capability could be integrated into daily life, office work, education, retail, and industrial settings, providing users with immediate help that is closely related to what they are currently doing.
For origami or cooking, an assistant can use completed steps to suggest the next one. In shopping, it can compare current products with previously seen styles, specifications and labels. After an office demonstration, it can review operation order and changes in a proposal. During industrial maintenance, it can combine earlier operations with current equipment state to suggest further checks. These applications share a need: connect current observation with historical memory, understand task progress and provide a basis for subsequent judgment.
This capability can also be applied to video creation. As a video is continuously generated and edited, the model can combine the current footage, progress on the work, and earlier content to continually assess how well the work matches its creative goals, suggest improvements to shot transitions, narrative pacing, and visual style, and dynamically plan the subsequent steps for generation, editing, and revision. Integrating continuous perception, memory, and reasoning into the creative process could reduce the time and computing costs of repeated generation and revision, make the work better reflect the creator's intended expression, and at the same time maintain coherent content and a consistent style.
An entry point for foundation models to perceive the world continuously
Foundation-model progress expands AI's role in complex work. OpenAI's GPT-6 family advances task handling, computer operation and cost efficiency. Anthropic's Claude Fable 5.2 release demonstrated robot tasks including unscrewing bottle caps and drawing.
In Weitu AI's view, as general-purpose models become better at reasoning and task planning, the ability to perceive changes in the environment efficiently and in real time, and bring relevant historical information into current judgments, will become an important foundation for robot applications. For example, when a household robot searches for an object or tidies a room, it needs to continually check whether objects have moved, whether an operation has been completed, and whether the conditions of the task have changed, while adjusting its actions in light of earlier observations. Continually updated perception and memory can provide the context needed for these judgments.
Weitu AI hopes to develop its streaming multimodal brain into an entry point that gives all kinds of general-purpose foundation models continuous perception of the world: the video brain would continually process footage, track the state of the scene, and update memory, providing timely, compact, task-relevant visual and multimodal information for complex reasoning and task planning. This is equivalent to giving general-purpose foundation models a pair of "eyes" that can keep observing and retain relevant memories, potentially reducing the burden of repeatedly processing lengthy stores of visual information.
This positioning also gives the streaming multimodal brain two types of application value: it can directly support specific scenarios, and it can also work alongside other general-purpose models as a perception and memory module. In the future, when this capability is further combined with robots' motion-control and execution systems, it could help robots continuously understand how a task is progressing in homes, offices, and service environments, and adjust their subsequent actions according to changes in their surroundings.
Starting with video, integrating more modalities
The team says the streaming-video brain is its starting point for multimodal interactive intelligence. It plans to incorporate more modalities into the architecture so perception, memory and reasoning adapt to different devices and settings.
For example, in an industrial setting, video can be combined with signals from sensors measuring temperature, pressure, vibration, and other variables to jointly reflect equipment operation and changes in the environment. The different signals provide complementary information: footage shows the process of an operation and changes in appearance, while sensors reflect temperature fluctuations, pressure changes, or abnormal vibration. The model needs to connect this information across time and space, understand the relationships between the signals, and form memories that can be used for subsequent judgments.
Continuously perceiving change, compressing important information into spatiotemporal memory, then reasoning and deciding with current perception and history is Weitu AI's central approach. From today's cameras, AI glasses and wearables to future robots and industrial multimodal systems, it hopes scalable architectures will let more devices understand surroundings continuously and give foundation models a fast, accurate window onto the world.
Weitu AI welcomes partners across industries to explore more global applications. Business users seeking access or trials should send requirements from a company email address to business@weitu.ai, and the team will respond as soon as possible.
For more Weitu AI updates, follow the WeChat account below.
[1] Huang Z, Li X, Li J, et al. Online video understanding: Ovbench and videochat-online [C]//2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025: 3328-3338.
[2] Wang X, Zhao X, Rodriguez P, et al. S-EMBER: A Large-Scale Benchmark for Streaming Egocentric Memory Retrieval [J]. arXiv preprint arXiv:2607.02689, 2026.
This article was originally published by Synced. Author: Synced.
Related topics
After reading
Join the AINRK community
Read how to join the community and connect with people working in AI.




