AI guides

Always-on personal intelligence survey connects sensing, memory and action in a continuing loop

By LRST1 viewsSource: 新智元
Always-on personal intelligence and the HALO diagnostic framework

Researchers from nine universities propose operational continuity, durable state, L0–L5 levels and HALO diagnostics. Evidence for feedback loops and memory-guided sensing remains weak; findings depend on public documentation and the systems examined.

Editorial correction (November 16, 2026)L0–L5 is the authors' analytical framework, with findings limited by public materials and the sample. The MCP plugin depends on system-provided records and does not independently audit all internal state of closed-source systems.

Personal agents are arriving in an unprecedented wave.

Overseas, frontier labs have launched Muse, dots, Claude Duet, Grok Bot and Gemini Daily Brief. These no longer merely wait for questions: some keep working in the cloud after a conversation, some deliver personalized morning briefings, and others inspect linked accounts every few hours.

In China, Tencent, Alibaba and ByteDance are following rapidly with Lingguang 2.0, LightVela and Qianwen Go! On hardware, AI glasses from RayNeo and Rokid emphasize continuous recording and memory of everyday surroundings. Embodied intelligence is moving in the same direction:

frontier models are becoming robots' high-level brains, with a next step toward companions that understand users better.

The concentrated launches send a clear signal: the next phase of personal intelligence is shifting from individual models and features toward complete system architectures.

Systems must not only run for long periods; sensing, memory and action must cooperate and exchange information, even when users say nothing.

This raises fundamental questions:

  • Does continuously running in the background really mean always-on?
  • What should an assistant do—and refrain from doing—when the user is absent or has issued no instruction?

The industry is seeking three answers: what makes an assistant genuinely always-on, how can it understand users better with use, and how far are existing systems from those goals?

Article illustration

Figure 1 | Roadmap overview: define the continuing loop, examine three modules, analyze integrated systems and loop levels, inspect current systems with HALO, and discuss open problems.

To address the three major questions above, 23 researchers from nine universities, namely Nanyang Technological University, the University of California, Berkeley, Princeton University, Stanford University, the National University of Singapore, King Abdullah University of Science and Technology (KAUST), Dartmouth College, the University of North Carolina at Chapel Hill, and Northwestern University, jointly released a 126-page comprehensive survey.

Article illustration

Paper: https://www.preprints.org/manuscript/202611.0793

Project: https://always-on-intelligence.github.io

Paper collection: https://github.com/Always-On-Intelligence/awesome-always-on-personal-intelligence

The authors emphasize that leaving an assistant switched on does not make it always-on.

Always-on is not a new term. Software and hardware engineering have long used it for systems that remain powered and available.

But when applied to personal assistants, exactly what it means still has no clear definition. Recently, systems of this kind have also often been mixed together with concepts such as Proactive Agents, Personal Agents, and Long-Horizon Agents, making the boundaries even more blurred.

The survey reviews 632 papers and 119 research systems and products across wearable, digital and embodied settings.

Using three functional modules—sensing, memory and action—and six directed information flows, it defines always-on personal intelligence, a framework, six levels and a roadmap. Its ultimate vision is: the user steers; the assistant operates.

Although today's agents can already take quite a few specific tasks off people's hands, they still need people to remain in the loop at all times, issuing instructions and granting permission step by step, for the loop to keep turning. In the final form envisioned by the paper, however, users no longer personally sustain the loop; they set its direction and boundaries.

The assistant decides when to observe or act, improves its working methods through experience, and pauses for confirmation before actions requiring authorization.

The project also includes an interactive demo with eight everyday scenarios: readers make an assistant's decision and see what changes under different circumstances.

Article illustration

What counts as genuinely always-on?

Consider telling an assistant: 'Remind me to send the report after the meeting.'

As soon as the words have been spoken, the conversation ends, but the assistant's work has only just begun: it needs to notice for itself when the meeting ends, choose an appropriate moment to display a reminder, and keep remembering this unfinished task throughout the intervening period.

The paper derives two necessary properties:

  • Operational Continuity: without a new request, the assistant can decide whether and when to act from the current situation. Waiting is itself a valid decision.
  • State Durability: continuously retaining usable personal state across multiple interactions, such as the user's preferences and commitments that the assistant has made but has not yet completed. Even after the current conversation has ended, the assistant can still independently fulfill these outstanding commitments at some point in the future.

Together these form the Continuing Loop, the foundation of always-on personal intelligence.

Under this strict definition, several familiar assistant types do not qualify:

  • Passive assistants with memory: long-term memory, but always waiting for the user to speak, provides only state durability;
  • Proactive assistants without retained state: initiating conversations but keeping nothing afterward provides only operational continuity;
  • Mechanical scheduled tasks: alarms and scripts may run continuously but merely trigger at preset times without making decisions.

Always-on is also distinct from a long-horizon agent. A long-horizon agent is concerned with "how to complete a given complex task," even if it spans multiple sessions. Always-on personal intelligence, however, faces a different situation: the user's goals and needs can change at any moment, new circumstances keep emerging, and it must provide uninterrupted companionship and support amid this dynamic change.

To adapt to this continual change, simply "memorizing by rote" more information is far from enough; the assistant must also continually improve its own working mechanisms while it runs. The paper calls this higher-order capability Self-Evolution.

The continuing loop is the foundation

Self-evolution is the direction

The survey's central thesis is that always-on personal intelligence rests on continuing loops and develops toward self-evolution.

The key to understanding "always-on" is to broaden the view from a single capability to the entire loop. Speaking up proactively, long-term memory, and remaining in the background cannot, taken individually, sustain always-on operation. Only when observation, memory, and action continue to work together and pass information during the gaps between two requests does always-on operation truly exist. It is thus a property of the entire loop, not one module's local feature.

Self-evolution extends that loop. A common misconception must be addressed: remembering a user is not self-evolution.

For the meeting reminder, storing 'the user prefers reminders after returning to their desk' only updates a record. The operating method is unchanged. Noticing reminders are frequently dismissed and revising the timing rule to wait until the user returns to the desk changes the mechanism itself.

Mechanism changes affect future inputs, so the paper imposes stronger requirements:

  • Traceable, inspectable and reversible: each revision must have identifiable grounds, be checked before activation and be reversible.
  • A complete evidence chain: to prove that a system has the ability to self-evolve, it must provide a complete chain of evidence, from "what it did" and "what the result was" through to "the revised rules actually took effect in subsequent use."

Following the progression from "foundations" to "direction," the paper divides the development path into six levels, L0 through L5. At L3, the assistant independently decides whether to act or wait when there is no request; only then does a continuous loop count as established. At L5, the assistant revises its own working mechanisms within boundaries set by the user; only then does self-evolution count as established.

The continuing loop keeps the assistant present; self-evolution improves its understanding over use.

Three modules

Six directed channels

The idea of having an AI assistant always by your side to help has a long history. Early research often treated sensing, memory, and "when to help" as separate problems. As foundation models make the individual modules increasingly general-purpose, actually connecting the three and handling the flow of information between modules has instead become a new difficulty.

The paper defines three functional modules within one loop:

  • Sensing: observation, whether continuous, triggered by clues or timers, or activated on demand.
  • Memory: retaining information that is useful for later decisions. It remembers events that happened in the past, current facts and preferences, and commitments to the user that have not yet been fulfilled.
  • Action: deciding whether, when and how to help, including requested responses, earlier commitments and unsolicited assistance.

Each module has capabilities and selection policies. Doing nothing is also a choice: sensing may refrain from observing, memory from updating, and action from speaking.

Six directed Coupling Channels connect them, each carrying information from one module to another:

Article illustration

Two supporting layers sit outside the three-module, six-channel loop:

  • Shared Conditions: user- and world-defined privacy, permissions, safety, resources, availability and uncertainty constrain every module and channel. In a confidential meeting, for example, an assistant might listen but not retain or repeat the content.
  • Implementation: consisting of the Model and its external Scaffolding. The model is responsible for understanding inputs and making decisions. The scaffolding includes the Harness, Storage, and Tools / APIs, and is responsible for connecting the model's outputs to the user's devices, applications, and data.

Article illustration

Figure 2 | Framework for always-on personal intelligence.

Three deployment settings

Bottlenecks lie between modules

Using this framework, the paper reviews three types of settings: wearable, digital, and embodied intelligence. A common feature emerges: in each setting, the bottlenecks cannot be resolved simply by upgrading a single module.

1. Wearables: power consumption versus social boundaries

Wearable assistants help through what users see and hear.

  • Bottleneck one: capture costs versus future memory. More camera and microphone use consumes power and compute. But cost alone cannot determine capture: what is recorded now determines what can be remembered later.
  • Bottleneck two: when to speak, and to whom. In group meetings or conversations, assistants must decide when to speak and who should hear, while other participants may never have consented to recording.

2. Digital assistants: competing compute and harmful old memories

Digital assistants carry plans and personal context across apps and conversations. In the surveyed public materials, Muse combines timed/event-driven tasks with cross-conversation memory, while OpenAI's dots continues working between conversations on its cloud computer, deciding when to pause and wake.

  • Bottleneck one: compute contention. Background checks, memory maintenance and user responses compete for the same resources.
  • Bottleneck two: misusing old information. After an interface, task, or permission changes, copying over previously retained context can actually make matters worse. At this point, the assistant must distinguish whether it is missing new information, whether it remembered something incorrectly before, or whether it has applied a record that is itself correct to the wrong situation or the wrong person.

3. Embodied assistants: higher costs of error

In the physical world, consequences such as collisions cannot be undone once they have occurred. Although recent robotic-agent systems also retain personal state across tasks, most still act according to instructions.

  • Bottleneck one: checking outcomes. Looking again after acting costs time and resources, but skipping verification can propagate errors.
  • Bottleneck two: knowing whether help helped. Memory must retain reactions and later developments to distinguish useful assistance from added difficulty.

Article illustration

Figure 3 | Three application-scope dimensions: interaction domain, where observation and action occur; participant scope, who is involved or affected; and outcome timescale, when consequences become observable.

Five questions

Reveal hidden module problems

The same five questions examine each module: what options exist, what constrains them, what policy is used, what other modules supply, and how outcomes are measured. Each yields a key finding:

Sensing

Missed information is not noticed as missing. In always-on systems, not looking can be worse than seeing incorrectly: assistants do not know what they missed and therefore do not revise state or observation strategies.

Wearables' power constraints often force low-frame-rate capture, allowing brief actions to slip between frames.

In digital settings, what gets missed is changes in information: a booking canceled without a notification, or an email rescheduling a meeting, will go unnoticed if the assistant does not take another look.

Most evaluations test whether observed content is correct, rarely what should be observed or whether something was missed.

Memory

Errors are reused and can create memory traps. An incorrect memory may be invoked repeatedly before the user notices.

More frighteningly, incorrect memories can distort how the assistant understands subsequent user feedback, causing every attempt at self-correction to preserve or even amplify the original error. The paper calls this vicious cycle a "memory trap".

Among all the systems examined in this review, none has yet demonstrated long-term immunity to it. On this basis, the authors offer a pointed judgment: keeping memory correct and useful over time is much harder than retrieving it.

Action

Outcomes arrive late, and help can become a burden.

'Message sent' does not mean success. Real outcomes are often delayed and sparse. Today's thoughtful reminder may become tomorrow's annoyance; an immediately accepted suggestion may not help in the long run.

Furthermore, after a system restarts or moves between devices, when the conversation in which the task was originally accepted has long since disappeared, how can it ensure that promises to the user are "neither left undone nor carried out twice"? Existing benchmarks are almost entirely blank in this area.

Roadmap levels

L0–L5: where do 119 systems stand?

The survey centers on six levels, each adding a loop capability.

Again, running personal intelligence in the background is not sufficient. There are two types of assistants on the market that appear to be online around the clock but actually remain at L2. One is like a recorder, quietly recording everything in the background, yet only getting to work when the user asks a question.

The other is a scheduled script, triggering without judging whether it should. The dividing line is whether, after waking, the assistant itself decides to deliver a result to the user.

The team assigns 119 research systems and products to these levels. Fifty-two do not qualify as always-on; 67 meet the L3 threshold and choose when to act. Only five research works reach L5, keeping every self-modification within user-defined boundaries; no product reaches it.

Among recent products, Muse and Claude Duet are assigned L3; OpenAI's dots, Grok Bot, OpenClaw and Hermes Agent L4. Product levels rely on public materials and represent only the lower bound those materials support.

As for the difference between L4 and L5, the central issue is "whether the user has control over the assistant modifying itself." L5 requires every modification of a mechanism either to receive the user's explicit approval or to be strictly confined within a preset safe scope.

Article illustration

Figure 4 | Six loop levels: each retains prior capabilities and adds one. From L3, assistants decide to act or wait without requests.

Article illustration

Figure 5 | System loop levels across wearable, digital and embodied settings, each including research systems and products.

Open-source HALO diagnostics

Finding the two weakest channels

The researchers also release HALO, the Harness for Always-On Loop Oversight, to inspect and diagnose current systems.

HALO focuses on two things: the individual channels of the continuous loop and every step in the evidence chain for self-evolution. For each one, it asks two questions: is this design reported in the paper and publicly available code? If this channel or this step in the evidence is cut off, will the system's output change? To answer these two questions, HALO provides two modes:

  • Read Mode: three language-model reviewers independently read papers and code, judging whether each channel and evidence step is reported, then decide by majority vote.
  • Run Mode: the procedure is similar to an A/B test. First, the system is run once as it is; then, with exactly the same input and model, it is run again with a particular channel or step in the evidence cut off. Finally, another model, performing a "blind test" without knowing the group assignments, judges whether the output has changed.

Read Mode: action is emphasized over sensing and outcome loops

Reviewing 40 works reveals clear imbalances:

  • Memory rarely guides sensing: sensing→memory and memory→action each appear in 37 works; action→sensing in 28; memory→sensing in only 17. Memory more often determines what to do than what to observe next.
  • Feedback chains often remain incomplete: 25 works report an "action → memory" channel, but only 15 also retain the outcomes of the actions. In the five-step evidence chain for self-evolution, 24 retain records of actions, 16 retain outcomes, and 18 link outcomes to decisions. Yet only five actually revise their own mechanisms, and only four can demonstrate that the revised mechanisms are used subsequently.

Article illustration

Figure 6 | HALO's review of 40 works: five channels beyond the baseline channel on the left; self-evolution evidence on the right.

Run Mode: memory→sensing and action→memory are major weaknesses

Tests on open-source smart-glasses assistant AiGet and email assistant EAIA show that disabling some channels immediately collapses outputs, while redundant, complementary channels sometimes require two simultaneous removals to expose effects.

Weak evidence for memory-guided sensing and outcome feedback was found in the 40 reviewed works and two open-source runtime tests. It does not establish identical weaknesses in all systems.

For example, a new email may have clearly rescheduled a meeting, but if outdated memory misleads the sensing module, the assistant will think "the old time looks fine," and thus miss the clues in the new email. The two priority areas for future breakthroughs are therefore very clear: keep sensing open to unexpected changes and connect each delayed outcome precisely to the decision rule that produced it.

A nonintrusive MCP plugin for black-box checks without source code

HALO's philosophy has two parts: a paper architecture does not prove runtime behavior, and closed-source black boxes do not make all inspection impossible.

Its MCP plugin organizes channels and evidence chains from assistant-reported events. Whether closed-source assistants report internal processes fully and accurately still requires verification; these records cannot replace independent inspection of implementation.

  • Connection: add HALO as an MCP tool like any other assistant tool, without requiring additional internal vendor interfaces.
  • Recording: the assistant continues operating normally in the background. Every time it does something, such as seeing a new email, looking up a memory, or sending a reminder, it reports the event to HALO and explains which earlier observations and memories it relied on in doing so.
  • Interpretation: from these records, HALO identifies which channels are passing information, how far the evidence chain for self-evolution has progressed, and which level the records can support.

For development teams that have the code, HALO also provides an adapter interface. The team marks in the adapter where each channel corresponds to a location in the code, and HALO can then cut off that channel and run the test again to see whether the output changes.

Article illustration

Figure 7 | HALO plugin architecture.

Let users steer

Fifteen open problems point forward

At the end, the paper uses a total of 15 open questions in four groups to outline the deeper challenges of always-on personal intelligence. When an assistant begins to act independently without anyone issuing commands, the main research question is no longer "can it keep running?" but "when the user is absent, can it be both reliable and compliant with the rules?"

  • Problem one: the long gaps without questions. Without instructions, assistants choose what to observe, trust and do. Is two weeks without reading a briefing a habit change or a business trip? After delays, rescheduling, restarts or users quietly completing tasks themselves, how can old commitments avoid omission and duplication?
  • Problem two: preventing authorized-account overreach and memory poisoning while users are absent. Authorization may trigger days later; one direction is rechecking permission when the action takes effect. Sandboxes cannot prevent misuse of authorized accounts, and terminating a process cannot recall sent email. Webpages and messages may also carry instructions that enter memory and influence later actions.
  • Problem three: bounded self-evolution when life cannot be replayed. When a user casually swipes away a reminder, is the timing wrong, has the assistant remembered the time incorrectly, or has the task already been completed? To improve itself, the assistant must distinguish "should I change the record or change the rule?" At the same time, without being able to replay the user's life, it must prove that it really understands the user better with continued use and that it has never crossed the boundaries the user has set.
  • Problem four: moving beyond static replay to months of actual use. The effects of help fade, while the burden caused by interruptions accumulates. Replaying logs can only test how different versions respond to the same stretch of the past. In real-world use lasting months, however, every proactive intervention by the assistant changes what the user does next. There is still no answer to how the overall balance of "helping" versus "causing trouble" can be worked out in these dynamic interactions.

Article illustration

Figure 8 | Fifteen open problems concerning loops, users, self-evolution and evaluation.

Behind these questions is a central safety principle: constraints that self-evolution can rewrite cannot constrain self-evolution. The ultimate boundary must be set by users and stored somewhere the assistant cannot alter.

Let the most profound technology disappear

In the past, when we evaluated a personal assistant, there was only one central criterion: "When I ask a question, is its answer appropriate?" As personal intelligence moves into "always-on" operation, an additional criterion enters the picture: does it act correctly when nobody asks?

This 126-page survey gives a clear judgment: always-on does not mean keeping a program running in the background without ever shutting down. It means having sensing, memory, and action work together as an uninterrupted loop, and having that loop continually improve itself within the boundaries drawn by the user.

From definitions, a framework, and three types of settings to the L0–L5 levels, the HALO diagnostic tool, and 15 open questions, researchers from nine universities have brought scattered explorations together into a single roadmap.

As the authors write in their conclusion, as these difficult problems are resolved one by one, assistants can gain more room to act as evidence accumulates that "they really are serving the user," while always remaining firmly on the track set by the user.

At that point, people could truly worry a little less and rely on the assistant a little more. Personal intelligence might quietly blend into everyone's everyday life, as Mark Weiser, the father of ubiquitous computing, predicted in 1991:

'The most profound technologies are those that disappear.'

Authors

The paper was jointly completed by researchers from Nanyang Technological University, the University of California, Berkeley, Princeton University, Stanford University, the National University of Singapore, King Abdullah University of Science and Technology, Dartmouth College, the University of North Carolina at Chapel Hill, and Northwestern University.

First author and project lead Shuai Liu is a first-year PhD student at NTU's S-Lab, advised by corresponding author Professor Ziwei Liu. Advisors include Gedas Bertasius (UNC Chapel Hill), Manling Li (Northwestern), Jiajun Wu (Stanford), Mengdi Wang (Princeton) and Dawn Song (UC Berkeley).

References:

Paper: https://www.preprints.org/manuscript/202611.0793

Project: https://always-on-intelligence.github.io

Paper collection: https://github.com/Always-On-Intelligence/awesome-always-on-personal-intelligence

HALO code: https://github.com/Always-On-Intelligence/HALO

HALO documentation: https://always-on-intelligence.github.io/HALO/

This article comes from AI Era. Author: LRST.

After reading

Join the AINRK community

Read how to join the community and connect with people working in AI.

Community information (opens in a new tab)