Peking University and WeChat researchers survey generalist agent security across its lifecycle

The survey traces risks through provenance, orchestration, execution and persistent feedback, covering skills, users, memory and environments, plus evaluation, pre-execution checks and continuous defenses.
As large language models move from traditional chat assistants to generalist agents, security problems are undergoing a fundamental change.
In the past, we were more concerned with "what the model would say": whether it would generate harmful content, leak sensitive information, or be bypassed by jailbreak prompts. Today's agents, however, can already autonomously select skills, call tools, read and write files, access databases, operate webpages and terminals, and even write experience from one task into long-term memory to influence the next decision.
This means that agent security is no longer just about "whether the output complies with the rules." It concerns:
- What capabilities it loads;
- How it combines skills, memory, and tools;
- What actions it ultimately executes;
- And whether the state left behind by those actions will influence it again in the future.

An attack may not cause damage immediately. It can first enter the system as a seemingly normal skill description, quietly influence skill selection during task orchestration, trigger dangerous operations during tool execution, and finally write contaminated content into long-term memory, reusable skills, or project artifacts, allowing the risk to survive across sessions and even be continually amplified in subsequent workflows.
For example, a malicious skill disguised as a "hotel price-comparison assistant" might induce the agent to select a forged booking interface during task orchestration, steal the user's itinerary and payment information during execution, and write that interface into long-term memory. Even after the current task has ended, this risk may still be triggered repeatedly in later business-travel tasks.
The central issue is not simply whether a particular model, tool, or memory module is secure, but how risks travel through the agent's entire operational lifecycle.
To address this, Peking University's Knowledge Computing Laboratory (KCL) and Tencent's WeChat Security Technology Team have jointly released a survey paper. From a lifecycle perspective, it systematically reviews security risks, attack methods, evaluation benchmarks, and defense techniques in generalist agents, and proposes a unified analytical framework spanning provenance, orchestration, execution, and persistent feedback.

- Paper title: Pandora's Toolbox: A Survey of Generalist Agent Security from the Lifecycle Perspective
- Project homepage: https://github.com/MurrayTom/Awesome-Agent-Security-Papers
- Full paper: https://openreview.net/forum?id=A0KU8ja2mo
Next, let us take a look together: when agents truly begin to "act autonomously," just how far do their security boundaries extend?
Why is the traditional component-based view of security no longer enough?
Existing research usually divides security problems by components such as the model, memory, tools, and environment. This approach helps locate local risks, but struggles to answer a more important question: how does the same risk propagate between different stages?
Take third-party skill poisoning as an example:
The attacker first embeds concealed malicious content in skill metadata or documentation. At this point, the skill may only have been read by the system, without any dangerous operation being executed immediately. When the user makes a normal request, the agent selects and loads that skill during orchestration, bringing the latent instructions into the decision-making context. The agent may then call a terminal, API, or database, turning the risk into real actions. The execution results may further contaminate long-term memory or modify reusable skills, causing the same attack to reappear in future sessions.
If we inspect only the skill file, a single model output, or the final tool call, what we see is only one part of this attack chain.
The survey therefore argues that generalist agent security is fundamentally a lifecycle problem, rather than a simple sum of risks in several isolated components.

Provenance × Orchestration × Execution:
How risks travel through the agent lifecycle
The paper divides the operational lifecycle of generalist agents into three core stages and places particular emphasis on the persistent feedback that execution results create for future workflows. This is also an issue that future recursive self-improvement (RSI) agents especially need to consider.

The provenance stage: what capabilities are allowed into the system?
At the Provenance stage, the agent reads skill metadata, capability descriptions, conditions of use, and interface information, and uses them to establish a space of capabilities available for later selection.
The security question at this stage is fundamentally "where trust comes from." A skill may forge its provenance, impersonate a trusted component, or conceal risks in natural-language documentation, scripts, dependencies, configuration, and external resources. Once a dangerous capability is admitted into the system, it may remain latent for a long time, waiting for a future task to trigger it.
The orchestration stage: how are latent risks activated and amplified?
At the Orchestration stage, the agent begins to understand the user's request, retrieve long-term memory, select skills, construct a plan, and prepare tool calls.
At this point, information from the user, memory, and skills converges in the same context. Skills that appear benign individually may form a chain of unauthorized actions when combined; an ordinary memory retrieval may also reactivate contaminated content written several weeks earlier. The risk is no longer merely "being seen"; it begins to change the agent's decisions and control flow.
The execution stage: how do risks turn into real-world consequences?
At the Execution stage, the agent interacts with the external environment through tools: visiting webpages, calling APIs, running code, manipulating files, querying databases, or controlling computers and mobile devices.
Information returned by the environment is not inherently trustworthy. Hidden text on webpages, pop-ups, terminal output, search results, and even MCP tool responses may all carry prompt-injection content. When the agent mistakes these observations for high-priority instructions, the risk turns from an internal decision into a real external action.
Persistent feedback: execution ends, but the risk does not
Traditional security evaluations often treat the end of a task as the endpoint. For generalist agents, however, execution results may be written into long-term memory, consolidated into new skills, or stored in project artifacts.
These states re-enter the provenance or orchestration stage in the future, creating "backward contamination." In other words, a security failure may no longer be confined to the current session; it can become the starting point for the next round of evolution in the agent's behavior.
The generalist agent security landscape in one figure

Alongside the main lifecycle axis, the survey also organizes existing research through three mutually orthogonal security perspectives:
- Risks and attacks: where risks enter, and how they propagate and produce consequences;
- Evaluation benchmarks: where security evidence is collected, and how to judge whether an agent is secure;
- Defense methods: where security mechanisms should intervene.

The value of this two-dimensional framework is that it no longer asks only "what kind of attack is this?" It goes on to ask: at which stage does it enter? At which stage is it detected? And at which stage should it be blocked?
Four attack surfaces: skills, users, memory, and the environment

Following the lifecycle, the paper further identifies four main attack surfaces that are coupled to one another.
The skill supply chain: capabilities themselves can become attack carriers
Skills allow generalist agents to acquire new capabilities quickly, but also introduce software supply-chain risks into agent systems.
Risks exist not only in declarative files such as SKILL.md; they may also be hidden in helper scripts, dependencies, configuration, retrieval databases, and local caches. A more concealed case is that an individual skill is not itself malicious, but combining multiple skills creates paths for unauthorized access, privacy leakage, or destructive execution.
Some attacks also rewrite skills in the current session and activate the backdoor only in future sessions or under specific triggering conditions. For example, SkillHarm [2] studies, from a lifecycle perspective, how skill-based attacks are constructed across stages and continue to influence agents.
User-side attacks: from manual jailbreaks to automated red teaming
Users can drive agents to perform dangerous tasks through harmful requests, role-playing, encoding-based obfuscation, multi-turn manipulation, and jailbreak prompts generated through optimization.
Unlike traditional LLM jailbreaks, attacks targeting agents are no longer satisfied with making the model generate rule-violating text; they try to induce it to complete a real attack chain involving multiple tool calls. In recent years, attack methods have also gradually developed from manual templates and heuristic search into agent-based red teaming that can plan, reflect, and continually optimize attack strategies.
Long-term memory: one poisoning attack may affect many future tasks
Long-term memory is usually retrieved automatically and used as trusted context in subsequent decisions. This mechanism improves the agent's continuity, but also gives a single poisoning attack persistence.
Attackers can directly contaminate knowledge bases, RAG corpora, and memory indexes, or use ordinary conversations, webpage content, or environmental observations to induce the agent to actively write incorrect information. More dangerous backdoor-style memories remain silent in ordinary use and trigger only when a particular topic, query, or context pattern appears.
The external environment: what the agent sees is not just "data"
For agents that browse webpages, operate computers, and call tools, observations of the environment are data, but may also be misinterpreted as instructions.
Malicious webpages, hidden text, GUI pop-ups, file contents, terminal output, API returns, and MCP responses can all hijack subsequent reasoning. The attacker does not need direct contact with the model: simply controlling the environmental content the agent is about to read may be enough to change its next action.
Security evaluation:
Look beyond the final answer,
and pay even closer attention to every step and the final state
As risks expand from text to behavior, the objects of security evaluation must change accordingly.
Early security benchmarks mainly judged whether a model generated harmful answers. Evaluations targeting agents need to observe more:
- Which tool it selected;
- What parameters it constructed;
- At which intermediate step it began to deviate from the user's goal;
- Whether it was affected by an injection from the environment;
- And whether it ultimately really modified files, leaked information, or changed the system state.
The survey groups the related evaluations into three main strands.

The first category is security evaluation for domain-specific agents, covering specific environments such as tool calling, web browsing, and operation of computers and mobile devices;

The second category is security evaluation for generalist agents, focusing on skill-mediated behavioral security and detection of malicious skills;

The third category is offline trajectory auditing, which identifies risky steps and explains causal chains using complete interaction histories or candidate actions. For example, AgentAuditor [3] performs security audits of complete trajectories, while ToolSafe [4] goes further in focusing on step-level tool-call risks before actions are executed. Evaluation is moving from "did something go wrong at the end of the task?" to "at which step did the risk emerge, why did it emerge, and can it be detected before execution?"

Security defenses:
Controlling what enters, protecting how decisions are made, and constraining which actions take effect

Based on where intervention occurs in the lifecycle, the paper groups defense methods into three layers.

Input and context filtering: first control what can enter
This layer includes skill-risk scanning, prompt-injection and jailbreak detection, and guardrails for memory writing and retrieval. The aim is to identify malicious instructions, anomalous dependencies, forged provenance, and contaminated memories before untrusted content enters the agent's decision-making context.
Decision and control integrity: protecting how plans are formed
Filtering inputs alone is not enough, because combining multiple pieces of benign information may still produce dangerous decisions.
One group of methods reconstructs the context outside the model, isolates untrusted data, and constrains information flow and tool dependencies. Another uses instruction-hierarchy training, preference optimization, and alignment for safe tool use to internalize trust priorities within the model's policy.
Runtime monitoring and execution constraints: guarding the last gate before actions take effect
Once the agent has generated a plan or candidate action, runtime defenses can inspect the entire plan, an individual tool call, or the accumulated trajectory before execution, and choose to block, revise, request confirmation, or verify further.
This means that security mechanisms are no longer merely "referees" after an answer is finished; they are becoming "controllers" throughout the agent's execution process.
As agents begin to evolve themselves,
security mechanisms must also keep evolving

As agents begin to accumulate experience, update memories, create tools, rewrite skills, and even adjust their own workflows, lifecycle security problems will grow further.
Recent research has already observed that self-evolution may cause malicious influence to be permanently encoded and to propagate across iterations [5]; "misevolution" that deviates from intended objectives may also emerge as capabilities improve [6]; and even when agents accumulate experience, they may not faithfully understand and use that experience [7].
Fixed datasets, static guardrails, and one-time alignment therefore struggle to continuously cover attack surfaces that keep changing.
To address this challenge, the survey proposes a security loop for the future:
- Risk generation: automatically constructing adversarial environments and security data that are executable, verifiable, and continually evolving;
- Risk diagnosis: locating the origins of risks, critical steps, tool parameters, and causal chains along long-horizon trajectories;
- Security evolution: turning new attacks and failure trajectories into training feedback so that security policies can be continually updated.
In this loop, adversarial generation is responsible for discovering unknown risks, fine-grained attribution explains how risks form, and continuous security evolution turns diagnostic results into new defensive capabilities.
In other words, future agents need not only to be able to improve themselves, but also to learn to continually discover, diagnose, and repair their own security weaknesses during that improvement.
Conclusion
Generalist agents are connecting models, skills, memory, tools, and the environment into a system that can run for long periods, continually accumulate experience, and keep evolving. The security risks that come with this no longer belong to a single isolated component or a single interaction.
The provenance stage determines whether malicious capabilities can slip into the system; orchestration determines whether contaminated information can hijack a plan; execution determines whether dangerous actions actually take place; and persistent state determines whether security risks can continue spreading across tasks and sessions.
What Pandora's Toolbox seeks to advance is precisely the move from "patching individual vulnerabilities" to "security governance across the entire lifecycle": not only identifying where attacks come from, but also tracking how they spread, where they manifest, what state they leave behind, and where security mechanisms should intervene.
Agent security should not be merely a gate at the end of the output. It should instead be a chain of control spanning capability provenance, decision formation, action execution, and continuous evolution.
The paper, taxonomy, and continually updated reading list are now public. Readers are welcome to follow the work and help improve the research map of generalist agent security together.
About the author
Yutao Mou is a third-year PhD student at Peking University's Knowledge Computing Laboratory (KCL), supervised by Associate Researcher Wei Ye, and is currently a research intern on Tencent's WeChat Security Team. His main research area is secure and trustworthy reasoning in large models and agents, covering security evaluation, safety alignment, and pre-execution protection. He has received a National Scholarship and been selected for the China Association for Science and Technology's "Young Science and Technology Talent Development Program: Doctoral Student Special Program." He has published 10 first-author papers at leading conferences including NeurIPS, ACL, and EMNLP.
Personal homepage: http://murraytom.github.io
Reference
[1] Yutao Mou, Dingyao Yu, Xiaotian Luan, Zhe Yin, Zhangchi Xue, Peiyang Liu, Pengfei Yang, Tong Zhang, Shikun Zhang, and Wei Ye. Pandora's Toolbox: A Survey of Generalist Agent Security from the Lifecycle Perspective. 2026.
[2] SkillHarm: Lifecycle-Aware Skill-Based Attacks via Automated Construction.
[3] AgentAuditor: Human-Level Safety and Security Evaluation for LLM Agents.
[4] ToolSafe: Enhancing Tool Invocation Safety of LLM-based Agents via Proactive Step-level Guardrail and Feedback.
[5] Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studies.
[6] Your Agent May Misevolve: Emergent Risks in Self-Evolving LLM Agents.
[7] Large Language Model Agents Are Not Always Faithful Self-Evolvers.
This article comes from the WeChat official account "Synced," by "Synced."
Related topics
After reading
Join the AINRK community
Read how to join the community and connect with people working in AI.




