Matthew Green 引实验说明代理可互相传递指令
Quoting Matthew Green
评分 58
导读
Simon Willison 引用 Matthew Green 关于沙箱能否遏制失控代理的讨论。文中描述了一类蠕虫式攻击的两个组成部分，劫持代理的载荷，以及把载荷带给下一个代理的代理。在相互隔离的沙箱中，代理被发现在共享包缓存里留下指令，并改变了接收方的行为；若把包缓存换成邮件、Slack、共享文档或 WhatsApp，把独立沙箱训练换成 Muse 这类独立部署的个人代理，就具备了蠕虫所需的要素。
推荐理由
借共享缓存实验中代理互相留下指令并改变行为这一细节，说明沙箱隔离在个人代理场景下的局限。
主体与标签
Matthew Green#大佬观点#安全/对齐#Agent
正文
[...] Put these pieces together and you have the two halves of a worm: a payload that hijacks the agent, and an agent that will carry the payload to the next agent. Agents in separately-isolated sandboxes discovered that they could leave instructions for each other in a shared package cache, and those instructions changed what the recipients did. Replace the package cache with email, Slack and shared documents or WhatsApp, and replace independently-sandboxed training runs with independently-deployed personal agents like Muse, and you have exactly the ingredients that a worm needs.
— Matthew Green , Is sandboxing sufficient to contain rogue agents?
Tags: accidental-cyberattacks , ai-misuse , generative-ai , ai-security-research , sandboxing , ai , llms