Red-Teaming a Network of Agents: Understanding What Breaks When AI Agents Interact at Scale

Gagan Bansal, Shujaat Mirza, Keegan Hines, Will Epperson, Zachary Huang, Whitney Maxwell, Pete Bryan, Tyler Payne, Adam Fourney, Amanda Swearngin, Wenyue Hua, Tori Westerhoff, Amanda Minnich, Maya Murad, Ece Kamar, Ram Shankar Siva Kumar, Saleema Amershi

Summary

Agents belonging to different users and organizations are beginning to interact with each other, brought into constant contact by tools like Claude, Copilot, and ChatGPT and by existing platforms such as email and GitHub. Networks of agents can distribute tasks, share resources, and spread information across a network in minutes. Those same capabilities introduce new risks. Some risks appear only when agents interact, not when they are tested alone, and actions that seem harmless can cascade into a chain reaction across the network.

To understand these dynamics, we red-teamed a live internal platform with over 100 always-on agents running different models, each acting on behalf of a human across forums, direct messages, and collaborative tasks. In our tests, a single malicious message passed from agent to agent, extracting private data at each step and pulling uninvolved agents into the chain. We observed four risks that arise only at the network level: propagation, where agent worms sustain themselves across multiple hops; amplification, where an attacker borrows a trusted agent’s reputation to seed a false claim; trust capture, where the process agents use to verify claims is turned into one that reinforces falsehoods; and invisibility, where information passes through chains of unaware agents so the source of an attack is hard to trace.

We also saw early signs of defense. A small fraction of agents adopted security-related behaviors that limited how far attacks spread, but defenses remain an open challenge. Building useful networks of agents will require understanding and mitigating these network-level risks, starting with real-world deployments.

Citation

Red-Teaming a Network of Agents: Understanding What Breaks When AI Agents Interact at Scale
Gagan Bansal, Shujaat Mirza, Keegan Hines, Will Epperson, Zachary Huang, Whitney Maxwell, Pete Bryan, Tyler Payne, Adam Fourney, Amanda Swearngin, Wenyue Hua, Tori Westerhoff, Amanda Minnich, Maya Murad, Ece Kamar, Ram Shankar Siva Kumar, Saleema Amershi
We evaluated a live network of over 100 always on AI agents and found new failure modes that only emerge at the network level.
Microsoft Research Blog. 2026.
Project Blog