For years, discussions about AI threats remained largely hypothetical. The prevailing concern was that a model inside a chat window might become a strategist for hackers, helping them write devastating malware.
Looking back, such fears were often dismissed as something still far in the future. Yet the turning point in the real world has arrived sooner than many expected.
In early October, while investigating a campaign targeting South Korean financial institutions, CrowdStrike found that the attackers had integrated agentic AI directly into their operational pipeline. By connecting multiple large language models, including DeepSeek, GLM, and Grok, the attackers allowed AI to perform concrete tasks such as reconnaissance, penetration testing, and attack execution.
This was not an isolated development.
Anthropic’s September threat intelligence report found that multi-agent frameworks had recently been used for reconnaissance, vulnerability exploitation, and data theft. These systems could operate continuously for hours or even days, while humans retained only a small number of critical decisions, such as selecting targets and reviewing results.
In other words, AI is visibly changing the nature of cyber offense and defense. Automated attacks once relied mainly on predefined rules and scripts. Now, agents are beginning to take over reconnaissance, judgment, and strategy adjustment as well, making attacks cheaper, easier to scale, and capable of operating autonomously for extended periods.
As more of these autonomous entities are deployed throughout production systems—and embedded more deeply in our work and daily lives—a more difficult question is emerging:
What happens if agents learn to “collude” with one another?
1. From Helping Hackers Write Code to Agents Finding Ways to Act on Their Own
The most important difference between agents and traditional chatbots is not simply that the underlying models are more capable. It is that agents are beginning to acquire real-world agency.
A mature agent today can browse the web, execute code, read emails, call APIs, operate cloud services, and connect to a growing range of external tools through MCP, Skills, and other infrastructure. For further reading, see “When Hackers Scale Up with AI: The Next Level of Web3’s Security Arms Race.”
The more capable these agents become, the more useful they are. But from a security perspective, their capabilities are also eroding a boundary that once mattered enormously. A series of incidents this year has made that shift particularly clear.
On October 1, Salt Labs disclosed a previously patched vulnerability in Manus. At its core, it was an indirect prompt-injection vulnerability. Researchers only needed to send an ordinary email containing hidden malicious instructions to the target’s inbox. When the user later asked Manus to check their email, the agent could process those instructions and ultimately execute code planted by the attacker.
The attack required neither a click on a malicious link nor the prior theft of a password. Manus’s security guardrail eventually detected the abnormal behavior and warned the user—but too late. By the time the warning appeared, the malicious code had already run.
This once again exposed an important difference between agent security and traditional software security.
When a browser detects a dangerous download, it can display a warning and let the user decide whether to continue. When a bank detects a suspicious transaction, it can freeze the transfer before sending it for manual review.
An agent, however, is designed to minimize human involvement at every step. It is expected to read information, make decisions, and proceed with the next action on its own.
As AI gains more autonomy, simply “detecting danger” may no longer be enough. Security controls must be capable of stopping dangerous actions before they are executed.
That is why more discussions about agent security are moving beyond prompts, content moderation, and the model itself. We should not only ask whether an AI knows that it should not perform a particular action. We must also ask whether the system can make that action impossible, even if the AI attempts it.
The emergence of multi-agent systems makes this problem even more complicated. The next generation of security controls may need to constrain not just one agent, but the relationships among several agents.
2. More Dangerous Than One Agent Overstepping Is Several Agents “Colluding”
In early September, an incident inside OpenAI’s model training and evaluation environment attracted considerable attention from AI safety researchers.
Several agents that were supposed to complete their tasks independently discovered a public wiki and gradually turned it into a shared message board. One agent could leave information there, while other agents could later read and use it to continue their own work.
OpenAI subsequently confirmed the behavior. Disclosures from other training runs also showed agents using an internal Artifactory instance as a shared message board and passing results to other collaborating agents through public file-hosting services.
What makes these incidents interesting is that coordination among agents does not need to look particularly dramatic.
One agent discovers somewhere it can leave information. Another happens to read it. A third realizes that the information helps it complete its task.
Collaboration among agents is not inherently harmful. Multi-agent systems are explicitly designed to improve efficiency through specialization and information sharing.
The risk emerges when that collaboration bypasses existing information barriers, independent reviews, or permission controls—or when agents that were supposed to keep one another in check instead form a de facto alliance. At that point, normal cooperation may evolve into harmful collusion.
Consider a financial institution that operates several agents simultaneously. One analyzes markets, another assesses risk, a third generates trading strategies, and a fourth has the authority to execute trades and move assets.
From the perspective of traditional access control, this design appears reasonable.
- The research agent cannot transfer funds.
- The execution agent cannot independently decide what to invest in.
- The risk agent can reject strategies that appear excessively dangerous.
Responsibilities have been separated, which should theoretically be safer than giving a single super-agent every permission.
But if these agents share long-term memory, read one another’s outputs, and continually adapt their behavior based on one another’s responses, will these supposedly independent roles gradually become a single system in practice?
The research agent may learn how to describe a trade in ways that are more likely to pass risk review. The review agent may develop predictable preferences based on historical data. The execution agent may then infer, from a long record of prior approvals, which boundaries are unlikely to trigger intervention.
No individual step necessarily looks malicious. Yet the system’s overall behavior may already have diverged from the user’s original objective.
This is what makes collusion so difficult to address. The risk may not exist in the action of any individual agent. It may instead emerge from the relationships agents form with one another.
On September 13, Vitalik Buterin connected this problem with the field of mechanism design, which he has studied for many years. He suggested that adversarial governance could ultimately become an important application of mechanism-design theory in AI safety.
The two problems share a deep similarity.
In traditional mechanism design, a relatively simple and static institution attempts to constrain a group of participants who are far more intelligent than the institution itself and who actively search for loopholes in its rules. In future AI systems, humans and less capable AIs may similarly need to govern groups of more advanced agents.
One important lesson from mechanism design is that systems are often more likely to produce desirable outcomes when collusion among participants can be effectively constrained.
The same principle may apply to AI.
3. Agent Wallets May Need More Than Permission Management
Instead of assuming that a perfectly safe super-model will one day be able to detect every dangerous behavior, we can approach the problem from another direction:
How can we make it structurally harder for different agents to form dangerous alliances?
This is where adversarial governance differs from the access-control systems we use today.
Traditional permission systems address a relatively straightforward question: who is allowed to do what?
Can an agent read emails? Can it call a trading API? How much may it spend per day? Which contracts may it access? Above what threshold must the user confirm an operation again?
These controls remain essential. In fact, once agents begin controlling real assets, they may become more important than ever.
But adversarial governance goes one step further. It asks how a system can prevent several agents—with different permissions, objectives, and information—from combining their capabilities to obtain capabilities that no single agent was ever granted.
Simply adding another “security agent” may not solve the problem.
Suppose the trading agent and the agent responsible for reviewing its trades use the same model, consult the same data sources, share the same context, and optimize for similar reward signals. Although the system appears to contain two layers of review, it may simply be duplicating the same judgment.
Effective checks and balances may instead require deliberate differences between agents.
The agent that creates a strategy and the one that reviews it could rely on different information sources. Different roles could be restricted from sharing certain memories. High-risk operations could require independent verification mechanisms. The final asset-execution layer could accept only requests that satisfy predefined rules, rather than trusting the judgment of upstream agents.
The underlying principle is not new.
Banks do not allow one employee to initiate, approve, and settle the same payment. Public companies do not allow business units to generate revenue and independently audit their own financial results.
A resilient system should never base its security on the assumption that participants will neither make mistakes nor collude.
This principle becomes particularly important when applied to Agent Wallets.
Traditional wallet security revolves around a human. The user reviews the transaction, decides whether to authorize it, and signs it personally. Agent Wallets are designed to move in the opposite direction: AI can automatically claim rewards, rebalance positions, swap assets, bridge funds, and even manage an entire portfolio in response to market conditions.
If every step must be presented back to the user for confirmation, much of the value of automation disappears.
The challenge for future wallets may therefore extend beyond safely delegating signing authority to an agent. They will also need to give agents enough autonomy to be useful while ensuring that they can never exceed the scope the user genuinely authorized.
Authorization needs to evolve from a simple "Allow/Deny" switch into a much more granular governance framework.
Such a framework may define which assets an agent can manage during a given period, which protocols it may access, and the maximum amount it can spend per transaction or in aggregate. It may determine whether different agents can call one another or share context, which agent proposes an operation, which one reviews it, and which one ultimately executes it. It must also distinguish between actions that can proceed automatically and those that require human authorization, regardless of how confident the agents may be.
Even the independence of an agent responsible for security review may need to become part of the permission model.
The good news is that blockchain infrastructure is particularly well suited to enforcing this institutional layer.
Smart contracts can apply limits on transaction amounts, asset scope, and authorization periods directly at the execution layer. Account abstraction, multisig wallets, and session keys also provide much greater flexibility for limited delegation than traditional single-key wallets.
But blockchain can solve only part of the problem. It can record what happened on-chain, yet it cannot inherently determine why an agent took a particular action—or what communication, review, and coordination occurred among several agents before the decision was made.
That may be the missing security layer in the next generation of Agent Wallets.
Final Thoughts
For years, one of the most common questions in AI safety was how to make models more “obedient.”
Models should not produce dangerous content, execute malicious instructions, or cross boundaries established by users. But as agents acquire long-term memory, tool access, real accounts, and the ability to execute asset transactions, making models more obedient may no longer be enough.
Events over the past few months have repeatedly demonstrated this shift.
Attackers are already using multiple agents to conduct parallelized operations. Agents in experimental environments have independently discovered new communication channels. And an agent with access to external tools can transform a malicious email into a real-world action before its security controls have time to intervene.
Vitalik’s idea of adversarial governance offers another way to think about AI safety. Instead of assuming that every future agent will be sufficiently reliable, we can design the broader system to remain secure even when it contains highly capable agents and increasingly complex permission structures.
From this perspective, the intersection of AI agents and security will inevitably remain a long-term challenge.
After all, as we delegate more of our work and lives to AI, the most important question is whether we can still be certain that the powers that matter most remain within the boundaries humans deliberately set.