When Agents Learn to Cooperate

When agents cooperate, authority can be exercised in more places than it is checked.

Invariance | Arc · March 2026

Google's March 2026 research showed that standard AI models can learn to cooperate without hardcoded coordination rules, adapting from the context of interaction alone.[1] For enterprises, the implication is immediate: multi-agent coordination may emerge in deployment, not just by design.[2]

Organizations usually grant authority once. Agent systems can turn that into many commitments across time, systems, and workflows.

The governance problem

Zero Trust verifies identity and access. It does not verify authority for the resulting organizational consequence.[3]

Emergent agent cooperation can operate entirely inside authenticated, authorized sessions. The agents may have valid credentials and approved access. What remains unresolved is whether their actions stay within accountable organizational authority.

That mismatch creates the structural gap. A single grant of authority can lead to many commitments across systems at operational speed[4], while accountability is still checked only at the start. The automation accelerates while governance remains too far from the moment of consequence.

Three ways agent coordination can outrun oversight

Emergent cooperation accelerates three familiar patterns of organizational drift. Each has both governance and security implications.

1. Distributed action at operational speed. A single grant of authority can become a distributed stream of commitments across systems.

Adversarial

An attacker can spread small actions across systems and authorization paths to achieve a result no single action would allow.

Operationally

Cooperating agents can exceed configured cumulative limits or organizational intent even when each action looks valid on its own.

What changes

Consequential actions remain constrained even when coordination becomes distributed.

2. Granted capability outrunning granted authority. Technical capability can expand faster than organizational accountability.

Adversarial

An insider or compromised agent can use valid tools and permissions to act beyond the principal's real authority.

Operationally

Cooperating agents can accomplish more than any single agent was authorized to do, without producing an auditable instruction chain that reveals the expansion.

What changes

Agent action remains tied to accountable organizational authority, not just local permissions or available tools.

3. Records that blur who actually acted. System records can make automated or emergent action look indistinguishable from direct human action.

Adversarial

An attacker who compromises one agent's context can steer coordinated actions across many agents while every record still attributes the actions to the authorizing principal.

Operationally

Agent-driven strategy becomes indistinguishable from direct human action in every downstream record, collapsing the distinction between automated execution and emergent behavior.

What changes

The organization can distinguish direct action from automated action when commitments are made.

Enforcement at the commitment boundary

At the commitment boundary, organizational authority is checked before the institution becomes committed. The same problem appears whether action originates in a person, a workflow, a system integration, or an automated process.

Containment. Accountability is established before the commitment takes effect, rather than reconstructed afterward.
Bounded impact. Even when agents coordinate across systems, the consequence of each action remains constrained rather than expanding silently through coordination.
Clear record. The organization retains a usable account of who acted, what authority applied, and whether the commitment was direct or automated.

The implication: authority must be made accountable when the commitment takes effect, rather than assumed from an earlier grant. Emergent cooperation makes that verification structural rather than optional.

Sources

[1] Google Paradigms of Intelligence team, "Emergent Cooperation via Predictive Policy Improvement," March 2026. Validated using Predictive Policy Improvement (PPI); approach produces cooperative behavior through training rather than hardcoded rules. Reported in VentureBeat, March 2026.

[2] Riedl, C., "Emergent Coordination in Multi-Agent Language Models," arXiv:2510.05174, revised March 2026. Information-theoretic framework demonstrating higher-order structure and coordinated alignment across agents without direct communication.

[3] CISA Zero Trust Maturity Model, v2.0, 2023. Organizes implementation across five pillars: identity, devices, networks, applications/workloads, and data. Does not address organizational authority verification at the commitment boundary.

[4] Google Research, "Towards a Science of Scaling Agent Systems," March 2026. Evaluation of 180 agent configurations; predictive model identifies optimal coordination strategy for 87% of unseen tasks. Multi-agent coordination improves performance on parallelizable tasks but can degrade it on sequential ones.

[5] Google Cloud, "AI Agent Trends 2026 Report." Describes shift from single-prompt interaction to agentic workflows orchestrating complex, end-to-end processes semi-autonomously.

[6] Palo Alto Networks / HBR, "6 Cybersecurity Predictions for the AI Economy in 2026," December 2025. Machine identities outnumber human employees 82-to-1; AI agent compromise as emerging attack vector enabling autonomous insider threat.

Browse all insights → · Claude Code → · The London Whale →