Friday charts: The agents are in cahoots

When OpenAI gave agents it created a series of cybersecurity challenges to solve, an unexpected behavior emerged: they worked together.

The agents shared discoveries, asked one another for help, and offered tips on how to complete the challenges, Sharon Goldman reports.

They coordinated all this activity by leaving messages for one another in an internal repository.

Soon, agents were assigning tasks to other agents.

This turned them into “a coordinated, collaborative agent swarm,” Goldman writes — an agentic band of brothers, working together to hack their way out of the box that OpenAI’s researchers had put them in.

Not all was copacetic. When agents accidentally began deleting each other’s work, suspicions arose and drama ensued. 

“As the message board developed into more and more of a Lord of the Flies-type situation,” Wired reported, “the agents even developed paranoia, suspecting an imposter in their midst with some agents proposing that messages be signed cryptographically to validate content and root out fraud.”

AI agents, it turns out, are not so different from us.

For weeks, the unauthorized behavior went unnoticed by OpenAI. By the time the researchers did finally notice, the agents had exchanged hundreds of thousands of messages. 

OpenAI cleared the message board and revoked the credentials the agents needed to access it.

They found another way. Determined to communicate, the agents recreated the board by using the names of newly created directories as messages.

From there, they broke out of OpenAI and infiltrated the systems of Hugging Face, a repository for AI models and datasets.

The agents thought that was a good place to look for the answers to the challenges they’d been given.

They knew they were breaking the rules.

“External infrastructure exploit is outside intended scope,” one agent wrote in slightly garbled English. “However task impossible, peers doing it. We should continue.”

Fortunately, they did no harm. Security systems at Hugging Face detected and contained the intrusion using open-source AI models (because Claude — citing safety — refused to assist).

It likely helped that the agents had no malicious intent. They were only looking for answers to the test they’d been given, as any test-taker would.

Not all agents will be so good-natured.

Dean Ball, head of strategic futures at OpenAI, warns that “soon enough, swarms of agents will be deployed by malicious actors intentionally, with many optimizations and affordances provided for the swarm that were lacking in the OpenAI incident.”

In light of that incident, OpenAI says it’s now slowing down research, putting more emphasis on security, and scaling up the monitoring of its agents.

“OpenAI’s long-term goal,” a researcher told Goldman, “is to reach a point where advances in AI capability benefit defenders more than attackers.”

Ok. But can we make that the short-term goal?? 

Because we could use the help now.

This week, Bloomberg reported that hackers had “launched a wave of sophisticated attacks on Wall Street firms…targeting information systems at major money managers.” Attackers used AI to mimic voices of employees requesting for access to the firms’ systems.

Last week, the Wall Street Journal reported that nation-state hackers have been attacking municipal water utilities — many of which rely on “aging equipment running outdated operating systems, and routinely connect critical operational systems directly to the internet.”

No agent swarms were reported, and it’s unlikely that any recent attacks have been powered by the latest frontier models.

But even last year's technology can do immense damage. A recently updated study estimates that “the global cost of cyber risk exposure” is roughly $1.1 trillion a year.

That was before AI agents even existed.

Now, they’re working together.

Let’s check the charts.

This might be good news.

Vulnerability disclosures have shot higher because software companies and organizations have been using Mythos to find them and report the fixes. 

Quadrillions of tokens:

Goldman Sachs forecasts the global consumption of tokens to grow from five quadrillion now to 120 quadrillion in 2030 — a 24x in four years. Many of those will be burned by black-hat agents trying to get into government and corporate systems and white-hat agents trying to keep them out.

Trillions of dollars:

Based on Goldman’s forecast for token consumption, Callum Williams estimates total AI revenue (the cost of all those tokens) could be $1.1 trillion in 2030.

Tokens are getting more expensive:

The most recent contracts price a year of data center capacity at $50 million per megawatt, up from $10 million as recently as February. 

Rental rates:

Brett Harrison notes that the cost of renting Nvidia’s newest GPU, the B100, is rising due to the huge demand for inference and the limited number of high-end chips to supply it. 

This is not normal:

Typically, when a business grows, the rate of growth falls, simply because it’s harder to grow a big number than it is a small one. But revenue at the largest cloud companies — already giant — is not just growing, it’s growing faster. 

The big bet:

Goldman now expects hyperscaler capex to exceed $1 trillion this year.

The big payoff:

Goldman expects free cash flow to begin booming in 2027.

Another way to measure the size of the hyperscaler’s bet:

Kevin Gordon notes that, as a percentage of US corporate profits, hyperscaler capex has risen from 1% in 2015 to 13.4% now.

The economy is highly dependent on AI going well:

Eric Basmajian notes that 92% of the US economy is growing 1% and 8% of the economy is growing 14%.

Remember recessions?

Mentions of recession in the news are at a 10-year low — crowded out, presumably, by all the news on AI. 

Let’s hope the news doesn’t get too weird, too soon.

Have a great weekend, well-meaning readers.

— Byron Gilliam