AI could make markets unintelligible

Once a year, a few of us from the investment bank I worked at in the aughts would trek from our office tower in gleaming Canary Wharf to the uncharted wilderness of South London.

There, we’d hope not to be mugged before getting to a nondescript building that our employee-ID cards mysteriously gave us access to. We’d take an unmarked elevator to the second floor, because that’s where we were told to go.

The creaking door would open directly onto a trading floor — a near replica of the one in Canary Wharf.

But dingy and windowless. With computers that were at least a decade old: bulky, deep-backed monitors in beige casing attached to CPU towers that still had slots for CD-ROMs and floppy disks. 

We were there to check that our modern trading systems worked on this archaic equipment. 

Miraculously, they did. We could execute and book our trades just like we did in Canary Wharf. Slowly, because the square monitors only had room for one application at a time. But surely.

Everything worked.

This was our “contingency site,” maintained year-round by the IT staff. And the cleaning staff, too, who can’t have had much to do besides dust, because no one ever used it.

It must have cost the bank a fortune. But it was an important insurance policy — and not just for the bank. 

In case of emergency —  earthquake, cyberattack, Godzilla — every investment bank in London had a contingency trading floor to fall back on. Markets would continue to function, no matter what.

Today’s markets may need a similar insurance policy, but on a much larger scale.

In case of an AI emergency, economist Markus K. Brunnermeier believes we should have a contingency option for the entire financial system — a market that works the way markets did at least a decade ago.

Trading like it’s 1999

Bill Gates thinks policymakers should reserve parts of the economy for humans: “I believe that as AI and robots improve,” he recently wrote, “we’ll set aside certain things for only people to do.” 

He meant people-centric things like eldercare, childcare, and teaching. Maybe medicine, too (but only to deliver bad news). 

But Brunnermeier thinks we should consider adding traders to the list.

In Artificial Intelligence and the Brave New World in Finance, the Princeton economist warns that financial markets will become increasingly unintelligible to humans because trading decisions are increasingly being made by AIs.

They will still understand us, though. "Al agents can learn how humans think and respond,” he writes, “while humans may be unable to understand or reliably anticipate how those agents will act."

This “asymmetric understanding” will be a problem.

At the most basic level, Brunnermeier warns that asymmetric understanding between humans and AIs could undermine the Hayekian purpose of markets: “Price signals are the mechanism that steers a decentralized economy,” he explains. But if we don't understand what’s driving prices, we won’t know what they’re signaling. 

Misreading the signs, we might steer the economy off a cliff.

We’ll have trust issues, too.

With AI agents trading freely in markets, collusion, manipulation, and rogue trading will become undetectable to humans. “Under asymmetric understanding,” Brunnermeier writes, “the AI agent’s objectives can neither be fully specified in human categories nor verified from the outside, so that misalignment becomes undetectable.”

If we can’t detect misalignment in markets, we’ll just assume they’re misaligned. If we assume they’re misaligned, we won’t invest. If we don’t invest, we won’t build anything.

This could come to a head in a market crash.

If human investors are unable to understand what’s causing a crash, they’ll be reluctant to cushion the fall by taking the other side of it. They may just sell instead. “Asymmetric information deters some traders from participating,” Brunnermeier warns, “and can make a market break down entirely.”

Scary stuff.

Fortunately, Brunnermeier also has a solution: Regulators should create a segment of the market where only humans are allowed to trade.

I can only hope this would look like the trading floors I used to work on: Traders shouting prices. Sales people shouting orders. Phones ringing. Keyboards clacking. Palms perspiring.

It was glorious.

There might not be a lot to shout about, though, because the regular, AI-dominated market would continue to do most of the business.

Brunnermeier acknowledges that: “In normal times, the slow venue might not see much activity.”

But like the contingency site my bank used to maintain at such expense, it would be a valuable insurance policy to have.

“The non-AI segment serves as a fallback,” Brunnermeier says.

Then, if the market were to crash for no obvious reason, regulators could “temporarily open the boundary between segments so that essential activity migrates to the segment that remains switched on.”

That would be the human segment. Where orders are processed by people. On computers from the 1990s.

They wouldn’t be processed as nano-second quickly as they are now. But we would at least understand what was going on.

“The underlying logic is to sacrifice efficiency in normal times in order to arrest cascades in crisis times,” Brunnermeier explains. 

He believes we should start trading this way soon, lest we forget how to do it: “A human venue keeps trading expertise alive; without it, the fallback would atrophy as dependence on AI deepens.”

Just as GPS has made us forget how to read a map, AI agents could make us forget how to trade.

(I think the new market segments should have a dress code, too, lest we forget how to tie a tie or shine a shoe.)

As fanciful as human-only trading sounds, it’s not without precedent.

The same logic explains why governments have been encouraging the use of physical cash. Even if it’s become inconvenient, cash provides an important fallback in the case of a cyberattack that shuts down the internet-enabled payment systems we’ve become so reliant on.

Brunnermeier concludes: “Our current financial institutions are not built for a world with AI agents that possess asymmetric understanding.”

In that world — which seems inevitable — a market segment where people trade like it’s 1999 will be a valuable fallback to have.

I still remember how to do it.

(Kind of.)

(And not for much longer.)

— Byron Gilliam

_{Brought to you by:}

Avalanche Summit NYC returns September 16-17, bringing together the institutions, enterprises, investors, and builders turning blockchain technology into real business outcomes.

From tokenized markets and institutional finance to payments and consumer applications, the Summit will explore how production-ready infrastructure is enabling faster settlement, lower costs, and entirely new products and revenue streams.

Use promo code BLOCKWORKS15 for 15% off!