AI is outperforming human CEOs, but it’s also getting fired faster

• Artificial intelligence largely outperformed human executives in an experiment conducted by researchers at the University of Cambridge.
• But the AI ​​was not as good at making decisions in unexpected “black swan” events.
• This led to the AI ​​being fired by a virtual board of directors faster than humans.

Can we throw CEOs out of their offices and replace them with AI? A new one experiment from the University of Cambridge suggests the answer is no.

The AI ​​actually outperformed human CEOs in most situations in a real business management simulation that pitted humans against computers, but there was one thing the AI ​​couldn’t handle, according to the experiment: so-called black swan events , such as a pandemic.

Because of this, the AI ​​was fired faster by a virtual board than its human counterparts, which navigated unexpected situations better.

Hamza Mudassir, one of the researchers behind the experiment, told Business Insider that the AI ​​outperformed human participants on most metrics, including profitability, product design, inventory management and price optimization — but that its performance wasn’t enough to save it from upon receipt. boot.

“He didn’t do well in terms of surviving in the C-suite just because he wasn’t very good at handling sudden changes or changes that require a new way of thinking,” Mudassir said.

Cambridge researchers conducted the experiment from February to July and included 344 people, some of whom were senior executives at a bank in South Asia. It also included students. And the last participant wasn’t a person at all, but rather OpenAI’s GPT-4o, the Large Language Model, or LLM.

Participants played a game designed to simulate real-world situations in which CEOs must make decisions. The game made him take on the role of CEO of a car company. It was designed by Cambridge researchers’ ed-tech startup Strategize.inc.

“The goal of the game was simple—survive as long as possible without getting fired by a virtual board while maximizing market capitalization,” the researchers wrote in the Harvard Business Review.

Mudassir told BI that the LLMs were excellent at analyzing data, recognizing patterns and making inferences. For example, when it came to designing a car based on factors such as available parts, price, consumer preferences and demand, there were 250,000 combinations that participants could come up with. The machines the AI ​​put together were significantly better than the ones humans invented, he said.

In part, he said, that’s because people have biases and personal taste in things like the shape of a car; for AI, it was simply a “puzzle of figuring out the most optimal value for what the customer wanted,” Mudassir said.

But that doesn’t mean AI was the optimal CEO. When a “black swan” event occurred, the bot was unable to resolve it as quickly – or as well – as the human directors and students. When there was a major change in market conditions, such as introducing a pandemic into the mix, the model failed, he said.

“How do you react to COVID if you are dealing with it for the first time? Many people and many CEOs have different strategies,” Mudassir said. “In this case, he didn’t have enough information about how to react in time to prevent himself from being fired,” he said of the AI.

So CEOs can rest easy for now. The researchers say that while AI’s performance as a virtual boss of a company was impressive, it wasn’t good enough to replace a human. However, AI has worked so well that it cannot be ignored in corporate strategy, Mudassir said.

In the future, Mudassir said LLMs could be tuned specifically for a particular company with real-time data, in which case they would likely perform even better than the AI ​​did in the experiment.

He said perhaps the best use case for artificial intelligence would be in business “war games” – or using multiple LLMs to represent different stakeholders, such as competitors, parliamentarians or activists, and then testing how certain decisions will actually play out. Some of these could, in theory, replace the work of strategy and management consultants, who often make recommendations to a CEO based on their own analysis of certain outcomes in certain situations.