Elon Musk has just launched ‘Colossus’ – the world’s largest Nvidia GPU supercomputer

Founded only last July, the latest artificial intelligence company, xAI, just brought a new supercomputer called Colossus online over Labor Day weekend, designed to train its large language model (LLM) known as Grok, a rival to the better known Open AI. GPT-4.

While Grok is limited to paying subscribers to Musk’s X social media platform, many Tesla experts speculate that it will eventually form the artificial intelligence that will power the electric vehicle maker’s Optimus humanoid robot.

Musk estimates that this strategic beacon project could bring Tesla a trillion dollars in annual profit.

Located in Tennessee, xAI’s new data center houses 100,000 Nvidia Hopper H100 benchmark processors, more than any other individual AI compute cluster.

“From start to finish, it was done in 122 days,” Musk wrote, calling Colossus “the world’s most powerful AI training system.”

this weekend, @xAI The team brought our Colossus 100k H100 training cluster online. From start to finish, it was done in 122 days.

Colossus is the world’s most powerful AI training system. Moreover, it will double its size to 200k (50k H200s) in a few months.

Excellent…

— Elon Musk (@elonmusk) September 2, 2024

It doesn’t end there for xAI either: Musk predicts it will double Colossus’ computing power in a few months, once it can procure 50,000 chips from Nvidia’s new, more advanced H200 series, which are about twice as powerful.

Musk and xAI did not respond to requests from wealth for comment.

Built to train Grok-3, potentially the next leader in AI models

The pace at which Colossus has been installed is lightning fast, given that xAI only chose its Memphis site in June.

Moreover, several powerful tech firms, including Microsoft, Google and Amazon, are competing to acquire Nvidia’s Hopper series AI chips in the current AI gold rush alongside Musk.

But the AI ​​entrepreneur is a valued Nvidia customer and has committed to spending $3 billion to $4 billion this year on CEO Jensen Huang’s hardware — just for Tesla.

What’s more, xAI enjoyed a head start by helping supply Tesla with AI chips already delivered to the electric vehicle maker.

The Memphis cluster will train Musk’s third-generation Grok.

“We hope to launch Grok-3 by December, and Grok-3 should be the most powerful AI in the world at that time,” he told conservative podcaster Jordan Peterson in July.

An early beta version of the Grok-2 was just released to users last month.

It was trained on only about 15,000 Nvidia H100s GPUs, but by some standards it is already among the most capable large-language AI models according to competitive chatbot rankings.

The nearly sevenfold increase in the number of GPUs suggests that Musk has no intention of abandoning the race to develop general artificial intelligence to OpenAI, which he helped found in late 2015 after growing concerned that Google was dominating the technology .

Musk subsequently fell out with CEO Sam Altman and is now suing the company a second time.

To help even the odds, xAI raised $6 billion in a Series B funding round in May, with the help of venture capitalists such as Andreessen Horowitz and Sequoia Capital, as well as deep-pocketed investors such as Fidelity and Saudi Prince Alwaleed bin Talal’s Kingdom Holding.

Tesla could be the next company to invest in Musk’s xAI

Musk also indicated that he would propose to Tesla’s board a vote on potentially investing $5 billion in xAI as well, a move welcomed by a number of shareholders.

Of the roughly $10 billion in AI spending we said Tesla would do this year, about half is internal, primarily the AI ​​inference computer Tesla designed and the sensors in all our cars, plus Dojo.

For building AI training superclusters, NVidia hardware is about…

— Elon Musk (@elonmusk) June 4, 2024

But xAI’s supercomputer cluster has caused alarm in Memphis, given the extreme haste with which city officials agreed to the project, which is bringing economic activity back to a part of the city that was last home to an Electrolux white goods factory.

A primary concern is the strain it will create on the city’s resources. MLGW municipality officials estimate that Colossus it requires up to 1 million gallons of water per day to cool the servers and will consume up to 150 megawatts of power.

But Musk is someone who only thinks big, and anything worth doing is worth doing quickly, otherwise you risk falling behind the competition.

Speaking to Lex Fridman after the podcaster toured xAI’s fast-growing operations, Musk said speed is a key part of his five-step management process.

“Anything can be accelerated. As fast as you think it can be done,” he said last month, “it can be done faster.”

Recommended reading:
In our new special edition, a Wall Street legend gets a makeover, a tale of crypto iniquity, poultry misbehavior and more.
Read the stories.