The California legislature just passed the AI ​​safety bill that has divided Silicon Valley

California Governor Gavin Newsom now has a month to veto the AI ​​bill or sign it.

AP Photo/Rich Pedroncelli

• California lawmakers just passed a controversial AI safety bill, SB 1047.
• The bill imposes safeguards on companies that spend $100 million or more on AI training.
• Silicon Valley is divided: OpenAI opposed the bill, Elon Musk supported it.

California lawmakers have voted to pass an artificial intelligence safety bill that has sent ripples through Silicon Valley.

The bill, SB 1047, was introduced seven months ago by Sen. Scott Weiner and passed the state assembly on Wednesday.

It’s now on Gov. Gavin Newsom’s desk, and he has until Sept. 30 to decide whether to sign it into law or reject it. He has not publicly signaled his position.

The bill aims to force companies that spend $100 million or more on training AI models to develop safeguards so their technology can’t be used to harm society, such as creating dangerous weapons or cyber attacks.

The measures include requiring companies operating in California to report any safety incidents to the government. protecting whistleblowers and allowing third parties to test their designs for safety. If necessary, they would also push the companies implement a complete shutdown.

Teams of choice

The bill drew a line through Silicon Valley, and some of tech’s biggest figures took part.

Jason Kwon, OpenAI’s chief strategy officer, warned in a letter to Sen. Wiener on Wednesday that the bill could stifle progress and drive companies out of California.

The maker of ChatGPT joined the meta heavy in its lobby against the bill. Meta said the bill could discourage the open-source movement by exposing developers to significant legal liability.

Former OpenAI employees said the company’s opposition to the bill was disappointing — but in line with its recent path.

“We joined OpenAI because we wanted to ensure the safety of the incredibly powerful AI systems the company is developing,” former OpenAI researchers William Saunders and Daniel Kokotajlo wrote in the letter. “But we resigned from OpenAI because we lost confidence that it would develop its AI systems safely, honestly and responsibly.”

Elon Musk also supported the bill.

In a social media post on Monday, Musk, who founded artificial intelligence company xAI last year and has a long-standing rivalry with OpenAI’s Altman, said that while it was “a tough call and it will upset him some people,” he believes “California should probably pass the SB 1047 AI safety bill.”

“For over 20 years, I have been an advocate for regulating AI, just as we regulate any product/technology that poses a potential risk to the public,” Musk wrote on X.

Amazon-backed Anthropic appeared to have switched sides in the middle of the debate after the bill was amended.

In a letter sent to California Governor Gavin Newsom last week, Anthropic CEO Dario Amodei said “the benefits of the bill likely outweigh the costs.” However, he added that “we are not sure about this and there are still some aspects of the bill that we find worrying or ambiguous”.

Amodei said the bill now “appears to us to be halfway between our suggested version and the original bill.”