SB 1047 Heads to Governor's Desk

California’s proposed SB 1047, the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act, is entering the final stages of the legislative process. Having passed the Assembly and Senate votes, all that’s needed now is Governor Gavin Newsom’s signature.

Numerous AI experts, tech industry figures, and the California public have weighed in with supportive comments, encouraging legislators to pass the bill and Gov. Newsom to sign off on it. Among them:

• AI pioneer and Turing Award winner Yoshua Bengio’s letter, and joint podcast with the bill’s author, State Sen. Scott Wiener

• Legal scholar Lawrence Lessig’s article explaining that, despite what they say, Big Tech wants to avoid regulation

• The open letter jointly authored by Bengio, Lessig, and fellow “AI godfathers” Geoffrey Hinton and Stuart Russell

• xAI founder Elon Musk’s endorsement:

• Notion co-founder Simon Last’s op-ed

• The Los Angeles Times’ editorial board’s endorsement

• Anthropic’s positive comments in a letter weighing the bill’s pros and cons, concluding: “In our assessment the new SB 1047 is substantially improved, to the point where we believe its benefits likely outweigh its costs.”

• Polling showing that a vast majority - 77% - of the California public support the rules SB 1047 puts forth

Contrasting the commitment to safety and desire for regulation that they’ve previously expressed, OpenAI is opposing the bill - much to the disappointment “but not [surprise]” of former OpenAI employees, who left due to safety concerns. You can read the letter penned by those whistleblowers in response to OpenAI’s opposition here.

Unfortunately, much of the opposition has centered around misleading and even inaccurate information about the bill. We’ve put together a document dispelling these myths and answering FAQs; I also encourage you to check out these bill summaries from Zvi Mowshowitz and SafeSecureAI respectively.

OpenAI Safety Team “Gutted”

OpenAI’s safety team has yet again made headlines - this time, with reports emerging that the team has effectively been “gutted” after a mass exodus.

Former employee and whistleblower Daniel Kokotajlo told Fortune that almost half of the team working on AGI safety has left since the start of the year. Although Daniel couldn’t speak to individual employees’ reasons for leaving, he shared that, “People who are primarily focused on thinking about AGI safety and preparedness are being increasingly marginalized” at the company.

See the infographic below for an overview of safety controversies at the company in just the past year:

More Progress Towards a Treaty

Updates from FLI

• FLI’s EU Policy Lead Risto Uuk, along with researchers from MIT, University of Queensland, and Harmony Intelligence, have created the first living AI risk repository, categorizing over 700 AI risks.

• AI Safety Summit Lead Imane Bello is shortly releasing the first edition of the “AI Action Summit Newsletter”. The biweekly newsletter will offer updates and analyses of the developments and political context leading up to the 2025 French AI Action Summit. Subscribe here and look out for it hitting your inbox shortly!

• Director of Policy Mark Brakel launched "Not Another Big Tech Stack”, a monthly newsletter offering his perspectives and takes on AI policy. Subscribe and read the latest edition here.

• FLI is expanding our policy team! Learn more and apply for our multiple U.S. policy team openings here, and our Multilateral Governance Lead role here. Apply by September 8th.

• FLI Executive Director Anthony Aguirre had an op-ed published in the Washington Post, explaining why OpenAI CEO Sam Altman’s advocacy for a race to AGI is dangerous: “There’s no way for humanity to win an AI arms race”.

• Anthony also was quoted in this Semafor newsletter on SB 1047.

• FLI President Max Tegmark joined the Rich on Tech show for a segment on SB 1047 and AI risk.

• Risto was interviewed on the EU AI Act Podcast about general purpose AI and systemic risks.

• On the FLI podcast, XPRIZE CEO Anousheh Ansari joined host Gus Docker for an episode of how XPRIZE uses "incentivized competition" to spur innovation aimed at addressing the greatest challenges humanity faces.

• Niskanen Center Senior Fellow Samuel Hammond also joined the podcast to discuss if AI is progressing or plateauing, and how governments should respond.

What We’re Reading

• Code Red on AI Bio-Risk: As covered in TIME, public health experts published a new paper in Science, calling for AI regulation to mitigate the near-future risk of advanced AI models engineering critical biosecurity threats.

• SamA Power Play: In The Guardian, AI expert Gary Marcus - who spoke alongside Sam Altman in front of the US Senate on AI oversight - outlines the causes for concern with the OpenAI CEO having as much power as he does.

• What We’re Watching: Kurzgesagt - In a Nutshell have a new video on the “race” to superintelligence, and its huge potential consequences for humanity:

• What We’re Watching: Also new from Digital Engine is a video on AI risks, particularly in the context of AI in warfare: