FLI’s Pause Letter, Six Months Later

• FLI Executive Director Anthony Aguirre had an op-ed published in The Hill, in which he discussed why it’s now more important than ever to approach AI development with immense caution. He was also interviewed in Axios.

• FLI President Max Tegmark spoke to Semafor, The Guardian, Wired, and MIT Technology Review about the letter’s expiration, and also participated in a Reuters panel on the anniversary.

• Jaan Tallinn, Skype and FLI co-founder, was interviewed on a TRT World segment about pausing AI development.

• Mark Brakel, FLI’s Director of Policy, was interviewed about the pause anniversary in Euronews, Le Figaro, and Les Echos.

• Landon Klein, FLI’s Director of U.S. Policy, spoke to The Hill in advance of the pause anniversary.

Our Recommendations for the UK AI Safety Summit

The UK AI Safety Summit from November 1-2 presents the biggest opportunity yet for global coordination on AI regulation - if organizers can prevent it from falling into the following three traps as identified by FLI:

1. Letting tech companies write the legislation.

2. Turning this into a geopolitical contest of the West versus China.

3. Focusing only on existential threats, or conversely, only on current events.

We propose the following three high-level objectives for the Summit:

1. Establish a common understanding of the severity and urgency of AI risks.

2. Make the global nature of the AI challenge explicit, recognizing that all of humanity has a stake in this issue, and that some solutions require a unified global response.

3. Embrace the need for urgent government intervention, including through hard law.

First published in Politico Pro, our full list of recommendations for the Summit, including a post-Summit road map, can be found here.

Have You Listened to ‘Imagine A World’ Yet?

‘Imagine A World’ is our new podcast series exploring eight different positive visions of what our reality could look like in 2045.

From greater inclusivity, to new democratic models, to solutions for the climate crisis, these worlds capture a plausible future in which human lives are transformed for the better by technology such as advanced AI, rather than disempowered by it.

The first five of eight episodes are now available here.

Tune in on YouTube, Spotify, or Apple Podcasts, and be sure to like and subscribe! We’ll be releasing a new episode every week for the next three weeks.

Apply Now for our Vitalik Buterin PhD and Postdoctoral Fellowships!

A reminder that applications are open for our PhD and postdoctoral fellowships focused on AI existential safety research. The fellowship is global and open to all regardless of nationality or background; we are seeking a diverse applicant pool. All Fellows will receive applicable tuition and fees, as well as a stipend and research/travel fund. Please share this excellent opportunity widely.

Current or future PhD student intending to research AI existential safety? The deadline for PhD fellowship applications is November 16, 2023 at 11:59 pm ET.

Current or future postdoc working on AI existential safety research? The deadline for postdoctoral fellowship applications is January 2, 2024 at 11:59 pm ET.

Governance and Policy Updates

AI policy:

• As encouraged by FLI, the British Government has confirmed that China has been invited to the UK AI Safety Summit.

• Reflective of the greater shift happening in conversations about AI safety, European Commission President Ursula von der Leyen’s 2023 State of the Union address referred to possible extinction risk from AI - and our “narrowing window of opportunity” to guide AI development responsibly.

• The first meeting of U.S. Senate Majority Leader Chuck Schumer’s AI Insight Forum was held September 13th, with over a dozen tech executives convened to help inform lawmakers on AI. FLI’s Landon Klein was interviewed about it on NBC News NOW - watch it here at the 7:43 mark.

Biosecurity:

• The 78th session of the UN General Assembly has wrapped up. Notably, this session included the Assembly’s first ever high-level meeting on pandemic prevention, preparedness, and response, resulting in the adoption of a political declaration to strengthen international collaboration and coordination on the topic.

Updates from FLI

• FLI’s Anna Hehir spoke to The Hill about the “Pandora’s box” of AI in military use.

• FLI President Max Tegmark was honoured as one of TIME’s 100 Most Influential People in AI 2023.

• FLI’s Carlos Ignacio Gutierrez, Risto Uuk, and Anthony Aguirre, along with Claire C. Boine and Matija Franklin, published an article proposing a functional definition of the term “general purpose AI systems”.

• We’ve built a compliance checker tool on our dedicated EU AI Act website, for “providers” and “deployers” of AI systems to assess which legal obligations their system may be subject to once the Act comes into effect.

• We’ve created a compilation of introductory resources on AI risk for those who are new to conversations around AI and AI risks.

• On the FLI podcast, host Gus Docker spoke to Tom Davidson from Open Philanthropy about AI takeoff speeds. He also interviewed Johannes Ackva from Founders Pledge about the main causes of, and most promising solutions for, climate change.

New Research: Provably safe AI systems

A path to controllable AGI?: Steve Omohundro and FLI’s Max Tegmark published a paper outlining how mathematical proofs of safety, e.g., written into “proof-carrying code”, are powerful, accessible tools for developing safe artificial general intelligences (AGIs).

What We’re Reading

• AGI Safety Weekly: A new weekly publication tracks and shares weekly updates on AI Safety - check it out and subscribe below:

• New AIPI polling: More polling has recently come out from the AI Policy Institute (AIPI), showing that Americans don’t want superintelligent AI, and favour restricting AI development - even at the risk of China outpacing the U.S.

• Evaluating international AI institutions: Matthijs M. Maas and José Jaime Villalobos present a literature review of models, examples, and proposals for governance through international AI institutions.

• Lessons from Oppenheimer: In TIME Magazine, Conjecture’s Andrea Miotti makes a compelling case for governing AI development like was done with nuclear development.

• What we’re watching: This video from 80,000 Hours captures the existential risks presented by AI, and outlines what we can do to mitigate them.

Hindsight is 20/20

On September 26th 1983, a USSR army officer named Stanislav Petrov may have saved the world.

This terrifying near-disaster is just one of many examples highlighting why we must keep under human control decisions like Petrov had to make.

Our recent short film, "Artificial Escalation", explores what can happen if we entrust such consequential decisions to algorithms. Let us heed these warnings from the past and work to safeguard our future.