Will A.I. Actually Want to Kill Humanity? | The Ezra Klein Show

What is discussed, and when

1. 0:00 – 2:15: humans, ant heap, extinction level event, oak trees

Notable quotes from "Will A.I. Actually Want to Kill Humanity? | The Ezra Klein Show"

• Wants is not us living happily ever after so we're dead. Like it's not that humans have been trying to cause side effects. When we build a skyscraper on top of where there used to be an ant heap, we're not trying to kill the ants.

  about humans · at 0:22

• Rough balance? Not a balance where there's never a problem, but a balance where there's not an extinction level event from a super smart AI that deviously plots to conduct a…

  about extinction level event · at 1:23

Who and what is mentioned in this episode?

Products & tech

• skyscraper — Product · 1 mention · first at 0:28
  After so we're dead. Like it's not that humans have been trying to cause side effects. When we build a skyscraper on top of where there used to be an ant heap, we're not trying to kill the ants.

Concepts & topics

• humans — Concept · 2 mentions · first at 0:26
• side effects — Concept · 1 mention · first at 0:26
  Wants is not us living happily ever after so we're dead. Like it's not that humans have been trying to cause side effects. When we build a skyscraper on top of where there used to be an ant heap, we're not trying to kill the ants.
• oak trees — Concept · 1 mention · first at 1:02
  At any point in history. Right? So, it's not like the relationship between us and ants or us and oak trees. It's more like the relationship between, I don't know, us and us or…
• extinction level event — Concept · 1 mention · first at 1:27
• super smart AI — Concept · 1 mention · first at 1:29
  There's never a problem, but a balance where there's not an extinction level event from a super smart AI that deviously plots to conduct a strategy to destroy us. I mean we've…
• training set — Concept · 1 mention · first at 1:51
  Direct answer there is something like one the relationship between what you optimize for that the training set you optimize over and what the entity the organism the AI ends up wanting has been and will be weird and twisty.
• AI alignment — Concept · 1 mention
• genie — Concept · 1 mention
• tools — Concept · 1 mention

Places & events

• ant heap — Place · 1 mention · first at 0:31
  Humans have been trying to cause side effects. When we build a skyscraper on top of where there used to be an ant heap, we're not trying to kill the ants.

Full transcript with timestamps

0:00 Your book is not called if anyone builds it there is a 1 to4% chance everybody dies. You you believe that the misalignment becomes catastrophic.

0:10 Yeah. Why do you think that is so likely? Um that's just like the the straight line extrapolation from it gets what it most wants and the thing that it most wants is not us living happily ever after so we're dead. Like it's not that humans have been trying to cause side effects. When we build a skyscraper on top of where there used to be an ant heap, we're not trying to kill the ants. We're trying to build the size skyscraper, but we are more dangerous to the small creatures of the earth than we used to be just because we're doing larger things. Humans were not designed to care about ants.

0:48 Humans were designed to care about humans. And for all of our flaws, and there are many, there are today more human beings than there have ever been at any point in history. Right? So, it's not like the relationship between us and ants or us and oak trees. It's more like the relationship between, I don't know, us and us or us and tools or us and dogs or something. It's, you know, maybe it maybe the metaphors begin to break down. Why don't you think in the back and forth of that relationship uh there's the capacity to maintain a a rough balance? Not a a balance where there's never a problem, but a balance where there's not an extinction level event from a super smart AI that deviously plots to conduct a strategy to destroy us. I mean we've already observed some amount of like slightly devious plotting in the existing systems but leaving that aside um the the more direct answer there is something like

1:48 one the relationship between what you optimize for that the training set you optimize over and what the entity the organism the AI ends up wanting has been and will be weird and twisty. It's not direct. It's not like making a wish to a genie inside a fantasy story. And second, ending up slightly off is predictably enough to kill everyone.

Questions about this episode

What is discussed in "Will A.I. Actually Want to Kill Humanity? | The Ezra Klein Show"?

In order: 0:00 humans, ant heap, extinction level event, oak trees.

What is mentioned in this The Ezra Klein Show episode?

11 people, works and ideas are mentioned, including humans, side effects, skyscraper, ant heap, oak trees, extinction level event. Each one on this page links to its own page with every episode that mentions it.

How long is "Will A.I. Actually Want to Kill Humanity? | The Ezra Klein Show"?

The episode runs 2 min; the transcript is about 375 words in 4 timestamped sections.

More from The Ezra Klein Show

• All analyzed episodes
• Books mentioned on The Ezra Klein Show
• People mentioned on The Ezra Klein Show
• Sponsors mentioned on The Ezra Klein Show
• Topics mentioned on The Ezra Klein Show