The Prisoner's Dilemma: How Game Theory Explains Cooperation and Conflict

Veritasium

Summary:

The Prisoner's Dilemma, a famous game theory problem, emerged during the Cold War to model nuclear arms races between the US and the Soviet Union.
The game involves two players choosing to cooperate or defect, with payoffs structured such that individual rational choice leads to a sub-optimal collective outcome.
In the context of the Cold War, this led to a massive nuclear arms buildup by both superpowers, despite both being worse off.
Robert Axelrod's computer tournaments in the 1980s tested various strategies for the iterated (repeated) Prisoner's Dilemma.
The simplest strategy, "Tit for Tat" (start by cooperating, then mirror the opponent's last move), won the first tournament.
Successful strategies were identified as being nice (never defect first), forgiving (don't hold grudges), and retaliatory (strike back immediately if provoked).
In a noisy environment, a "Generous Tit for Tat" (more forgiving) performed best, highlighting the importance of breaking out of mutual defection cycles.
The Prisoner's Dilemma helps understand cooperation and conflict in nature (e.g., impala grooming) and human society, showing that self-interest can still lead to cooperation.

The Origins of the Prisoner's Dilemma [0:30]

The Cold War and the Nuclear Arms Race [0:34]
- In September 1949, an American weather plane detected radioactive material over Japan, confirming the Soviet Union had developed a nuclear bomb [0:34].
- This news ended American military supremacy and raised fears of nuclear war [1:20].
- John von Neumann, a founder of game theory, suggested a preemptive strike against the Soviets [1:51].
The RAND Corporation and Game Theory [2:11]
- In 1950, the RAND Corporation studied how to address the nuclear weapons threat [2:11].
- Mathematicians Merrill Flood and Melvin Dresher invented a game that resembled the US-Soviet conflict, later known as the Prisoner's Dilemma [2:21].

Playing the Prisoner's Dilemma [2:36]

Game Mechanics [2:36]
- Two players, you and an opponent, each choose to Cooperate (C) or Defect (D) simultaneously.
- If both Cooperate (C, C): Each player gets 3 coins [2:47].
- If one Cooperates and one Defects (C, D): The defector gets 5 coins, the cooperator gets 0 coins [2:51].
- If both Defect (D, D): Each player gets 1 coin [2:58].
Rational Choice Leads to Sub-optimal Outcome [3:07]
- If your opponent cooperates, you are better off defecting (5 coins vs. 3 coins) [3:10].
- If your opponent defects, you are better off defecting (1 coin vs. 0 coins) [3:21].
- Therefore, a purely rational player will always choose to defect, regardless of the opponent's choice [3:34].
- This leads to both players defecting, resulting in 1 coin each, which is worse than if both had cooperated (3 coins each) [3:44].
Real-World Impact on Nuclear Armaments [3:54]
- The US and Soviet Union, acting rationally, developed tens of thousands of nuclear weapons [3:54].
- This left both countries in a worse-off state, having spent trillions of dollars on unusable weapons [4:07].
Natural Examples: Impalas and Grooming [4:42]
- Impalas need mutual grooming to remove ticks, but grooming costs time and resources [5:04].
- A single interaction would lead to defection, as it's individually rational not to groom [5:34].
The Iterated Prisoner's Dilemma [5:48]
- When the game is played repeatedly, the strategy changes because past actions can influence future ones [5:57].

The Tournament [6:15]

Axelrod's Computer Tournament [1980] [6:18]
- Robert Axelrod invited game theorists to submit computer programs (strategies) to play the iterated Prisoner's Dilemma [6:18].
- Each strategy played against every other strategy and itself for 200 rounds, repeated 5 times [6:43].
Sample Strategies [7:07]
- Sample: Cooperates initially, defects if opponent defects twice in a row, then cooperates again [7:07].
- Friedman: Starts cooperating, but if opponent defects once, it defects for the rest of the game (unforgiving) [7:33].
- Joss: Starts cooperating, copies opponent's last move, but occasionally defects randomly (sneaky) [7:44].
- Graaskamp: Probes opponent's strategy by defecting in the 50th round to see if it can exploit weaknesses [7:57].
- Name Withheld: The most complex strategy with 77 lines of code [8:13].
Tournament Results [8:18]
- The simplest program, "Tit for Tat" (submitted by Anatol Rapoport), won the tournament [8:26].
- Tit for Tat Strategy: Cooperates on the first move, then copies the opponent's last move [8:29].
- Against Friedman (unforgiving), both Tit for Tat and Friedman cooperated for perfect scores [8:46].
- Against Joss (sneaky), Tit for Tat started cooperating, but Joss's defection led to mutual back-and-forth defections, resulting in poor scores for both [8:55].
Qualities of Successful Strategies [10:08]
- Axelrod identified four qualities shared by the best-performing strategies:
  1. Nice [10:15]: Never be the first to defect (e.g., Tit for Tat) [10:20].
  - The top 8 strategies were all nice [10:37].
  1. Forgiving [10:48]: Retaliates but doesn't hold a grudge; forgets past defections after one retaliation (e.g., Tit for Tat) [10:51].
  - Friedman is unforgiving; it defects forever after one defection [11:05].
  1. Retaliatory [14:39]: If your opponent defects, strike back immediately; don't be a pushover (e.g., Tit for Tat is hard to take advantage of) [14:42].
  - "Always Cooperate" is a pushover strategy and performs poorly [14:47].
  1. Clear [14:58]: Predictable behavior; opponents can understand your strategy and adapt [15:01].
  - Programs that were too opaque or random struggled to establish trust [15:02].

Emergence [15:55]

Second Tournament and The Evolution of Cooperation [12:20]
- Axelrod announced a second tournament, identical to the first except for variable rounds per game [12:20].
- Knowledge of the final round in a fixed-round game encourages defection through backward induction [12:28].
- With unknown ending points, cooperation becomes rational [12:58].
- Tit for Tat again won this tournament, and nice strategies generally performed better [14:17].
Ecological Simulation [16:35]
- Axelrod ran simulations where successful strategies multiplied, and unsuccessful ones died out [16:35].
- Only nice strategies survived and dominated the population [17:15].
- Harrington, a nasty strategy, initially thrived by exploiting nice strategies but then declined as its prey became extinct [16:51].
Cooperation Emerges from Self-Interest [17:39]
- A small cluster of cooperating Tit for Tat players in a nasty environment can emerge and spread, eventually dominating the population [17:46].
- This demonstrates that cooperation can emerge among self-interested players, even without altruism or conscious thought [18:14].
- The "Tit for Tat" principle (an eye for an eye) explains cooperation in various natural systems (impalas, fish, monkeys) [18:51].

Does This Apply to the Real World? [19:26]

The Importance of Noise [19:38]
- Real-world interactions involve "noise" or errors (e.g., trying to cooperate but it's perceived as defection) [19:43].
- Stanislav Petrov's decision to dismiss a false missile alarm in 1983 prevented nuclear war, highlighting the impact of a single error [19:50].
Generous Tit for Tat [21:31]
- A "Generous Tit for Tat" (more forgiving, retaliating 9 out of 10 times instead of every time) can break out of mutual defection cycles caused by noise [21:31].
- Axelrod found that in noisy environments, generous Tit for Tat strategies outperformed strict Tit for Tat [21:49].
Beyond Zero-Sum Games [22:27]
- Most of life is not a zero-sum game (one person's gain is another's loss) [22:44].
- Instead, the world is a non-zero-sum game where win-win situations are possible [22:50].
Disarmament Through Gradual Cooperation [23:08]
- The US and Soviet Union eventually reduced nuclear arsenals by disarming small numbers of nukes each year, inspecting each other, and repeating the process [23:08].
- This gradual, verifiable approach of mutual cooperation (similar to Generous Tit for Tat) allowed them to resolve conflict without turning it into a single Prisoner's Dilemma [23:23].