How Software Complication Threatens Civilization and What Programmers Can Do to Simplify and Improve It

DevGAMM

The Cyclical Nature of Technological Progress and Decline [00:00:00]

• Historical Examples of Rapid Progress:
  • The Space Race: The speaker begins by highlighting the rapid advancements in space technology in the mid-20th century.
    • Sputnik's Launch [1957]: The Soviet Union's launch of Sputnik spurred immense effort in the USA to catch up, leading to a successful launch within 120 days. [00:00:13]
    • Yuri Gagarin's Orbit [1961]: One month after Gagarin's orbit, President Kennedy committed the US to landing a man on the moon before the decade was out. [00:00:53]
    • Apollo 11 [1969]: The successful Apollo 11 mission, achieved within 12 years from almost no space program, demonstrated an incredible feat of coordinated engineering and scientific advancement. [00:01:53]
      • Visuals from a documentary of Apollo 11 preparation emphasize the massive scale of the undertaking, including giant crawler transporters. [00:02:31]

        

  • Post-Apollo Decline: The Space Shuttle, despite its innovative reusability, proved expensive and unreliable, leading to its retirement. [00:03:37]
    • This marked a period of decline in US space capabilities, forcing reliance on other nations for orbital access, leading to a pervasive sense of "Isn't it a shame?" about the lost vision of a spacefaring future. [00:04:02]
  • Modern Resurgence (SpaceX): Elon Musk initiated a private space company despite no prior rocket experience, driven by the belief that technological progress is not inevitable and requires hard work. [00:04:42]
    • Musk noted the historical trend of technological degradation, citing ancient Egypt forgetting pyramid construction and Romans forgetting aqueduct building. [00:06:01]
    • Today, SpaceX is achieving reusable rockets and discussing new moon missions, showing a reversal of the previous decline. [00:06:23]

      

• Historical Examples of Lost Technology:
  • The Lycurgus Cup (300 AD): An ancient Roman glass goblet that changes color from green to red depending on the light source. [00:06:51]
    • This effect is due to nanometer-sized particles of silver and gold embedded in the glass, a nanotechnology far ahead of its time, whose knowledge was lost after the fall of the Roman Empire and only rediscovered around 1990. [00:07:36]
    • The complexity and craftsmanship involved indicate a sophisticated understanding of materials science, not mere accident. [00:08:03]

      

  • Byzantine Flamethrowers: The Byzantine Empire possessed advanced "Greek Fire" flamethrowers capable of projecting napalm-like substances from pressurized vessels on ships. [00:09:16]
    • This powerful military secret was crucial for defending Constantinople but eventually faded from knowledge. [00:09:38]

      

  • Antikythera Mechanism (200-80 BC): A complex mechanical calendar discovered from a sunken Greek ship. [00:10:00]
    • This intricate device used numerous gears to track celestial movements, predict eclipses, and even schedule Olympic Games. [00:10:21]
    • Its sophistication implies a developed scientific process in ancient Greece, yet this knowledge was completely lost. [00:12:00]

      

• The Principle of Degradation:
  • Technology tends to degrade over time, contrary to the modern assumption of continuous improvement. [00:12:31]
  • Knowledge is lost due to failures in generational transfer, either through sudden civilizational collapse or gradual institutional decay. [00:15:10]
  • The Late Bronze Age collapse serves as an example where a sophisticated network of civilizations and trade (dependent on bronze, made from widely sourced copper and tin) fell apart due to environmental stressors, leading to the loss of entire cities, languages, and cultures because crucial skills were not widespread. [00:15:41]

    

The Decline of Modern Software [00:18:12]

• Thesis: Software is currently in decline, a "soft decline" that could lead to a "hard decline" because civilization increasingly depends on it (e.g., communication, vehicles). [00:18:12]
  • Collapses often appear slow from the inside, meaning current issues might be early signs of a larger problem. [00:18:56]

    

• Software Riding on Hardware Improvements:
  • The perceived advancement of software is largely due to immense improvements in hardware speed and capability over decades. [00:19:48]
  • Machine learning algorithms and face filters are impressive due to the quantity of computation enabled by fast hardware, rather than fundamental software breakthroughs. [00:20:41]
  • The complex underlying software (e.g., loading bitmaps, handling user input) that supports these impressive but simpler algorithms is actually where the degradation lies. [00:21:40]
• Evidence of Declining Software Quality (Personal Anecdotes): The speaker recounts numerous instances of everyday software failures experienced within just a few days:
  • Command-line console errors (Lua script issue). [00:23:19]
  • Emacs file reloading bug, causing truncated content. [00:23:31]
  • Gmail formatting bugs, replicating old Microsoft Word issues. [00:23:47]

    

  • Visual Studio failing to save simple command-line arguments. [00:24:26]

    

  • Game store download errors (Epic Games Store) and Steam's black installation window. [00:25:12]

    

  • In-game full-screen display issues and HUD bugs (e.g., "undefined" player in Counter-Strike). [00:25:36]

    

  • Website form validation issues requiring full session resets (e.g., Russian visa application). [00:26:18]

    

    

  • Hotel room climate control triggering phone rings. [00:27:06]
  • Newly installed Photoshop failing to create a new document due to a "program error." [00:27:42]
  • Laptop unexpectedly rebooting due to updates or OS failure, leading to loss of work. [00:29:07]
• Loss of Quality Rhetoric: The industry no longer talks about "Five 9s" (99.999% uptime), a common metric for reliability, suggesting a decline in quality standards. [00:28:24]
• Industry Excuses and Blind Spots:
  • The common excuse "The market won't pay for it!" is challenged by the speaker, who questions if the industry even possesses the capability to produce robust software anymore. [00:29:46]
  • The definition of a "tech company" has shifted from hardware innovators to software startups focused on market niches, often neglecting core software technology advancements. [00:30:42]
• Declining Programmer Productivity:
  • The move towards higher levels of abstraction in programming languages (e.g., from machine language to JavaScript) is often justified by increased productivity. [00:31:40]

    

  • However, this comes with a loss of low-level capability and understanding, leading to programmers being less productive now than in the past, approaching zero productivity per programmer when considering large companies like Twitter and Facebook and their rate of functional change. [00:33:57]

    

The Root Cause: Excessive Complication [00:37:18]

• "You Can't Just..." Phenomenon: Software has become excessively complicated, making simple tasks impossible without workarounds. [00:37:26]
  • Deployment: You can't just copy a program; you need installers, flatpaks, or containers, which are complex solutions to self-imposed operating system incompatibilities. [00:37:35]
    • Operating systems, while adding capabilities, also remove basic capabilities like direct program portability from the CPU. [00:38:59]
  • Graphics: Writing shaders involves different programming languages for each platform, even with identical hardware, forcing complex auto-translation or multiple rewrites. [00:39:38]
  • Development Tools: Simple tasks like finding installed libraries (e.g., Microsoft's vswhere program, which is 7,000 lines of code just to locate libraries) are absurdly complicated. [00:41:36]
  • Language Server Protocol (LSP): LSP turns simple editor functions (like finding a variable's declaration) into distributed systems running multiple servers on a local machine, adding immense, unnecessary complexity, bugs, and latency. [00:42:31]

    

    

• Complication Accelerates Knowledge Loss:
  • Reduced Individual Knowledge: Increased complexity means individuals know a smaller percentage of the overall system, hindering good work and knowledge transfer. [00:45:36]
  • Deep Knowledge Replaced by Trivia: Fundamental concepts (e.g., cache coherency) are replaced by specific, often temporary, workarounds for bugs in large frameworks (e.g., "How to work around Unity junk"). [00:45:58]
  • Drowning of Good Information: The difficulty of understanding complex systems leads to a higher prevalence of bad or outdated information in online searches and educational materials. [00:46:42]

The Path Forward: Simplification at Every Level [00:52:58]

• The Danger of Unchecked Complexity: More complexity reduces a system's resilience to disasters and accelerates institutional decay. [00:47:15]
  • Modern society acts as if it can handle infinite complexity, but there is a limit, especially considering the difficulty of generational knowledge transfer. [00:47:26]
  • Parallels to the Bronze Age collapse, where the fragility of elite-held knowledge contributed to widespread loss of skills like reading and writing. Today, the detailed understanding of CPUs is similarly a fragile, non-widespread skill. [00:51:27]
  • Potential stressors like internet shutdowns due to cyberattacks or supply chain restrictions could severely impact a fragile, complex system. [00:52:16]
• The Mandate for Simplification:
  • Technology will degrade naturally; therefore, a conscious effort to simplify is necessary. [00:53:01]
  • Simplification must occur at every level: hardware, operating systems, libraries, application code, communication systems (e.g., Internet), compilation, debugging, distribution, and how people interface with software. [00:53:11]

    

  • The "good news" is that current systems are so ridiculously complicated that opportunities for improvement are abundant and only require the "will" to act. [00:53:33]
• Why Programmers Should Care:
  • Even if civilization doesn't collapse, the future will be "deeply mediocre" if the current path continues. [00:53:53]
  • Programmers themselves are often unhappy due to the constant struggle with stupid, complicated tasks instead of engaging in interesting work. Simplifying will lead to personal fulfillment. [00:54:09]
  • For individual game developers, removing complexity is the right "short-term play," even if it seems counterintuitive or costly initially. [01:00:16]
    • Sticking with a buggy, complex system always ends up taking longer and causing more suffering than addressing the complexity head-on. [00:55:05]
  • Simplifying one's own code contributes to building institutional knowledge about how to simplify, a skill that is currently lacking. [00:55:54]