The 35-Year Mistake in Object-Oriented Programming: A Deep Dive into Software Architecture and Historical Context

Better Software Conference

Introduction to the "Mistake" [00:00:00]

The speaker introduces a significant "35-year mistake" in software architecture, specifically within Object-Oriented Programming (OOP).

• Looking Glass [1998] and the Entity Component System (ECS) [00:00:50]

  • Looking Glass, an influential game company, developed "Thief: The Dark Project" in 1998.

    

  • The game looked primitive by today's standards but was state-of-the-art at the time.

    

  • Unlike traditional game development practices using "fat structs" (single large structures for entities) or hierarchical OOP models (like entity -> npc -> mech), Looking Glass pioneered a different approach.
  • Their innovation was the Entity Component System (ECS), where entities are merely IDs, and data (e.g., physics, combat, fuel) is stored and managed by separate systems, with entities looking up their data across these systems.

    

  • Encapsulation boundaries in ECS are drawn around the systems, promoting data-oriented design, unlike traditional OOP where boundaries are around objects.

    

• The Core Problem: Compile-Time Hierarchy Matching Domain Model [00:05:47]

  • Marc LeBlanc, a key implementer of Looking Glass's system, identified the prevalent OOP mindset: "A compile-time hierarchy that matches the domain model."
  • This means software structures are designed to directly reflect real-world classifications (e.g., a "guard" subclassing an "NPC"), even if it's not optimal for the code.
  • The speaker argues that this approach, while seemingly intuitive, misplaces encapsulation boundaries, making code harder to write and maintain.
  • This "mistake" is explicitly defined as "a compile-time hierarchy of encapsulation that matches the domain model was a mistake."

    

The Genesis of the "Mistake": C++ and Simula [00:16:32]

The historical roots of this problematic OOP paradigm are explored, starting with C++ and its precursor, Simula.

• Bjarne Stroustrup and C with Classes [00:16:51]
  • Bjarne Stroustrup, creator of C++, began his work on distributed systems simulation at Cambridge University in 1978 for his PhD.
  • He learned Simula, a simulation language, and found its class concept effective for directly mapping application concepts (e.g., different computer types) into language constructs.
  • Simula provided strong type-checking, which was a significant advantage over weaker type systems of the time (like Pascal).

    

  • However, the Simula implementation faced severe performance issues, particularly abysmal link times and a garbage collector consuming over 80% of runtime, even when no garbage was produced.
  • This led Stroustrup to abandon Simula for his primary work and rewrite his simulator in BCPL, a language with minimal type checking.
  • Upon moving to Bell Labs, Stroustrup sought to combine C's efficiency with Simula's beneficial features, leading to "C with Classes" (later C++). He aimed for modularity and concurrency analysis, leveraging classes for protection mechanisms inspired by hardware (e.g., virtual memory protection leading to private access specifiers, and OS read/write permissions leading to const).
  • Initially, C with Classes did not have virtual functions but later added them from Simula, prioritizing efficiency.
  • Stroustrup also initially used inheritance for code reuse (e.g., a link class for linked lists) but later advocated for templates for this purpose, acknowledging that inheritance was not ideal for generic container classes.

Tracing Back to Simula and Its Origins [00:31:47]

To understand the core ideas, the lineage is traced further back to Simula's development.

• Norwegian Computing Center [1962]: Kristen Nygaard and Ole-Johan Dahl [00:32:14]
  • Kristen Nygaard and Ole-Johan Dahl developed Simula.

    

  • The initial version, Simula I [1963], was designed as a "scripting language for system simulation," used for operations research to model queuing systems (e.g., airport departures).
  • The language allowed direct description of system components and their processes, generating statistics for optimization.
  • Simula I was successful, leading to the development of Simula 67 to address shortcomings and enable greater reusability.
  • The primary motivation for Simula 67's class concept was code reuse, exemplified by a "toll booth on a bridge" scenario (cars, trucks, buses). They wanted to define a generic car class from which truck and bus could inherit, and a link class to enable linked list functionality.
  • This shows that both domain modeling and code reuse (via inheritance) were intertwined from the start of Simula's class concept.
  • Simula also introduced mechanisms for dynamic dispatch, akin to virtual functions, to handle operations based on object type.

The Deeper Roots: C.A.R. Hoare and Discriminated Unions [00:41:59]

The origin of the "subclass" idea is attributed to C.A.R. Hoare, revealing a crucial divergence.

• C.A.R. Hoare [1966] and "Record Handling" [00:42:32]
  • Nygaard and Dahl explicitly credited C.A.R. Hoare for the "subclass idea," specifically from his work on "record class constructs."
  • Hoare's paper explored modeling real-world objects with mutually exclusive subclasses (e.g., algebraic expressions as constants, variables, dyadic expressions).

    

  • Crucially, Hoare's proposal included discriminated unions (or tagged unions), where a record had a type field and a consider when ... then construct for type-safe branching and access to specific subtype fields.

    

    

  • This mechanism provided compile-time type safety and was a powerful tool for dynamic polymorphism, allowing code to safely differentiate and operate on different sub-types.
  • This direct, type-safe approach to handling polymorphic data was, however, deliberately omitted from C++ by Stroustrup (who saw Simula's INSPECT statement as breaking modularity) and was thus largely lost in mainstream OOP.

The True Genesis: Douglas Ross and Ivan Sutherland [00:50:50]

The chain of influence is traced further back, revealing the ultimate origins of these foundational ideas.

• Douglas T. Ross (MIT Servomechanisms Laboratory, 1950s) [00:55:19]

  • Douglas T. Ross, an influential figure in early computing, developed the concept of "plexes" in the 1950s.
  • His work at MIT focused on Computer-Aided Design (CAD), aiming to translate human descriptions into manufactured parts.
  • Ross criticized List Processing languages (like Lisp) for their inefficient two-component list structures, advocating for plexes which were structured data with fixed sizes, enabling direct storage and faster processing of complex entities.

    

  • Plexes contained data members, pointers, and crucially, function pointers, allowing for dynamic behavior (early virtual functions).

    

  • This early work in CAD involved direct, data-oriented representations with dynamic dispatch mechanisms, fundamentally different from the later object-oriented encapsulation model.

• Ivan Sutherland (MIT Lincoln Laboratory, 1960) and Sketchpad [01:06:48]

  • Ivan Sutherland, working at the MIT Lincoln Laboratory, created Sketchpad as part of his PhD thesis (published 1963).

    

  • Sketchpad was a revolutionary interactive graphical program running on the TX-2 computer (one of the few with a display at the time).

    

  • Sutherland directly sought advice from Douglas Ross on data modeling for his CAD system, adopting Ross's "n-component elements" (plexes).
  • The program demonstrated groundbreaking interactive capabilities like moving and modifying drawn elements without erasing, and dynamic constraint satisfaction (e.g., making lines perpendicular).

    

    

  • Sutherland's constraint solver was "omniscient," iterating over and manipulating components directly through their shared memory space, demonstrating immense power and flexibility.

    

    

  • The speaker highlights that Sketchpad's architecture, particularly its "omniscient" handling of components, was a precursor to modern ECS, yet its true power was overlooked by subsequent OOP pioneers. Alan Kay himself noted the power of Sketchpad but focused on "messaging" and encapsulated objects rather than the underlying data structure and global access.

The Speaker's Personal Encounter with the "Mistake" [01:19:47]

Casey Muratori shares his personal experience encountering the "mistake" in 1997.

• Developing "Negaman" [1997] [01:20:29]
  • As a young, inexperienced programmer, Casey was tasked with creating a level editor for a game, involving "CSG on spheres" (later "negative metaballs").

    

  • He created a modeler called "Negaman" for this task.

    

  • He attempted to implement a selection-based UI using the then-popular compile-time OOP hierarchies (like those in C++).

    

  • This led to an extremely complex and convoluted codebase (e.g., parallel hierarchies of "property universes" with callbacks to get/set data) in an attempt to handle polymorphic interactions across selected objects.

    

    

  • He realized in hindsight that he was effectively trying to implement an Entity Component System without knowing it, struggling against the rigid OOP paradigm he was taught.

    

  • He contrasts his inefficient implementation with the elegant solutions found in Sketchpad and Looking Glass's ECS, lamenting the lost knowledge.

The Lasting Impact and The Path Forward [01:46:58]

• Continuing the Cycle [2025] [01:47:04]

  • The 35-year gap between Sketchpad and ECS represents a missed opportunity for architectural evolution.

    

  • Modern OOP tutorials still predominantly teach the problematic compile-time hierarchies that directly map domain models (e.g., IronMan extending IronMan, Vehicle extending Car, Bicycle extending MountainBike).
  • This perpetuates an architectural approach that is often over-engineered and less efficient for many common problems.

• The Treacherous Metaphor [01:48:50]

  • The core issue is that "the most treacherous metaphors are the ones that seem to work for a time, because they can keep more powerful insights from bubbling up."
  • The intuitive but rigid compile-time OOP hierarchy, while seemingly functional for simple cases, has suppressed the broader understanding and adoption of more flexible, data-oriented architectures that were, in fact, discovered decades earlier.
  • The emphasis on strict encapsulation based on real-world objects (influenced by backgrounds in molecular biology and distributed systems) prevented the acceptance of "omniscient" systems that accessed and manipulated data more broadly.

Q&A: Further Insights [01:50:11]

The speaker answers questions, providing additional context and personal reflections.

• Looking Glass's Evolution to ECS [02:03:33]
  • Ultima Underworld entities were "fat struct overlays" (union overlays) for memory efficiency, leading to bugs (e.g., empty lanterns disappearing due to fuel overlaying HP).

    

  • Marc LeBlanc, a key implementer, encountered these bugs directly.

    

  • Such overlays required manual checks to prevent issues.

    

  • For System Shock, they moved to an "outlinking" structure where entities had a shared base and one pointer to an extended data block. This reduced memory fragmentation and offered some type safety but limited combination possibilities.

    

    

    

  • Flight Unlimited faced severe memory fragmentation issues due to heavy C++ new/delete usage, leading to forced game restarts ("out of fuel" message).

    

    

    

  • The notorious "out of fuel" message was actually an indicator of memory fragmentation, prompting a game restart.

    

  • These real-world performance and memory issues drove the Looking Glass team to innovate towards the Entity Component System in Thief.
  • Tom Leonard is credited with the key ECS insight: shifting from entity.getHitPoints() to hitPointsSystem.getEntityHitPoints(entityID), allowing systems to iterate efficiently over specific components and manage arbitrary-sized properties, while still enabling designers to define types at runtime.

    

  • Their motivations included avoiding fragmentation, supporting arbitrary-sized properties, enabling runtime types, and ensuring the fastest possible performance.