Beyond Moore's Law: Harnessing the Redshift of Generative AI with Effective Hardware-Software Co-Design

TL;DR

The paper argues that the slowdown of Moore’s Law necessitates hardware-software co-design as a core design principle rather than a peripheral consideration. It presents a historical, epoch-based narrative from primordial co-design to AI-driven co-design, diagnosing current challenges in efficiency, adaptability, and complexity within generative AI workloads. A central contribution is the articulation of the hardware lottery and a set of paradigm-shifting recommendations—reconfigurable hardware, architectural diversity, rapid iteration, and cross-disciplinary culture shifts—to sustain momentum. The work emphasizes practical impact for industry and academia, advocating integrated, cross-stack strategies to achieve robust performance gains in next-generation AI systems.

Abstract

For decades, Moore's Law has served as a steadfast pillar in computer architecture and system design, promoting a clear abstraction between hardware and software. This traditional Moore's computing paradigm has deepened the rift between the two, enabling software developers to achieve near-exponential performance gains often without needing to delve deeply into hardware-specific optimizations. Yet today, Moore's Law -- with its once relentless performance gains now diminished to incremental improvements -- faces inevitable physical barriers. This stagnation necessitates a reevaluation of the conventional system design philosophy. The traditional decoupled system design philosophy, which maintains strict abstractions between hardware and software, is increasingly obsolete. The once-clear boundary between software and hardware is rapidly dissolving, replaced by co-design. It is imperative for the computing community to intensify its commitment to hardware-software co-design, elevating system abstractions to first-class citizens and reimagining design principles to satisfy the insatiable appetite of modern computing. Hardware-software co-design is not a recent innovation. To illustrate its historical evolution, I classify its development into five relatively distinct ``epochs''. This post also highlights the growing influence of the architecture community in interdisciplinary teams -- particularly alongside ML researchers -- and explores why current co-design paradigms are struggling in today's computing landscape. Additionally, I will examine the concept of the ``hardware lottery'' and explore directions to mitigate its constraining influence on the next era of computing innovation.