Incremental Computation: What Is the Essence?

TL;DR

The paper addresses the challenge of efficiently recomputing results as input changes, proposing incrementalization as the discrete analogue of differentiation. It unifies three research strands—incremental algorithms, program-evaluation frameworks, and derivation methods—under a single systematic approach, III, to generate incremental programs from non-incremental ones. It articulates a three-step method (P1–P3) for systematic incrementalization and discusses how this translates across loops, sets, recursion, logic rules, and object-oriented programming, with concrete graph and distributed-algorithm examples. A key contribution is illuminating the strong role of high-level data/control abstractions in enabling precise complexity guarantees and scalable, automated transformations, while also tying incrementalization to partial evaluation and integration concepts. The work points to future directions for practical language design and compilers that support rich abstractions and rigorous correctness/efficiency guarantees.

Abstract

Incremental computation aims to compute more efficiently on changed input by reusing previously computed results. We give a high-level overview of works on incremental computation, and highlight the essence underlying all of them, which we call incrementalization -- the discrete counterpart of differentiation in calculus. We present the gist of a systematic method for incrementalization, and a systematic method centered around it -- called Iterate-Incrementalize-Implement -- for program design and optimization, as well as algorithm design and optimization. We illustrate the methods with example applications in arithmetic computations, recursive functions, graph analysis, and distributed algorithms. At a meta-level, with historical contexts and for future directions, we stress the power of high-level data, control, and module abstractions in developing new and better algorithms and programs as well as their precise complexities.