Learning Is a Kan Extension
Matthew Pugh, Jo Grundy, Corina Cirstea, Nick Harris
TL;DR
This work formalises error minimisation within a category-theoretic framework by proving that all such problems can be represented as a left Kan extension, specifically $Lan_iota d$, which acts as a global minimiser. It introduces $S$ flavoured error as a lax 2-functor $Err:D\to S$ and shows that when $Inf:M\to D$ has a left adjoint $Alg:D\to M$, then $Alg(d)$ is a global minimiser for each $d$, while left Kan extensions provide a universal minimising construction. The paper also presents a universal representation: any set-theoretic error minimisation problem can be recast as a 2-category extension problem whose left Kan extension yields the global minimisers. Overall, the framework offers an algebraic lens for interpreting and guiding optimization in data transformations, with inference treated as a pseudo-inverse and Kan extensions serving as a unifying tool for existence and construction of optimal solutions.
Abstract
Previous work has demonstrated that efficient algorithms exist for computing Kan extensions and that some Kan extensions have interesting similarities to various machine learning algorithms. This paper closes the gap by proving that all error minimisation algorithms may be presented as a Kan extension. This result provides a foundation for future work to investigate the optimisation of machine learning algorithms through their presentation as Kan extensions. A corollary of this representation of error-minimising algorithms is a presentation of error from the perspective of lossy and lossless transformations of data.