Learning Geometry: A Framework for Building Adaptive Manifold Models through Metric Optimization

TL;DR

The paper addresses the limitation of static geometry in conventional learning by proposing metric-optimization on a fixed-topology manifold, effectively making the geometry of the model space learnable. It introduces a variational framework that minimizes $L(g)=L_{\text{data}}(g;D)+\lambda L_{\text{geometry}}(g)$, with $L_{\text{data}}$ defined via a differentiable generator $f$ and data projections, and $L_{\text{geometry}}$ comprising curvature, smoothness, and volume terms. To make this tractable, the continuous theory is discretized on a triangular mesh with edge-length parametrization of the metric, enabling end-to-end automatic differentiation and projected-gradient optimization under triangle-inequality constraints. The work draws a deep analogy to the Einstein-Hilbert action, argues for greater expressiveness over fixed-geometry models, and charts a course toward fully dynamic topology-evolving meta-learners, with potential impact on scientific model discovery and robust representation learning.

Abstract

This paper proposes a novel paradigm for machine learning that moves beyond traditional parameter optimization. Unlike conventional approaches that search for optimal parameters within a fixed geometric space, our core idea is to treat the model itself as a malleable geometric entity. Specifically, we optimize the metric tensor field on a manifold with a predefined topology, thereby dynamically shaping the geometric structure of the model space. To achieve this, we construct a variational framework whose loss function carefully balances data fidelity against the intrinsic geometric complexity of the manifold. The former ensures the model effectively explains observed data, while the latter acts as a regularizer, penalizing overly curved or irregular geometries to encourage simpler models and prevent overfitting. To address the computational challenges of this infinite-dimensional optimization problem, we introduce a practical method based on discrete differential geometry: the continuous manifold is discretized into a triangular mesh, and the metric tensor is parameterized by edge lengths, enabling efficient optimization using automatic differentiation tools. Theoretical analysis reveals a profound analogy between our framework and the Einstein-Hilbert action in general relativity, providing an elegant physical interpretation for the concept of "data-driven geometry". We further argue that even with fixed topology, metric optimization offers significantly greater expressive power than models with fixed geometry. This work lays a solid foundation for constructing fully dynamic "meta-learners" capable of autonomously evolving their geometry and topology, and it points to broad application prospects in areas such as scientific model discovery and robust representation learning.