Edgeworth Expansion for Semi-hard Triplet Loss
Masanari Kimura
TL;DR
This work analyzes the semi-hard triplet loss used in metric learning through a higher-order asymptotic lens by applying the Edgeworth expansion to the distance-difference variable $\Delta$. By refining the central limit theorem with skewness corrections, it derives explicit expressions for the Edgeworth expansion and shows how the margin parameter $\alpha$ and the distributional shape of $\Delta$ influence training dynamics. The authors provide detailed formulas for the loss expansion $\mathcal{L}_{semi}(\alpha) = \mathcal{L}^{(0)}_\alpha + \frac{1}{\sqrt{N}}\mathcal{L}^{(1)}_\alpha + O(N^{-1})$ and for the derivative $\frac{d}{d\alpha}\mathcal{L}_{semi}(\alpha) = P_{sh}(\alpha)$, including a uniform error bound and conditions under which first-order corrections vanish. These results offer theoretical guidance for margin tuning to improve training stability and embedding quality in metric learning contexts.
Abstract
We develop a higher-order asymptotic analysis for the semi-hard triplet loss using the Edgeworth expansion. It is known that this loss function enforces that embeddings of similar samples are close while those of dissimilar samples are separated by a specified margin. By refining the classical central limit theorem, our approach quantifies the impact of the margin parameter and the skewness of the underlying data distribution on the loss behavior. In particular, we derive explicit Edgeworth expansions that reveal first-order corrections in terms of the third cumulant, thereby characterizing non-Gaussian effects present in the distribution of distance differences between anchor-positive and anchor-negative pairs. Our findings provide detailed insight into the sensitivity of the semi-hard triplet loss to its parameters and offer guidance for choosing the margin to ensure training stability.