Differentiable Integer Linear Programming is not Differentiable & it's not a mere technical problem

Abstract

We show how the differentiability method employed in the paper ``Differentiable Integer Linear Programming'', Geng, et al., 2025 as shown in its theorem 5 is incorrect. Moreover, there already exists some downstream work that inherits the same error. The underlying reason comes from that, though being continuous in expectation, the surrogate loss is discontinuous in almost every realization of the randomness, for the stochastic gradient descent.

Figures (2)

• Figure 1: In 1-dimensional case where $a>b>0$, we will have that the loss proposed in geng2025differentiable is $\hat{\varphi}$, which is a transformation of the function $f_1$, which is independent from $a$ and $b$. Both functions are bounded and convex, so the minimizer is unique. Strict convexity suggests that the unique minimizer of $\hat{\varphi}$ will be close to that of $f_1$ when $\frac{b}{a-b}$ is close to $0$, so such minimizer will still be greater than $0.5$.
• Figure 2: In these two figures, we set $(a,b) = (1, 0.95)$. To evaluate the actual loss $\hat{\phi}\left( \hat{x} \right)$, we start from encoding $\hat{x}$ to be $z = \sigma^{-1}\left( \hat{x} \right)$, and sample a logistic distribution to be added to $\sigma^{-1}\left( \hat{x} \right)$ (as shown as the yellow density in the top figure). It is easy to see that $\hat{\phi}\left( \hat{x} \right)$ will be the integration of the step function (in green) with respect to the density (in yellow). The paper geng2025differentiable suggests a surrogate function (as the bold blue line), which is $0$ when lower than $0$ and converge to $1$ when $z \to \infty$. The integration of such function with respect to the yellow density will yield $\hat{\varphi}\left( \hat{x} \right)$. When the value of the encoded $z = \sigma^{-1}\left( \hat{x} \right)$ shifts, the yellow density will also shift by the same distance. Thus, if a function is continuous together with some regularity conditions, we will have that the derivative (with respect to $z$) of the integration of it with respect to the yellow density will be the same as the integration of its derivative (with respect to $z$) with respect to the yellow density. However, the bold blue function is discontinuous, so such equality does not hold. We note that the dotted blue line is the bold blue function with an upward shift for the positive region of $b - \frac{a}{2} > 0$. Now, we have that the dotted blue function is absolutely continuous and has the same derivative as the bold blue function almost everywhere. Thus, we have that the algorithm in geng2025differentiable would not be changed if the bold blue function were to be the dotted one. For the dotted function, we can swap the integration and differentiation, so the algorithm will be as if we perform stochastic gradient on $\hat{\hat{\varphi}}$, which is the integral of the dotted blue line, and is different from the proposed surrogate $\hat{\varphi}$.

Theorems & Definitions (3)

• Theorem 1: Summarization of Theorem 1 and Theorem 2 of geng2025differentiable
• Theorem 2: Corrected Theorem 3 of geng2025differentiable
• proof