Exploring the holographic entropy cone via reinforcement learning

TL;DR

This work introduces a reinforcement learning framework to explore the holographic entropy cone (HEC) by targeting graph realizations whose min-cut entropies reproduce given vectors. The method precisely classifies targets inside the HEC and, for targets outside, navigates toward the nearest facet, enabling gradient-based discovery of holographic inequalities. Demonstrations on ${\sf N}=3$ recover monogamy of mutual information and on ${\sf N}=6$ resolve 3 of 6 mysterious SAC extreme rays, providing evidence for additional HEIs yet to be found. The results illustrate RL as a powerful, scalable tool for uncovering the facet structure of high-dimensional entropy cones and guide future directions toward a complete ${\sf N}=6$ characterization.

Abstract

We develop a reinforcement learning algorithm to study the holographic entropy cone. Given a target entropy vector, our algorithm searches for a graph realization whose min-cut entropies match the target vector. If the target vector does not admit such a graph realization, it must lie outside the cone, in which case the algorithm finds a graph whose corresponding entropy vector most nearly approximates the target and allows us to probe the location of the facets. For the $\sf N=3$ cone, we confirm that our algorithm successfully rediscovers monogamy of mutual information beginning with a target vector outside the holographic entropy cone. We then apply the algorithm to the $\sf N=6$ cone, analyzing the 6 "mystery" extreme rays of the subadditivity cone from arXiv:2412.15364 that satisfy all known holographic entropy inequalities yet lacked graph realizations. We found realizations for 3 of them, proving they are genuine extreme rays of the holographic entropy cone, while providing evidence that the remaining 3 are not realizable, implying unknown holographic inequalities exist for $\sf N=6$.

Figures (10)

• Figure 1: The $S_3$-symmetric slice of the ${\sf N}=3$ entropy space in the $(s,t)$ plane. The color gradient shows the analytically computed reward $R$ (cosine similarity). On this symmetric slice, the HEC (pink region, $R = 1$) is constrained by MMI, SA group 1, and SA group 2. The black circle is the boundary $u = 0$.
• Figure 2: Zoomed in view of the reward landscape near the HEC boundary, showing the gradient field in the region $s \in [0.15, 0.55]$ and $t \in [0.35, 0.55]$. Arrows indicate the direction of steepest reward increase, colored by magnitude. The gradient consistently points towards the nearest HEC boundary (SA group 1, SA group 2, or MMI).
• Figure 3: 3D view of the reward landscape on the ${\sf N}=3$ symmetric slice, showing agreement between analytical predictions (surface) and RL results (points). Rewards are plotted as $\log(1-R)$ for visibility near the HEC boundary. The red dots lie outside the HEC, whereas the green dots lie inside the HEC.
• Figure 4: Classification comparison on the ${\sf N}=3$ symmetric slice with points sorted by analytical reward. Rewards are plotted as $\log(1-R)$ for visibility near the HEC boundary. Green points (inside HEC) cluster at high rewards near the analytical ceiling, while red points (outside HEC) follow the analytical curve with small gaps due to finite training.
• Figure 5: Trajectory of gradient-based optimization projected onto the symmetric slice $(s, t)$, shown until it first enters the HEC (pink region). The initial point lies exactly on the symmetric slice, and since the reward is higher on the symmetric slice than off it, the trajectory remains close to the slice throughout (projection distance $< 0.04$, shown by marker color). Black arrows indicate the gradient direction at each iteration; since we use fixed step size, all arrows are drawn with equal length. The 7D gradient $\vec{G}$ is projected to 2D via $\partial_s = G_A + G_B + G_C - 3s G_{ABC}/u$ and $\partial_t = G_{AB} + G_{AC} + G_{BC} - 3t G_{ABC}/u$, accounting for how $u$ varies with $s$ and $t$ on the unit sphere. These projected gradients appear to be not tangent to the trajectory due to the nonlinearity of the projection, but they align well with the analytical gradient field (gray arrows), confirming consistency between RL sampling and theoretical predictions. Detailed gradient evolution analysis is provided in \ref{['fig:mmi-evolution']}.