Instabilities in heterotic M-theory induced by open membrane instantons

TL;DR

This work analyzes how open membrane instantons dominate nonperturbative effects in strongly coupled heterotic M-theory with M5-branes, deriving explicit ΔW contributions and the resulting moduli potential in a controlled regime. By computing membrane vertex operators and leveraging holomorphy, the authors obtain closed-form expressions for the instanton-induced superpotential and Kahler potentials, and they examine the resulting dynamics, revealing attraction toward boundaries or decompactification tendencies depending on the moduli and charged-scalar vevs. The study highlights Dine-Seiberg-type instabilities in this setting, discusses multiple-cover effects relevant to chirality-changing transitions, and generalizes to multiple M5-branes, outlining future directions for stabilization and richer phenomenology in heterotic M-theory. Overall, the results illuminate how nonperturbative open membrane effects shape the moduli space and potential landscape, with implications for chirality, phase transitions, and the structure of the effective theory.

Abstract

We study the effective low energy supergravity of the strongly coupled heterotic string compactified on a Calabi-Yau 3-fold with generic E8 x E8 gauge bundle. We focus on the effective potential for the chiral scalars. The effective superpotential can be studied using the dual 11-dimensional M-theory background involving insertions of M5 branes along an interval. In such backgrounds, in some regions of moduli space, the leading nonperturbative contributions are due to open membrane instantons. These instantons lead to both attractive and repulsive forces between the 5-branes and the orientifold ``M9-branes,'' depending on the region of moduli space. The resulting dynamics on moduli space include a strong coupling dual to the Dine-Seiberg instability, in which the interval grows. We discuss conditions under which the 5-branes are attracted to the wall and comment on the relevance of these results to the study of chirality-changing phase transitions in heterotic M-theory.