Constraining the CP4-invariant three-Higgs-doublet model via top quark decays

TL;DR

This work analyzes the CP4-invariant three-Higgs-doublet model (CP4 3HDM), where a single order-$4$ CP symmetry induces tight correlations between the scalar and Yukawa sectors and guarantees tree-level flavor-changing neutral currents. By employing a physics-driven inversion/scanning approach, the authors identify that only the (A,B2) Yukawa pattern remains viable under meson-oscillation and top-quark constraints, ruling out (B1,B1). They parametrize the scalar sector in a Higgs basis with misalignment angles and study non-standard top decays to light neutral and charged Higgses, including decays of the SM-like Higgs to up-type quarks, while imposing experimental bounds on top widths and FCNC processes. The analysis yields viable points in all scalar-regime scenarios (all heavy, one charged, one neutral, full sample) and presents benchmark points illustrating consistent spectra and constrained top-decay signatures. Overall, top-quark observables—especially the h_SM t tbar coupling and total top width—provide powerful probes of CP4 3HDM, guiding future tests at the LHC and flavor experiments.

Abstract

CP4 3HDM is a peculiar three-Higgs-doublet model in which a single symmetry leads to tight constraints on the scalar and Yukawa sectors. In this models, tree-level flavor-changing neutral couplings are unavoidable; however, as previously shown, their contributions to neutral meson oscillations can be suppressed. Here, we explore the remaining quark flavor violating effects, which give rise to the top quark decays to light scalars, including the 125 GeV Higgs boson $h_{SM}$, as well as the magnitude of the $h_{SM} t\bar t$ coupling. Utilizing the recently developed scanning procedure, in which observables are used as input, we narrow down the viable options to a unique CP4-invariant Yukawa sector capable of satisfying all meson oscillation and top quark constraints. We present benchmark models that feature neutral or charged Higgs bosons lighter than the top quark, and we look forward to testing them further at the LHC and through flavor physics observables.

Figures (6)

• Figure 1: The contribution to the neutral $D$-meson mass splitting $\Delta m_D$ coming from the tree-level FCNC as a function of $M_{\min}$, the minimal mass of any non-SM Higgs boson. The Yukawa sector is of type $(A,B_2)$. The light pink points correspond to the full Yukawa scan, while the dark red points emerge from the restricted Yukawa scan with $\theta_{max} = \pi/10^4$. The dashred line corresponds to the experimentally measured $\Delta m_D$.
• Figure 2: The quantity $R_{(B_1,B_1)}$ defined in Eq. \ref{['RB1B1']} that shows the magnitude of the tree-level FCNC contributions to the neutral meson oscillation parameters relative to their experimental values. The Yukawa sector is of type $(B_1,B_1)$; the labels are the same as in Fig. \ref{['fig-D-meson']}. The dashed line corresponds to $R_{(B_1,B_1)} = 1$, the target value for this ratio.
• Figure 3: The Higgs-top coupling relative to its SM value ($\kappa_t$) and the branching fraction of $t\to h_{\hbox{\tiny SM}} u$ in a restricted scalar and Yukawa scans. The color encodes the magnitude of $|\epsilon|$, with larger-$|\epsilon|$ points plotted on top of lower-$|\epsilon|$ ones. The box corresponds to the tightest LHC bounds.
• Figure 4: The total top quark decay width as given in Eq. \ref{['top-width']} for the case $(A, B_{2})$. The four panels correspond to the four scalar sector samples listed in section \ref{['subsection-scan-procedure']}. The band between the dashed lines corresponds to the $1\sigma$ experimental range established by PDG ParticleDataGroup:2024cfk.
• Figure 5: The impact of light charged Higgs bosons on the non-standard top quark decays. Shown are the ratios of the decay chain probabilities, ending in $H^+ \to c\bar{b}$ and $H^+ \bar{s}$, to the corresponding experimental upper limits defined in \ref{['R-cbcs']}. Light and dark blue points correspond to the results of a full and restricted Yukawa scan, respectively. The left panel contains only one charged Higgs boson, the right panel corresponds to the full sample. Only points with both $R_{cb} < 1$ and $R_{cs} < 1$, marked by the dashed lines, pass this check.