Relativistic Mott transition and high-order van Hove singularity in twisted double bilayer WSe${}_2$: mean-field and functional renormalization group study

TL;DR

This work investigates relativistic Mott physics and high-order van Hove singularities in twisted double bilayer WSe$_2$ by combining a continuum model for the moiré Gamma valley, a honeycomb moiré lattice Hubbard mapping, unrestricted Hartree-Fock mean-field theory, and fermionic functional renormalization group calculations. Twist-angle tuning modifies the Dirac velocity, bandwidth, and density of states, enabling a relativistic Mott transition near $ heta_c \uparrow 2.7^ extdeg$ and a high DOS regime at a high-order VHS around $ heta_ ext{HOVHS}\nobreak\approx obreak 3.58^ extdeg$, with corresponding magnetic and superconducting tendencies. HF results predict Néel order at half filling for $ heta< heta_c$ and various three-mode stripe orders near van Hove fillings, while fRG finds competition between density-wave and unconventional superconducting instabilities, including chiral $d$-wave pairing and a ferromagnetic state at the HOVHS. Together, these findings establish tdWSe$_2$ as a tunable quantum simulator for Dirac semimetals, relativistic Mott transitions, and competing correlated phases, guiding future experiments.

Abstract

Experiments on twisted double bilayer tungsten diselenide have demonstrated that moir'e semiconductors can realize a relativistic Mott transition, i.e., a quantum phase transition from a Dirac semimetal to a correlated insulating state, by twist-angle tuning. In addition, signatures of van Hove singularities were observed in the material's moir'e valence bands, suggesting further potential for the emergence of strongly-correlated states. Based on a continuum model, we provide a detailed analysis of the twist-angle dependence of the system's band structure, focusing on the evolution of the Dirac excitations and the Fermi-surface structure with its Lifshitz transitions across the van Hove fillings. We exhibit that the twist angle can be used to band engineer a high-order van Hove singularity, which can be accessed by gate tuning. We then study the magnetic phase diagram of an effective Hubbard model for twisted double bilayer tungsten diselenide on the effective honeycomb superlattice with tight-binding parameters fitted to the two topmost bands of the continuum model. To that end, we employ a self-consistent Hartree-Fock mean-field approach in real space. Fixing the angle-dependent Hubbard interaction based on the experimental findings, we explore a broad parameter range of twist angle, filling, and temperature. We find a rich variety of magnetic states that we expect to be accessible in future experiments, including, e.g., a non-coplanar spin-density wave with non-zero spin chirality and a half-metallic uniaxial spin-density wave. Finally, we employ a functional renormalization group approach to also study the competition between density-wave and superconducting instabilities. For twist angles of $θ=2.0^\circ, 2.5^\circ$, as well as $θ\approx 3.5^\circ$ -- where the high-order van Hove-singularity is found -- we find clear indications for unconventional superconductivity.

Figures (8)

• Figure 1: Band structure from the continuum model for various twist angles. (a) Band structure along the $\gamma$--$m$--$\kappa$--$\gamma$-path in the moiré BZ from the continuum model (black bands) and from the effective tight-binding model (blue bands), (b) corresponding density of states, and (c) Fermi surface at van Hove filling of the lower band of tdbWSe${}_2$ for twist angles $\theta \in \{2.0^\circ, 2.5^\circ, 2.7^\circ, 3.0^\circ, 3.58^\circ, 4.0^\circ\}$ from the continuum model with cutoff radius $4 G$. A magnification of these Fermi surfaces is shown in Fig. \ref{['fig:VHSsplitting']}. Panel (d) shows the hopping amplitudes determined by fitting the honeycomb lattice tight-binding model defined in Eq. \ref{['eq:tbhamil']} to the two topmost bands of the continuum model.
• Figure 2: Evolution of Fermi surface at van Hove fillings. We show the evolution of the Fermi surface at the Van-Hove filling of the topmost (second-to-topmost) moiré band in the top (bottom) row for angles ${\theta \in \{3.00^\circ,3.58^\circ,4.00^\circ\}}$. In the lower band, each saddle point splits up into two, when crossing the angle $\theta_c\sim 3.58^\circ$ from below. Right at $\theta_c$ there is a degeneracy, leading to a high-order saddle point.
• Figure 3: Hubbard interaction and relativistic Mott transition from Hartree-Fock mean-field approximation. We show the antiferromagnetic gap $\Delta = 2 U m$ for different twist angles $\theta$ determined with the effective Hubbard model described in Sec. \ref{['sec:effhubbard']} and $\mathcal{N} = 24 \times 24$ unit cells (black dots), and the angle-dependent quantities $U(\theta)/t_1(\theta)$ (red curve) and $\alpha_\text{eff}$ (red dashed curve). We rescaled the effective fine-structure constant by a factor of 1/10 for better visibility. At $\theta_c = 2.7^\circ$, a transition from an antiferromagnet ($\theta < \theta_c$) to a Dirac semimetal ($\theta > \theta_c$) occurs. We note that the small but finite gap at $\theta_c$ is a finite-size effect, as we determined an estimate for $U_c$ in the thermodynamic limit from the correlation ratio in Eq. \ref{['eq:corrrat']}.
• Figure 4: Absolute magnetization, momentum transfer, and phase diagram in the $n$-$\theta$ -- plane for $T = 20\,\mathrm{mK}$ and $T = 2\,\mathrm{K}$. Absolute magnetization on the finite lattice (left panels), momentum transfer (middle panels), and phase diagram (right panels) in the filling--twist-angle plane at $T = 20\,\mathrm{mK}$ (bottom panels) and $T = 2\,\mathrm{K}$ (top panels) from Hartree-Fock calculation for a lattice with $18 \times 18$ unit cells. The dashed white lines mark the van Hove fillings of the upper and lower band, and the star marks the position of the higher-order van Hove singularity at $\theta = 3.51^\circ$. We note that the paramagnetic regions (grey) in the middle and right panels have been determined by using a threshold to consider finite-size effects, see App. \ref{['app:class']}. In the legend, "1 Stripe" refers to single-mode stripe order, "2 Orth." to two orthogonal stripes, "3 Modes" to three modes with the same amplitude, "3 Orth." to orthogonal stripes, and "3 Coll." to collinear stripes. The latter two are special cases of the "3 Modes" order, see the main text and App. \ref{['app:class']} for more details.
• Figure 5: Three orthogonal stripes and three collinear stripes spin-ordering patterns at commensurate wavevector $\vec{Q_k}=|\vec{m}|$ on the real-space moiré honeycomb lattice. Left panel: This state corresponds to a non-coplanar SDW with non-zero spin chirality and was previously discussed in Refs. Li_2012PhysRevLett.101.156402. We show the projection of the spins to the $x-z$-plane. The color of the arrow indicates the value of the $S_y$ component. Right panel: This state corresponds to a uniaxial SDW with only one spin branch being gapped out, i.e., it is a "half metal", which was described in Ref. PhysRevLett.108.227204.