Methods for Detecting Gravitational Waves from mini-Extreme-Mass-Ratio Inspirals II: A Spectral-Leakage-Aware Framework

TL;DR

This paper tackles the challenge of detecting long-lived mini-EMRI gravitational-wave signals in ground-based detectors by modeling spectral leakage from frequency evolution and removing the quasi-monochromatic constraint. It introduces a spectral-leakage-aware framework built on an STFT-based leakage function $\eta(o,w)$ and the ΣR detection statistic, enabling dynamic, frequency-layered searches that optimize coherence time across the band. The authors derive the statistical properties of the ΣR statistic, establish robust weighting via averaged leakage, and map detection confidence to a maximum distance $d_{\max}$, ultimately showing an order-of-magnitude increase in effective detection volume over baseline Hough methods. The approach significantly expands the accessible volume for mini-EMRIs and sub-solar exotic compact objects, with potential applicability to future space-based GW detectors; it also outlines paths to handle non-stationary noise and to construct a practical template bank.

Abstract

Mini-Extreme-Mass-Ratio Inspirals (mini-EMRIs), comprising a sub-solar exotic compact object (such as a primordial black hole or boson star) orbiting a much heavier stellar-origin or exotic compact object, represent key targets for ground-based gravitational-wave detectors to probe the early universe and the nature of dark matter. However, detecting such systems, which could spend hours to years in LIGO, Virgo and KAGRA data, poses a computational challenge to standard matched-filtering methods. However, semi-coherent methods are constrained by the quasi-monochromatic assumption, which restricts the coherence time to avoid spectral leakage caused by frequency evolution. In this work, we extend the development of our method, $Σ$Track, to the regime in which the quasi-monochromatic approximation is relaxed, in two ways. First, we establish an analytical model for the spectral leakage, extending the validity of conventional analyses beyond the quasi-monochromatic regime. Second, we propose the $ΣR$ statistic -- a novel detection metric formed by a weighted summation of power ratios -- which effectively recovers the signal energy dispersed across adjacent frequency bins. Building on this framework, we further introduce an innovative frequency-layered search strategy that dynamically optimizes the coherence time across the observation band. We benchmark our method against a globally optimized Hough transform pipeline using a fiducial mini-EMRI signal from a binary with masses $(1.5, 10^{-5})\,M_\odot$. The results demonstrate that our framework achieves an order-of-magnitude enhancement in the effective detection volume, significantly expanding the horizon for discovering mini-EMRIs and sub-solar exotic compact objects with ground-based gravitational wave detectors. This approach can be similarly applied to EMRI searches for future space-based gravitational wave detectors.

Figures (7)

• Figure 1: Evolution of $\lambda[i,\kappa]$ and $\mathcal{L}_i$ (upper) and the spectral leakage factor $\eta_i[\kappa]$ (lower) over STFT time segments for a simulated mini-EMRI signal. In both panels, the red dots represent the value in the anchoring frequency bin ($\kappa=0$), while the blue and green dots show the values in the adjacent bins ($\kappa=\pm1$). The yellow curve in the upper panel shows the total power statistic $\mathcal{L}_i$, calculated using a smoothed LIGO-H1 O3 PSD. The simulation uses a signal with $m_1 = 1.5 M_\odot$, $m_2 = 1 \times 10^{-5} M_\odot$, and $d = 8$ kpc, analyzed in the 100-200 Hz band. STFT parameters include $T_\mathrm{DFT} = 64$s, sampling frequency $f_s = 512$Hz, a Tukey window ($\alpha=0.5$), and 50% overlap between segments.
• Figure 2: Frequency evolution of the critical widening (see text) factors $w_{\mathrm{crit},m}$ (for orders $m=3, 4$), assuming a zeroth-order post-Newtonian inspiral model ($n=11/3$). The two panels correspond to different representative binary system parameters.
• Figure 3: The spectral leakage function $\eta(o, w)$ plotted against the dimensionless frequency offset $o$ for representative widening factors $w \in \{0, 2, 5, 10\}$. The limiting case $w=0$ represents a monochromatic signal. The upper panel illustrates the leakage pattern for a Rectangular window, while the lower panel shows the pattern for a Tukey window.
• Figure 4: The averaged spectral leakage function and its comparison with smoothed simulation data. (Upper panel) The average spectral leakage function $\hat{\eta}(w,o)$. The blue curve shows the power fraction in the central bin ($\kappa=0$), while the red curve shows the power in the first adjacent bin ($\kappa=1$). As widening increases, power clearly leaks from the central bin into its neighbors. (Lower panel) A direct comparison validating the averaged model. The rapidly oscillating data from \ref{['fig:stats']} is smoothed using a 100-point moving average. This empirically smoothed result shows excellent agreement with the theoretical averaged leakage function, confirming its validity as a robust model for the expected power distribution.
• Figure 5: The effective detection volume $V_\mathrm{eff}$ as a function of coherent integration time, $T_\mathrm{DFT}$, for the fiducial mini-EMRI system, computed using the O3 PSD for the LIGO Hanford detector. The performance is evaluated for a set $P_\mathrm{fd}=5\%$ and $P_\mathrm{fa}=1\%$. STFT uses the Tukey window function ($\alpha=0.5$) with a 50% overlap. The red curve illustrates the performance of our novel $\Sigma R$ statistic. In comparison, the blue curve depicts the performance of a statistic based on the Hough transform strategy. Instead of a full pipeline execution, this peak statistic is computed by summing binary pixels along the signal trajectory, mimicking the standard Hough approach which selects peaks based on a power ratio threshold (here $\theta=2.5$) and a spectral local maximum condition. The performance at $T=8$s, the optimal choice under the quasi-monochromatic assumption, is marked in the panel. Our new method, which transcends this limitation, finds a true optimal coherent time at $T=64$s.