Catching the Nebular Needle in a Polluted Haystack: Line-emission Signatures from Population III-forming Pockets around Massive Galaxies at the End of Reionization

Abstract

Finding the first generation of (Population III or Pop III) stars is one of the most ambitious and exciting challenges of astrophysics. JWST opened concrete prospects for their detection during the Epoch of Reionization (EoR), where increasing evidence suggests that residual Pop III formation may persist, even within pristine pockets of high-mass halos, due to inhomogeneous enrichment. However, the identification of Pop III stars within globally enriched environments will be challenging. We investigate the detectability of a subdominant Pop III component in/around massive ($M_\star \gtrsim 10^9 ~\mathrm{M_\odot}$) galaxies at $z \approx 6.5 - 9$ from the dustyGadget cosmological simulation suite, and the confusion arising from second-generation (Pop II) stars in their surroundings. We find that young ($\lesssim 1$ Myr), massive ($M_\mathrm{III} \sim 6 \times 10^5 ~\mathrm{M_\odot}$) Pop III clusters forming within these galaxy environments are responsible for strong HeII1640 line emission ($L_\mathrm{HeII1640} \gtrsim 10^{41} ~\mathrm{erg \, s^{-1}}$), which would be detectable with $\approx 10 (50)$ h of medium-resolution observations with NIRSpec/IFU at $z \approx 6 (10)$. These bright luminosities cannot be produced by standard Pop II populations alone. On the other hand, the dominant Pop II component within massive ``hybrid'' Pop III hosts powers strong metal line emission ($L_\mathrm{[OIII]5007} \gtrsim 10^{42} ~\mathrm{erg \, s^{-1}}$), indicating that the detection of metal lines alone cannot exclude the presence of Pop IIIs in high-$z$ galaxy environments. We further discuss candidate selection strategies based on Ly$α$, H$α$ and H$β$ emission, and how spatially resolved observations may enable the detection of isolated, pristine pockets in the outskirts of massive halos.

Figures (9)

• Figure 1: SFR averaged over 100 Myr ($\langle \mathrm{SFR} \rangle_\mathrm{100~Myr}$) as a function of total (Pop III + Pop II) stellar mass ($M_\star$) for the Pop III-hosting halos considered in this study (red circles), compared with benchmark Pop II-only halos (grey dots). The right axis shows the corresponding UV magnitudes ($M_\mathrm{UV}$) assuming a simple SFR-UV luminosity conversion as in Madau_Dickinson_2014 (see text). We compare with the $M_\mathrm{UV}$ vs $M_\star$ of faint, "pure" Pop III galaxy candidates (LAP1, Nakajima_2025, and AMORE6, Morishita_2025, filled, colored circles), and with brighter, "hybrid" Pop III candidates (RXJ2129-z8HeII, Wang_2024, EXCELS-63107 Cullen_2025, GN-z11, Bunker_2023Maiolino_2024Maiolino_2026, and GNHeII J1236+6215, Mondal_2025, empty, colored circles) proposed at $z \gtrsim 3$. Observed candidates cover a broad range of $M_\star$ and $M_\mathrm{UV}$, with our halo sample lying in a regime largely unexplored by "classical" Pop III models: while classical models typically focus on low-mass, Pop III-dominated systems, here we consider massive halos dominated by Pop II star formation, but hosting sub-dominant Pop III-forming clumps in their environment. Note that both axes have a broken range marked by "$\approx$" symbols, in order to fit the faint LAP1 and AMORE6 constraints on the bottom-left corner.
• Figure 2: HeII emission line luminosities from individual Pop III-forming clumps at 1640 Å (left) and 4686 Å (right), as a function of the age of the enclosed stellar populations, assuming a H number density around the stellar populations of $n_\mathrm{H} = 10^{4} ~cm^{-3}$ (dark-red, smaller dots) and $n_\mathrm{H} = 10^{3.7} ~cm^{-3}$ (red, larger dots) respectively, and an efficiency factor $\eta_\mathrm{III} = 0.3$ (corresponding to a Pop III mass of approximately $6 \times 10^5 ~\mathrm{M_\odot}$). For a stellar population of age $\sim 1.5$ Myr, other values of $n_\mathrm{H}$ explored in the model are shown with progressively lighter shades of red and progressively larger dots (with a zoom around the blended points in the top-right insets). Dark-red/cyan horizontal, solid lines show a comparison with the simple analytical model of Venditti_2024_HeIIAnalyticalModel, assuming no/strong mass loss, and the same $\eta_\mathrm{III} = 0.3$. Sensitivity thresholds at a $\mathrm{SNR} \sim 3$ for medium-resolution observations with JWST/NIRSpec/IFU are also shown as dotted lines, considering $\sim 10$ h (grey) and $\sim 50$ h (gold) of total integration time for sources at $z = 10$ (thin) and $z = 6$ (thick). The HeII line luminosity constraints of the observed Pop III candidates RXJ2129-z8HeII Wang_2024, GNHeII-J1236+621 Mondal_2025, LAP1 Vanzella_2020Vanzella_2023Nakajima_2025, and Hebe Maiolino_2026 are also shown as colored circles, with arbitrary positioning on the $x$ axis. The HeII emission decreases by up to $\sim 3$ dex with stellar age within the first 3 Myr. Only populations younger than $\sim 1$ Myr remain marginally detectable with $\sim 10$ h of observations up to $z \approx 10$ (HeII1640), or $\sim 50$ h up to $z \approx 6$ (HeII4686). In contrast, a much smaller variation of $\sim 0.5$ dex is found when varying $n_\mathrm{H}$ by 3 orders of magnitude.
• Figure 3: Integrated HeII line luminosity at 1640 Å ($L_\mathrm{HeII1640}$) vs 4686 Å ($L_\mathrm{HeII4686}$) arising from each halo within our Pop III halo sample and from benchmark Pop II-only halos at $z \approx 6.5 - 9$ (Section \ref{['sec:methods_haloSample']}), assuming our reference Cloudy configurations (Section \ref{['sec:methods_NEL']}). The total (Pop III + Pop II) HeII emission arising from Pop III-hosting halos and the contribution to the emission coming from confusing Pop II populations in these halos is shown by red/black circles, respectively; the HeII emission of benchmark Pop II-only halos is shown instead by black dots. We identify two main regions of interest for Pop III-hosting halos: a "confused" region (orange shaded area), in which the HeII emission arising from Pop III stars cannot be disentangled from a potential underlying Pop II floor, and an "unconfused" region (purple shaded area), which can only be reached through a Pop III contribution, allowing a more straightforward identification. The HeII luminosity of observed Pop III candidates (RXJ2129-z8HeII, Wang_2024, GNHeII-J1236+6215, Mondal_2025, Hebe, Maiolino_2026, and LAP1, Nakajima_2025) and sensitivity thresholds for JWST/NIRSpec are shown as in Figure \ref{['fig:HeIILuminosities_PopIIIClumps']}, as well as a comparison with the simple analytical model of fig. 2 of Venditti_2024_HeIIAnalyticalModel. Note that for GNHeII-J1236+6215, Hebe and LAP1, no constraints are available on the HeII4686 line, therefore measures of the HeII1640 line luminosity are shown out of the $x$-axis boundaries of the plot to avoid confusion.
• Figure 4: Fraction of the total HeII1640 line emission of Pop III-hosting halos from Figure \ref{['fig:HeIILuminosities_haloSum']} that is contributed by Pop III stars, as a function of the mass-weighted age of the Pop III stellar populations in each halo. Data points are color-coded according to the level of embedding in the assumed Cloudy configurations (as in Table \ref{['tab:number_density_config']}). The fractional contribution of Pop IIIs to the emission strongly decreases with age, with only a factor two scatter typically resulting from different assumed Cloudy configurations. All halos host a single Pop III stellar population, with the exception of the halos shown as squares (hosting two Pop III populations) and the halos shown as in triangles (hosting four Pop III populations).
• Figure 5: Integrated HeII1640 line emission ($L_\mathrm{HeII1640}$) from our Pop III halo sample and from benchmark Pop II-only systems (same as in Figure \ref{['fig:HeIILuminosities_haloSum']}) as a function of their Ly$\alpha$ ($L_\mathrm{Ly\alpha}$, first panel), H$\alpha$ ($L_\mathrm{H\alpha}$, second panel), H$\beta$ ($L_\mathrm{H\beta}$, third panel) and H$\beta$ + [OIII]5007 ($L_\mathrm{H\beta + [OIII]5007}$, last panel) line luminosity. Data points are color-coded according to the level of embedding in the assumed Cloudy configurations (as in Table \ref{['tab:number_density_config']}). Pop III systems (red-scale) are fully degenerate with the emission from Pop II stellar populations (grey-scale) in terms of their HI line luminosities, making these lines alone ineffective for distinguishing the two populations. However, Pop III sources with strong HeII emission above the confusion limit (red, dashed, horizontal line, corresponding to the purple shaded region in Figure \ref{['fig:HeIILuminosities_haloSum']}) can still be identified. These "unconfused" Pop III systems occupy the red shaded regions of the diagram, which therefore highlights promising areas of HI luminosities for selecting potential Pop III systems above the confusion limit. Measurements of the HeII and HI lines for the observed "hybrid" Pop III candidates RXJ2129-z8HeII Wang_2024 and GNHeII-J1236+621 Mondal_2025 are also shown as a comparison.