Multi-type $Ξ$-coalescents from structured population models with bottlenecks
Marta Dai Pra, Alison Etheridge, Jere Koskela, Maite Wilke-Berenguer
TL;DR
The paper advances population-genetic theory for structured populations undergoing bottlenecks by deriving two scaling regimes, drastic and soft, in which the forward allele-frequency process converges to jump-diffusions and the backward genealogies converge to multi-type Ξ-coalescents with simultaneous mergers and migrations. A novel d_λ metric enables convergence analysis through bottleneck intervals, and the two regimes are connected by moment duality between the forward diffusion and the coalescent processes. Drastic bottlenecks yield a diffusion with jumps whose size distribution is bottleneck-driven, while soft bottlenecks produce a coalescent with an α-time window of coalescent activity and corresponding jump-diffusion dynamics. Simulations show the model can generate diverse site frequency spectra with a small set of interpretable parameters, offering a flexible framework for fitting data from structured populations such as Atlantic cod.
Abstract
We introduce an individual-based model for structured populations undergoing demographic bottlenecks, i.e. drastic reductions in population size that last many generations and can have arbitrary shapes. We first show that the (non-Markovian) allele-frequency process converges to a Markovian diffusion process with jumps in a suitable relaxation of the Skorokhod J1 topology. Backward in time we find that genealogies of samples of individuals are described by multi-type $Ξ$-coalescents presenting multiple simultaneous mergers with simultaneous migrations. These coalescents are also moment-duals of the limiting jump diffusions. We then show through a numerical study that our model is flexible and can predict various shapes for the site frequency spectrum, consistent with real data, using a small number of interpretable parameters.
