Accurate forecasting of passenger flows is critical for maintaining the efficiency and resilience of airport operations. Recent advances in patch-based Transformer models have shown strong potential in various time series forecasting tasks. However, most existing methods rely on fixed-size patch embedding, making it difficult to model the complex and heterogeneous patterns of airport passenger flows. To address this issue, this paper proposes a deformable temporal-spectral transformer named DTSFormer that integrates a multiscale deformable partitioning module and a joint temporal-spectral filtering module. Specifically, the input sequence is dynamically partitioned into multiscale temporal patches via a novel window function-based masking, enabling the extraction of heterogeneous trends across different temporal stages. Then, within each scale, a frequency-domain attention mechanism is designed to capture both high- and low-frequency components, thereby emphasizing the volatility and periodicity inherent in airport passenger flows. Finally, the resulting multi-frequency features are subsequently fused in the time domain to jointly model short-term fluctuations and long-term trends. Comprehensive experiments are conducted on real-world passenger flow data collected at Beijing Capital International Airport from January 2023 to March 2024. The results indicate that the proposed method consistently outperforms state-of-the-art forecasting models across different prediction horizons. Further analysis shows that the deformable partitioning module aligns patch lengths with dominant periods and heterogeneous trends, enabling superior capture of sudden high-frequency fluctuations.