Millisecond Cadence Radio Frequency Interference Filters

TL;DR

RFI severely limits sensitivity in radio transient searches, motivating robust mitigation at millisecond cadences. The paper introduces three high-cadence filters—time-domain MAD, FFT-MAD in the Fourier domain, and a high-pass filter—and a composite that combines all three to address multiple RFI morphologies while preserving astrophysical bursts. It demonstrates, across synthetic Gaussian noise, RFI-injected spectra, and four pulsar observations, that these filters can increase sensitivity and reduce false positives, while flagging a small fraction of data (less than 5%), e.g., achieving a 53% increase in detected pulses and a 34% decrease in RFI candidates compared with a baseline pipeline. The findings provide guidance on when and how to deploy each filter or the composite in millisecond-scale transient searches and highlight practical benefits for FRB/pulsar surveys.

Abstract

Radio Frequency Interference (RFI) greatly reduces sensitivity of radio observations to astrophysical signals and creates false positive candidates in searches for radio transients. Real signals are missed while considerable computational and human resources are needed to remove RFI candidates. Effective RFI removal is vital to carry out successful searches for fast radio bursts and pulsars. Mitigation techniques that excise RFI on short timescales account for a changing radio frequency and pulse environments. We evaluate the effectiveness of three filters, as well as a novel composite of the three, that excises RFI at the cadence that the data is recorded. Each of these filters operates in a different domain and thus excises as a different RFI morphology. The composite filter removes RFI not accessible to other filtering methods. We analyze the performance of these four filters in three different situations: (i) synthetic pulses in Gaussian noise; (ii) synthetic pulses injected into observed spectra; (iii) test observations of four pulsars. From these tests, we gain insight into how the filters affect both the pulse and the noise level. This allows us to outline which and how the filters should be used based on the RFI present and the characteristics of the source signal. These filters both increase sensitivity and reduce the number of false positives. By flagging less than 5% of the spectrum, we demonstrate a 53% increase in detected pulses and 34% decrease in the number of RFI candidates relative to the Heimdall-Fetch-Your search pipeline.

Figures (11)

• Figure 1: GREENBURST dynamic spectra with three pulses from PSR B0329+54. The top row shows three time series de-dispersed at the pulsar's DM, these plots share the same scale. The bottom left plot shows the power (blue) and standard deviation (orange) of unfiltered dynamic spectra. We see very bright spikes in both metrics between 1.5 and 1.6 GHz from satellite communications and radar. Some channels have a standard deviation close to 60, adding a correlated signal that is ten times larger than the thermal background. The next three plots show dynamic spectra that are: (i) unfiltered; (ii) filtered using MAD across all frequencies (13.2% flagged); (iii) filtered using MAD on frequency-time blocks (7.2% flagged). The range of the dynamic spectra are set at +5$\sigma$ to --3$\sigma$ around the median (with the standard deviation, $\sigma$, calculated via MAD in Eq. \ref{['eq:mad-to-sigma']}). The MAD filter across all frequencies significantly reduces the pulse intensity. Using MAD on frequency-time blocks of 256 channels by 32 samples helps preserve the pulse. The block shape is indicated by white spaces, the time axis blocks are difficult to distinguish, so every tenth block is longer. The rightmost plot shows the channel statistics after cleaningṪhe band-pass has been flattened and changes in standard deviation are now smooth.
• Figure 2: The frequency structure of a typical GREENBURST observation, computed by taking the median along the time axis. The median is a robust to outliers estimator, however some channels (around frequencies 1550 MHz and 1030 MHz) are still significantly affected by RFI. The four offset curves are of robust to outlying channels fitting schemes we investigated.
• Figure 3: This flow chart shows the steps of the MAD filter. Data is read from file, de-trened, outliers are flagged, de-trened, more outliers are flagged. Then there is a final detrend followed by a replacement of flagged data.
• Figure 4: This flow chart shows the steps of the FFT-MAD filter. Data is read from file or accepted post MAD filtering. The data is FFTed along the time axis. Median and MAD is calculated along across channels. Outliers are flagged and replaced with zero. The data is inverse FFTed and written to file or given to the highpass filter.
• Figure 5: The dynamic spectrum of PSR B0329+54 shown in Fig. \ref{['fig:green_burst']} cleaned using the FFT-MAD filter, which flagged 3.3% of the Fourier domain. The left-side white blocks indicate subbands of 256 channels. The top plot shows the de-dispersed time series. The right plot shows the standard deviation of each channel. We compare three filters, RFICleanMann-2021 run on defaults, RFIClean run with the pulsar frequency protected, and FFT-MAD. In the dynamic spectra, we see that the periodic RFI has been removed, but the pulse is intact. The large spikes in standard deviation at 1500 MHz, due to commutation and RADAR have been removed. The magenta block shows a band with large variance that is reduced but not completely de-trended by FFT-MAD. We also show the effects of RFIClean with and without protecting the pulsar's frequency. Unlike the time-domain MAD, we do not flatten the dynamic spectra.