Mitigating incoherent excess variance in high-redshift 21 cm observations with multi-output cross-Gaussian process regression

Systematic effects that limit the achievable sensitivity of current low-frequency radio telescopes to the 21 cm signal are among the foremost challenges in observational 21 cm cosmology. The standard approach to retrieving the 21 cm signal from radio interferometric data separates it from bright astrophysical foregrounds by exploiting their spectrally smooth nature, in contrast to the finer spectral structure of the 21 cm signal. Contaminants exhibiting rapid frequency fluctuations, on the other hand, are difficult to separate from the 21 cm signal using standard techniques and the power from these contaminants contributes to low-level systematics that can limit our ability to detect the 21 cm signal. Many of these low-level systematics are incoherent across multiple nights of observation, resulting in an incoherent excess variance above the thermal noise sensitivity of the instrument. In this work, we developed a method called cross-covariance Gaussian process regression (cross-GPR) that exploits the incoherence of these systematics to separate them from the 21 cm signal, which remains coherent across multiple nights of observation. We developed and demonstrated the technique on synthetic signals in a general setting, then we applied it to gridded interferometric visibility cubes. We performed realistic simulations of visibility cubes containing foregrounds, 21 cm signal, noise, and incoherent systematics. The simulations show that the method can successfully separate and subtract incoherent contributions to the excess variance. Furthermore, its advantages over standard techniques become more evident when the spectral behavior of the contaminants resembles that of the 21 cm signal. Simulations performed on a variety of 21 cm signal shapes also reveal that the cross-GPR approach can subtract incoherent contributions to the excess variance, without suppressing the 21 cm signal. The codes underlying this article are publicly available in the Python library crossgp and will soon be integrated into the LOFAR and NenuFAR foreground removal and power spectrum estimation framework ps_eor.