Stellar halo subtraction alternative for accreting companions' characterization with integral field spectroscopy: Analytical and on-sky demonstration on the PDS70, HTLup, and YSES1 systems

Context. Medium-resolution IFS, such as MUSE at the VLT, are equipped to detect the emission lines of faint accreting companions when associated with dedicated stellar halo subtraction methods. We recently proposed a new approach based on polynomial modulations of a stellar spectrum estimate across the field of view, with orthogonal polynomials and lines masking. Aims. We seek to highlight and quantify analytically and on real data the benefits of this new approach over the one classically used, particularly with regard to distortions of the extracted spectra. Methods. We carried out analytical calculations based on simple toy models. Simulations of the most extreme situations identified were used to highlight these problems and corrections. Archival VLT/MUSE data of the young PDS70 and HTLup systems were used to vet the detection and characterization capabilities using on-sky observations. New images of the YSES1 planetary system were used to further illustrate the gains. Results. We show that the state-of-the-art method, based on low-pass filtering, can lead to the self-subtraction of the emission lines and modify the neighboring continua. We show that the proposed technique corrects these characterization problems, while maintaining the same detection capabilities. The two protoplanets PDS70 b and c were detected with 5sigma significance. The Halpha line estimate of the HTLup B stellar companion was improved by ~30% for the integrated flux and by ~8% for the 10%-width. As for YSES1 b, we found it uniquely displays a combination of Halpha, Hbeta, CaII H&K triplet, and HeI lines in emission. Conclusions. The proposed method better preserves the spectral information, notably the emission line fluxes and profiles, while achieving similar detection performance. Based on a linear and parametric approach, it can be extended and/or combined with additional faint signal search algorithms.