Detecting the third family of compact stars with normalizing flows

We explore the anomaly detection framework based on Normalizing Flows (NF) models introduced in PhysRevC. 106. 065802 to detect the presence of a large (destabilising) dense matter phase transition in neutron star (NS) observations of masses and radii, and relate the feasibility of detection with parameters of the underlying mass-radius sequence, which is a functional of the dense matter equation of state. Once trained on simulated data featuring continuous M (R) solutions (i. e. , no phase transitions), NF is used to determine the likelihood of a first-order phase transition in a given set of M (R) observations featuring a discontinuity, i. e. , perform the anomaly detection. Different mock test sets, featuring two branch solutions in the M (R) diagram, were parameterized by the NS mass at which the phase transition occurs, Mc, and the radius difference between the heaviest hadronic star and lightest hybrid star, R. We analyze the impact of these parameters on the NF performance in detecting the presence of a first-order phase transition. Among the results, we report that given a set of 15 stars with radius uncertainty of 0. 2 km, a detection of a two-branch solution is possible with 95\% accuracy if R > 0. 4 km.