Excitonic instability towards a Potts-nematic quantum paramagnet

Magnetic frustration can lead to peculiar magnetic orderings that break a discrete symmetry of the lattice in addition to the fundamental magnetic symmetries (i.e., spin rotation invariance and time-reversal symmetry). In this work, we focus on frustrated quantum magnets and study the nature of the quantum phase transition between a paramagnet and a magnetically ordered state with broken threefold (Z3) crystal rotation symmetry. We show that the transition can occur in two stages, giving rise to an intermediate nematic phase in which rotation symmetry is broken but the system remains magnetically disordered. Since the nematic transition is described by the three-state Potts model, the intermediate phase is a Z3 Potts-nematic phase. Our prediction of the existence of a Potts-nematic phase is based on an analysis of bound states formed from two-magnon excitations in the paramagnet, which become gapless while single-magnon excitations remain gapped. By considering three different lattice models, we demonstrate a generic instability towards two-magnon bound state formation in the Potts-nematic channel. We present both numerical results and a general analytical perturbative formula for the bound state binding energy similar to BCS theory. We further discuss a number of different materials that realize key features of the model considered, and thus provide promising venues for possible experimental observation. Published by the American Physical Society 2024