Quantum phase transition in Heisenberg-Kitaev model

We explore the nature of the quantum phase transition between a magnetically ordered state with collinear spin pattern and a gapless Z₂ spin liquid in the Heisenberg-Kitaev model. We construct a slave particle mean-field theory for the Heisenberg-Kitaev model in terms of complex fermionic spinons. It is shown that this theory, formulated in the appropriate basis, is capable of describing the Kitaev spin liquid as well as the transition between the gapless Z₂ spin liquid and the so-called stripy antiferromagnet. Within our mean-field theory, we find a discontinuous transition from the Z₂ spin liquid to the stripy antiferromagnet. We argue that subtle spinon confinement effects, associated with the instability of gapped $U(1)$ spin liquid in two spatial dimensions, play an important role at this transition. The possibility of an exotic continuous transition is briefly addressed.