Particle-hole condensates of higher angular momentum in hexagonal systems

Hexagonal lattice systems (e.g., triangular, honeycomb, kagome) possess a multidimensional irreducible representation corresponding to d_x²-y² and dxy symmetry. Consequently, various unconventional phases that combine these d-wave representations can occur, and in so doing may break time-reversal and spin-rotation symmetries. We show that hexagonal lattice systems with extended repulsive interactions can exhibit instabilities in the particle-hole channel to phases with either d_x²-y²+dxy or $d+id$ symmetry. When lattice translational symmetry is preserved, the phase corresponds to nematic order in the spin channel with broken time-reversal symmetry, known as the β phase. On the other hand, lattice translation symmetry can be broken, resulting in various d_x²-y²+dxy density wave orders. In the weak-coupling limit, when the Fermi surface lies close to a van Hove singularity, instabilities of both types are obtained in a controlled fashion.