Unveiling Excitonic Insulator Signatures in Ta₂NiSe₅

The high-temperature phase of Ta₂NiSe₅, a near-zero-gap semiconductor (EG = 0), is a promising candidate for an excitonic insulator. Given the dome-like evolution expected for an excitonic insulator around EG, we investigated Ta₂NiSe₅, the more semi-metallic Ta₂(Ni,Co)Se₅, and semiconducting Ta₂NiS₅ using high-resolution single-crystal x-ray diffraction and near-edge x-ray absorption fine structure (NEXAFS). Our findings reveal a second-order structural phase transition from orthorhombic (space group: $Cmcm$) to monoclinic (space group: $C2/c$) in Ta₂NiSe₅ and Ta₂(Ni,Co)Se₅, but no transition in Ta₂NiS₅ down to 2 K. This transition breaks two mirror symmetries, enabling and enhancing the hybridization of Ta, Ni, and Se atoms, shortening bond lengths, and strengthening orbital interactions. NEXAFS data confirm stronger hybridization, significant changes in excitonic binding energies, and a key alteration in orbital character, suggesting an excitonic insulating state in Ta₂NiSe₅ and emphasizing the crucial electronic role of orbitals in the formation of the excitonic insulator state.