Large second-harmonic generation and linear electro-optic effect in the bulk kagome lattice compound Nb3MX7 ( M=Se…

Layered Nb₃MX₇ (M=Se,S,Te; X=I,Br) compounds with a noncentrosymmetric kagome lattice structure constitute a promising candidate for exhibiting optical anisotropy and nonlinearity. Nevertheless, there has been limited research on their nonlinear optical properties both theoretically and experimentally. In this paper, we systematically study the linear and nonlinear optical responses of the bulk Nb₃MX₇ (M=Se,S,Te; X=I,Br) family. Previously, two distinct stacking sequences (i.e., AA and AB stacking) in Nb₃MX₇ compounds had been identified in experiments. Therefore, the influence of the stacking sequence and the isoelectronic substitution on the linear and nonlinear optical response of the Nb₃MX₇ compounds are investigated. It reveals that all four materials (AB-Nb₃TeI₇, AA-Nb₃TeI₇, AB-Nb₃SeI₇, and AA-Nb₃SBr₇) exhibit significant second-harmonic generation and linear electro-optic effects. Notably, the second-harmonic generation coefficient (χzzz⁽²⁾) of AB-Nb₃TeI₇ reaches 3081 pm/V at 0.91 eV, which is nearly 14 times as large as that of the known nonlinear optical material GaN. Furthermore, AB-Nb₃TeI₇ exhibits a large linear electro-optic coefficient rzzz(0) of 5.94 pm/V, which outperforms that of GaN by almost tenfold. The prominent nonlinear optical responses observed in the Nb₃MX₇ compounds are attributed to the intralayer directional covalent bonding. Our findings indicate that the Nb₃MX₇ materials would have potential applications in nonlinear optical and linear electro-optic devices.