Fractionalized quantum spin Hall effect

Effects of electron correlations on a two-dimensional quantum spin Hall (QSH) system are studied. We examine possible phases of a generalized Hubbard model on a bilayer honeycomb lattice with a spin-orbit coupling and short-range electron-electron repulsions at half filling, based on the slave-rotor mean-field theory. Besides the conventional QSH phase and a broken-symmetry insulating phase, we find a third phase, a fractionalized quantum spin Hall phase, where the QSH effect arises for fractionalized spinons which carry only spin but not charge. Experimental manifestations of the exotic phase and effects of fluctuations beyond the saddle-point approximation are also discussed.