Layered nickelates are believed to exhibit superconductivity similar to that found in the cuprates. However, the precise crystal structure of the superconducting phase of the layered nickelates has not been fully clarified. Here, I use first principles calculations to study the pressure dependence of the structural instabilities in the single-layer-trilayer La3Ni2O7, which is one member of the layered nickelates family that also shows signatures of superconductivity. I find a nearly dispersionless nondegenerate phonon branch in the parent P4/mmm phase that is unstable along the Brillouin zone edge M (½, ½, 0) → A (½, ½, ½) at all investigated pressures up to 20 GPa. Calculations show additional doubly-degenerate instabilities along the edge MA at lower pressures. I used group-theoretical analysis to identify the distinct low-symmetry distortions possible due to these instabilities and generated them using the eigenvectors of the unstable modes. Structural relaxations show that the lowest energy structures at 0 and 10 GPa involve condensation of both the nondegenerate and doubly-degenerate instabilities, which is in contrast to the experimental refinements that involve condensation of only the doubly-degenerate branch. I also find that structural distortions are energetically favorable at 20 GPa, contrary to the experiments that do not observe any distortions of the parent P4/mmm structure at high pressures.
Alaska Subedi (Thu,) studied this question.