Search for pair production of heavy resonances in final states with a photon and large-radius jets in proton-proton collisions at √s = 13 TeV

A search for the pair production of heavy spin-1/2 or spin-3/2 resonances (t^*) in proton-proton collisions at √s = 13 TeV is presented. Data collected with the CMS detector at the CERN LHC from 2016 to 2018 corresponding to an integrated luminosity of 138 fb⁻¹ are used. The analysis targets benchmark signal scenarios where one t^* decays into a top quark (t) and a photon ($γ$), and the other into a t quark and a gluon (g), i.e., pp → t^*t̄^*→$ tt$γ$g. All-hadronic final states from the t pair decay chain are selected using jet substructure techniques. The signal is probed as a function of the t$^*$ candidate mass, which is reconstructed using the photon and a top quark candidate jet. No significant deviation from the background-only hypothesis is found. Observed (expected) upper limits on the signal cross section at 95% confidence level are set, excluding masses of spin-1/2 t^* particles below 930 (930) GeV and spin-3/2 t^* particles below 1330 (1390) GeV. This analysis marks the first search for heavy resonances in the tt̄γg channel. Exploiting the high-energy photon to reduce the backgrounds, this search achieves sensitivity competitive with pp → t^*t̄^* → tt̄γg searches for spin-1/2 t^* despite the small expected t^* → t$γ$ branching fraction.