Quantum Circuit Discovery for Fault-Tolerant Logical State Preparation with Reinforcement Learning

One of the key aspects in the realization of large-scale fault-tolerant quantum computers is quantum error correction (QEC). The first essential step of QEC is to encode the logical state into physical qubits in a fault-tolerant manner. Recently, flag-based protocols have been introduced that use ancillary qubits to flag harmful errors. However, there is no clear recipe for finding a compact quantum circuit with flag-based protocols for fault-tolerant logical state preparation. It is even more difficult when we consider the hardware constraints, such as qubit connectivity and gate set. In this work, we propose and explore reinforcement learning (RL) to automatically discover compact and hardware-adapted quantum circuits that fault-tolerantly prepare the logical state of a QEC code. We show that RL discovers circuits with fewer gates and ancillary qubits than published results without and with hardware constraints of up to 15 physical qubits. Furthermore, RL allows for straightforward exploration of different qubit connectivities and the use of transfer learning to accelerate the discovery. More generally, our work opens the door towards the use of RL for the discovery of fault-tolerant quantum circuits for addressing tasks beyond state preparation, including magic state preparation, logical gate synthesis, or syndrome measurement.