Key result
The Deep Q-Network (DQN) algorithm outperformed Proximal Policy Optimization (PPO) and a greedy search baseline in optimizing the free energy of RNA secondary structures.
Why the study?
Rising costs of developing new drugs and treatments have motivated optimization techniques in biomolecular design, where directed evolution is currently the most widely used approach.
Deep Q-Network (DQN) outperforms other reinforcement learning algorithms like PPO in optimizing RNA secondary structure design based on free energy.
Rising costs in recent years of developing new drugs and treatments have led to extensive research in optimization techniques in biomolecular design. Currently, the most widely used approach in biomolecular design is directed evolution, which is a greedy hill-climbing algorithm that simulates biological evolution. In this paper, we propose a new benchmark of applying reinforcement learning to RNA sequence design, in which the objective function is defined to be the free energy in the sequence's secondary structure. In addition to experimenting with the vanilla implementations of each reinforcement learning algorithm from standard libraries, we analyze variants of each algorithm in which we modify the algorithm's reward function and tune the model's hyperparameters. We show results of the ablation analysis that we do for these algorithms, as well as graphs indicating the algorithm's performance across batches and its ability to search the possible space of RNA sequences. We find that our DQN algorithm performs by far the best in this setting, contrasting with, in which PPO performs the best among all tested algorithms. Our results should be of interest to those in the biomolecular design community and should serve as a baseline for future experiments involving machine learning in molecule design.
No takes yet. Share an insight, caveat, or question.
Whatley et al. (2021) studied RNA secondary structure design. Deep Q-Network (DQN) vs. Proximal Policy Optimization (PPO) and greedy search baseline was evaluated on Free energy in the sequence's secondary structure. The Deep Q-Network (DQN) algorithm outperformed Proximal Policy Optimization (PPO) and a greedy search baseline in optimizing the free energy of RNA secondary structures.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: