HDAC8 Assistant: a QSAR-guided molecular generation platform for de novo design and virtual screening of HDAC8 inhibitors

molecular generation module based on recurrent neural networks (SMILES-RNN) and reinforcement learning (REINVENT 2). For targeted discovery of novel inhibitors, an original reward function was proposed that combines QSAR-predicted activity with applicability domain control for generated compounds. This approach enables simultaneous optimization of predicted inhibitory activity while limiting extrapolation beyond the model applicability domain. HDAC8 Assistant also implements assessment of drug-likeness according to the Muegge criteria, as well as PAINS, toxicophore, and Brenk structural-alert detection. Molecular docking was performed for the most promising compounds generated by HDAC8 Assistant to analyse their interactions with the HDAC8 active site.