Molecular generators enable the exploration of chemical space to identify novel compounds with desirable properties. However, assessing their performance remains challenging due to the structural diversity and volume of the generated molecules. Commonly used evaluation metrics, focusing on chemical validity and novelty, do not fully align with the primary goal of molecular generation: the discovery of new biologically active compounds. To address this limitation, we introduce scaffold-based metrics that enable a fair comparison by evaluating a generator's ability to recover biologically relevant scaffolds absent from the input set. We applied the scaffold Recovery Score (RS), SEt scaffold Diversity (SED), and Absolute SEt scaffold Recall (ASER) metrics to compare several molecular generators, including Molpher, DrugEx, REINVENT, and Graph-based genetic algorithm. The proposed scaffold-based metrics provide a realistic framework for evaluating and optimizing molecular generators for their practical use in drug discovery scenarios, particularly in the design of focused virtual chemical libraries. The metrics are available as open-source in a GitHub repository at https://github.com/filvaleriia/scaffold-based-metrics.
Fil et al. (Fri,) studied this question.