Assessing the predictive limits of machine learning on cyclic peptides

Machine learning (ML) has become a key tool in molecule discovery and optimization. Yet cyclic peptides, which occupy an intermediate chemical space between small molecules and proteins, have received limited attention in ML research, leaving the predictive value of current methods and future research needs unclear. To address this gap, we systematically evaluated ten model families (from classical ML approaches to pretrained deep-learning models) and nine molecular representations (from molecular descriptors to strings and graphs) across twelve prediction tasks spanning three properties and 9,341 peptides. Membrane permeability showed the clearest predictive signal, whereas binding-affinity and half-life prediction remained challenging. Evaluating models against explicit ‘no-skill’ references was central to interpreting these results and contextualizing performance across endpoints and data splits. No single model family performed best across all tasks: tree ensembles were strong in regression, while pretrained chemical language models performed best on average in classification. We further show that data splitting, task formulation, and dataset composition can influence apparent performance. Together, these findings highlight data availability and assay consistency, limited statistical power, and evaluation design as key challenges for cyclic-peptide modeling. To support progress in these areas, we release CycPepNet (https://github.com/molML/CycPepNet), an open framework that brings together curated datasets, evaluation protocols, and full results to enable reproducible comparisons and future method development.