A Unified Peptide Generative Framework via a Weakly Order-Dependent Autoregressive Language Model and Lifelong Learning

Bioactive peptides have become strong candidates for a variety of clinical therapies due to their diverse advantages, which promote the development of deep generative models for peptide generation. Considering that existing methods cannot effectively deal with the conformational flexibility of peptides and find it difficult to capture accurate residue-to-residue interaction dependencies, we propose a unified peptide generative framework, PepGenWL, via a weakly order-dependent autoregressive language model and lifelong learning. PepGenWL introduces tolerance for out-of-order input as an inductive bias into the autoregressive language model, coupled with Mixture-of-Experts-style plugins to maintain the optimal trade-off between memory stability and learning plasticity across multiple rounds of fine-tuning. The superiority of PepGenWL was demonstrated by generating three classes of therapeutic peptides, including antimicrobial peptides, anticancer peptides, and peptide binders. Under performance evaluation on raw and permuted peptide sequences, PepGenWL not only surpassed state-of-the-art baseline models across the board but also exhibited a significant propensity to incorporate specific residues that are beneficial for antimicrobial or anticancer bioactivity. Furthermore, the property-guided peptide binder generation, screening, and in vitro experimental validation pipeline was presented, achieving a target binding rate of 28.6% with binding specificity. More importantly, the applicability of PepGenWL can be broadened to encompass the peptide SMILES chemical space, thereby facilitating the generation of chemically modified peptides as well as cyclic peptides. Overall, PepGenWL is a unified framework for general-purpose peptide generation that can be flexibly customized for different task requirements.