LoRA-BERT: a natural language processing model for robust and accurate prediction of long non-coding RNAs

Long non-coding RNAs (lncRNAs) are key regulators in many biological processes. While they share some sequence features with messenger RNAs (mRNAs), lncRNAs have distinct functions, opening new research opportunities. Recent advances in next-generation sequencing have made it easier to detect lncRNAs, and deep learning methods are increasingly used to classify these transcripts. Although these methods have significantly improved the efficiency of identifying lncRNAs, they often lack robustness, and the prediction accuracy tends to vary significantly depending on the quality of the transcript. To tackle this issue, we introduce LoRA-BERT, a novel lncRNA prediction algorithm built on the bidirectional encoder representation from transformers (BERT). LoRA-BERT is designed to effectively capture information at the nucleotide level that is important for lncRNA classification, leading to more robust and accurate prediction outcomes that are not significantly affected by the quality of the input transcript. Through performance evaluations on comprehensive benchmarks, we demonstrate that LoRA-BERT outperforms existing schemes in terms of accuracy, efficiency, and robustness. Especially, unlike other methods, LoRA-BERT retains good predictive capability for partial transcripts, a critical feature that makes it applicable for reliable lncRNA prediction even when the sequencing depth is relatively low. Availability and implementation: The code repository is available at https://github.com/Nick-Jeon/LoRA_BERT .