Comprehensive assessment of disordered linker predictions using data from the third critical assessment of protein intrinsic disorder

Disordered flexible linkers (DFLs) are an abundant type of intrinsically disordered regions that contribute to many cellular functions by facilitating movement between adjacent functional domains. Several new sequence-based predictors of DFL regions have been published in recent years, prompting the need to evaluate their predictive performance comparatively. We used the recently released data from the third Critical Assessment of protein Intrinsic Disorder prediction (CAID3) to comprehensively evaluate nine DFL predictors, including seven methods that participated in CAID3 and two additional, recently published DFL predictors. We expanded the CAID3 evaluation by examining predictions on a larger dataset and, arguably, more practical test scenarios that assess predictions at the residue, region, and protein levels. We also evaluated on a subset of CAID3 proteins that share less than 30% sequence similarity with the training sets of the included DFL predictors, yielding more robust estimates of predictive performance. We showed that several DFL predictors accurately identify DFL residues among other types of disordered residues. We also found that different methods perform better in different scenarios, with several methods producing very accurate predictions of DFL residues and a few performing well when predicting proteins with DFLs. While our results suggest that progress has been made in recent years, driven by the use of deep learning and protein language models, we also identified a major weakness in current tools, as they are unable to reproduce the correct sizes and numbers of DFL regions. We also suggested several ways to develop new DFL predictors that would address this issue.