Answers are grounded in this paper and cite their sources.
Assesses enzyme family classification using homology-aware benchmarks, suggesting improved evaluation metrics.
The CAZy database organizes carbohydrate-active enzymes into more than 800 families with distinct biochemical roles, making automatic family assignment from sequence a high-value task in metagenomics and enzyme engineering. Although recent deep learning studies have reported strong classification performance, we identify a critical evaluation bias that has not been systematically quantified in this setting: commonly used random train/test splits allow homologous sequences to appear in both training and evaluation sets, thereby inflating apparent performance by 5.9–12.0%. We refer to this effect as homology leakage and quantify its magnitude across five representative classification approaches. To enable more rigorous evaluation, we construct a homology-aware benchmark comprising 30,000 sequences, 60 families, and 6 classes, split using MMseqs2 clustering at 20% sequence identity. Under this protocol, the method most strongly affected by leakage is homology-based inference itself ( \(Δ \) F1 \(= +0.076\) ), whose performance would be reported as 0.894 under a random split but drops to 0.818 under homology-aware evaluation. Against this benchmark, our proposed model, ESM-2+Mean+MTL (two-stage selective fine-tuning with joint family/class supervision), achieves a family Macro-F1 of 0.805 ± 0.004, narrowing the gap to homology-based inference while providing calibrated uncertainty estimates (ECE \(= 0.028\) ) and compatibility with 8 GB consumer GPU hardware. These findings support homology-aware evaluation as a more reliable standard for protein sequence classification.
No takes yet. Share an insight, caveat, or question.
Yurttakal et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: