Answers are grounded in this paper and cite their sources.
Background . Resistance genes form an abundant reservoir of resistance factors within the oral microbiome. Scalable attention alternatives may be able to annotate them; however, none has been evaluated for annotating them without learning from memorization of the data. Methods . We curated 26,324 resistome proteins from UniProtKB, CARD, and NCBI’s AMRFinderPlus datasets, clustered by 40% sequence similarity and 80% coverage. The proteins were partitioned into clusters. Eight causal residue-level architectures (Transformer, Funnel-Transformer, Performer, Hyena, causal state-space model, Mixture-of-Recursions MoR with a causal token choice router, parameter-matched MoR, and ablation of pairwise merge gating) were trained with identical data (up to 100 epochs with early stopping) for up to three random seeds and evaluated on held-out clusters and on independent databases with autoregressive causality audit. We used antibiotic class prediction with frozen representations as a measure of transferability. Results . 26,324 proteins aggregated into 1,437 clusters (the largest being 3,333). On held-out clusters, all architectures outperformed the residue composition benchmark (perplexity 17.50), with Hyena outperforming Transformer (14.93 ± 0.58 vs 15.33); MoR (0.79M parameters) achieved 16.37, a sixth of the baseline value. On external validation (against the CARD dataset), only 4 of 8 architectures exceeded the composition benchmark perplexity of 17.99; the Transformer degrades by 27%; constrained architectures did better, with the MoR reaching 17.40, parameter-matched to the Transformer. Perplexity on external validation increased as similarity between the evaluation and training sets increased. MoR is 5.7x faster than Transformer on 4096 residues; the router uses 42% fewer block applications than the fixed depth. Conclusions . Performance of architectures trained on the oral resistome critically depends on the way evaluation is done: homology-aware partitions and external validation reverse the order of architectures seen when evaluated on held-out clusters only. Recursive architectures perform best on external validation and are the least costly to compute, using less than 1/7th of the parameters compared to Transformers. Yet, for proteins without apparent homology to training sets, none of the eight architectures outperform residue composition.
No takes yet. Share an insight, caveat, or question.
Saeed et al. (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: