Segment Anything Model-Guided Dual-Branch Swin Transformer with Cross-Attention Fusion for Knee Osteoarthritis Severity Grading on X-ray Images

We investigate a Segment Anything Model (SAM)-guided dual-branch Swin Transformer for knee osteoarthritis severity grading from X-ray images. A threshold-assisted SAM pipeline generates bone-region masks without task-specific SAM fine-tuning. Separate Swin-B branches process the image and mask, and two unidirectional cross-attention modules integrate their features. In the reported five-class evaluation, the model achieves 78.80\% accuracy, a macro-averaged one-vs-rest AUC of 0.9588, and quadratic weighted kappa of 0.8780. The corresponding three-class accuracy is 79.58\%. Expanded comparisons include KOA-specific methods and ablations of mask generation, weight sharing, and fusion configuration. These results support the usefulness of combining image context with mask-derived structural information within the evaluated setting.