A dual-phase transformer with multi-dimensional fusion for real-time steel surface defect detection

Accurate detection of small and low-contrast objects remains a major challenge for modern object detectors, particularly in defect detection. Inherent limitations, including shape variation, size variation, and low-contrast defects in defective images, severely hinder the performance of existing detection frameworks. To tackle these issues, we develop a real-time, end-to-end transformer detector, named dual-phase hybrid multi-dimensional DETR (DHM-DETR). Firstly, we design a multi-dimensional feature fusion (MDFF) block, which leverages element-wise multiplication and residual connections to enhance implicit feature dimensions, particularly improving detection sensitivity for low-contrast defects and small objects. Secondly, a hybrid query propagation (HQP) mechanism is developed to redistribute supervision across decoder stages, improving feature refinement and convergence stability. Thirdly, we devise a dual-phase hybrid strategy (DHS) to dynamically adapt the decoder architecture based on training epochs, boosting convergence efficiency while minimising computational overhead. Extensive experiments are performed on the NEU-DET and GC10-DET defect detection datasets to assess the effectiveness of the proposed method, achieving significant improvements of 3.1% and 5.1% in mAP50 over state-of-the-art methods. Data perturbation study and model generalisation analysis on ASSDD dataset validate the robustness and transferability of our model, highlighting its effectiveness in challenging industrial inspection scenarios.