Transfer-Learning Audio Spectrogram Transformer for Multi-Label Stem Detection with DSP Stabilization

The automatic detection of musical stems within polyphonic mixtures is a vital preliminary step in contemporary Music Information Retrieval (MIR) tasks, notably stem-based source separation applications. While prior research has largely focused on isolated vocal activity detection (VAD) or frame-level instrument tagging, modern engineering requirements necessitate robust Stem Activity Detection (SAD) systems capable of real-time gating. In this study, we propose a hybrid framework combining a Transfer-Learning Audio Spectrogram Transformer (AST) with a strictly causal Digital Signal Processing (DSP) post-processor. To overcome the data-starvation limitations of training Transformers from scratch, we leverage an existing HuggingFace pre-trained backbone, dynamically padded to process 5.0-s high-resolution log-mel spectrograms. To explicitly model the absence of instruments and to mitigate significant segment-level class imbalance, we adapt a physics-aware stochastic source-dropping augmentation combined with an optimized Asymmetric Loss (ASL) function. To eliminate transient classification chatter and bridge micro-silences for live deployment, we introduce a non-linear, zero-look-ahead causal median filter. To ensure statistical rigor and eliminate overlapping-window data leakage, the models are evaluated across three random seeds utilizing track-level bootstrapping (1000 iterations) to generate 95% Confidence Intervals (CIs). Experimental results on the MUSDB18 test set demonstrate that our hybrid AST-DSP framework surpasses an official audio-pretrained PANN CNN14 baseline, achieving a global Macro F1-score of 0.914 and a Macro MCC of 0.462, highlighted by a robust AUPRC of 0.997 for Drums and 0.978 for Vocals. Operating at an end-to-end neural inference and DSP speed of approximately 11.62 ms per segment via Graphics Processing Unit (GPU) acceleration, the proposed system provides a highly stable, real-time streaming-capable gating mechanism for multi-stem presence detection in dense polyphonic mixtures.