Blind source separation in complex marine soundscapes: An unsupervised two-stage clustering based on non-negative matrix factorization

Soundscape monitoring assesses biodiversity by analyzing environmental acoustic signals, but overlapping sound sources in complex environments limit the performance of traditional methods. We propose an unsupervised blind source separation algorithm using nonnegative matrix factorization (NMF) and a two-stage coarse-to-fine clustering strategy. First, NMF decomposes the mixed spectrogram into spectral bases and temporal activations. In the clustering stage, coarse clustering is first performed via a second NMF with sparsity constraints using the spectral bases, temporal activations, or their derived features. Subsequently, fine clustering is performed using hierarchical clustering-guided K-means, which leverages complementary feature dimensions to refine the initial groups. Performance was evaluated on both simulated data and real-world recordings using separation quality metrics, detection metrics, and a composite score. Robustness was further examined under different mixture complexities. Results demonstrate that the proposed method achieves superior separation performance compared to one-stage clustering on both simulated and real-world data, particularly in successfully recovering a greater number of source components. This work provides a practical approach for fine-grained source separation in complex soundscapes and supports quantitative ecoacoustic analysis.