An end-to-end deep learning framework for automated transcription of humpback whale songs

Humpback whales produce complex, hierarchically structured songs during the breeding season, composed of units, phrases, and themes. These songs are essential for understanding whale behavior, cultural transmission, and population dynamics. However, song analysis is challenging due to the interference from environmental and anthropogenic sounds. This study presents an end-to-end deep learning framework for automated detection and transcription of song units. By training to predict the occurrence of song units directly from raw waveforms, the model achieved strong performance (AUC: 0.94), accurately detecting whale songs in noisy conditions dominated by shipping noise and fish chorus. Each detected unit was represented by embeddings generated from the model. Uniform Manifold Approximation and Projection was applied to cluster similar units, revealing patterns corresponding to recurring phrases and themes. This embedding-based method enables unsupervised transcription by capturing fine spectral-temporal distinctions between song units. Applied to recordings from Sesoko Island, Okinawa, the approach successfully revealed complex song structures, reducing manual workload and enabling comparison across song sequences. The framework provides a robust, scalable solution for passive acoustic monitoring of humpback whales in breeding ground and facilitates deeper insight into the variation of their songs.