What question did this study set out to answer?

The aim is to develop a vocal imitation system that leverages ensemble learning to enhance audio recognition.

January 20, 20260 citationsOpen Access

Class-Aware Hybrid Ensemble for Query-by-Vocal Imitation

Key Points

The aim is to develop a vocal imitation system that leverages ensemble learning to enhance audio recognition.
Utilized four pretrained audio encoders (MobileNetV3, PANNs, PaSST, BEATs) for audio feature extraction.
Implemented two inference systems: a class-aware mixture-of-experts and a global fusion approach.
Fine-tuned models on VimSketch pairs using contrastive learning techniques.
System 1 achieved a Mean Reciprocal Rank (MRR) of 0.3191, outperforming all compared methods.
Class-wise metrics improved significantly with the class-aware approach over the global fusion method.

Abstract

We present a Query-by-Vocal Imitation (QBV) system that ensembles four pretrained audio encoders (MobileNetV3,PANNs, PaSST, BEATs), each contrastively fine-tuned on VimSketch pairs. Two inference fusions are compared:System 1, a novel class-aware mixture-of-experts using per-class Mean Reciprocal Rank (MRR) for soft weights;and System 2, global fusion (weighted average) without class information. System 1 attains the best MRR (0.3191) andclass-wise metrics, outperforming System 2 and single-model baselines. Code is available at: https://github.com/RP335/qvim-challenge-aalto

问 AI

Bookmark

View Full Paper