A Resource-Aware Multi-Fidelity Optimization Framework for Multimodal Music Information Analysis

This study develops an offline, budget-aware, resource-efficient multi-fidelity configuration optimization technique for music information analysis for music information analysis with the Latent-Kernel Surrogate Ensemble capable of fidelity-based performance prediction and uncertainty measurement, and a Budget-Aware Model-Based Halving algorithm for budget-aware configuration pruning. Configuration options include combinations of music feature clusters, fusion strategies, model types, and their hyperparameters. Fidelity levels depend on the computational-budget parameters of the number of epochs for learning, training data subset size, feature resolution, and the validation budget. The effectiveness of the proposed technique is illustrated using four music information retrieval datasets – RWC Pop, SALAMI, Isophonics, and MedleyDB. The corresponding metrics include: Accuracy, Recall, F1 score, and Area under the Curve on RWC Pop – 86.92%, 83.79%, 85.03%, and 89.17%; on SALAMI – 88.75%, 85.91%, 86.57%, and 88.40%; on Isophonics – 83.26%, 80.14%, 81.37%, and 86.59%; and on MedleyDB – 78.34%, 79.82%, 78.95%, and 83.27%, respectively. This means that the presented LKSE-BAMBH technique enables the identification of competitive multimodal music information analysis configurations given limited resources for configuration evaluation. The proposed framework is limited to offline model selection and does not evaluate real-time musical performance, latency, synchronization, or human-in-the-loop feedback.