A Reinforcement-Driven Multiple Instance Learning Framework for Multi-Task Speaker Attribute Prediction

Abstract Standard models for predicting speaker attributes from text often fail to manage multiple, interdependent attributes simultaneously, and existing Reinforced Multiple Instance Learning frameworks are typically limited to single-task prediction. To address this, we propose and evaluate a Reinforced, Multi-Task, Multiple Instance Learning framework, a novel framework that enhances Reinforced Multiple Instance Learning with Multi-Task Learning to predict a speaker’s age, gender, and political party from congressional speeches. A central goal of our work was to investigate the optimal parameter-sharing strategy. We compared a fully shared architecture against a dynamic task clustering mechanism designed to mitigate negative transfer by adaptively grouping related tasks. Our results demonstrate that the multi-task approach significantly outperforms single-task baselines. Interestingly, the model with a fully shared representation achieved the highest macro average F1-score of 0.668, suggesting the tasks in this dataset were sufficiently correlated to benefit from shared features without needing adaptive separation. This work contributes a more effective method for weakly-supervised, multi-attribute prediction and provides crucial insights into the trade-offs between different parameter-sharing strategies in Multi-Task Learning.