Ensemble of deep architectures for SWARM AI-based decision-making

This work studies how an artificial agent can participate in swarm-based decision-making alongside human experts by learning collective decision dynamics. We formulate this problem as a multimodal sequential prediction task that combines two complementary sources of information: (i) spatiotemporal interaction traces generated by experts through the HANS web platform and (ii) the corresponding image. We propose a hybrid architecture with an LSTM branch for interaction sequences and a WideResNet branch for image encoding, followed by feature fusion and joint optimisation. The model predicts the evolution of the collective decision at each time step, thereby acting as an autonomous swarm AI agent that receives the same evidence as a human participant. We evaluate the approach on four use cases from environmental microscopy and biomedical imaging: diatoms, cyanobacteria, HER2 and Ki67. The analysis considers interaction-level prediction error, measured by RMSE, and decision-level metrics, including precision, sensitivity, specificity, F1-score and balanced accuracy under Top- settings. Across the datasets, the model achieves low interaction error and strong balanced accuracy, while the temporal analysis shows that it captures meaningful convergence patterns in expert consensus. These results support the feasibility of hybrid multimodal models for embedding artificial agents in collaborative decision environments.