Diversity in the impact of heterogeneities on recurrent networks performing a cognitive task

Artificial recurrent networks are widely used to model the complex dynamics of biological neural circuits performing cognitive tasks. Yet, most studies assume homogeneous units, ignoring functionally important neural heterogeneities. We present a systematic framework to quantify the impact of heterogeneities on recurrent networks performing a cognitive task. We trained a population of networks, differing in hyperparameters, with six graded levels of intrinsic heterogeneities. Our analyses unveiled a surprising diversity in the impact of intrinsic heterogeneities on training, task-execution dynamics, and robustness, despite identical training conditions. First, the dependence of training performance on heterogeneity levels showed pronounced network-to-network variability. Second, task-execution dynamics exhibited marked variability in their dependence on heterogeneities. Finally, post-training perturbations strongly affected robustness, manifesting substantial network-specific variability. This striking diversity in the impact of heterogeneities underscores the need to consider global structure, interactions, and adaptations across multiple heterogeneities to understand collective function in complex neural circuits.