Multiagent Federated Learning, Interoperability, and Virtual-Physical Co-simulation

The composition of different types of terrains and presence of a variety of objects and artifacts in a real environment are always evolving. Not every aspect of the environment can be learned, modeled, and synthesized a priori in an artificial intelligence (AI)-enabled, decision-making pipeline. If collaboration can be enabled among several deployed robots in different remote zones, it is possible to develop a generalized terrain and object detection model capturing greater uncertainty and variability in an environment. Conventionally, to realize this, data sharing between agents and servers is warranted; however, that may introduce a higher risk of an adversarial attack.Federated learning can be a useful approach to mitigate this issue. The object detection model can be trained using federated learning in which training data will not be explicitly shared between the robots performing terrain reconnaissance in various geolocations. The robots can learn generalized features while training the model onboard and share the model updates from one location to another to support the collaborative training for distributed learning and real-time situational awareness. Also, simulating physical autonomous systems with virtual entities allows exploring complex interactions between collaborating agents at scale. Virtual-physical co-simulation mitigates costly environments populated by large numbers of autonomous entities. A high-level overview of these approaches and a case study to overcome challenges of real-time adaptability and experiment scalability in multiagent teaming are presented in this article.