Distributed Formation Control of Satellite Swarms Under Orbital Perturbations Using Multi-Agent Reinforcement Learning

This paper presents a multi-agent reinforcement learning (MARL) framework for autonomous distributed formation control of satellite swarms under representative low-Earth-orbit-analogue perturbations. A 2-D planar simulator built on MuJoCo and Gymnasium models four in-plane orbital disturbances—solar radiation pressure, atmospheric drag, thruster noise, and residual aerodynamic wind—as equivalent planar force proxies, consistent with the air-bearing testbed tradition for spacecraft GNC validation. A Proximal Policy Optimization (PPO) controller with centralized training and decentralized execution (CTDE) is trained progressively, scaling from one satellite to a four-agent ring formation. The final model (PPO_final) achieves 99.1 % success with only 0.9 % collision rate over 1 000 evaluation episodes at full perturbation magnitude. Complementarily, a Ufactory Lite6 robotic arm on an air-bearing platform is used as a terrestrial analogue of an on-orbit manipulator; the trained policy transfers successfully to the physical hardware. The key contributions are a progressive four-phase scaling curriculum, an explicit angular role assignment that resolves combinatorial ambiguity beyond two agents, and a linear reward funnel that avoids the quadratic deadlock trap, providing a Sim-to-Real validation of the pipeline.