RflyBench-FD: The Benchmark for Quadcopter Fault Diagnosis

The reliability of quadcopters is becoming increasingly important. However, the field of quadcopter fault diagnosis lacks a standardized, reproducible benchmark for fair and comprehensive algorithm evaluation. To address this gap, this paper presents RflyBench-FD, a standardized, open benchmark for quadcopter fault diagnosis built upon the RflyMAD dataset, publicly accessible at https://www.rflybench.com/competitions/5/ . RflyBench-FD establishes a complete evaluation framework that covers eight representative fault types and provides standardized simulation/real dual-domain splits, a well-defined task formulated as eight parallel classification sub-tasks, publicly released scoring scripts, and a multi-dimensional metric system (accuracy, F1-score, and robustness under Gaussian noise at 8/4/2 dB SNR and physically motivated sensor degradations) aggregated into a weighted composite score whose ranking stability is verified by a sensitivity analysis. To validate the benchmark, we evaluate twelve baselines spanning classical machine learning, temporal deep learning, domain adaptation, transfer learning, and training-mode ablations, complemented by noise tests, physically motivated sensor-degradation tests, leakage-oriented checks, and paired significance tests; the results show that the benchmark produces stable and discriminative rankings across method families. The platform further supports daily submissions, a governed real-time leaderboard with anti-overfitting safeguards, full result traceability, and an LLMassisted analysis report. By providing a fair, reproducible, and insight-rich evaluation infrastructure, RflyBench-FD aims to support the development and real-world deployment of robust fault diagnosis solutions for autonomous quadcopters.