Blinking like Fireflies: Convolutional neural networks for bio-inspired visible light communication between nano-drones

We present a novel visible light communication (VLC) system to enable swarms of pocket-sized nano-drones to exchange messages through light-emitting diodes’ (LEDs) blinking, like fireflies. While a nano-drone is sending a message encoded via LED’s blinking, a receiver one reconstructs it employing only a low-resolution camera and an ultra-low-power GreenWaves application processor 8 (GAP8) system-on-chip running a compact (7500 parameters) fully convolutional neural network (FCNN) that achieves 0.87 area under the curve (improving upon prior nano-drone VLC work by +0.27) and predicts both the LEDs’ state and the image position of the sender nano-drone. A stream of LEDs’ state (on/off) is then continuously fed to a synchronization-free decoder, which also runs aboard the nano-drone. Our approach, only leveraging inexpensive onboard hardware (camera and LEDs), achieves competitive accuracy compared to state-of-the-art VLC methods designed for larger drones while consuming orders of magnitude less power (101 milliwatt compared to more than 25 watt). By employing a pair of Crazyflie nano-drones, our FCNN reaches 39 frames per second, which allows from 2.8 to 8.6 bits per second throughput with a per-bit accuracy of 93 percent and from 0.6 to 1.6 bits per second with a per-bit accuracy of 99.8 percent. Finally, our closed-loop system is experimentally demonstrated in the field, where two fully autonomous nano-drones exchange messages with our VLC technique while following each other thanks to the predicted image position.