A multimodal framework for UAV-assisted wildfire detection and contextual alert generation using vision-language models and LLM fusion

The increasing frequency and intensity of wildfires pose a critical threat to ecosystems, property, and human life, creating a need for monitoring systems that combine rapid visual detection with interpretable and actionable reporting. This study presents an integrated multimodal framework for UAV-assisted wildfire monitoring that combines YOLOv12n-based wildfire and smoke detection, Moondream2-based vision-language captioning, and GPT-4o mini-based alert summarization. YOLOv12n is evaluated against six YOLO baselines on an aerial wildfire and smoke dataset and achieves a favorable balance between detection performance and computational efficiency, with an overall precision of 0.649, recall of 0.628, mAP@0.5 of 0.628, an inference time of 1.8 ms, and a model size of 2.52 million parameters. For semantic scene understanding, Moondream2 achieves the highest average BLEU-4, ROUGE-L, and METEOR scores among the evaluated vision-language models. GPT-4o mini similarly outperforms the evaluated summarization baselines in summary-reference alignment, achieving average BLEU-4, ROUGE-L, and METEOR scores of 0.1501, 0.5595, and 0.5461, respectively. Qualitative analysis across five wildfire video scenarios further illustrates the pipeline’s ability to generate contextually relevant descriptions and concise draft alerts under varied visual conditions. Computational measurements for the locally executed models were obtained on a backend workstation, whereas GPT-4o mini was accessed through an external API; therefore, embedded UAV performance and network-inclusive end-to-end latency remain to be validated. Overall, the results demonstrate the potential of combining lightweight visual detection, vision-language reasoning, and LLM-based summarization to support human-verified wildfire situational awareness and emergency decision-making.