A probabilistic vision-language framework for habitat mapping within biodiversity projects using remote sensing data, ecological text and prior maps

Accurate and time-sensitive spatial ecological information is essential for biodiversity policies and environmental planning, yet existing remote sensing (RS) classification workflows may struggle to integrate ecological semantics and prior domain knowledge, limiting their interpretability and performance for statutory habitat assessments such as Biodiversity Net Gain (BNG). In this study we explore the potential of Vision-Language Models (VLM) for UK Habitat (UKHab) classification, the standard framework for BNG assessments. Through integrating textual descriptors with spectral and structural RS data from Sentinel-2 and UK National lidar datasets, we address two methodological hypotheses: (1) how does the level of ecological detail in textual habitat descriptions influence VLM classification performance, and (2) can probabilistic priors derived from existing land cover products improve reclassification to UKHab categories within a VLM framework? Five model configurations were evaluated ranging from a vision-only approach to four text models, representing increasing semantic detail. Our results show that incorporating carefully designed text consistently improves classification performance, particularly for habitats with limited training data that typically underperform in conventional classification schemes. These habitats often correspond to higher biodiversity value and greater relevance for BNG delivery. For example, Wetland showed an improvement of 19.36% and Heathland and Shrub improved by 9.7% relative to the vision-only approach. The highest performing model, Text Level 3, included core ecological details and achieved the highest macro-averaged scores in precision (0.93), recall (0.87), F1 score (0.90), and overall accuracy (98.1%), representing an improvement of 5%, 3%, 4%, and 0.4%, respectively, compared to the vision-only approach. This study also tested whether the inclusion of probabilistic priors derived from an existing (but differently classified) land cover map (Living England map) improved VLM performance in classifying habitats to UKHab. While improvements were observed for certain classes, such as Heathland and Shrub (+4.54% in overall accuracy), priors produced inconsistent performance including a 16.13% decrease in Wetland accuracy due to errors within the prior map. Overall, this study demonstrates that VLMs, when paired with carefully designed textual inputs provide a promising tool for ecological mapping, offering advantages in low-data scenarios and supporting biodiversity assessments and conservation decision making.