GLoRA: a novel parameter-efficient fine-tuning framework for GIS large language models

Since large language models (LLMs) generally have a large parameter scale and acquire knowledge across multidisciplinary corpora, their parameters tend to exhibit varying levels of importance for GIS knowledge alignment. Specifically, certain model layers of LLMs may already align well with GIS knowledge, while others may require significant adaptation and thus play a more critical role during the fine-tuning process. However, representative fine-tuning methods, such as low-rank adaptation (LoRA), apply a uniform treatment to all parameters, overlooking their varying importance for GIS downstream tasks. This strategy may disrupt well-adapted layers or limit less-adapted ones from effectively encoding GIS knowledge. Therefore, we develop a novel GIS knowledge-aware LoRA allocation (GLoRA) scheme that adaptively allocates trainable parameters to layers based on their importance. Specifically, GLoRA first investigates a GIS knowledge-aware strategy to identify the importance of model layers for a given GIS task in a data-dependent way. Second, it dynamically adjusts the size of LoRA modules, allocating more parameters to more important layers to enhance their representation capability. This adaptive approach ensures efficient parameter utilization while preserving the strengths of well-adapted layers. We evaluated GLoRA on three GIS-related tasks, and the results show its improved performance compared to recent baselines with comparable parameter budgets.