A Fine-Tuned Lightweight Vision-Language Safety Layer with Weather-Generalising Decisions at Signalized Intersections

Conventional traffic signal controllers sense traffic only through point detectors, so they cannot recognize safety-critical events such as jaywalking or red-light running. This study tests whether a vision-language model (VLM) can act as a semantic safety layer for signal control. The VLM analyzes short camera clips of an intersection and may request only bounded, conflict-free interventions (temporary all-red holds); a rule-constrained controller keeps the phase plan and clearance times fixed, so each intervention can be audited. In a CARLA world model, 1020 event clips were automatically generated, covering nine weather and lighting combinations, and a lightweight 3-billion-parameter model (Qwen2.5-VL-3B) was fine-tuned on it using low-rank adaptation. On three adverse conditions unseen during training, the fine-tuned model always produced decisions in the correct format and range, made the correct intervention decision in all 313 held-out clips, and set hold times within 0.15 s of the target. These results reflect reliable imitation of a bounded expert policy, not general-purpose visual reasoning. Two zero-shot baselines requested an intervention in nearly every scene, the untrained 3B model produced no output the controller could accept, and both were slower than the fine-tuned model. In closed-loop simulation, the fine-tuned model reduced pedestrian exposure to conflicting vehicle green by 68% and 73% at low and medium demand, outperforming both fixed-time and actuated pedestrian-clearance control. The main barrier is perception: pedestrian detection degrades at night and in heavy rain. Within that limit, a small fine-tuned VLM provides an auditable safety layer whose decisions are reliably formatted and bounded.