PulseExploreJournal ClubDebatesTrendingResearchersJournals
Instagram
HomeExploreJournal ClubTrending
Synapse
⌘+K
Synapse
October 2, 20250 citationsOpen Access

GUI-ARP: Enhancing Grounding with Adaptive Region Perception for GUI Agents

View Full Paper
XYXuehan YeYLYiqing LiWDWei Dai

Key Points

  • Achieving 60.8% accuracy on the ScreenSpot-Pro benchmark, GUI-ARP demonstrates significant improvements in fine-grained localization.
  • GUI-ARP integrates adaptive region perception, enhancing visual attention methods for crop tasks in GUI agents and boosts performance.
  • Utilizing a two-phase training pipeline, the framework combines supervised fine-tuning and reinforcement strategies for optimized outcomes.
  • Results showcase GUI-ARP's leading position against competitor models, including a substantial gap over open-source models in benchmark tests.

Abstract

Existing GUI grounding methods often struggle with fine-grained localization in high-resolution screenshots. To address this, we propose GUI-ARP, a novel framework that enables adaptive multi-stage inference. Equipped with the proposed Adaptive Region Perception (ARP) and Adaptive Stage Controlling (ASC), GUI-ARP dynamically exploits visual attention for cropping task-relevant regions and adapts its inference strategy, performing a single-stage inference for simple cases and a multi-stage analysis for more complex scenarios. This is achieved through a two-phase training pipeline that integrates supervised fine-tuning with reinforcement fine-tuning based on Group Relative Policy Optimization (GRPO). Extensive experiments demonstrate that the proposed GUI-ARP achieves state-of-the-art performance on challenging GUI grounding benchmarks, with a 7B model reaching 60.8% accuracy on ScreenSpot-Pro and 30.9% on UI-Vision benchmark. Notably, GUI-ARP-7B demonstrates strong competitiveness against open-source 72B models (UI-TARS-72B at 38.1%) and proprietary models.

Ask AI
Helpful
Bookmark
Share
View Full Paper

Cite This Study

Ye et al. (2025) studied this question.

synapsesocial.com/papers/68de5da783cbc991d0a20ab1https://doi.org/10.48550/arxiv.2509.15532
Ask AI
Helpful
Bookmark
Share
View Full Paper