Answers are grounded in this paper and cite their sources.
Demonstrates improved layout completion in mobile app screens using an encoder-only transformer, suggesting effective design refinement strategies.
The growing complexity of user interface (UI) design calls for effective methods to understand, complete, and refine layout structures. While prior work has focused pre-dominantly on generating UI layouts from scratch, completing partially designed interfaces is equally critical, particularly in iterative design workflows and scenarios involving incomplete prototypes. In this study, we address the UI layout completion task for mobile app screens using an encoder-only transformer architecture with masked modeling and a layout tokenizer. By representing UI elements as discrete tokens, we formulate layout completion as a sequence prediction problem that leverages global context to infer missing components. We evaluate our approach on subsets of the RICO dataset designed with varying constraints on UI element types and spatial overlap, and report results using standard layout metrics: Coverage, Intersection over Union (IoU), Max IoU, and Alignment. The experiments demonstrate that the proposed method achieves substantial improvements over LayoutFormer++, a widely adopted baseline in UI layout generation, particularly in Coverage, and in several cases, IoU, Max IoU, and Alignment. Additional experiments on noise-reduced subsets reveal that dataset curation can enhance spatial consistency but may also reduce Coverage, reflecting an inherent trade-off between completeness and structural precision. These findings highlight both the effectiveness and the limitations of encoder-only masked modeling for layout completion, and underscore the importance of balancing model design with dataset construction when tackling complex UI design tasks.
No takes yet. Share an insight, caveat, or question.
Salama et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: