Methodological study demonstrates automated QA and LLM proxy-user UX evaluation in a data-driven visualizer, indicating weaker models effectively surface usability barriers.
Data-driven UIs — dashboards, assessment reports, solver visualizers — must be both correct (they show the right data in the right state) and usable (a first-time user reaches the answer without excessive effort). Both properties are typically checked by hand, which is slow and error-prone. We present a five-part, human-effort-minimizing playbook: (1) coverage round-trip testing, which enumerates the full data/state space, programmatically reconstructs the application into each state, and asserts correct display and timing; (2) test-of-the-test meta-validation, which deliberately corrupts data/UI to prove the tests are not vacuously passing; (3) automatable accessibility checks (colour-vision-deficiency distance, contrast, label completeness); (4) weak-model proxy-user testing, in which LLM agents weaker than the developer, given only a goal and the live UI (no source), act as naive users while logging surprises, retries, and click/scroll/time metrics; and (5) metrics-driven UX improvement targeting reductions in clicks-, scrolls-, and time-to-answer. We report a case study on a data-driven poker-strategy visualizer backed by a public GTO dataset. Coverage round-trip testing verified 100% of 23,368 data nodes; meta-validation caught a real bug in our own test; and weak-model proxy users reproduced, and quantified, usability barriers a human developer had personally hit — including a confidently-wrong weak model that reached a high-confidence but incorrect conclusion by misreading UI affordances. We distill design principles: prefer design-level clarity over textual guidance; separate "strictly correct" from "correct but unkind"; and treat a weaker model as a cognitive-load meter.
No takes yet. Share an insight, caveat, or question.
Shinobu Tsuchiya (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: