Synapse
⌘+K
Synapse
PulseExploreClubsResearchersJournals
Instagram
HomeClubsExplore
September 5, 2025Proceedings of the Human Factors and Ergonomics Society Annual Meeting

Smarter UX Evaluations? Comparing AI and Human Experts in Usability Analysis

View Full Paper
Ask AI
Bookmark
Share

Authors

PTPhuoc ThaiFMFernando MontalvoPSPromise Stephens

Discussion

Loading...

Member takes

Overview

This study compares generative AI and human experts in usability analysis, suggesting AI's potential benefits.

Key Points

  • The AI model showed moderate agreement with human experts in usability evaluations, indicating effectiveness.
  • Test-retest reliability was good for the custom AI model, supporting its stability over time.
  • AI identified usability issues effectively but struggled with ambiguous feedback, highlighting limits.
  • The findings suggest AI could streamline usability assessments while still needing human oversight.

Cite This Study

Thai et al. (2025) studied this question.

synapsesocial.com/papers/68bb3d4e2b87ece8dc955b96https://doi.org/10.1177/10711813251369790
View Full Paper
Ask AI
Bookmark
Share

Also Consider

Synapse has enriched 3 closely related papers on similar clinical questions. Consider them for comparative context:

  1. 1The Measurement of Observer Agreement for Categorical Data1977 · 81,468 citations
  2. 2Mitigating Bias in Generative AI: The Role of Explainable AI for Ethical Deployment2024 · 4 citations
  3. 3Damaged Merchandise? A Review of Experiments That Compare Usability Evaluation Methods1998 · 490 citations