Towards HCXAI: Explainability Preferences of Healthcare Professionals

This study examined how healthcare professionals (HCP) perceive and trust AI in clinical settings, and what kinds of explanations they need to use it effectively. Using interviews with six HCPsand a questionnaire (N=41), the findings show that trust was higher for administrative tasks and lower for direct clinical decisions, regardless of their prior AI or clinical experience or AI literacy. Clinical validation and algorithmic bias ranked above explainability as trust factors, indicating that explainability is not a primary trust-building mechanism. However, HCP consistently demanded explanations as a tool to support their own clinical reasoning, preferring to receive them across all clinical contexts rather than only under disagreement with the system, and valued grounding in medical evidence and consistency with clinical protocols over clarity or simplicity. These findings argue for an HCXAI design approach that treats explainability as a critical-reasoning tool rather than a primary trust mechanism.