Ethical Awareness in the Use of Large Language Models: Development and Validation of a Scale for Healthcare Professionals

Background The integration of large language models (LLMs) into healthcare offers transformative potential but raises significant ethical challenges. Addressing these challenges requires a comprehensive framework to assess healthcare professionals' ethical awareness of LLMs usage. Objective To develop and validate a scale designed to evaluate healthcare professionals' ethical awareness regarding the integration of LLMs into clinical practice. Methods This two‐phase methodological study was conducted in 2024 across nine healthcare institutions—five in Egypt and four in Saudi Arabia. In Phase I, a comprehensive literature review and semi‐structured interviews with healthcare professionals guided the development of the initial scale and item pool. In Phase II, the psychometric properties of the scale were evaluated using data collected from 658 healthcare professionals. Construct validity was assessed through exploratory and confirmatory factor analyses, while internal consistency reliability was examined using Cronbach's alpha ( α ) coefficients and item–total correlation metrics. Results An initial pool of 36 items was refined to 21 items across 6 dimensions: data privacy and confidentiality, consent and autonomy, transparency and accountability, bias and equity, safety and professional integrity, and education and sustainability. EFA identified a six‐factor structure accounting for 71.5% of the variance. CFA confirmed the scale's structure with strong model fit indices for first‐order analysis (CMIN/DF = 1.798, CFI = 0.967, RMSEA = 0.050) and second‐order analysis (CMIN/DF = 2.862, CFI = 0.927, RMSEA = 0.058). The scale demonstrated excellent internal consistency (overall Cronbach's α = 0.90; dimensions ranging from 0.780 to 0.964) and achieved satisfactory composite reliability, convergent validity and discriminant validity. Moderate statistically significant inter‐factor correlations confirmed the multidimensional nature of the scale. Conclusion The developed scale is a valid and reliable tool for assessing healthcare professionals' ethical awareness in the use of LLMs in healthcare. It provides a comprehensive framework for evaluating and enhancing ethical considerations, promoting the responsible and informed use of LLMs technologies in clinical practice.