Design and psychometrics of assessment tool for health system preparedness on disaster risk communication

Effective risk communication is a critical component of disaster management, enabling appropriate responses and enhancing health system preparedness. Regular evaluation of risk communication processes ensures the accurate dissemination of information, which is vital for mitigating the impact of disasters. This study aimed to develop and validate a comprehensive tool to assess health system preparedness for disaster risk communication. This mixed-methods study was conducted in two phases. First, a systematic review and a qualitative study were performed to identify the core components of disaster risk communication. These findings informed the development of an initial item pool. The tool’s validity was then rigorously assessed. Content validity (relevance, clarity, and necessity) was evaluated by 15 experts using the Content Validity Ratio (CVR) and Content Validity Index (CVI) based on Lawshe and Waltz & Bausell methods. Face validity was assessed both qualitatively and quantitatively (item impact method)(1). Construct validity was established through an exploratory factor analysis (EFA) with 508 randomly selected health system managers and experts. The tool’s reliability was measured using Cronbach’s alpha for internal consistency and the intraclass correlation coefficient (ICC) for stability via a test-retest with 15 participants. The initial pool of 309 items was refined to 109. Content and face validity assessments resulted in a 93-item tool. EFA reduced this to a final 91-item instrument, which revealed a clear four-factor structure explaining 61.67% of the total variance. The four factors were: “Management of Relational Network and Information,” “Planning, Monitoring, and Evaluation,” “Education, Coordination, and Support,” and “Ethical Principles and Social Values.” The tool demonstrated excellent reliability, with a Cronbach’s alpha of 0.98 and an ICC of 0.98. This study presents a valid and reliable 91-item tool for evaluating the preparedness of health systems for disaster risk communication. By providing a comprehensive assessment across four key domains, this instrument can help identify strengths and gaps in current practices. It is designed for use by RCCE (Risk Communication and Community Engagement) officials in hospitals, health networks, and pre-hospital emergency settings to guide improvements, inform planning, and ultimately strengthen disaster risk communication throughout the disaster management cycle.