Conceptualization and Preliminary Testing of the Research Instrument Validation Framework (RIVF) for Quantitative Research in Education, Psychology, and Social Sciences: A Modified Delphi Method Approach

Background: The quality of instruments used in quantitative research is crucial for scientific rigor in education, psychology, and social sciences. However, current validation approaches often lack comprehensiveness and fail to integrate multiple validity dimensions.Objective: This study aims to develop and validate the Research Instrument Validation Framework (RIVF) using a modified Delphi approach. The RIVF is grounded in contemporary validity theories, including Messick's Unified Theory, Kane's argument-based approach, principles from Classical Test Theory, Item Response Theory, and cross-cultural validity.Methods: A three-round modified Delphi study was conducted with 22 experts from educational measurement (n=7), psychological assessment (n=6), social science methodology (n=5), and cross-cultural validation (n=4). The study, spanning December 2023 to June 2024, employed open-ended questions, structured questionnaires, and consensus-building rounds. Qualitative data were analyzed using MAXQDA 2020, while quantitative analyses used IBM SPSS Statistics version 26.Results: The final RIVF comprises 21 criteria across the face, content, and construct validity domains. Consensus was achieved on all criteria (greater than 80 percent agreement), with Kappa statistics indicating slight to moderate agreement between domains (0.04–0.11).Conclusion: The RIVF offers a comprehensive, theoretically grounded approach to instrument validation, addressing gaps in existing practices. While achieving consensus, the Kappa statistics suggest areas for refinement. Future research could focus on empirical testing, reliability assessment, user-friendly implementation guides, and exploring artificial intelligence integration to enhance accessibility and streamline the validation process, ultimately improving research quality across disciplines.