Psychometric validity and reliability of a methods of research attitude scale: an item response theory analysis

Abstract The acquisition of data literacy skills is crucial for enhancing teaching practices and educational outcomes. Given that many education students exhibit negative attitudes towards research methods, it is essential to develop tools that accurately assess these perceptions to foster positive changes. This study evaluates the psychometric properties of the Attitudes Toward Research Methodology Questionnaire (ATRMQ), a tool specifically designed to measure education students’ attitudes toward research methodologies. The ATRMQ was administered to a sample of 377 undergraduate education students in Andalusia, Spain. Psychometric analysis, based on Item Response Theory (IRT), assessed reliability, validity, item difficulty, and discrimination indices within its three-dimensional structure: anxiety, interest, and perceived usefulness. Results indicate that the ATRMQ effectively distinguishes between levels of attitude towards research methodology, demonstrating high internal consistency across dimensions. Findings provide valuable insights for refining the ATRMQ, and an updated version is proposed. The ATRMQ offers a robust tool for identifying attitudinal barriers in research methodology training, with implications for curriculum design aimed at improving data literacy and methodological competence among future educators. The study underscores the value of IRT in enhancing scale precision over Classical Test Theory (CTT), advocating its application in educational assessments.