Beyond self-reports: Addressing bias and improving data quality in educational research

The use of self-report in educational research has facilitated data collection by providing easy access to diverse populations in a short period of time. However, research has shown that these opportunities come with challenges. From inadequate response efforts to culturally influenced responses, there are numerous failures that can call into question the validity of our findings. For example, discrepancies between self-reports and objective data often reveal underlying biases. Poor data quality, exacerbated by social desirability in sensitive constructs, individual and environmental factors, and changes in scale structure, has highlighted that our methods may have some limitations that reduce generalisability and trigger the replicability crisis. However, these limitations can also lead to improvements in both survey design and data interpretation. Our experiences point to the need to integrate multiple data sources, to improve survey development and adaptation methods, and to use true experimental studies more frequently. By reflecting on these challenges, we suggest new directions for survey implementation in educational research studies to improve the reliability and replicability of our findings and deepen our understanding of complex human dynamics.