A study evaluation framework for measuring cognition: lessons learned in cross-national contexts from four English-speaking aging cohorts

The Harmonized Cognitive Assessment Protocol (HCAP) is a detailed battery assessing cognition among older people used by studies across the world. Data harmonization is a key priority for HCAP studies. Errors or artefacts introduced during data collection become embedded in the dataset and cannot be fully resolved through post-hoc statistical adjustment alone. We used a mixed-methods approach using established theories from the existing literature on methodologies of longitudinal studies and the implementation of HCAP in four English-speaking studies adopting the same protocol. Through a detailed investigation involving the English Longitudinal Study of Ageing (ELSA), the Health and Retirement Study (HRS), The Irish Longitudinal Study on Ageing (TILDA), and the Northern Ireland Cohort for the Longitudinal Study of Ageing (NICOLA), we identified 60 factors contributing to the development of a conceptual framework for the evaluation and implementation of HCAP. We present this framework and a prototype checklist as a tool for providing a transparent and structured approach to improve data quality, cross-country comparability and for identifying, mitigating, and monitoring sources of bias. The framework consisting of four broad headings: (1) Organisation and design, (2) Competency of personnel and systems, (3) Implementation and outputs, and (4) Feedback and communication. This framework aims improves data quality at the point of collection, designed to complement and not replace, post-hoc statistical harmonization. By strengthening data at the outset enables subsequent harmonization to be more robust. We recommend studies seeking cross-national comparability to give careful consideration to operational aspects of fieldwork.