Quantifying data quality after removing respondents who fail data quality checks

Check Questions (CQs) are used to scrutinise whether respondents are providing quality data and these are becoming more important to ensure data quality, especially with the use of online panels. Often the proportion of people passing these questions are used as a summary of data quality, but this is flawed and of limited value when assessing the quality of a study. This paper introduces a new statistic, the data quality index Q^, to measure the quality of data after respondents who fail CQs are removed. This index is interpretable as the estimated proportion of respondents (after excluding respondents who fail the CQs) who answer the survey conscientiously. The index is derived using simple probability, its properties explored, and directions provided on how it should be used and interpreted. The use of Q^ in published research will increase transparency and confidence in the quality of research findings.