Reliability in multi‐site psychiatric studies

Multi‐site psychiatric studies need to take account of cultural and sometimes also linguistic differences and the reliability of the instruments used may need to be re‐established. The analysis of reliability data in these circumstances presents special problems, arising from the requirements to make the best use of all the available data and to compare reliability coefficients across sites, interpreting any differences found in terms of measurement errors and/or sample heterogeneity. Focusing on statistical issues, this paper discusses some topics that should ideally be considered when planning, conducting and analysing such studies. A number of practical examples using data from a recent European study involving the translation and cultural adaptation of five instruments used in mental health service research (the EPSILON Study) are used to illustrate the issues.