Ophthalmic statistics note 14: method agreement studies in ophthalmology: the intraclass correlation coefficient?

Our previous note outlined why method agreement studies are so important in ophthalmology.1 Technology moves at a relentless pace and clinicians are keen to adopt innovative techniques that may offer benefits to their patients, such as shorter or less invasive testing, in addition to creating richer data sets that may increase the research potential of the data captured. Researchers and clinicians must, however, use caution to ensure that any differences observed between measurements made on a patient with different methods of measurement are truly due to changes in pathology rather than the method of measurement, the observer making the measurement or other variables that might influence the measurement. Even if two machines appear to report the same characteristic it is possible that one machine is measuring a different anatomical feature than another machine but using the same name. An example is in studies of keratometry and topography where the term Kmax is used to describe both the steepest meridian of the cornea in the central 3 mm (also called K2) and the power of the steepest point of the cornea.2 3 This is of particular importance in trials investigating treatments for keratoconus where Kmax may be the primary outcome measure or used to determine subject eligibility.4 We commented also in our previous note on the differences between the regulation of devices as compared with that of medicines.1 There is a tension between innovation and safety and while measurement reproducibility may not immediately be seen as relevant to harm, measurements are used to make decisions about diagnosis, progression and treatment. For example, a large change in intraocular pressure (IOP) between two visits in a child with glaucoma may indicate the need for examination under anaesthesia, while a large change in K2 readings in an individual with keratoconus might …