Selecting diagnostic tests for ruling out or ruling in disease: the use of the Kullback-Leibler distance

BACKGROUND: To select a proper diagnostic test, it is recommended that the most specific test be used to confirm (rule in) a diagnosis, and the most sensitive test be used to establish that a disease is unlikely (rule out). These rule-in and rule-out concepts can also be characterized by the likelihood ratio (LR). However, previous papers discussed only the case of binary tests and assumed test results already known. METHODS: The author proposes using the 'Kullback-Leibler distance' as a new measure of rule-in/out potential. The Kullback-Leibler distance is an abstract concept arising from statistics and information theory. The author shows that it integrates in a proper way two sources of information--the distribution of test outcomes and the LR function. The index predicts the fate of an average subject before testing. RESULTS: Analysis of real and hypothetical data demonstrates its applications beyond binary tests. It works even when the conventional methods of dichotomization and ROC curve analysis fail. CONCLUSIONS: The Kullback-Leibler distance nicely characterizes the before-test rule-in/out potentials. It offers a new perspective from which to evaluate a diagnostic test.