Reliability of the fellowship examination of the royal Australian college of general practitioners

The examination of the Royal Australian College of General Practitioners is a battery of eight subtests using different methods of assessment to assess different components of competence (domains) regarded as appropriate for Australian general practice. A pass/fail decision is made by combining subtest scores. This article reports an analysis of the reliability of the 1991 examination using multivariate generalizability theory. Results demonstrate that although reliability scores of individual tests vary, the combination of several tests, each providing information about a different component of competence, may produce reliable information about performance of candidates. Replication of this study with data from other candidate cohorts is required to enhance the generalizability of its findings.