Answers are grounded in this paper and cite their sources.
This study determined the extent to which medical faculty agreed when rating options in two written patient management problems in diabetes mellitus. Another purpose was to determine whether option weights (used for scoring) based on the consensus ratings of faculty actually predict the choices of well-qualified physicians. Experts showed better-than-chance agreement but with considerable variation from one part of a problem to another (31% to 73%). Nevertheless, consensus ratings were very accurate predictors of the decisions of endocrinology fellows (correlations from .60 to .97). When scoring weights are assigned to options in patient management problems, consensus (average) ratings of experts are likely to demonstrate high concurrent validity for well-qualified clinicans.
No takes yet. Share an insight, caveat, or question.
Mazzuca et al. (1982) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: