Answers are grounded in this paper and cite their sources.
The development of a scoring key for the situational judgment test often requires subject matter experts (SMEs) to identify the best responses for a hypothetical situation. And yet, there is no gold standard for identifying the SMEs. This paper describes an empirical and context-free approach: the Cochran–Weiss–Shanteau (CWS) method, which does not rely on external criteria such as tenure or credential. We first describe the theory behind the empirical approach of expertise. We also outline the CWS method and provide an R script for calculating the CWS index. Next, we demonstrate how the CWS index can be used for improving interrater agreement and the efficiency of SME selection. Finally, we examined the nomological network of the CWS index. We found that the CWS index was associated with reflective thinking and intuition avoidance.
No takes yet. Share an insight, caveat, or question.
Zhang et al. (2021) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: