Answers are grounded in this paper and cite their sources.
Background Competency models provide useful frameworks for organizing learning and assessment programs, but their construction is both time intensive and subject to perceptual biases. Some aspects of model development may be particularly well‐suited to automation, specifically natural language processing (NLP), which could also help make them more generalizable and thus more learner‐centric. Aims In this study, we sought to evaluate the potential for NLP techniques be applied to competency framework development. Materials & Methods Using NLP, we developed a set of new competency frameworks from a sample of existing leadership competency models from the health professions (e.g. nursing, medicine, healthcare management, social work, spiritual care). We then arranged for a human reviewer who was blind to the frameworks’ sources to evaluate their relative coherence. Results The human‐developed frameworks tended to be viewed as more coherent than the NLP‐generated frameworks, however the coherence advantage was greatest for the least complex models, and there was no apparent advantage in the most complex model we tested. Discussion Although NLP did not consistently outperform the human‐developed model structures, the pattern of results suggested directions for further model refinement and future study. Conclusion Replicating this research with a broader sample of competency models will be important for establishing whether the observed relationship between NLP performance and model size is a more widely generalizable principle.
No takes yet. Share an insight, caveat, or question.
Garman et al. (2021) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: