Complexity science studies systems in which properties and behaviors emerge at meso- and macroscales that are difficult to predict and model by observing the properties and behaviors exhibited by the system’s components at smaller scales. The set of relationships that exist among post-secondary school curricula and job markets is one example of such a system. Prior work has undertaken the challenge of modeling this system for several purposes, one of which has been to develop retrieval and ranking algorithms in the education–career domain. A particular emergent property that is closely bound up with this prior work, and that is the focus of the present work, is the salience of a course with respect to a specific objective. The specific objective that we are interested in here is career usefulness, which means that our overall task is to rank order courses based on their usefulness in helping a student to obtain and perform a specific job. One aspect of this overall task that remains understudied concerns how it can best be performed in an interpretable manner and whether existing interpretable methods that may be applied to it, such as text-based similarity measures and document-ranking functions, represent workable solutions or whether an approach involving more detailed modeling of the underlying complex system may prove more effective. The purpose of this article is to answer this question, and, in order to do this, most of this article’s content is devoted to the latter kind of approach, because the former kind is described in detail in the existing literature. The specific approach of the latter kind that we investigate is based on, first, developing a heterogeneous knowledge graph model of the overall complex system, and, second, developing a procedure that quantifies salience using the strength of the skill-dependency chains that link a course to a specified job. To evaluate our methods, we perform a human subjects study in which we leverage the domain expertise of fifty participants. The results of the study demonstrate that the latter approach produces career-motivated course recommendations, as well as accompanying explanations, which systematically exceed those produced by the former approach, in terms of both their quality and usability.
Striebel et al. (Mon,) studied this question.