The Quest for Ground Truth in Musical Artist Similarity

It would be interesting and valuable to devise an automatic measure of the similarity between two musicians based only on an analysis of their recordings. To develop such a measure, however, presupposes some 'ground truth' training data describing the actual similarity between certain pairs of artists that constitute the desired output of the measure. Since artist similarity is wholly subjective, such data is not easily obtained. In this paper, we describe several attempts to construct a full matrix of similarity measures between a set of some 400 popular artists by regularizing limited subjective judgment data. We also detail our attempts to evaluate these measures by comparison with direct subjective similarity judgments collected via a web-based survey in April 2002. Overall, we find that subjective artist similarities are quite variable between users—casting doubt on the concept of a single 'ground truth'. Our best measure, however, gives reasonable agreement with the subjective data, and forms a useable stand-in. In addition, our evaluation methodology may be useful for comparing other measures of artist similarity.