Key points are not available for this paper at this time.
Evaluating music information retrieval systems is acknowledged to be a difficult problem. We have created a database and a software testbed for the systematic evaluation of various query-by-humming (QBH) search systems. As might be expected, different queries and different databases lead to wide variations in observed search precision. Natural queries from two sources led to lower performance than that typically reported in the QBH literature.
Dannenberg et al. (Tue,) studied this question.