Answers are grounded in this paper and cite their sources.
Conversations with stakeholders, as well as remarks in the literature, suggest that there may be confusion about what can be concluded when a canine behavior evaluation has been described as being “validated,” “reliable,” or “predictive.” To assess the evidence, we searched PubMed and ScienceDirect using the terms “canine,” “behavior evaluation,” “temperament test,” and “shelter” to identify articles that assessed the validity or reliability of evaluations based on battery of tests used or intended for screening shelter dogs for behavior labeled aggressive and/or for adoption suitability. Despite 25+ years of publications, including solid studies performed under good to ideal conditions by skilled investigators, findings indicate there is no evidence that any canine behavior evaluation or individual subtest has come close to meeting accepted standards justifying claims that it is validated for routine use in shelters. Furthermore, the mean reported false-positive error rate in study populations was 35.1%, whereas in more typical shelter populations, it was estimated at 63.8%. We propose that the discrepancy between the actual state of the science and what people assume has been accomplished is primarily due to the following: [1] confusion from mixing colloquial with scientific uses of words such as “validated,” “predictive,” “reliable,” and “agreement”; [2] the limitations of correlation and regression as statistical methods for demonstrating agreement or predictive ability; [3] failure to account for the difference between predictive validity of an instrument in populations of dogs in a research exercise versus predictive ability and error rate for individual dogs in real-world settings; [4] conflating statistical significance with clinical significance; and, as a result of 1-4 aforementioned, [5] conferring overall validation status, despite the results of studies being much more circumscribed. Given their published error rates, one explanation may be that behavior evaluations lack basic face validity and/or a clear focus as to what is being measured and its relevance to postadoption outcomes. This argues against use of any behavior evaluation to make important decisions for shelter dogs, especially if the behavior(s) of concern were only observed during provocative testing. These findings indicate an opportunity to acknowledge what has been learned and bring together all stakeholders to consider the real needs of shelter dogs and what the future might look like.
No takes yet. Share an insight, caveat, or question.
Patronek et al. (2019) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: