Custom Contrast Testing: Current Trends and a New Approach

ABSTRACT Contrast analysis has become prevalent in experimental accounting research since Buckless and Ravenscroft (1990) introduced it to the accounting literature over 25 years ago. Since its initial introduction, the scope of contrast testing has expanded, yet guidance as to the most appropriate methods of specifying, conducting, interpreting, and exhibiting these tests has not. We survey the use of contrast analysis in the recent literature and propose a three-part testing approach that provides a more comprehensive picture of contrast results. Our approach considers three pieces of complementary evidence: the visual evaluation of fit, traditional significance testing, and quantitative evaluation of the contrast variance residual. Our measure of the contrast variance residual, q2, is proposed for the first time in this work. After proposing our approach, we walk through six common contrast testing scenarios where current practices may fall short and our approach may guide researchers. We extend Buckless and Ravenscroft (1990) and contribute to the accounting research methods literature by documenting current contrast analysis practices that result in elevated Type I error and by proposing a potential solution to mitigate these concerns.