When is a balanced scorecard a balanced scorecard?

Purpose The Balanced Scorecard (BSC) is widely applied as a performance measurement and strategy implementation tool by organizations. Research has revealed that the term “balanced scorecard” may be understood differently by managers both within as well as across organizations implying that the performance measurement systems implemented in organizations may not be similar to the construct envisioned by Kaplan and Norton. Using Kaplan and Norton's Balanced Scorecard construct as a basis, the paper aims to develop and test a five‐level taxonomy to classify firms' performance measurement systems. Design/methodology/approach A Balanced Scorecard taxonomy is validated using a large sample of professional accountants working in Canadian organizations. Findings The five‐level taxonomy is used to categorize the performance measurement systems of 149 organizations. It is found that 111 organizations' (74.5 percent) performance measurement systems met the criteria to be classified as a Basic Level 1 BSC, while 61 (40.9 percent) organizations have structurally complete Level 3 BSCs, and 36 (24.2 percent) organizations have fully developed Level 5 BSCs. The paper also discusses differences between Level 1 and Level 5 BSC organizations. Research limitations/implications While many researchers assume that organizations' performance measurement systems are similar in implementation level and use, the paper demonstrates that organizations are at different levels of BSC implementation and use, a factor that should be taken into consideration when designing empirical studies to test the efficacy of Kaplan and Norton's BSC. Practical implications The five‐level BSC taxonomy scheme provides managers working with Kaplan and Norton's BSC with a tool to plan their implementation steps and then benchmark their progress towards implementing a fully developed Level 5 BSC. Originality/value In developing and empirically validating a BSC taxonomy, the paper builds on and extends previous research on BSC implementation and its potential implications.