Pursuing coherence in software process assessment and improvement

The improvement of software development processes is certainly one of the top priorities of the IT industry. Software has become the most critical component in any modern product and service. All sectors of our society have been affected and influenced by the introduction of software and software‐based features. To support the improvement of software development processes, a number of process assessment and improvement techniques have been proposed and applied in a significant number of cases. However, the evaluation of the effects produced by the introduction of assessment and improvement methods and models is raising much controversy among researchers and practitioners. In particular, it is extremely difficult to evaluate if, under what circumstances, and to what extent an assessment approach (and the consequent improvement plan) has had a direct, significant and positive impact on the company performance. This paper reports our experiences in conducting process assessment/improvement initiatives in three companies. The assessments have been based on SEI's Capability Maturity Model for software (Software CMM). The main finding of this study is that effective process assessment (and consequently improvement) models/methods must necessarily pursue the coherent improvement of all the aspects of a company and not just of the engineering factors. In the paper we provide a detailed discussion of the case studies and of the motivations that justify our position. Also, as a synthesis of our experiences, we outline a strategy to software process assessment and improvement that takes into account the concerns and requirements we have identified in our work.