Replication of Empirical Studies in Software Engineering: An Update of a Systematic Mapping Study

Context: Current empirical research highlight the need for replications of empirical studies because replications plays an important role in the construction of scientific knowledge. Objective: Considering the importance of replications in the consolidation of the knowledge produced in the software engineering research, this study aims to update and extend the results produced in a previous mapping study seeking to discuss the current state of the replication work of empirical studies performed in software engineering research between 2011 and 2012. Method: We applied the systematic review method to search and select published papers, to extract, and synthesize data from reported replications. Results: This study analyzed more than 7,000 articles, from which 39 articles that published replications between 2011 and 2012 were selected. Data extracted from these studies were used to update the information about the replications work in software engineering. Conclusion: The number of replications increased significantly in the period, when compared to the previous mapping study. In particular, the percentage of external replications also increased, with respect to internal ones. However, several other limitations identified in the previous mapping studies are still observed in this new set of replications.