Software product lines and variability modeling: A tertiary study

Context: A software product line is a means to develop a set of products in which variability is a central phenomenon captured in variability models. The field of SPLs and variability have been topics of extensive research over the few past decades. Objective: This research characterizes systematic reviews (SRs) in the field, studies how SRs analyze and use evidence-based results, and identifies how variability is modeled. Method: We conducted a tertiary study as a form of systematic review. Results: 86 SRs were included. SRs have become a widely adopted methodology covering the field broadly otherwise except for variability realization. Numerous variability models exist that cover different development artifacts, but the evidence is insufficient in quantity and immature, and we argue for better evidence. SRs perform well in searching and selecting studies and presenting data. However, their analysis and use of the quality of and evidence in the primary studies often remains shallow, merely presenting of what kinds of evidence exist. Conclusions: There is a need for actionable, context-sensitive, and evaluated solutions rather than novel ones. Different kinds of SRs (SLRs and Maps) need to be better distinguished, and evidence and quality need to be better used in the resulting syntheses. (C) 2019 The Authors. Published by Elsevier Inc.