UnWise: High T-Wise Coverage from Uniform Sampling

Configuration spaces of industrial product lines are typically too large to be tested exhaustively. Therefore, testing in practice is often carried out on samples, sets of configurations which satisfy the requirements of the testing scenario. For t-wise sampling, the objective is to cover all t-wise interactions between configurable options with as few configurations as possible. However, a trade-off needs to be made between t, sampling time, sample size, and achieved coverage. In addition, it is infeasible for larger systems to even compute the set of all 2-wise interactions in practicable time. In this work, we reevaluate the performance of uniform samplers in terms of 2-wise coverage and come to a more positive result than previous research. We also present completion and reduction algorithms that greatly improve said performance. As a baseline for comparison, we additionally evaluate the two state-of-the-art dedicated t-wise samplers Baital and YASA. In doing so, we are the first to evaluate and compare these samplers on a large set of industrial feature models.