On parameter tuning in search-based software engineering: a replicated empirical study

Abstract

Multiobjective Evolutionary Algorithms are increasingly used to solve optimization problems in software engineering. The choice of parameters for those algorithms usually follows the "default" settings, often accepted as "rule of thumb" or common wisdom. The fact is that each algorithms needs to be tuned for the problem at hand. Previous work [Arcuri and Fraser, 2011] has shown that variations in parameter values had large effects on the performance of the algorithms. This project seeks to partially replicate the statistical analysis performed by Arcuri and Fraser. We seek to investigate the effects of parameter tuning on the performance of the two algorithms: Indicator-Based Evolutionary Algorithm (IBEA), and Nondominated Sorting Genetic Algorithm (NSGA-II) when applied to the problem of configuring Software Product Lines (SPLs) in the presence of stakeholder preferences such as cost and reliability. The results of this study confirm and strengthen the findings in the original study by Arcuri and Fraser