Enhancing Parallel Genetic Algorithms for Efficient Distributed Data Analysis: Automatic Termination and Population Sizing Integration

Abstract Unleashing the potential of large-scale data analysis requires advanced computational methods for managing distributed data’s size and complexity. Genetic algorithms (GAs) have demonstrated adaptability through parallelization, prompting further performance enhancements. This study proposes integrating automatic termination and population sizing mechanisms to augment parallel GAs’ flexibility, and effectiveness. We extend PDMS-BioHEL and PDMD-BioHEL, two parallel GAs-based classifiers implemented on the Spark platform. Through extensive experimentation, we demonstrate the efficacy of our approach, enhancing efficiency and user-friendliness. Our techniques enable more efficient and effective large-scale data analysis using Parallel GAs.