Reaching optimal performance for particle simulations: from performance engineering to automatic code generation

Many physics simulations today can be described by particle interactions, where particles represent point masses (as in molecular dynamics), rigid bodies (as in discrete element methods), or even massive bodies such as planets. Evaluating and calculating the interactions between pairs of particles for real-world simulations is computationally expensive, so the system size and timescale are crucially limited by the available computing power. It is therefore important to understand the performance of such simulations and to develop efficient algorithms and optimisation strategies to exploit the parallelism available in modern HPC hardware. However, it is difficult to provide flexible implementations that can handle complex cases while maintaining state-of-the-art performance as the complexity of the code increases significantly. This thesis addresses the aspects of performance engineering and software portability with a focus on particle simulation codes. In the first part, we introduce MD-Bench, a performance-focused prototyping harness for short-range MD kernels that implements state-of-the-art algorithms from several production applications such as LAMMPS and GROMACS. The MD-Bench source code is simple, understandable and extensible, making it well suited for benchmarking, teaching and research on MD algorithms. We describe the design, structure and implemented algorithms of MD-Bench. Finally, we show many use cases of MD-Bench and describe how these are useful for gaining a deeper understanding of the performance of MD kernels. In the remaining parts of this thesis, we address the software engineering challenges and present two solutions to approach performance portability. We present tinyMD: an efficient, scalable and portable implementation for particle simulations. We compare tinyMD with the proxy application miniMD, which scales very well on parallel supercomputers. We discuss the differences between the two implementations and contrast miniMD’s performance for single-node CPU and GPU targets, as well as its scalability on the SuperMUC-NG and Piz Daint supercomputers. In addition, we demonstrate tinyMD’s flexibility by coupling it to the waLBerla multi-physics framework. This allows us to run tinyMD simulations using the load balancing mechanism implemented in waLBerla. Finally, we present P4IRS, an intermediate representation and compiler for particle simulations that aims to achieve good performance on modern HPC clusters. We describe P4IRS, show usage examples and features, and discuss the benefits we can achieve with code generation. We then evaluate the performance and scalability of code generated by P4IRS on modern processors, accelerators and supercomputers in both the MD and DEM domains.