Massively Parallel Simulations of Binary Black Hole Intermediate-Mass-Ratio Inspirals

We present a highly scalable framework that targets problems of interest to the numerical relativity and broader astrophysics communities. This framework combines a parallel octree-refined adaptive mesh with a wavelet adaptive multiresolution and a physics module to solve the Einstein equations of general relativity. The goal of this work is to perform advanced, massively parallel numerical simulations of intermediate-mass-ratio inspirals of binary black holes with mass ratios on the order of 100:1. These studies will be used to generate waveforms as used in the data analysis of the Laser Interferometer Gravitational-Wave Observatory and to calibrate semianalytical approximate methods. Our framework consists of a distributed memory octree-based adaptive meshing framework in conjunction with a node-local code generator. The code generator makes our code portable across different architectures. The equations corresponding to the target application are written in symbolic notation, and generators for different architectures can be added independently of the application. Additionally, this symbolic interface also makes our code extensible and as such has been designed to easily accommodate many existing algorithms in astrophysics for plasma dynamics and radiation hydrodynamics. Our adaptive meshing algorithms and data structures have been optimized for modern architectures with deep memory hierarchies. This enables our framework to achieve excellent performance and scalability on modern leadership architectures. We demonstrate excellent weak scalability up to 131K cores on the Oak Ridge National Laboratory's Titan for binary mergers for mass ratios up to 100.