A high-performance, memory efficient micro-FE solver for large scale heterogeneous image-based models on consumer hardware

Micro finite element modelling is commonly used in bone research to predict strains, stresses, forces and displacements. To accurately capture the underlying bone microstructure, microCT can be used to generate large scale heterogeneous image-based models. However, these large models quickly overwhelm traditional solvers, even when efficiently optimized sparse matrices are used. Matrix-free methods can be combined with preconditioned conjugate gradient (PCG) methods to further decrease memory usage, however, the unstructured solvers required for porous bone still require significant amounts of memory for the mesh. Additionally, typical FE solvers designed for solving large models are centered around supercomputer clusters and not consumer hardware, making these large-scale simulations infeasible for smaller research teams or clinical use. To solve these problems, a custom structural linear finite element solver was built and tested to conform to accessible consumer hardware. Maximizing performance, memory efficiency and hardware compatibility were the top priorities for the solver, and thus special care was taken to decrease the memory usage of the mesh by exploiting patterns that occur during meshing, utilizing bi-directional element and node index buffers for faster per node parallelization, all while including both GPU OpenCL and CPU solving support. The developed software solved models up to 1 billion degrees-of-freedom (DOF) on a single consumer GPU with single precision, and models of nearly 600 million DOF were solved when utilizing double precision. When compared to other bone oriented solvers, the new solver outperformed both in solving time and memory usage. The mesh compression allowed the unstructured mesh data to be reduced to just a tiny fraction of the PCG solving vectors, resulting in a memory efficiency ranging from 35 to 37 bytes per DOF. Overall, the solver demonstrated the ability to quickly solve large porous FE models that would typically be unsolvable on regular consumer hardware.