High-performance nonlinear least square optimization via domain specific language

The least square method is common and classical in the regression analysis. It is often used to solve the convex optimization problem, but the traditional solving routine for least squares which is done by hand-written codes shows the disadvantages when dealing with common least square problems. One significant drawback for traditional solving routine is it is hard to work along with and produce high performance solver by non-professional users who do not have the knowledge of CPU/GPU architecture, and it is also a tough job to review or improve the solvers which already have been written, since many fine details that relate to the processor structure may be hard-coded in to the source code. In this paper, we propose a new domain specific language (DSL) for the producing of non-linear least square solver for research purpose with a back end of Gauss-Newton and Levenberg-Marquardt methods implemented in cuSPARSE and cuBLAS. The DSL paired with a C/C++ interface has a user-friendly syntax which can be easily used to write energy functions and generate GPU solvers which have the performance close to hand-written CUDA solvers.