ChiSao: A fully GPU-resident optimizer for cataloguing the modes of multimodal black-box functions

Bayesian inference and scientific computing often call for cataloguing all modes of a multimodal black-box objective, not just its global optimum. Classical multimodal and niching methods keep their search strategy on the host, parallelizing at most one generation, and pay a per-generation cost, an O ( d 2 ) covariance update or O ( N 2 ) pairwise distances. We present ChiSao , a memetic multimodal optimizer whose entire search strategy resides on the device: seeding, batched L-BFGS convergence, deduplication, reseeding, smoothing, and anti-convergence all run on the GPU with no expensive host round-trips. To our knowledge, it is the first fully GPU-resident multimodal optimizer. GPU throughput rather than dimension becomes the limiting factor: the per-evaluation cost stays O ( d ) against the baselines’ O ( d 2 ) to O ( d 3 ) , a full polynomial order lower on identical hardware. On a single NVIDIA RTX 3080, ChiSao solves d = 512 in about twenty seconds where restart CMA-ES needs over four minutes, and recovers shifted optima to d = 2048 , a regime the host-resident baselines cannot enter. On single-global recovery, a memetically fair, L-BFGS-polished restart CMA-ES matches ChiSao at equal budget, and dedicated niching methods lead on rotated-composition landscapes. However, recovery quality is not the claim; the contribution is the device-resident regime itself, in which cataloguing coverage scales with a GPU-sized population at high dimension, plus an on-device configuration portfolio at constant-factor cost. We map reach and limits across the 42 Simon Fraser University (SFU) functions, the CEC’2013 niching suite, a dimensional multi-Ackley benchmark, and a noisy landscape. ChiSao is released as open-source.