Reduction of the effective population size in a branching particle system in the moderate mutation-selection regime

We consider a system of particles performing a one-dimensional dyadic branching Brownian motion with positive drift (0, 1), branching rate 1/2, killed at L () >0, and reflected at 0. The killing boundary L () is chosen so that the total population size is approximately constant, proportional to N. This branching system is interpreted as a population accumulating deleterious mutations. We prove that, when the typical width of the cloud of particles is of order c (N), c (0, 1), the demographic fluctuations of the system converge to a Feller diffusion on the time scale N^1-c. In addition, we show that the limiting genealogy of the system comprises only binary mergers and that these mergers are concentrated in the vicinity of the reflective boundary. This model is a version of the branching Brownian motion with absorption studied by Berestycki, Berestycki and Schweinsberg to describe the effect of natural selection on the genealogy of a population accumulating beneficial mutations. In the latter case, the genealogical structure of the system is described by a Bolthausen-Sznitman coalescent on a logarithmic time scale. In this work, we show that, when the population size in the fittest class is mesoscopic, namely of order N^1-c, the genealogy of the system is given by a Kingman coalescent on a polynomial time scale.