Too good to be true: underdispersion in geochronology

Statistical tests of data dispersion are widely used in geochronology to assess whether analytical uncertainties adequately explain the scatter of repeated measurements. Whereas excess scatter (overdispersion) is commonly regarded as a problem to be identified and removed, this paper draws attention to the opposite phenomenon: underdispersion, in which measurements cluster more closely than expected from their analytical uncertainties. Underdispersion may result from incorrectly estimated uncertainties, but can also arise from over-zealous outlier rejection, whereby measurements are selectively removed until the remaining data meet a predefined p value cutoff or mean square weighted deviation (MSWD) target. Such practices can produce apparently precise but potentially inaccurate geochronological estimates. This paper introduces a simple graphical method for detecting underdispersion in collections of geochronological data. The method is based on the empirical cumulative distribution function (ECDF) of p values from chi-squared tests of sample homogeneity. Under the null hypothesis, p values should be uniformly distributed. Although genuine geological complexity can shift the distribution towards low p values, there is no natural mechanism that can produce an excess of high p values. Consequently, the region below the 1:1 line of a p value ECDF represents a “forbidden zone” that provides a diagnostic for excessive outlier rejection (“Type-II cherry picking”). Observer drift, iterative outlier removal and optimised LA-ICP-MS signal windows are mechanisms that can produce underdispersion. Applied to real data, underdispersion appears in every geochronometer examined. For fission track analysis (where cherry picking is the only possible cause of underdispersion), about one third of p values in the geochron.org database plot in the forbidden zone. So do 15 % of 650 p values from Nature journals (rising from 11.5 % before 2019 to 17.2 % since) and 25 % of those underlying the 2020 edition of the Geologic Time Scale, including 90 % of Ar–Ar constraints. Addressing the problem requires greater emphasis on geological evidence when identifying outliers, preferably through procedures that are blind to statistical dispersion, and greater transparency through FAIR sharing of raw data and data-reduction workflows.