On Misuses of the Kolmogorov–Smirnov Test for One-Sample Goodness-of-Fit

The Kolmogorov–Smirnov (KS) test is widely employed to assess the goodness-of-fit of a hypothesized continuous distribution to a sample. Despite its popularity, the test is frequently misused in the literature and practice. While originally intended for independent, continuous data with precisely specified hypothesized distributions, it is erroneously applied to scenarios with dependent, discrete, or rounded data, with hypothesized distributions requiring estimated parameters. For example, it has been "discovered" multiple times that the test is too conservative when the hypothesized distribution has parameters that need to be estimated. We demonstrate misuses of the one-sample KS test in three scenarios through simulation studies: 1) the hypothesized distribution has unspecified parameters; 2) the data are serially dependent; and 3) a combination of the first two scenarios. For each scenario, we provide remedies for practical applications using appropriate bootstrap approaches. The whole demonstration can be used as hands-on education materials on both goodness-of-fit tests and bootstrap.