A Python module to combine p values arising from discrete tests.

Most popular hypothesis tests are designed for datasets with a simple structure.For example, in a simple medical trial, this structure would be a treatment and a control group, and a test would answer whether some health score exhibits a significant difference between their members.However, many datasets are naturally segmented or benefit from being analysed in segments.Continuing our example, if we know a priori that females score higher than males on average, analysing each sex separately should allow us to see more clearly whether the treatment is effective.To draw an overall conclusion in such a case, we need to combine the values for each sub-dataset.Starting with Fisher's method, several ways have been proposed to combine values (Heard & Rubin-Delanchy, 2018).Their usual implementations assume continuous tests, i.e., tests whose values under the null hypothesis follow the uniform distribution on the unit interval.However, rank tests and many others are discrete tests, i.e., they can only yield values from a finite selection (for a given sample size).Assuming continuous tests when combining values from discrete tests can lead to considerably misleading outcomes (Kincaid, 1962;Mielke et al., 2004), in particular when the sub-datasets are small.We here present the Python module combine_pvalues_discrete, which provides a toolbox for combining values from discrete tests.Results from individual tests are stored with the respective null distribution of values and the combined value is accurately estimated using a Monte Carlo simulation based on these null distributions.