Using Bayesian test relationship between categorical random variables on small sample sizes: application on the relationship between the weight status of parents and their children in understudied populations living in Pacific Island countries and territories

Background Within the traditional frequentist framework, the emphasis is often on reporting the p-value. Obtaining significant results is infrequent in small sample size contexts, implying a major concern when reporting results in scientific research because rejecting the null hypothesis does not provide evidence of the absence of an effect. Methods We explored the association between child weight status and parental weight status within an understudied Pacific Island population. The sample was divided in two groups: Group 1 with children whose parents were not both overweight, and Group 2 containing children whose both parents were overweight. We compared: • the Fisher’s exact test and computed its p-value; • the Bayesian A/B test with a variety of indicators including the Bayes factors (BF), posterior probabilities, and density functions. Two Bayesian approaches were compared in order to evaluate the hypothesis that the probability of success was higher in Group 2 than in Group 1 ( H+ ): the Independent Beta Estimation (IBE) and the Logit Transformation Testing (LTT). Furthermore, the data were slightly modified and the indicators were re-examined to understand the effects of such minor alterations on the results and conclusions derived from small samples sizes. Results • Estimates of the success probabilities: 0.17 in Group 1, 0.52 in Group 2. • Fisher’s exact test: not significant ( p≈0.182 ) with a large effect size associated ( h=0.78 ). • Evidence for H+ with IBE: moderate to strong (BF from 8.5 to 14.9). • Evidence for H+ with LTT: negligible to moderate (BF from 2.2 to 13.9). • Adding a single observation: conclusions not changed for both the Fisher’s exact test and the Bayesian approaches. • Doubling the sample size: significant result for the Fisher’s exact test and the evidence for H+ strengthened. Conclusions The Bayesian testing framework offers additional insights that enrich interpretation alongside frequentist testing. Scientific studies would benefit from giving them greater consideration.