Answers are grounded in this paper and cite their sources.
Background : Clinical prediction models in critical care and toxicology are frequently developed with sample sizes below formal minimum requirements and without explicit quantification of the resulting overfitting, contributing to a broader reproducibility problem in this literature. We present a transparent methodological pipeline for developing, internally validating, and honestly reporting such a model, using a real clinical question-predicting the need for invasive mechanical ventilation (IMV) in patients with acute poisoning treated with renal replacement therapy-as the applied case study. No prediction model specific to this population, and none developed in a Latin American setting, has previously been reported. Methods : We conducted a retrospective, multi-institutional cohort study (11 centres, one predominant centre contributing 76% of cases) of patients with acute poisoning who required renal replacement therapy (hemoperfusion 68.6%, hemodialysis 17.2%, or both 14.2%). After excluding non-toxicological indications for renal replacement therapy, 169 patients were analysed (82 IMV events, 48.5%). Sample size adequacy was assessed using the Riley et al. simulation-based approach; the available sample fell below the calculated minimum, and this model is presented as a preliminary internal derivation rather than a finalised clinical tool. Missing values in APACHE II and SOFA (14.8% each), found to be missing-at-random conditional on treating centre, were addressed with multiple imputation by chained equations (20 imputations). A Firth-penalised logistic regression with restricted cubic splines for APACHE II and SOFA (informed by a Box-Tidwell non-linearity test) was developed as the primary model, with a random forest as a non-linear comparator. Internal validity was assessed via 5-fold cross-validation and bootstrap optimism correction (200 resamples). Model interpretability was assessed with Shapley Additive Explanations (SHAP) on the random forest comparator. A simplified point-based risk score was derived, with bootstrap 95% confidence intervals for observed event rates per score level. Sensitivity analyses examined treatment modality and sex as potential sources of heterogeneity. Quasi-external validation was performed by training the model exclusively on the predominant centre and freezing it for application to the remaining satellite centres, including full logistic recalibration (intercept and slope). Results : SOFA (p<0.001), APACHE II (p<0.001), vasoactive drug use (p<0.001), and time from ingestion to first dialysis (p=0.0009) differed significantly between patients with and without IMV. The Firth-penalised spline model achieved a cross-validated AUC of 0.821 (SD 0.014); bootstrap optimism correction of the apparent model (AUC 0.898) yielded a corrected AUC of 0.856, consistent with the theoretical shrinkage factor (0.856) derived from the Riley sample-size calculation. The random forest achieved a cross-validated AUC of 0.831 (SD 0.011). Decision curve analysis showed net benefit over 'treat-all' and 'treat-none' strategies across the 5-75% threshold probability range. Treatment modality did not materially change model performance (AUC 0.830 with modality included vs. 0.829 without) despite its uneven distribution across centres. Discrimination showed a point-estimate difference by sex (AUC 0.883 in men vs. 0.785 in women) that did not reach statistical significance on a formal DeLong test (difference 0.098, 95% CI -0.022 to 0.218, p=0.109), an inconclusive result discussed as an open equity question rather than a confirmed disparity. A four-variable exploratory point score (VM-Risk, range 0-7) showed a gradient from 0% to 100% observed IMV incidence, with wide, overlapping bootstrap confidence intervals at adjacent score levels reflecting the small per-category sample sizes; its apparent AUC (0.848, bootstrap 95% CI 0.793-0.901) has not been independently validated. Quasi-external validation (training on the predominant centre, testing on satellite centres, n=41) preserved discrimination (AUC 0.833); full logistic recalibration (intercept -0.024, slope 1.603) improved the Brier score from 0.166 to 0.153, though the substantial departure of the slope from 1 indicates meaningful risk-stratification distortion beyond a simple level shift. Conclusions : This case study demonstrates a reproducible pipeline for developing and transparently reporting an underpowered clinical prediction model: formal sample-size calculation, missing-data mechanism identification, triangulated overfitting quantification, full recalibration, and explicit subgroup equity testing. Applied to acute poisoning requiring renal replacement therapy, the resulting model showed encouraging discrimination but remains a preliminary, hypothesis-generating derivation; no specific score cut-off is recommended for clinical decision-making pending prospective, adequately powered external validation.
No takes yet. Share an insight, caveat, or question.
Mendoza et al. (2026) studied this question.