Interpreting regression models in clinical outcome studies

Bone & Joint ResearchVol. 4, No. 9 EditorialOpen AccessOpen Access licenseInterpreting regression models in clinical outcome studiesCrossmarkD. F. Hamilton, M. Ghert, A. H. R. W. SimpsonD. F. HamiltonResearch Fellow, Department of Trauma and OrthopaedicsUniversity of Edinburgh, FU413 Chancellor’s Building, 49 Little France Crescent, Edinburgh, EH16 4SB, UK.Search for more papers by this author, M. GhertAssociate Professor, Department of Surgery Deputy Editor,The Bone and Joint Journal, 22 Buckingham Street, London, WC2N 6ET, UK.Search for more papers by this author, A. H. R. W. SimpsonEditor-in-ChiefCorrespondence should be sent to Professor A. H. R. W. Simpson; e-mail: E-mail Address: [email protected]The Bone and Joint Journal, 22 Buckingham Street, London, WC2N 6ET, UK.Search for more papers by this authorPublished Online:21 Sep 2015https://doi.org/10.1302/2046-3758.49.2000571AboutSectionsPDF/EPUB ToolsDownload CitationsTrack CitationsAdd to Favourites ShareShare onFacebookTwitterLinked InRedditEmail Measuring the outcome of an intervention is central to the practice of evidence based medicine, and most research papers evaluating patient outcomes now incorporate some form of patient-based metric, such as questionnaires or performance tests. Once an outcome has been defined, researchers typically want to know if any other factors can influence the result. This is typically assessed with regression analysis.Regression analysis1 determines the relationship of an independent variable (such as bone mineral density) on a dependent variable (such as ageing) with the statistical assumption that all other variables remain fixed. The calculation of the relationship results in a theoretical straight line, and the correlation co-efficient (r) measures how closely the observed data are to the theoretical straight line that we have calculated.In such a linear model, we can judge how well the line fits the data (‘goodness of fit’) by calculating the coefficient of determination (or square of the regression line, R2). R2 is a measure of the percentage of total variation in the dependant variable that is accounted for by the independent variable. An R2 of 1.0 indicates that the data perfectly fit the linear model. Any R2 value less than 1.0 indicates that at least some variability in the data cannot be accounted for by the model (e.g., an R2 of 0.5 indicates that 50% of the variability in the outcome data cannot be explained by the model).Given these statistical tools, we can use the regression equation to predict the value of the dependent variable based on the known value of independent variable. Since many variables may contribute to the outcome (dependent variable), further statistical analysis can be achieved with multiple regression analysis. These models are essentially the same as simple regression analysis, except that the multiple regression analysis equation describes the interrelationship of many variables and allows us to evaluate the joint effect of these variables on the outcome variable in question. Poitras et al2 report an interesting study this month that aims to predict length of stay and early clinical function following joint arthroplasty. Multiple linear regression analyses produced an equation based on the timed-up-and-go test, which was associated with length of stay. In addition, models based on the pre-operative WOMAC function sub-score produced the best model for describing early post-operative function (as calculated by the Older American Resources and Services ALD score). As such the authors were able to conclude that the outcomes assessments (timed-up-and-go and WOMAC) were predictive of outcome, and further modelling identified thresholds of the outcome assessment scores that related to better and worse outcomes.How should we interpret these findings? The authors quite correctly suggest that models such as these could be of value in discharge planning and resource utilisation by targeting the patients that most need intervention and rehabilitation. The reported R2 for the models, however, was 0.18. Bearing in mind that R2, the coefficient of determination, measures the percentage of the variation in the dependent variable that is explained by variation in the independent variable,3 taking the compliment (100 – R2) we see that 82% of the variation in the outcome parameter assessed is unexplained by the model. The principal problem is that the variance in the population studied can strongly influence R2 magnitude. Therefore, there is no guarantee that a high coefficient of determination is indicative of ‘goodness of fit’. Similarly there is no guarantee that a small R2 indicates a weak relationship, given that the statistic is largely influenced by variation in the independent variable.4Therefore, there is no rule for interpreting the strength of R2 in its application to clinical relevance. Useful high values of R2 can be obtained with clinical data sets,5 however, a low R2 can still provide a useful clinical model with respect to data trends, but may be low in precision. In this study there is an association between the performance tests and length of stay; and, using the equations, we can indeed predict one from the other. The accuracy of this prediction though, needs to be borne in mind when using it as a clinical tool. Furthermore, it is not rational to compare R2 across different samples, which given clinical populations, are likely to differ significantly in the variance of the independent and dependent variables.6In controlled environments, such as biomechanical tests on cadaveric bones, the variance across predictive measurements is likely to be low, and therefore R2 values can be expected to lie in the 0.8 range.7 In clinical studies, however, R2 values vary widely depending on the nature of the analysis. For example, when comparing radiographic parameters or associating surgical technical factors, values of R2 are reported in the 0.2 to 0.4 range.8,9 Whereas, comparing data between separate (but intrinsically similar) outcome assessment questionnaires can yield higher values in excess of 0.7.10As such, further validation of the Poitras study2 using new datasets and, ideally, confirmatory analysis of the findings using a much larger sample size, would be required before their regression model could be recommended for use clinically. This does not devalue the appropriateness – or indeed ‘worthiness’ – of reporting these findings in the literature, as the important clinical tools typically start as ideas in small datasets. As with all research papers, the reader requires a basic understanding of methodology to evaluate how relevant the results are to wider practice. References 1 Draper NR, Smith HApplied regression analysis. Wiley-Interscience, 1998. Google Scholar2 Poitras S, Wood KS, Savard J, Dervin GF, Beaule PE. Predicting early clinical function after hip or knee arthroplasty. Bone Joint Res 2015;4:145–151. Link, ISI, Google Scholar3 Schroeder LD, Sjoquist DL, Stephen PEUnderstanding regression analysis: an introductory guide. 1986, Sage Publications; Beverly Hills, California. Google Scholar4 Filho DBF, Silva JA, Rocha E. What is R2 all about? Leviathan – Cadernos de Pesquisa Política 2011;3:60–68. Google Scholar5 Maempel JF, Clement ND, Brenkel IJ, Walmsley PJ. Validation of a prediction model that allows direct comparison of the Oxford Knee Score and American Knee Society clinical rating system. Bone Joint J 2015;97-B:503–509. Link, ISI, Google Scholar6 Kennedy PA guide to econometrics. 2008, Wiley-Blackwell; San Francisco, California:27. Google Scholar7 Eckstein F, Wunderer C, Boehm H, et al. Reproducibility and side differences of mechanical tests for determining the structural strength of the proximal femur. J Bone Miner Res 2004;19:379–385. Crossref, ISI, Google Scholar8 Weber M, Lechler P, von Kunow F, et al. The validity of a novel radiological method for measuring femoral stem version on anteroposterior radiographs of the hip after total hip arthroplasty. Bone Joint J 2015;97-B:306–311. Link, ISI, Google Scholar9 Kuwashima U, Okazaki K, Tashiro Y, et al. Correction of coronal alignment correlates with reconstruction of joint height in unicompartmental knee arthroplasty. Bone Joint Res 2015;4:128–133. Link, ISI, Google Scholar10 Parsons N, Griffin XL, Achten J, Costa ML. Outcome assessment after hip fracture: is EQ-5D the answer? Bone Joint Res 2014;19;3:69–75. Link, ISI, Google ScholarFiguresReferencesRelatedDetailsCited byForecasting the rate of hand injuries in Singapore4 May 2022 | Journal of Occupational Medicine and Toxicology, Vol. 17, No. 1Enhancing psychosomatic health using artificial intelligence-based treatment protocol: A data science-driven approachInternational Journal of Information Management Data Insights, Vol. 2, No. 2The relationship between trust and outcomes during physical therapy care for chronic low back pain25 October 2022 | Physiotherapy Theory and PracticeAn intrinsic analysis on the nature of alumina (Al2O3) reinforced hydroxyapatite nanocompositePhysica B: Condensed Matter, Vol. 642A behavioural study of hydrothermally derived novel alumina/magnesia/hydroxyapatite (Al2O3/MgO/HA) bioceramic nanocompositeJournal of the Mechanical Behavior of Biomedical Materials, Vol. 133Examining the factors influencing adoption of over the top (OTT) services among Indian consumers27 August 2021 | Journal of Science and Technology Policy Management, Vol. 13, No. 3Neurovascular coupling on trial: How the number of trials completed impacts the accuracy and precision of temporally derived neurovascular coupling estimates25 February 2022 | Journal of Cerebral Blood Flow & Metabolism, Vol. 42, No. 8Reliability of non-invasive arterial blood pressure measurement in patients with aneurysmal subarachnoid haemorrhage7 July 2022 | Physiological Measurement, Vol. 43, No. 7Factors Related to Expectations in Individuals Waiting for Total Knee Arthroplasty13 July 2022 | Physiotherapy Canada, Vol. 97Effects of anode position on pedicle screw testing during lumbosacral spinal fusion surgeryThe Spine Journal, Vol. 22Assessing Conservation and Mitigation Banking Practices and Associated Gains and Losses in the United States29 May 2022 | Sustainability, Vol. 14, No. 11Current capacity, bottlenecks, and future projections for offsetting habitat loss using Mitigation and Conservation banking in the United StatesJournal for Nature Conservation, Vol. 67Risk Factors for Potential Mental Illness Among Brazilians in Quarantine Due To COVID-1910 February 2021 | Psychological Reports, Vol. 125, No. 2Predicting height from ulna length for the determination of weight status in New Zealand adolescents: A cross‐sectional study16 June 2021 | Journal of Human Nutrition and Dietetics, Vol. 35, No. 2The determination of skin surface pH via the skin volatile emission using wearable colorimetric sensorsSensing and Bio-Sensing Research, Vol. 35Personalized prediction of rehabilitation outcomes in multiple sclerosis: a proof-of-concept using clinical data, digital health metrics, and machine learning25 November 2021 | Medical & Biological Engineering & Computing, Vol. 60, No. 1Optimization and design of machine learning computational technique for prediction of physical separation processArabian Journal of Chemistry, Vol. 3The Social Media Break-Up: Psycho-Behavioral Measures and ImplicationsIEEE Access, Vol. 10Continuous diagnostic models for volume deficit in patients with acute diarrhea6 September 2021 | Tropical Medicine and Health, Vol. 49, No. 1Optimization of substrate composition for pectinase production from Satkara (Citrus macroptera) peel using Aspergillus niger-ATCC 1640 in solid-state fermentationHeliyon, Vol. 7, No. 10An integrated model for customer equity estimation based on brand equity17 September 2020 | International Journal of Market Research, Vol. 63, No. 5Genetic Variations Associated with Long-Term Treatment Response in Bipolar Depression18 August 2021 | Genes, Vol. 12, No. 8Optimum machine learning algorithm selection for forecasting vegetation indices: MODIS NDVI & EVIRemote Sensing Applications: Society and Environment, Vol. 23Analysis of Bubbles Size Produced in Electroflotation Using Graphite and Stainless Steel Electrode With DinoCapture 2.0Symptom response to mobilization and outcomes in patients with subacromial pain syndrome: A cohort study24 May 2021 | Physiotherapy Research International, Vol. 26, No. 3The validity and reliability of ultra-short-term heart rate variability parameters and the influence of physiological covariatesJournal of Applied Physiology, Vol. 130, No. 6PROMIS Scores Should Not be Used in Isolation to Measure Outcomes in Lumbar Stenosis PatientsSpine, Vol. Publish Ahead of PrintA Multivariate Time-Series Based Approach for Quality Modeling in Wireless Networks12 March 2021 | Sensors, Vol. 21, No. 6Cyanocobalamin Ultraflexible Lipid Vesicles: Characterization and In Vitro Evaluation of Drug-Skin Depth Profiles20 March 2021 | Pharmaceutics, Vol. 13, No. 3Torque Teno Virus plasma level as novel biomarker of retained immunocompetence in HIV-infected patients3 February 2021 | Infection, Vol. 18Impacts of Patient Characteristics and Care Fragmentation on the Value of HIEs18 November 2020 | Production and Operations Management, Vol. 30, No. 2Improvement of Bioethanol Production in Consolidated Bioprocessing (CBP) via Consortium of Aspergillus niger B2484 and Trichoderma asperellum B15811 January 2021 | Pertanika Journal of Science and Technology, Vol. 29, No. 1Use of shear wave elastography on the maternal cervix to recognise cervical insufficiency using a transabdominal ultrasound approach18 December 2020 | Australasian Journal of Ultrasound in MedicineA data-driven framework for selecting and validating digital health metrics: use-case in neurological sensorimotor impairments29 May 2020 | npj Digital Medicine, Vol. 3, No. 1Auditory Development of Frequency Discrimination at Extended High FrequenciesEar & Hearing, Vol. Publish Ahead of PrintComorbidity-dependent changes in alpha and broadband electroencephalogram power during general anaesthesia for cardiac surgeryBritish Journal of Anaesthesia, Vol. 125, No. 4Maternal Anaemia in Pregnancy: A Significantly Greater Risk Factor for Anaemia in Australian Aboriginal Children than Low Birth Weight or Prematurity3 June 2020 | Maternal and Child Health Journal, Vol. 24, No. 8Resistant measures in assessing the adequacy of regression modelsScientific African, Vol. 8Development and temporal validation of a prognostic model for 1-year clinical outcome after decompression surgery for lumbar disc herniation27 February 2020 | European Spine Journal, Vol. 29, No. 7Inadequacy and impact of facility design for adolescents and young adults with cancerJournal of Environmental Psychology, Vol. 69Using accelerations of single inertial measurement units to determine the intensity level of light-moderate-vigorous physical activities: Technical and mathematical considerationsJournal of Biomechanics, Vol. 107Validation and Extension of the Ventral Hernia Repair Cost Prediction ModelJournal of Surgical Research, Vol. 244The Pelagic Habitat of Swordfish (Xiphias gladius) in the Changing Environment of the North Indian Ocean10 December 2019 | Sustainability, Vol. 11, No. 24The self-care profiles and its determinants among adults with hypertension in primary health care clinics in Selangor, Malaysia6 November 2019 | PLOS ONE, Vol. 14, No. 11Effect of printing layer thickness on water absorption and mechanical properties of 3D-printed wood/PLA composite materials23 January 2019 | The International Journal of Advanced Manufacturing Technology, Vol. 102, No. 5-8Changes in prospectively collected longitudinal patient-generated health data are associated with short-term patient-reported outcomes after total joint arthroplasty: a pilot studyArthroplasty Today, Vol. 5, No. 1Evaluation tool for a gastroenterostomy simulated training1 January 2019 | Acta Cirurgica Brasileira, Vol. 34, No. 3It is merely subjective opinion that patient-reported outcome measures are not objective toolsD. F. Hamilton, J. M. Giesinger, K. Giesinger6 December 2017 | Bone & Joint Research, Vol. 6, No. 12Experiences of adult patients using primary care services in Poland – a cross-sectional study in QUALICOPC study framework22 November 2017 | BMC Family Practice, Vol. 18, No. 1Non linear relationship between change in awareness in municipal solid waste management and domestic wastewater management – A case of the Jodipan and Ksatrian village, Malang, East Java Vol. 4, No. 9 Metrics History Published online 21 September 2015 Information© 2015 The author(s)KeywordsRegressionOutcomeStudiesPDF download