Segmentation of high-cost patients in England: insights from a machine learning analysis

Background Segmentation of high-cost patients can be used to inform service design. We aimed to cluster high-cost patients in the English healthcare system into distinct groups based on diagnoses and socioeconomic characteristics. Methods We examined primary and secondary care costs in the financial year 2018/19 for 10,119,490 adult patients registered with general practices in England contributing to the Clinical Practice Research Datalink and linked to Hospital Episode Statistics, representing around 23% of the adult population in England. High-cost patients were defined as those in the top 1% of total care costs. We used multiple unsupervised machine learning approaches to group patients based on diagnoses, age, sex, deprivation, and ethnicity, and evaluated the resulting clusters for validity and stability using established indices. We also examined several patient subgroups, including age-stratified groups, in-year decedents and patients with the highest unplanned care costs. Results High-cost patients accounted for 26.8% of total healthcare costs in the study population. They had average annual costs of £17,485 (median: £14,609; interquartile range: £12,028£19,633, 2018 prices) compared with £653 (median: £103, interquartile range: £14–£352) for the population. The optimal clustering method was hierarchical clustering, which produced nine clusters with distinct multimorbidity and sociodemographic profiles. Across those clusters, mean age varied between 56 and 79 years and mean annual costs per cluster ranged between £15,792 (95% CI: £15,629–£15,955) and £19,107 (95%CI: £18,784–£19,430). Several core clinical profiles recurred across different clustering approaches and subgroup analyses, including younger people with liver disease and mental health conditions, patients with nodal metastases, patients with prostate cancer and nodal metastases, and older people with cardiovascular disease and dementia. Conclusions High-cost patients are a heterogeneous population with distinct clinical and sociodemographic profiles and utilisation patterns. Incorporating cluster-based approaches into population health management may help inform the targeting of case management programmes, resource allocation, and health system planning.