Ensemble learning for poor prognosis predictions: A case study on SARS-CoV-2

Objective: Risk prediction models are widely used to inform evidence-based clinical decision making. However, few models developed from single cohorts can perform consistently well at population level where diverse prognoses exist (such as the SARS-CoV-2 [severe acute respiratory syndrome coronavirus 2] pandemic). This study aims at tackling this challenge by synergizing prediction models from the literature using ensemble learning. Materials and Methods: In this study, we selected and reimplemented 7 prediction models for COVID-19 (coronavirus disease 2019) that were derived from diverse cohorts and used different implementation techniques. A novel ensemble learning framework was proposed to synergize them for realizing personalized predictions for individual patients. Four diverse international cohorts (2 from the United Kingdom and 2 from China; N = 5394) were used to validate all 8 models on discrimination, calibration, and clinical usefulness. Results: Results showed that individual prediction models could perform well on some cohorts while poorly on others. Conversely, the ensemble model achieved the best performances consistently on all metrics quantifying discrimination, calibration, and clinical usefulness. Performance disparities were observed in cohorts from the 2 countries: all models achieved better performances on the China cohorts. Discussion: When individual models were learned from complementary cohorts, the synergized model had the potential to achieve better performances than any individual model. Results indicate that blood parameters and physiological measurements might have better predictive powers when collected early, which remains to be confirmed by further studies. Conclusions: Combining a diverse set of individual prediction models, the ensemble method can synergize a robust and well-performing model by choosing the most competent ones for individual patients.

PubMed ID

33185672 (pubmed)

Additional Links

https://academic.oup.com/jamia/article/28/4/791/5981140

Type

Journal article

Language

en

Description

© 2020 The Authors. Published by Oxford University Press. This is an open access article available under a Creative Commons licence. The published version can be accessed at the following link on the publisher’s website: https://doi.org/10.1093/jamia/ocaa295

ISSN

1067-5027

EISSN

1527-974X

Sponsors

HW and HZ are supported by a Medical Research Council and Health Data Research UK Grant (MR/S004149/1), an Industrial Strategy Challenge Grant (MC_PC_18029), and a Wellcome Institutional Translation Partnership Award (PIII054). AK is supported by a Medical Research Council and Health Data Research UK Grant (MR/S003991/1). XW is supported by the National Natural Science Foundation of China (81700006). DMB is funded by a UKRI Innovation Fellowship (Health Data Research UK MR/S00310X/1).TV, FG-S, and TW are funded by National Institute for Health Research (NIHR) covid/non-covid research grants and Queen Elizabeth Hospital Charities. KD is supported by the LifeArc STOPCOVID award. VRC and GVG acknowledge support from the NIHR Birmingham Experimental Cancer Medical Centre, NIHR Birmingham Surgical Reconstruction and Microbiology Research Centre, Nanocommons H2020-EU (731032), and the NIHR Birmingham Biomedical Research Centre and Medical Research Council Health Data Research UK (HDRUK/CFC/01). RJBD is supported by NIHR Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King’s College London; Health Data Research UK; the BigData@Heart Consortium, funded by the Innovative Medicines Initiative-2 Joint Undertaking under grant agreement no. 116074; the National Institute for Health Research University College London Hospitals Biomedical Research Centre; the UK Research and Innovation London Medical Imaging and Artificial Intelligence Centre for Value Based Healthcare; and the NIHR Applied Research Collaboration South London at King’s College Hospital NHS Foundation Trust.

Rights

Licence for published version: Creative Commons Attribution 4.0 International

cb

Collections

Faculty of Science and Engineering