Development and validation of automated computer aided-risk score for predicting the risk of in-hospital mortality using first electronically recorded blood test results and vital signs for COVID-19 hospital admissions: a retrospective development and validation study

Abstract

Objectives 

There are no established mortality risk equations specifically for unplanned emergency medical admissions which include patients with the novel coronavirus SARS-19 (COVID-19). We aim to develop and validate a computer-aided risk score (CARMc19) for predicting mortality risk by combining COVID-19 status, the first electronically recorded blood test results and latest version of the National Early Warning Score (NEWS2).

Design 

Logistic regression model development and validation study using a cohort of unplanned emergency medical admissions to hospital.

Setting 

York Hospital (YH) as model development dataset and Scarborough Hospital (SH) as model validation dataset.

Participants 

Unplanned adult medical admissions discharged over three months (11 March 2020 to 13 June 2020) from two hospitals (YH for model development; SH for external model validation) based on admission NEWS2 electronically recorded within ±24 hours and/or blood test results within ±96 hours of admission. We used logistic regression modelling to predict the risk of in-hospital mortality using two models: 1) CARMc19_N: age + sex + NEWS2 including subcomponents + COVID19; 2) CARMc19_NB: CARMc19_N in conjunction with seven blood test results and acute kidney injury score. Model performance was evaluated according to discrimination (c-statistic), calibration (graphically), and clinical usefulness at NEWS2 thresholds of 4+, 5+, 6+.

Results 

The risk of in-hospital mortality following emergency medical admission was similar in development and validation datasets (8.4% vs 8.2%). The c-statistics for predicting mortality for Model CARMc19_NB is better than CARMc19_N in the validation dataset (CARMc19_NB = 0.88 (95%CI 0.86 to 0.90) vs CARMc19_N = 0.86 (95%CI 0.83 to 0.88)). Both models had good internal and external calibration (CARMc19_NB: 1.01 (95%CI 0.88 vs 1.14) & CARMc19_N: 0.95 (95%CI 0.83 to 1.06)). At all NEWS2 thresholds (4+, 5+, 6+) model CARMc19_NB had better sensitivity and similar specificity.

Conclusions 

We have developed a validated CARMc19 score with good performance characteristics for predicting the risk of in-hospital mortality following an emergency medical admission using the patient’s first, electronically recorded vital signs and blood tests results. Since the CARMc19 scores place no additional data collection burden on clinicians and is readily automated, it may now be carefully introduced and evaluated in hospitals with sufficient informatics infrastructure.

Citation

Muhammad F, Mohammed A .M., Richardson D.,  Massimo F. &  Beatson K (2020), Development and validation of automated computer aided-risk score for predicting the risk of in-hospital mortality using first electronically recorded blood test results and vital signs for COVID-19 hospital admissions: a retrospective development and validation study, MedRxiv 11.30.20241273;