Background: A recent systematic review recommends against the use of any of the current COVID-19 prediction models in clinical practice. To enable clinicians to appropriately profile and treat suspected COVID-19 patients at the emergency department (ED), externally validated models that predict poor outcome are desperately needed.

Objective: Our aims were to identify predictors of poor outcome, defined as mortality or ICU admission within 30 days, in patients presenting to the ED with a clinical suspicion of COVID-19, and to develop and externally validate a prediction model for poor outcome.

Methods: In this prospective, multi-centre study, we enrolled suspected COVID-19 patients presenting at the EDs of two hospitals in the Netherlands. We used backward logistic regression to develop a prediction model. We used the area under the curve (AUC), Brier score and pseudo-R2 to assess model performance. The model was externally validated in an Italian cohort.

Results: We included 1193 patients between March 12 and May 27 2020, of whom 196 (16.4%) had a poor outcome. We identified 10 predictors of poor outcome: current malignancy (OR 2.774; 95%CI 1.682-4.576), systolic blood pressure (OR 0.981; 95%CI 0.964-0.998), heart rate (OR 1.001; 95%CI 0.97-1.028), respiratory rate (OR 1.078; 95%CI 1.046-1.111), oxygen saturation (OR 0.899; 95%CI 0.850-0.952), body temperature (OR 0.505; 95%CI 0.359-0.710), serum urea (OR 1.404; 95%CI 1.198-1.645), C-reactive protein (OR 1.013; 95%CI 1.001-1.024), lactate dehydrogenase (OR 1.007; 95%CI 1.002-1.013) and SARS-CoV-2 PCR result (OR 2.456; 95%CI 1.526-3.953). The AUC was 0.86 (95%CI 0.83-0.89), with a Brier score of 0.32 and, and R2 of 0.41. The AUC in the external validation in 500 patients was 0.70 (95%CI 0.65-0.75).

Conclusion: The COVERED risk score showed excellent discriminatory ability, also in external validation. It may aid clinical decision making, and improve triage at the ED in health care environments with high patient throughputs.

References

1. Coronavirus Update (Live) - Worldometers. Available:https://www.worldometers.info/coronavirus/
2. New Cases of COVID-19 In World Countries. In: Johns Hopkins Coronavirus Resource Center [Internet]. [cited 9 Jun 2020]. Available: https://coronavirus.jhu.edu/data/new-cases.
3. Anderson RM, Vegvari C, Truscott J, Collyer BS. Challenges in creating herd immunity to SARS-CoV-2 infection by mass vaccination. The Lancet. 2020; 396: 1614–6. doi:10.1016/S0140-6736(20)32318-7.
4. Abu-Raya B. Predictors of Refractory Coronavirus Disease (COVID-19) Pneumonia. Clinical Infectious Diseases. 2020; ciaa409. doi:10.1093/cid/ciaa409.
5. Guan W, Ni Z, Hu Y, et al. Clinical Characteristics of Coronavirus Disease 2019 in China. N Engl J Med. 2020; 382: 1708–20. doi:10.1056/NEJMoa2002032.
6. Huang C, Wang Y, Li X, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. The Lancet. 2020; 395: 497–506. doi:10.1016/S0140-6736(20)30183-5.
7. Zhou F, Yu T, Du R, et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. The Lancet. 2020; 395: 1054–62. doi:10.1016/S0140-6736(20)30566-3.
8. Larremore DB, Wilder B, Lester E, et al. Test sensitivity is secondary to frequency and turnaround time for COVID-19 screening. Science Advances. 2020; eabd5393. doi:10.1126/sciadv.abd5393.
9. Sethuraman N, Jeremiah SS, Ryo A. Interpreting Diagnostic Tests for SARS-CoV-2. JAMA. 2020 [cited 6 May 2020]. doi:10.1001/jama.2020.8259.
10. Wang W, Xu Y, Gao R, et al. Detection of SARS-CoV-2 in Different Types of Clinical Specimens. JAMA. 2020 [cited 30 Apr 2020]. doi:10.1001/jama.2020.3786.
11. Yang HS, Hou Y, Vasovic LV, et al. Routine Laboratory Blood Tests Predict SARS-CoV-2 Infection Using Machine Learning. Clinical Chemistry. 2020; 66: 1396–404. doi:10.1093/clinchem/hvaa200
12. Wynants L, Van Calster B, Bonten MMJ, et al. Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal. BMJ. 2020; m1328. doi:10.1136/bmj.m1328.
13. Petrilli CM, Jones SA, Yang J, et al. Factors associated with hospital admission and critical illness among 5279 people with coronavirus disease 2019 in New York City: prospective cohort study. BMJ. 2020; 369: m1966. doi:10.1136/bmj.m1966
14. Galloway JB, Norton S, Barker RD, et al. A clinical risk score to identify patients with COVID-19 at high risk of critical care admission or death: An observational cohort study. Journal of Infection. 2020; 81: 282–8. doi:10.1016/j.jinf.2020.05.064.
15. Gong J, Ou J, Qiu X, et al. A Tool for Early Prediction of Severe Coronavirus Disease 2019 (COVID-19): A Multicenter Study Using the Risk Nomogram in Wuhan and Guangdong, China. Clinical Infectious Diseases. 2020; 71: 833–40. doi:10.1093/cid/ciaa443.
16. Ji D, Zhang D, Xu J, et al. Prediction for Progression Risk in Patients with COVID-19 Pneumonia: the CALL Score. Clinical Infectious Diseases. 2020; ciaa414. doi:10.1093/cid/ciaa414.
17. Wu C, Chen X, Cai Y, et al. Risk Factors Associated With Acute Respiratory Distress Syndrome and Death in Patients With Coronavirus Disease 2019 Pneumonia in Wuhan, China. JAMA Intern Med. 2020 [cited 22 May 2020]. doi:10.1001/jamainternmed.2020.0994.
18. Wu Z, McGoogan JM. Characteristics of and Important Lessons From the Coronavirus Disease 2019 (COVID-19) Outbreak in China: Summary of a Report of 72 314 Cases From the Chinese Center for Disease Control and Prevention. JAMA. 2020;323:doi:10.1001/jama.2020.2648.
19. Yang J, Zheng Y, Gou X, et al. Prevalence of comorbidities and its effects in patients infected with SARS-CoV-2: a systematic review and meta-analysis. International Journal of Infectious Diseases. 2020; 94: 91–5. doi:10.1016/j.ijid.2020.03.017.
20. Zheng Z, Peng F, Xu B, et al. Risk factors of critical & mortal COVID-19 cases: A systematic literature review and meta-analysis. Journal of Infection. 2020; S0163445320302346. doi:10.1016/j.jinf.2020.04.021.
21. Zhao Z, Chen A, Hou W, et al. Prediction model and risk scores of ICU admission and mortality in COVID-19. PLoS One. 2020;15: e0236618. doi:10.1371/journal.pone.0236618.
22. Sperrin M, McMillan B. Prediction models for covid-19 outcomes. BMJ. 2020; 371: m3777. doi:10.1136/bmj.m3777.
23. What prognostic clinical risk prediction scores for COVID-19 are currently available for use in the community setting? In: CEBM [Internet]. [cited 24 May 2020]. Available: https://www.cebm.net/covid-19/what-prognostic-clinical-risk-prediction-scores-for-covid19-are-currently-available-for-use-in-the-community-setting/
24. Ryan L, Lam C, Mataraso S, et al. Mortality prediction model for the triage of COVID-19, pneumonia, and mechanically ventilated ICU patients: A retrospective study. Ann Med Surg (Lond). 2020; 59: 207–16. doi:10.1016/j.amsu.2020.09.044.
25. Siontis GCM, Tzoulaki I, Ioannidis JPA. Predicting Death: An Empirical Evaluation of Predictive Tools for Mortality. Archives of Internal Medicine. 2011; 171: 1721–6. doi:10.1001/ archinternmed.2011.334.
26. Wang J-Y, Chen Y-X, Guo S-B, et al. Predictive performance of quick Sepsis-related Organ Failure Assessment for mortality and ICU admission in patients with infection at the ED. Am J Emerg Med. 2016; 34: 1788–93. doi:10.1016/j.ajem.2016.06.015.
27. Handreiking ‘Diagnostiek bij verdenking COVID-19 en opnameindicatie.’ In: Federatie Medisch Specialisten [Internet]. 11 May 2020 [cited 8 Jul 2020]. Available: https://www.demedischspecialist.nl/nieuws/handreiking-%E2%80%98diagnostiek-bijverdenking-covid-19-en-opname-indicatie%E2%80%99.
28. Collins GS, Reitsma JB, Altman DG, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ. 2015; 350:g7594–g7594. doi:10.1136/bmj.g7594.
29. Buuren S van. Flexible imputation of missing data. Second edition. Boca Raton: CRC Press, Taylor and Francis Group; 2018.
30. Jr DWH, Lemeshow S, Sturdivant RX. Applied Logistic Regression. John Wiley & Sons; 2013.
31. Clift AK, Coupland CAC, Keogh RH, et al. Living risk prediction algorithm (QCOVID) for risk of hospital admission and mortality from coronavirus 19 in adults: national derivation and validation cohort study. BMJ. 2020; 371: m3731. doi:10.1136/bmj.m3731
32. Liang W, Liang H, Ou L, et al. Development and Validation of a Clinical Risk Score to Predict the Occurrence of Critical Illness in Hospitalized Patients With COVID-19. JAMA Intern Med. 2020;180: 1081. doi:10.1001/jamainternmed.2020.2033.
33. Royston P, Altman DG, Sauerbrei W. Dichotomizing continuous predictors in multiple regression: a bad idea. Statist Med. 2006; 25: 127–141. doi:10.1002/sim.2331.
34. Van Calster B, Nieboer D, Vergouwe Y, et al. A calibration hierarchy for risk models was defined: from utopia to empirical data. Journal of Clinical Epidemiology. 2016; 74: 167–76. doi:10.1016/j.jclinepi.2015.12.005.
35. Bleeker SE, Moll HA, Steyerberg EW, et al. External validation is necessary in prediction research: a clinical example. J Clin Epidemiol. 2003; 56: 826–32. doi:10.1016/s0895-4356(03)00207-5.
36. Halpin DMG, Faner R, Sibila O, Badia JR, et al. Do chronic respiratory diseases or their treatment affect the risk of SARSCoV-2 infection? The Lancet Respiratory Medicine. 2020; 8: 436–8. doi:10.1016/S2213-2600(20)30167-3.
37. Singer AJ, Morley EJ, Meyers K, et al. Cohort of Four Thousand Four Hundred Four Persons Under Investigation for COVID-19 in a New York Hospital and Predictors of ICU Care and Ventilation. Annals of Emergency Medicine. 2020; 76: 394–404. doi:10.1016/j.annemergmed.2020.05.011.
38. Lieveld AWE, Azijli K, Teunissen BP, et al. Chest CT in COVID-19 at the ED: Validation of the COVID-19 Reporting and Data System (CO-RADS) and CT severity score. Chest. doi:10.1016/j.chest.2020.11.026.
39. Lichter Y, Topilsky Y, Taieb P, et al. Lung ultrasound predicts clinical course and outcomes in COVID-19 patients. Intensive Care Med. 2020 [cited 22 Sep 2020]. doi:10.1007/s00134-020-06212-1.
40. Knight T, Edwards L, Rajasekaran A, et al. Point-of-care lung ultrasound in the assessment of suspected COVID-19: a retrospective service evaluation with a severity score. Acute Medicine. 2020; 19 (4):192-200.