Comparison of Mortality Predictive Models of Sepsis Patients Based on Machine Learning

Ziyang Wang; Yushan Lan; Zidu Xu; Yaowen Gu; Jiao Li

doi:10.24920/004102

Your Location：

Home >

Browse articles >

Comparison of Mortality Predictive Models of Sepsis Patients Based on Machine Learning

Chinese Medical Sciences Journal Vol. 37, Issue 3, Pages: 201-209(2022)

Scientific Data Sharing and Reuse:Original Article | Updated：2024-04-10

- Comparison of Mortality Predictive Models of Sepsis Patients Based on Machine Learning
- Affiliations：
  
  Institute of Medical Information/Medical Library, Chinese Academy of Medical Science & Peking Union Medical College, Beijing 100020, China
- Author bio：
- Funds：
- DOI：10.24920/004102
  CLC：
- Received：21 April 2022，
  
  Accepted：2022-8-10，
  
  Published Online：20 September 2022，
  
  Published：30 September 2022
- Accepted：
Scan QR Code
Ziyang Wang, Yushan Lan, Zidu Xu, et al. Comparison of Mortality Predictive Models of Sepsis Patients Based on Machine Learning[J]. Chinese medical sciences journal, 2022, 37(3): 201-209.
DOI：

Ziyang Wang, Yushan Lan, Zidu Xu, et al. Comparison of Mortality Predictive Models of Sepsis Patients Based on Machine Learning[J]. Chinese medical sciences journal, 2022, 37(3): 201-209. DOI： 10.24920/004102.

摘要

目的

比较五个机器学习模型和SAPS II评分在预测脓毒症患者30天内死亡率方面的表现。

方法

从MIMIC-IV数据库中提取败血症患者相关数据

生成临床特征

并通过互信息法和网格搜索进行特征筛选。构建逻辑回归、随机森林、LightGBM、XGBoost等机器学习模型

预测脓毒症患者30天内死亡率。此外

还获得了包括准确率、精确度、召回率、F1得分和受试者工作特性曲线下面积（area under the curve

AUC）在内的五个模型评估指标。最后

在外部数据集中验证了模型的效果。

结果

LightGBM的表现优于其他方法

取得了最高的AUC（0.900）、准确率（0.808）和精确度（0.559）。所有机器学习模型的表现都优于SAPS II评分（AUC=0.748）。在外部数据集的验证中LightGBM的AUC达到0.883。

结论

机器学习模型在预测败血症患者的死亡率方面被认为是比传统的SAPS II评分更有效的方法。

Abstract

Objective

To compare the performance of five machine learning models and SAPS II score in predicting the 30-day mortality amongst patients with sepsis.

Methods

The sepsis patient-related data were extracted from the MIMIC-IV database. Clinical features were generated and selected by mutual information and grid search. Logistic regression

Random forest

LightGBM

XGBoost

and other machine learning models were constructed to predict the mortality probability. Five measurements including accuracy

precision

recall

F1 score

and area under curve (AUC) were acquired for model evaluation. An external validation was implemented to avoid conclusion bias.

Results

LightGBM outperformed other methods

achieving the highest AUC (0.900)

accuracy (0.808)

and precision (0.559). All machine learning models performed better than SAPS II score (AUC=0.748). LightGBM achieved 0.883 in AUC in the external data validation.

Conclusions

The machine learning models are more effective in predicting the 30-day mortality of patients with sepsis than the traditional SAPS II score.

关键词

Keywords

references

Singer M , Deutschman CS , Seymour CW , et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) . JAMA 2016 ; 315 ( 8 ): 801 - 10 . doi: 10.1001/jama.2016.0287 https://dx.doi.org/10.1001/jama.2016.0287 . DOI: 10.1001/jama.2016.0287 http://doi.org/10.1001/jama.2016.0287

Martin GS . Sepsis, severe sepsis and septic shock: changes in incidence, pathogens and outcomes . Expert Rev Anti Infect Ther 2012 ; 10 ( 6 ): 701 - 6 . doi: 10.1586/eri.12.50 https://dx.doi.org/10.1586/eri.12.50 . DOI: 10.1586/eri.12.50 http://doi.org/10.1586/eri.12.50

Song J , Park DW , Moon S , et al. Diagnostic and prognostic value of interleukin-6, pentraxin 3, and procalcitonin levels among sepsis and septic shock patients: a prospective controlled study according to the Sepsis-3 definitions . BMC Infect Dis 2019 ; 19 : 68 . doi: 10.1186/s12879-019-4618-7 https://dx.doi.org/10.1186/s12879-019-4618-7 . DOI: 10.1186/s12879-019-4618-7 http://doi.org/10.1186/s12879-019-4618-7 https://doi.org/10.1186/s12879-019-4618-7 https://doi.org/10.1186/s12879-019-4618-7

Xie J , Wang H , Kang Y , et al. The epidemiology of sepsis in Chinese ICUs: A national cross-sectional survey . Crit Care Med 2020 ; 48 ( 3 ): e209 - e218 . doi: 10.1097/CCM.0000000000004155 https://dx.doi.org/10.1097/CCM.0000000000004155 . DOI: 10.1097/CCM.0000000000004155 http://doi.org/10.1097/CCM.0000000000004155 http://journals.lww.com/10.1097/CCM.0000000000004155 http://journals.lww.com/10.1097/CCM.0000000000004155

Zhao L , Zhao L , Wang YY , et al. Platelets as a prognostic marker for sepsis: a cohort study from the MIMIC-III database . Medicine 2020 ; 99 ( 45 ): e23151 . doi: 10.1097/MD.0000000000023151 https://dx.doi.org/10.1097/MD.0000000000023151 . DOI: 10.1097/MD.0000000000023151 http://doi.org/10.1097/MD.0000000000023151 https://journals.lww.com/10.1097/MD.0000000000023151 https://journals.lww.com/10.1097/MD.0000000000023151

Chen H , Zhu Z , Zhao C , et al. Central venous pressure measurement is associated with improved outcomes in septic patients: an analysis of the MIMIC-III database . Crit Care 2020 ; 24 ( 1 ): 433 . doi: 10.1186/s13054-020-03109-9 https://dx.doi.org/10.1186/s13054-020-03109-9 . DOI: 10.1186/s13054-020-03109-9 http://doi.org/10.1186/s13054-020-03109-9 https://doi.org/10.1186/s13054-020-03109-9 https://doi.org/10.1186/s13054-020-03109-9

Zhu C , Xu Z , Gu Y , et al. Prediction of post-stroke urinary tract infection risk in immobile patients using machine learning: an observational cohort study . J Hosp Infect 2022 ; 122 : 96 - 107 . doi: 10.1016/j.jhin.2022.01.002 https://dx.doi.org/10.1016/j.jhin.2022.01.002 . DOI: 10.1016/j.jhin.2022.01.002 http://doi.org/10.1016/j.jhin.2022.01.002

Wang Y , Sun F , Hong G , et al. Thyroid hormone levels as a predictor marker predict the prognosis of patients with sepsis . Am J Emerg Med 2021 ; 45 : 42 - 7 . doi: 10.1016/j.ajem.2021.02.014 https://dx.doi.org/10.1016/j.ajem.2021.02.014 . DOI: 10.1016/j.ajem.2021.02.014 http://doi.org/10.1016/j.ajem.2021.02.014

Hou N , Li M , He L , et al. Predicting 30-days mortality for MIMIC-III patients with sepsis-3: a machine learning approach using XGboost . J Transl Med 2020 ; 18 : 462 . doi: 10.1186/s12967-020-02620-5 https://dx.doi.org/10.1186/s12967-020-02620-5 . DOI: 10.1186/s12967-020-02620-5 http://doi.org/10.1186/s12967-020-02620-5

Feng M , McSparron JI , Kien DT , et al. Transthoracic echocardiography and mortality in sepsis: analysis of the MIMIC-III database . Intens Care Med 2018 ; 44 ( 6 ): 884 - 92 . doi: 10.1007/s00134-018-5208-7 https://dx.doi.org/10.1007/s00134-018-5208-7 . DOI: 10.1007/s00134-018-5208-7 http://doi.org/10.1007/s00134-018-5208-7 https://doi.org/10.1007/s00134-018-5208-7 https://doi.org/10.1007/s00134-018-5208-7

Wang D , Li J , Sun Y , et al. A machine learning model for accurate prediction of sepsis in ICU patients . Front Public Health 2021 ; 9 : 754348 . doi: 10.3389/fpubh.2021.754348 https://dx.doi.org/10.3389/fpubh.2021.754348 . DOI: 10.3389/fpubh.2021.754348 http://doi.org/10.3389/fpubh.2021.754348 https://www.frontiersin.org/articles/10.3389/fpubh.2021.754348/full https://www.frontiersin.org/articles/10.3389/fpubh.2021.754348/full

Johnson A , Bulgarelli L , Pollard T , et al. MIMIC-IV (version 0.4) . PhysioNet 2020 . https://doi.org/10.13026/a3wn-hq05 https://doi.org/10.13026/a3wn-hq05 https://doi.org/10.13026/a3wn-hq05.

Uusitalo L . Advantages and challenges of Bayesian networks in environmental modelling . Ecol Modell 2007 ; 203 ( 3 ): 312 - 18 . doi: 10.1016/j.ecolmodel.2006.11.033 https://dx.doi.org/10.1016/j.ecolmodel.2006.11.033 . DOI: 10.1016/j.ecolmodel.2006.11.033 http://doi.org/10.1016/j.ecolmodel.2006.11.033 https://linkinghub.elsevier.com/retrieve/pii/S0304380006006089 https://linkinghub.elsevier.com/retrieve/pii/S0304380006006089

Mihaljević B , Bielza C , Larrañaga P . Bayesian networks for interpretable machine learning and optimization . Neurocomputing 2021 ; 456 : 648 - 65 . doi: 10.1016/j.neucom.2021.01.138 https://dx.doi.org/10.1016/j.neucom.2021.01.138 . DOI: 10.1016/j.neucom.2021.01.138 http://doi.org/10.1016/j.neucom.2021.01.138 https://linkinghub.elsevier.com/retrieve/pii/S0925231221009644 https://linkinghub.elsevier.com/retrieve/pii/S0925231221009644

Hanko M , Grendár M , Snopko P , et al. Random forest-based prediction of outcome and mortality in patients with traumatic brain injury undergoing primary decompressive craniectomy . World Neurosurg 2021 ; 148 : e450 - e458 . doi: 10.1016/j.wneu.2021.01.002 https://dx.doi.org/10.1016/j.wneu.2021.01.002 . DOI: 10.1016/j.wneu.2021.01.002 http://doi.org/10.1016/j.wneu.2021.01.002

Davagdorj K , Pham VH , Theera-Umpon N , et al. XGBoost-based framework for smoking-induced noncommunicable disease prediction . Int J Environment Res Public Health 2020 ; 17 ( 18 ): e6513 . doi: 10.3390/ijerph17186513 https://dx.doi.org/10.3390/ijerph17186513 . DOI: 10.3390/ijerph17186513 http://doi.org/10.3390/ijerph17186513

Zhang C , Lei X , Liu L . Predicting metabolite-disease associations based on lightgbm model . Front Genet 2021 ; 12 : 660275 . doi: 10.3389/fgene.2021.660275 https://dx.doi.org/10.3389/fgene.2021.660275 . DOI: 10.3389/fgene.2021.660275 http://doi.org/10.3389/fgene.2021.660275 https://www.frontiersin.org/articles/10.3389/fgene.2021.660275/full https://www.frontiersin.org/articles/10.3389/fgene.2021.660275/full

Le Gall JR , Lemeshow S , Saulnier F . A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study . JAMA 1993 ; 270 ( 24 ): 2957 - 63 . doi: 10.1001/jama.270.24.2957 https://dx.doi.org/10.1001/jama.270.24.2957 . DOI: 10.1001/jama.270.24.2957 http://doi.org/10.1001/jama.270.24.2957 http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.1993.03510240069035 http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.1993.03510240069035

Moreno-Torres V , Royuela A , Múñez E , et al. Better prognostic ability of NEWS2, SOFA and SAPS-II in septic patients . Medicina Cinica 2021 ; 159 ( 5 ): 224 - 9 . doi: 10.1016/j.medcli.2021.10.021 https://dx.doi.org/10.1016/j.medcli.2021.10.021 . DOI: 10.1016/j.medcli.2021.10.021 http://doi.org/10.1016/j.medcli.2021.10.021

Cohen J , Vincent JL , Adhikari NKJ , et al. Sepsis: a roadmap for future research . Lancet Infect Dis 2015 ;15 ( 5 ): 581 - 614 . doi: 10.1016/S1473-3099(15)70112-X https://dx.doi.org/10.1016/S1473-3099(15)70112-X . DOI: 10.1016/S1473-3099(15)70112-X http://doi.org/10.1016/S1473-3099(15)70112-X

Zhang ZQ . Effect of changes in urine volume on prognosis of patients with sepsis and acute kidney injury after continuous renal replacement therapy . Chin Med Pharm 2021 ; 11 ( 12 ): 178 - 82 .

Lundberg S , Lee SI . A Unified Approach to Interpreting Model Predictions, carXiv: 1705.07874 . Available from: http://doi.org/1048550/arXiv.1705.07874 http://doi.org/1048550/arXiv.1705.07874 http://doi.org/1048550/arXiv.1705.07874.

Dugar S , Choudhary C , Duggal A . Sepsis and septic shock: Guideline-based management . Cleveland Clin J Med 2020 ; 87 ( 1 ): 53 - 64 . doi: 10.3949/ccjm.87a.18143 https://dx.doi.org/10.3949/ccjm.87a.18143 . DOI: 10.3949/ccjm.87a.18143 http://doi.org/10.3949/ccjm.87a.18143 https://www.ccjm.org//lookup/doi/10.3949/ccjm.87a.18143 https://www.ccjm.org//lookup/doi/10.3949/ccjm.87a.18143

Vincent JL , Ferguson A , Pickkers P , et al. The clinical relevance of oliguria in the critically ill patient: analysis of a large observational database . Criti Care 2020 ; 24 : 171 . doi: 10.1186/s13054-020-02858-x https://dx.doi.org/10.1186/s13054-020-02858-x . DOI: 10.1186/s13054-020-02858-x http://doi.org/10.1186/s13054-020-02858-x https://doi.org/10.1186/s13054-020-02858-x https://doi.org/10.1186/s13054-020-02858-x

Views

1266

Downloads

CSCD

Alert me when the article has been cited

Submit

Publicity Resources

Intelligent Electrocardiogram Analysis in Medicine: Data, Methods, and Applications

Radiomics in Antineoplastic Agents Development: Application and Challenge in Response Evaluation

Advances of Artificial Intelligence Application in Medical Imaging of Ovarian Cancers

Detection of Asymptomatic Carotid Artery Stenosis in High-Risk Individuals of Stroke Using a Machine-Learning Algorithm

Artificial Intelligence in Healthcare and Medicine: Promises, Ethical Challenges and Governance

Related Author

Ying An

Feng-Yi Guo

Wei-Bai Pan

Jian-Xin Wang

Yu-Xia Guan

Tang Lei

Li Jiazheng

Zhe Wu

Related Institution

Hunan Provincial Key Lab on Bioinformatics, School of Computer Science and Engineering, Central South University

Big Data Institute, Central South University

Department of Radiology, Peking University Cancer Hospital & Institute, Key Laboratory of Carcinogenesis and Translational Research (Ministry of Education)

Department of Radiology, Beijing United Family Hospital

Department of Radiology, Fushun Central Hospital

⁰