An Interpretable Prediction Model for Acute Kidney Injury Based on XGBoost and SHAP

LUO Yan; WANG Cong; YE Wenling

doi:10.11999/JEIT210931

Volume 44 Issue 1

Jan. 2022

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2022 > 44(1): 27-38

LUO Yan, WANG Cong, YE Wenling. An Interpretable Prediction Model for Acute Kidney Injury Based on XGBoost and SHAP[J]. Journal of Electronics & Information Technology, 2022, 44(1): 27-38. doi: 10.11999/JEIT210931

Citation:

LUO Yan, WANG Cong, YE Wenling. An Interpretable Prediction Model for Acute Kidney Injury Based on XGBoost and SHAP[J]. Journal of Electronics & Information Technology, 2022, 44(1): 27-38. doi: 10.11999/JEIT210931

Citation:

PDF( 1430 KB)

An Interpretable Prediction Model for Acute Kidney Injury Based on XGBoost and SHAP

doi: 10.11999/JEIT210931

LUO Yan¹,
WANG Cong²,
YE Wenling^{3
,
,}

1.
School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Beijing 100876, China
2.
Key Laboratory of Trustworthy Distributed Computing and Service, Beijing University of Posts and Telecommunications, Ministry of Education, Beijing 100876, China
3.
Department of Nephrology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing 100730, China

Received Date: 2021-09-02
Rev Recd Date: 2021-10-19

Available Online: 2021-10-25

Publish Date: 2022-01-10

Abstract

Abstract

The development of Acute Kidney Injury (AKI) during admission to the Intensive Care Unit (ICU) is associated with increased morbidity and mortality. The objective of this study is to develop a machine learning-based framework for interpretable AKI prediction in critical care that can achieve both good prediction and interpretation capability. Data extracted from the Medical Information Mart for Intensive Care III (MIMIC-III) include patient age, gender, vital signs and lab values during the first day of ICU admission and subsequent hospitalization. In this study, the prediction performance of the XGBoost model is demonstrated by comparing it to four other machine learning models. In addition, the SHapley Additive exPlanation (SHAP) framework is used to provide individualized evaluation and explanations to enable personalized clinical decision support. The results show that XGBoost can predict AKI robustly with an Accuracy and the area Under the receiver operating Characteristic curve (AUC) of 0.824 and 0.840, respectively, which are higher than previous prediction models. Furthermore, a feature dependency analysis is conducted for two pairs of features and found decrease in urine volume and elevation of blood urea nitrogen indicates an increase of AKI risk. To sum up, this interpretable predictive model may help clinicians more accurately identify patients at risk of AKI in intensive care and provide better treatment for patients. In addition, the use of this interpretability framework increases model transparency and facilitates clinicians to analyze the reliability of predictive models.
- Acute kidney injury,
- Intensive Care Unit (ICU),
- Model interpretation,
- Clinical decision support,
- Prediction model

FullText(HTML)

References(67)

References

[1]	LUNDBERG S M and LEE S I. A unified approach to interpreting model predictions[C]. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, USA, 2017: 4765–4774.
[2]	ZHOU Jiaojiao, LIU Yun, TANG Yi, et al. A comparison of RIFLE, AKIN, KDIGO, and Cys-C criteria for the definition of acute kidney injury in critically ill patients[J]. International Urology and Nephrology, 2016, 48(1): 125–132. doi: 10.1007/s11255-015-1150-6
[3]	KELLUM J A, LAMEIRE N, ASPELIN P, et al. Kidney disease: Improving global outcomes (KDIGO) acute kidney injury work group. KDIGO clinical practice guideline for acute kidney injury[J]. Kidney International Supplements, 2012, 2(1): 1–138. doi: 10.1038/kisup.2012.1
[4]	COCA S G, YUSUF B, SHLIPAK M G, et al. Long-term risk of mortality and other adverse outcomes after acute kidney injury: A systematic review and meta-analysis[J]. American Journal of Kidney Diseases, 2009, 53(6): 961–973. doi: 10.1053/j.ajkd.2008.11.034
[5]	CHERTOW G M, BURDICK E, HONOUR M, et al. Acute kidney injury, mortality, length of stay, and costs in hospitalized patients[J]. Journal of the American Society of Nephrology, 2005, 16(11): 3365–3370. doi: 10.1681/ASN.2004090740
[6]	CHAWLA L S, BELLOMO R, BIHORAC A, et al. Acute kidney disease and renal recovery: Consensus report of the Acute Disease Quality Initiative (ADQI) 16 Workgroup[J]. Nature Reviews Nephrology, 2017, 13(4): 241–257. doi: 10.1038/nrneph.2017.2
[7]	KELLUM J A and PROWLE J R. Paradigms of acute kidney injury in the intensive care setting[J]. Nature Reviews Nephrology, 2018, 14(4): 217–230. doi: 10.1038/nrneph.2017.184
[8]	HOSTE E A J and SCHURGERS M. Epidemiology of acute kidney injury: How big is the problem?[J]. Critical Care Medicine, 2008, 36(4): S146–S151.
[9]	ZENG Xiaoxi, MCMAHON G M, BRUNELLI S M, et al. Incidence, outcomes, and comparisons across definitions of AKI in hospitalized individuals[J]. Clinical Journal of the American Society of Nephrology, 2014, 9(1): 12–20. doi: 10.2215/CJN.02730313
[10]	YANG Li, XING Guolan, WANG Li, et al. Acute kidney injury in China: A cross-sectional survey[J]. The Lancet, 2015, 386(10002): 1465–1471. doi: 10.1016/S0140-6736(15)00344-X
[11]	LIBÓRIO A B, LEITE T T, DE OLIVEIRA NEVES F M, et al. AKI complications in critically ill patients: Association with mortality rates and RRT[J]. Clinical Journal of the American Society of Nephrology, 2015, 10(1): 21–28. doi: 10.2215/CJN.04750514
[12]	DE OLIVEIRA MARQUES F, OLIVEIRA S A, DE LIMA E SOUZA P F, et al. Kinetic estimated glomerular filtration rate in critically ill patients: Beyond the acute kidney injury severity classification system[J]. Critical Care, 2017, 21(1): 280. doi: 10.1186/s13054-017-1873-0
[13]	SUSANTITAPHONG P, CRUZ D N, CERDA J, et al. World incidence of AKI: A meta-analysis[J]. Clinical Journal of the American Society of Nephrology, 2013, 8(9): 1482–1493. doi: 10.2215/CJN.00710113
[14]	SRISAWAT N, SILEANU F E, MURUGAN R, et al. Variation in risk and mortality of acute kidney injury in critically ill patients: A multicenter study[J]. American Journal of Nephrology, 2015, 41(1): 81–88. doi: 10.1159/000371748
[15]	DASTA J F and KANE-GILL S. Review of the literature on the costs associated with acute kidney injury[J]. Journal of Pharmacy Practice, 2019, 32(3): 292–302. doi: 10.1177/0897190019852556
[16]	OSTERMANN M, CHANG R, and The Riyadh ICU Program Users Group. Correlation between the AKI classification and outcome[J]. Critical Care, 2008, 12(6): R144. doi: 10.1186/cc7123
[17]	THAKAR C V, CHRISTIANSON A, FREYBERG R, et al. Incidence and outcomes of acute kidney injury in intensive care units: A veterans administration study[J]. Critical Care Medicine, 2009, 37(9): 2552–2558. doi: 10.1097/CCM.0b013e3181a5906f
[18]	SAWHNEY S, MARKS A, FLUCK N, et al. Intermediate and long-term outcomes of survivors of acute kidney injury episodes: A large population-based cohort study[J]. American Journal of Kidney Diseases, 2017, 69(1): 18–28. doi: 10.1053/j.ajkd.2016.05.018
[19]	WANG J H, JOSLIN J, JENKINS R, et al. Outcomes of elderly patients with acute kidney injury on a renal high dependency unit[J]. International Journal of Clinical Practice, 2015, 69(10): 1209–1210. doi: 10.1111/ijcp.12683
[20]	WU V C, WU C H, HUANG T M, et al. Long-term risk of coronary events after AKI[J]. Journal of the American Society of Nephrology, 2014, 25(3): 595–605. doi: 10.1681/ASN.2013060610
[21]	VILLENEUVE P M, CLARK E G, SIKORA L, et al. Health-related quality-of-life among survivors of acute kidney injury in the intensive care unit: A systematic review[J]. Intensive Care Medicine, 2016, 42(2): 137–146. doi: 10.1007/s00134-015-4151-0
[22]	LIN Ke, HU Yonghua, and KONG Guilan. Predicting in-hospital mortality of patients with acute kidney injury in the ICU using random forest model[J]. International Journal of Medical Informatics, 2019, 125: 55–61. doi: 10.1016/j.ijmedinf.2019.02.002
[23]	PONCE D, DE PIETRO FRANCO ZORZENON C, DOS SANTOS N Y, et al. Early nephrology consultation can have an impact on outcome of acute kidney injury patients[J]. Nephrology Dialysis Transplantation, 2011, 26(10): 3202–3206. doi: 10.1093/ndt/gfr359
[24]	FLECHET M, FALINI S, BONETTI C, et al. Machine learning versus physicians’ prediction of acute kidney injury in critically ill adults: A prospective evaluation of the AKIpredictor[J]. Critical Care, 2019, 23(1): 282. doi: 10.1186/s13054-019-2563-x
[25]	ANDREU-PEREZ J, POON C C Y, MERRIFIELD R D, et al. Big data for health[J]. IEEE Journal of Biomedical and Health Informatics, 2015, 19(4): 1193–1208. doi: 10.1109/JBHI.2015.2450362
[26]	XIAO Cao, CHOI E, and SUN Jimeng. Opportunities and challenges in developing deep learning models using electronic health records data: A systematic review[J]. Journal of the American Medical Informatics Association, 2018, 25(10): 1419–1428. doi: 10.1093/jamia/ocy068
[27]	GONG Kaidi, LEE H K, YU Kaiye, et al. A prediction and interpretation framework of acute kidney injury in critical care[J]. Journal of Biomedical Informatics, 2021, 113: 103653. doi: 10.1016/j.jbi.2020.103653
[28]	TOMAŠEV N, GLOROT X, RAE J W, et al. A clinically applicable approach to continuous prediction of future acute kidney injury[J]. Nature, 2019, 572(7767): 116–119. doi: 10.1038/s41586-019-1390-1
[29]	FLECHET M, GÜIZA F, SCHETZ M, et al. AKIpredictor, an online prognostic calculator for acute kidney injury in adult critically ill patients: Development, validation and comparison to serum neutrophil gelatinase-associated lipocalin[J]. Intensive Care Medicine, 2017, 43(6): 764–773. doi: 10.1007/s00134-017-4678-3
[30]	MOHAMADLOU H, LYNN-PALEVSKY A, BARTON C, et al. Prediction of acute kidney injury with a machine learning algorithm using electronic health record data[J]. Canadian Journal of Kidney Health and Disease, 2018, 5: 1–9.
[31]	LEI V J, LUONG T, SHAN E, et al. Risk stratification for postoperative acute kidney injury in major noncardiac surgery using preoperative and intraoperative data[J]. JAMA Network Open, 2019, 2(12): e1916921. doi: 10.1001/jamanetworkopen.2019.16921
[32]	KOYNER J L, CAREY K A, EDELSON D P, et al. The development of a machine learning inpatient acute kidney injury prediction model[J]. Critical Care Medicine, 2018, 46(7): 1070–1077. doi: 10.1097/CCM.0000000000003123
[33]	CHURPEK M M, CAREY K A, EDELSON D P, et al. Internal and external validation of a machine learning risk score for acute kidney injury[J]. JAMA Network Open, 2020, 3(8): e2012892. doi: 10.1001/jamanetworkopen.2020.12892
[34]	SIMONOV M, UGWUOWO U, MOREIRA E, et al. A simple real-time model for predicting acute kidney injury in hospitalized patients in the US: A descriptive modeling study[J]. PLoS Medicine, 2019, 16(7): e1002861. doi: 10.1371/journal.pmed.1002861
[35]	XU Zhenxing, LUO Yuan, ADEKKANATTU P, et al. Stratified mortality prediction of patients with acute kidney injury in critical care[J]. Studies in Health Technology and Informatics, 2019, 264: 462–466.
[36]	ZIMMERMAN L P, REYFMAN P A, SMITH A D R, et al. Early prediction of acute kidney injury following ICU admission using a multivariate panel of physiological measurements[J]. BMC Medical Informatics and Decision Making, 2019, 19(S1): 16. doi: 10.1186/s12911-019-0733-z
[37]	RASHIDI H H, SEN S, PALMIERI T L, et al. Early recognition of burn- and trauma-related acute kidney injury: A pilot comparison of machine learning techniques[J]. Scientific Reports, 2020, 10(1): 205. doi: 10.1038/s41598-019-57083-6
[38]	JOHNSON A E W, POLLARD T J, SHEN Lu, et al. MIMIC-III, a freely accessible critical care database[J]. Scientific Data, 2016, 3: 160035. doi: 10.1038/sdata.2016.35
[39]	RALIB A, PICKERING J W, SHAW G M, et al. The urine output definition of acute kidney injury is too liberal[J]. Critical Care, 2013, 17(3): R112. doi: 10.1186/cc12784
[40]	RAZAVIAN N and SONTAG D. Temporal convolutional neural networks for diagnosis from lab tests[J]. arXiv preprint arXiv: 1511.07938, 2015.
[41]	HAN Hui, WANG Wenyuan, and MAO Binghuan. Borderline-SMOTE: A new over-sampling method in imbalanced data sets learning[C]. 2005 International Conference on Intelligent Computing, Hefei, China, 2005: 878–887.
[42]	CHAWLA N V, BOWYER K W, HALL L O, et al. SMOTE: Synthetic minority over-sampling technique[J]. Journal of Artificial Intelligence Research, 2002, 16: 321–357. doi: 10.1613/jair.953
[43]	CHEN Tianqi and GUESTRIN C. Xgboost: A scalable tree boosting system[C]. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, USA, 2016: 785–794.
[44]	NGUYEN H, BUI X N, BUI H B, et al. Developing an XGBoost model to predict blast-induced peak particle velocity in an open-pit mine: A case study[J]. Acta Geophysica, 2019, 67(2): 477–490. doi: 10.1007/s11600-019-00268-4
[45]	MA Xiaojun, SHA Jinglan, WANG Dehua, et al. Study on a prediction of P2P network loan default based on the machine learning LightGBM and XGboost algorithms according to different high dimensional data cleaning[J]. Electronic Commerce Research and Applications, 2018, 31: 24–39. doi: 10.1016/j.elerap.2018.08.002
[46]	MOUSA S R, BAKHIT P R, and ISHAK S. An extreme gradient boosting method for identifying the factors contributing to crash/near-crash events: A naturalistic driving study[J]. Canadian Journal of Civil Engineering, 2019, 46(8): 712–721. doi: 10.1139/cjce-2018-0117
[47]	SYARIF I, PRUGEL-BENNETT A, WILLS G. SVM parameter optimization using grid search and genetic algorithm to improve classification performance[J]. Telkomnika, 2016, 14(4): 1502–1509. doi: 10.12928/telkomnika.v14i4.3956
[48]	ARLOT S and LERASLE M. Choice of V for V-fold cross-validation in least-squares density estimation[J]. The Journal of Machine Learning Research, 2016, 17(1): 7256–7305.
[49]	HANLEY J A and MCNEIL B J. The meaning and use of the area under a receiver operating characteristic (ROC) curve[J]. Radiology, 1982, 143(1): 29–36. doi: 10.1148/radiology.143.1.7063747
[50]	FLACH P, HERNÁNDEZ-ORALLO J, and FERRI C. A coherent interpretation of AUC as a measure of aggregated classification performance[C]. Proceedings of the 28th International Conference on International Conference on Machine Learning, Bellevue, USA, 2011.
[51]	WILSON F P. Machine learning to predict acute kidney injury[J]. American Journal of Kidney Diseases, 2020, 75(6): 965–967. doi: 10.1053/j.ajkd.2019.08.010
[52]	ŠTRUMBELJ E and KONONENKO I. Explaining prediction models and individual predictions with feature contributions[J]. Knowledge and Information Systems, 2014, 41(3): 647–665. doi: 10.1007/s10115-013-0679-x
[53]	RIBEIRO M T, SINGH S, and GUESTRIN C. "Why should I trust you?": Explaining the predictions of any classifier[C]. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, USA, 2016: 1135–1144.
[54]	LUNDBERG S M, NAIR B, VAVILALA M S, et al. Explainable machine-learning predictions for the prevention of hypoxaemia during surgery[J]. Nature Biomedical Engineering, 2018, 2(10): 749–760. doi: 10.1038/s41551-018-0304-0
[55]	LUNDBERG S M, ERION G, CHEN H, et al. From local explanations to global understanding with explainable AI for trees[J]. Nature Machine Intelligence, 2020, 2(1): 56–67. doi: 10.1038/s42256-019-0138-9
[56]	PARSA A B, MOVAHEDI A, TAGHIPOUR H, et al. Toward safer highways, application of XGBoost and SHAP for real-time accident detection and feature analysis[J]. Accident Analysis & Prevention, 2020, 136: 105405.
[57]	LI Yikuan, YAO Liang, MAO Chengsheng, et al. Early prediction of acute kidney injury in critical care setting using clinical notes[C]. Proceedings of 2018 IEEE International Conference on Bioinformatics and Biomedicine, Madrid, Spain, 2018: 683–686.
[58]	BASILE D P, ANDERSON M D, and SUTTON T A. Pathophysiology of acute kidney injury[J]. Comprehensive Physiology, 2012, 2(2): 1303–1353.
[59]	SØVIK S, ISACHSEN M S, NORDHUUS K M, et al. Acute kidney injury in trauma patients admitted to the ICU: A systematic review and meta-analysis[J]. Intensive Care Medicine, 2019, 45(4): 407–419. doi: 10.1007/s00134-019-05535-y
[60]	OH H J, SHIN D H, LEE M J, et al. Urine output is associated with prognosis in patients with acute kidney injury requiring continuous renal replacement therapy[J]. Journal of Critical Care, 2013, 28(4): 379–388. doi: 10.1016/j.jcrc.2012.11.019
[61]	HUSSEIN R H, CALICE-SILVA V, EVANS R, et al. Diagnosis of acute kidney injury in children hospitalized in a sub-saharan african unit by saliva urea nitrogen dipstick test[J]. Blood Purification, 2020, 49(1/2): 185–196.
[62]	SHAWWA K, GHOSH E, LANIUS S, et al. Predicting acute kidney injury in critically ill patients using comorbid conditions utilizing machine learning[J]. Clinical Kidney Journal, 2021, 14(5): 1428–1435. doi: 10.1093/ckj/sfaa145
[63]	SUN Mengxin, BARON J, DIGHE A, et al. Early prediction of acute kidney injury in critical care setting using clinical notes and structured multivariate physiological measurements[J]. Studies in Health Technology and Informatics, 2019, 264: 368–372.
[64]	CHEN Daonan, DU Jiang, XIE Yun, et al. Relationship between early serum sodium and potassium levels and AKI severity and prognosis in oliguric AKI patients[J]. International Urology and Nephrology, 2021, 53(6): 1171–1187. doi: 10.1007/s11255-020-02724-3
[65]	HURSH B E, RONSLEY R, ISLAM N, et al. Acute kidney injury in children with type 1 diabetes hospitalized for diabetic ketoacidosis[J]. JAMA Pediatrics, 2017, 171(5): e170020. doi: 10.1001/jamapediatrics.2017.0020
[66]	KUMAR I E, VENKATASUBRAMANIAN S, SCHEIDEGGER C, et al. Problems with shapley-value-based explanations as feature importance measures[C]. Proceedings of the 37th International Conference on Machine Learning, 2020: 5491–5500.
[67]	BELLOMO R, KELLUM J A, and RONCO C. Acute kidney injury[J]. The Lancet, 2012, 380(9843): 756–766. doi: 10.1016/S0140-6736(11)61454-2