RTI uses cookies to offer you the best experience online. By clicking “accept” on this website, you opt in and you agree to the use of cookies. If you would like to know more about how RTI uses cookies and how to manage them please view our Privacy Policy here. You can “opt out” or change your mind by visiting: http://optout.aboutads.info/. Click “accept” to agree.
development and validation of machine learning models using multimodal data across multiple health-care sites
Al-Garadi, M., Desai, R. J., Ngan, K., LeNoue-Newton, M., Reeves, R. M., Park, D., Hernández-Muñoz, J. J., Wang, S. V., Maro, J. C., Fuller, C. C., Kueiyu, J. L., Kuzucan, A., Coughlin, K., Pillai, H., McPheeters, M., Whitaker, J., Buckner, J. A., McLemore, M. F., Westerman, D. M., & Matheny, M. E. (2026). Enhancing cause of death prediction: development and validation of machine learning models using multimodal data across multiple health-care sites. JAMIA Open, 9(1), ooaf175. https://doi.org/10.1093/jamiaopen/ooaf175
OBJECTIVES: To develop and validate machine learning (ML) models that predict probable cause of death (CoD) using structured electronic health record (EHR) data, unstructured clinical notes, and publicly available sources.
MATERIALS AND METHODS: This multi-institutional retrospective study was conducted across Vanderbilt University Medical Center (VUMC) and Massachusetts General Brigham (MGB), including deceased patients with encounters between October 1, 2015, and January 1, 2021, and confirmed death records. The cohort included 13 708 patients from VUMC and 34 839 from MGB.The primary outcome was underlying CoD categorized into the top 15 National Center for Health Statistics rankable causes, with others grouped as "Other." Performance was assessed using weighted area under the receiver operating characteristic curve (AUC) and F-measure.
RESULTS: The XGBoost model using structured EHR data alone achieved weighted AUCs of 0.86 (95% CI, 0.84-0.88) at VUMC and 0.80 (95% CI, 0.79-0.80) at MGB. Adding unstructured notes improved performance, with weighted AUCs of 0.90 (95% CI, 0.88-0.93) at VUMC and 0.92 (95% CI, 0.91-0.92) at MGB. Adding publicly available data did not further improve performance. Cross-institutional validation revealed significant performance degradation.
DISCUSSION: Models integrating structured and unstructured EHR data show strong within-institution performance but limited generalizability across healthcare systems, highlighting challenges related to institutional data heterogeneity.
CONCLUSIONS: Machine learning models combining structured and unstructured EHR data accurately predict CoD within institutions but perform poorly across sites. Health-care institutions may benefit from adopting robust processes for locally tailored models, and future research should focus on enhancing model generalizability while addressing unique institutional data environments.
RTI shares its evidence-based research - through peer-reviewed publications and media - to ensure that it is accessible for others to build on, in line with our mission and scientific standards.
