Multitask learning and benchmarking with clinical time series data

Abstract

Health care is one of the most exciting frontiers in data mining and machine learning. Successful adoption of electronic health records (EHRs) created an explosion in digital clinical data available for analysis, but progress in machine learning for healthcare research has been difficult to measure because of the absence of publicly available benchmark data sets. To address this problem, we propose four clinical prediction benchmarks using data derived from the publicly available Medical Information Mart for Intensive Care (MIMIC-III) database. These tasks cover a range of clinical problems including modeling risk of mortality, forecasting length of stay, detecting physiologic decline, and phenotype classification. We propose strong linear and neural baselines for all four tasks and evaluate the effect of deep supervision, multitask training and data-specific architectural modifications on the performance of neural models.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 78

1. 1.

   Introduction to the HCUP National Inpatient Sample (NIS) 2012. (Agency for Healthcare Research and Quality, 2014).

   Not in our library

2. 2.

   Henry, J., Pylypchuk, Y., Talisha Searcy, M. & Patel, V. Adoption of electronic health record systems among US non-federal acute care hospitals: 2008–2015. ONCData Brief35 (Office of the National Coordinator for Health Information Technology, Washington DC, USA, 2015).

   Not in our library

3. 3.

   Bates DW, Saria S, Ohno-Machado L, Shah A, Escobar G. Big data in health care: using analytics to identify and manage high-risk and high-cost patients. Health Affairs. 2014;33:1123-1131.

   Not in our library10.1377/hlthaff.2014.0041

4. 4.

   Zimmerman JE, Kramer AA, McNair DS, Malila FM. Acute physiology and chronic health evaluation (apache) iv: hospital mortality assessment for today’s critically ill patients. Crit. Care Med. 2006;34:1297-1310.

   Not in our library10.1097/01.ccm.0000215112.84523.f0

5. 5.

   Williams, B. et al. National Early Warning Score (NEWS): Standardising the assessment of acute-illness severity in the NHS. (London: The Royal College of Physicians, 2012).

   Not in our library

6. 6.

   Dahl D, et al. The high cost of low-acuity icu outliers. Journal of Healthcare Management. 2012;57:421-433.

   Not in our library10.1097/00115514-201211000-00009

7. 7.

   Saria S, Goldenberg A. Subtyping: What it is and its role in precision medicine. IEEE Intelligent Systems. 2015;30:70-75.

   Not in our library10.1109/mis.2015.60

8. 8.

   Iserson KV, Moskop JC. Triage in medicine, part i: concept, history, and types. Ann. Emerg. Med. 2007;49:275-281.

   Not in our library10.1016/j.annemergmed.2006.05.019

9. 9.

   Apgar V. A proposal for a new method of evaluation of the newborn. Curr. Res. Anesth. Analg. 1952;32:260-267.

   Not in our library

10. 10.

    Ferrucci D, Levas A, Bagchi S, Gondek D, Mueller ET. Watson: beyond jeopardy! Artificial Intelligence. 2013;199:93-105.

    Not in our library10.1016/j.artint.2012.06.009

11. 11.

    Silver D, et al. Mastering the game of go with deep neural networks and tree search. Nature. 2016;529:484-489.

    Not in our library10.1038/nature16961

12. 12.

    Caballero Barajas, K. L. & Akella, R. Dynamically modeling patient’s health state from electronic medical records: A time series approach. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 69–78 (ACM, Sydney, Australia, 2015).

    Not in our library

13. 13.

    Ghassemi, M. et al. A multivariate timeseries modeling approach to severity of illness assessment and forecasting in icu with sparse, heterogeneous clinical data. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, 446–453 (AAAI Press, Austin, Texas, 2015).

    Not in our library

14. 14.

    Luo, Y., Xin, Y., Joshi, R., Celi, L. & Szolovits, P. Predicting icu mortality risk by grouping temporal trends from a multivariate panel of physiologic measurements. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, 42–50 (AAAI Press, Phoenix, Arizona, 2016).

    Not in our library

15. 15.

    Lee J, Maslove DM. Customization of a severity of illness score using local electronic medical record data. Journal of intensive care medicine. 2017;32:38-47.

    Not in our library10.1177/0885066615585951

16. 16.

    Johnson, A., Pollard, T. & Mark, R. Reproducibility in critical care: a mortality prediction case study. In Proceedings of the 2nd Machine Learning for Healthcare Conference, vol. 68, 361–376 (PMLR, Boston, Massachusetts, 2017).

    Not in our library

17. 17.

    Quinn JA, Williams CK, McIntosh N. Factorial switching linear dynamical systems applied to physiological condition monitoring. IEEE Trans. Pattern Anal. Mach. Intell. 2009;31:1537-1551.

    Not in our library10.1109/tpami.2008.191

18. 18.

    Laxmisan A, et al. The multitasking clinician: Decision-making and cognitive demand during and after team handoffs in emergency care. Int. J. Med. Inform. 2007;76:801-811.

    Not in our library10.1016/j.ijmedinf.2006.09.019

19. 19.

    Horn SD, et al. The relationship between severity of illness and hospital length of stay and mortality. Med. Care. 1991;29:305-317.

    Not in our library10.1097/00005650-199104000-00001

20. 20.
    MIMIC-III, a freely accessible critical care database · 2016

    No notice
21. 21.

    Laboratory For Computational Physiology, M. I. T. The MIMIC-III clinical database. PhysioNet., 10.13026/C2XW26 (2015).

    Not in our library

22. 22.

    Harutyunyan H, et al. MIMIC-III benchmark repository. Zenodo. 2018.

    Not in our library10.5281/zenodo.1306527

23. 23.

    Caruana, R., Baluja, S. & Mitchell, T. Using the future to “sort out” the present: Rankprop and multitask learning for medical risk evaluation. In Advances in Neural Information Processing Systems8, 959–965 (MIT Press, Denver, Colorado, 1996).

    Not in our library

24. 24.

    Clermont G, Angus DC, DiRusso SM, Griffin M, Linde-Zwirble WT. Predicting hospital mortality for patients in the intensive care unit: a comparison of artificial neural networks with logistic fmultion models. Crit. Care Med. 2001;29:291-296.

    Not in our library10.1097/00003246-200102000-00012

25. 25.

    Celi LA, et al. A database-driven decision support system: customized mortality prediction. J. Pers. Med. 2012;2:138-148.

    Not in our library10.3390/jpm2040138

26. 26.
    Scalable and accurate deep learning with electronic health records · 2018

    No notice
27. 27.

    Grigsby J, Kooken R, Hershberger J. Simulated neural networks to predict outcomes, costs, and length of stay among orthopedic rehabilitation patients. Arch. Phys. Med. Rehabil. 1994;75:1077-1081.

    Not in our library10.1016/0003-9993(94)90081-7

28. 28.

    Mobley BA, Leasure R, Davidson L. Artificial neural network predictions of lengths of stay on a post-coronary care unit. Heart & Lung: The Journal of Acute and Critical Care. 1995;24:251-256.

    Not in our library10.1016/s0147-9563(05)80045-7

29. 29.

    Buchman TG, Kubos KL, Seidler AJ, Siegforth MJ. A comparison of statistical and connectionist models for the prediction of chronicity in a surgical intensive care unit. Crit. Care Med. 1994;22:750-762.

    Not in our library10.1097/00003246-199405000-00008

30. 30.

    Yousefi, S., Song, C., Nauata, N. & Cooper, L. Learning genomic representations to predict clinical outcomes in cancer. Preprint at, https://arxiv.org/abs/1609.08663 (2016).

    Not in our library

31. 31.

    Ranganath, R., Perotte, A., Elhadad, N. & Blei, D. Deep survival analysis. In Proceedings of the 1st Machine Learning for Healthcare Conference, vol. 56 (PMLR, Los Angeles, California, USA, 2016).

    Not in our library

32. 32.
    Computational Phenotype Discovery Using Unsupervised Feature Learning over Noisy, Sparse, and Irregular Clinical Data · 2013

    Corrected
33. 33.

    Che, Z., Kale, D., Li, W., Bahadori, M. T. & Liu, Y. Deep computational phenotyping. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 507–516 (ACM, Sydney, Australia, 2015).

    Not in our library

34. 34.

    Choi, E., Bahadori, M. T., Schuetz, A., Stewart, W. F. & Sun, J. Doctor AI: Predicting clinical events via recurrent neural networks. In Proceedings of the 1st Machine Learning for Healthcare Conference, vol. 56 (PMLR, Los Angeles, California, USA, 2016).

    Not in our library

35. 35.

    Razavian, N., Marcus, J. & Sontag, D. Multi-task prediction of disease onsets from longitudinal lab tests. In Proceedings of the 1st Machine Learning for Healthcare Conference, vol. 56 (PMLR, Los Angeles, California, USA, 2016).

    Not in our library

36. 36.

    Lipton, Z. C., Kale, D. C., Elkan, C. & Wetzel, R. Learning to diagnose with LSTM recurrent neural networks. In International Conference on Learning Representations (San Juan, Puerto Rico, 2016).

    Not in our library

37. 37.

    Ngufor, C., Upadhyaya, S., Murphree, D., Kor, D. & Pathak, J. Multi-task learning with selective cross-task transfer for predicting bleeding and other important patient outcomes. In IEEE International Conference on Data Science and Advanced Analytics (DSAA), 1–8 (IEEE, Paris, France, 2015).

    Not in our library

38. 38.

    Collobert, R. & Weston, J. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Machine learning, 160–167 (ACM, Helsinki, Finland, 2008).

    Not in our library

39. 39.

    Gupta, P., Malhotra, P., Vig, L. & Shroff, G. Using features from pre-trained timenet for clinical predictions. In Proceedings of the 3rd International Workshop on Knowledge Discovery in Healthcare Data at IJCAI-ECAI, 38–44 (Stockholm, Sweden, 2018).

    Not in our library

40. 40.

    Gupta, P., Malhotra, P., Vig, L. & Shroff, G. Transfer learning for clinical time series analysis using recurrent neural networks. In Machine Learning for Medicine and Healthcare Workshop at ACM KDD 2018 Conference (London, United Kingdom, 2018).

    Not in our library

Cited by papers we hold 7

• Bias in medical AI: Implications for clinical decision-making · 2024
  No notice
• A comparison of approaches to improve worst-case predictive model performance over patient subpopulations · 2022
  Not checked
• A systematic review of the prediction of hospital length of stay: Towards a unified framework · 2022
  No notice
• A New Hybrid Predictive Model to Predict the Early Mortality Risk in Intensive Care Units on a Highly Imbalanced Dataset · 2020
  Not checked
• Combining structured and unstructured data for predictive models: a deep learning approach · 2020
  No notice
• Deep learning and alternative learning strategies for retrospective real-world clinical data · 2019
  No notice
• Scalable and accurate deep learning with electronic health records · 2018
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.