Translating radiology reports into plain language using ChatGPT and GPT-4 with prompt learning: results, limitations, and potential

Abstract

The large language model called ChatGPT has drawn extensively attention because of its human-like expression and reasoning abilities. In this study, we investigate the feasibility of using ChatGPT in experiments on translating radiology reports into plain language for patients and healthcare providers so that they are educated for improved healthcare. Radiology reports from 62 low-dose chest computed tomography lung cancer screening scans and 76 brain magnetic resonance imaging metastases screening scans were collected in the first half of February for this study. According to the evaluation by radiologists, ChatGPT can successfully translate radiology reports into plain language with an average score of 4.27 in the five-point system with 0.08 places of information missing and 0.07 places of misinformation. In terms of the suggestions provided by ChatGPT, they are generally relevant such as keeping following-up with doctors and closely monitoring any symptoms, and for about 37% of 138 cases in total ChatGPT offers specific suggestions based on findings in the report. ChatGPT also presents some randomness in its responses with occasionally over-simplified or neglected information, which can be mitigated using a more detailed prompt. Furthermore, ChatGPT results are compared with a newly released large model GPT-4, showing that GPT-4 can significantly improve the quality of translated reports. Our results show that it is feasible to utilize large language models in clinical education, and further efforts are needed to address limitations and maximize their potential.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 17

1. 1.

   ChatGPT sets record for fastest-growing user base-analyst note. https://www.marketscreener.com/news/latest/ChatGPT-sets-record-for-fastestgrowing-user-base-analyst-note--42873811/. Accessed 20 Feb 2023

   Not in our library

2. 2.

   ChatGPT reaches 100 million users two months after launch. https://www.theguardian.com/technology/2023/feb/02/chatgpt-100-million-usersopen-ai-fastest-growing-app. Accessed 20 Feb 2023

   Not in our library

3. 3.

   Devlin J, Chang MW, Lee K, Toutanova K (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (Long and Short Papers), Association for Computational Linguistics, Minneapolis, 2-7 June 2019

   Not in our library

4. 4.

   Yang ZL, Dai ZH, Yang YM, Carbonell J, Salakhutdinov R, Le QV (2019) XLNet: Generalized autoregressive pretraining for language understanding. In: Proceedings of the 33rd international conference on neural information processing systems, Curran Associates Inc., Vancouver, 8 December 2019

   Not in our library

5. 5.

   Radford A, Narasimhan K, Salimans T, Sutskever I (2018) Improving language understanding by generative pre-training

   Not in our library

6. 6.

   Ouyang L, Wu J, Jiang X, Almeida D, Wainwright CL, Mishkin P et al. (2022) Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155

   Not in our library

7. 7.

   Wang S, Scells H, Koopman B, Zuccon G (2023) Can ChatGPT write a good Boolean query for systematic review literature search? arXiv preprint arXiv:2302.03495. 10.1145/3539813.3545143

   Not in our library

8. 8.
   Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models · 2023

   No notice
9. 9.

   Liebrenz M, Schleifer R, Buadze A, Bhugra D, Smith A (2023) Generating scholarly content with ChatGPT: ethical challenges for medical publishing. Lancet Digit Health 5(3):E105–E106. 10.1016/S2589-7500(23)00019-5

   Not in our library10.1016/s2589-7500(23)00019-5

10. 10.

    Patel SB, Lam K (2023) ChatGPT: the future of discharge summaries? Lancet Digit Health 5(3):E107–E108. 10.1016/S2589-7500(23)00021-3

    Not in our library10.1016/s2589-7500(23)00021-3

11. 11.

    Biswas S (2023) ChatGPT and the future of medical writing. Radiology 307(2):e223312. 10.1148/radiol.223312

    Not in our library10.1148/radiol.223312

12. 12.
    ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports · 2023

    No notice
13. 13.

    Rao A, Kim J, Kamineni M, Pang M, Lie W, Succi MD (2023) Evaluating ChatGPT as an adjunct for radiologic decision-making. medRxiv, 2023-02. 10.1101/2023.02.02.23285399

    Not in our library10.1016/j.jacr.2023.05.003

14. 14.

    Sarraju A, Bruemmer D, Van Iterson E, Cho L, Rodriguez F, Laffin L (2023) Appropriateness of cardiovascular disease prevention recommendations obtained from a popular online chat-based artificial intelligence model. JAMA 329(10):842–844. 10.1001/jama.2023.1044

    Not in our library10.1001/jama.2023.1044

15. 15.

    PromptPerfect: elevate your prompts to perfection. https://promptperfect.jina.ai/. Accessed 20 Feb 2023

    Not in our library

16. 16.

    OpenAI: GPT-4 technique report (2023) https://cdn.openai.com/papers/gpt-4.pdf. Accessed 14 Mar 2023

    Not in our library

17. 17.

    GPT-4. https://openai.com/research/gpt-4. Accessed 14 Mar 2023

    Not in our library

Cited by papers we hold 3

• Applications and Concerns of ChatGPT and Other Conversational Large Language Models in Health Care: Systematic Review · 2024
  No notice
• Assessing the research landscape and clinical utility of large language models: a scoping review · 2024
  No notice
• Large language models in patient education: a scoping review of applications in medicine · 2024
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.