No retraction, correction or concern found when last checked on Oct 21, 2026.

Large Language Models for Chatbot Health Advice Studies

Bright Huo, Amy Boyle, Nana Marfo and 10 more

JAMA Network Open · 2025 · Systematic review

doi:10.1001/jamanetworkopen.2024.57879

Abstract

Question. What do studies report when evaluating the performance of large language models (LLMs) providing health advice?

Findings. In this systematic review of 137 articles, 99.3% of the studies assessed closed-source models and did not provide enough information to identify the LLM. Most (64.5%) studies used subjective means as the ground truth to define the successful performance of the LLM, while less than a third addressed the ethical, regulatory, and patient safety implications of clinically integrating LLMs.

Meaning. The findings of this study suggest that the extent of reporting varies considerably among studies evaluating the clinical accuracy of LLMs providing health advice.

Importance. There is much interest in the clinical integration of large language models (LLMs) in health care. Many studies have assessed the ability of LLMs to provide health advice, but the quality of their reporting is uncertain.

Objective. To perform a systematic review to examine the reporting variability among peer-reviewed studies evaluating the performance of generative artificial intelligence (AI)–driven chatbots for summarizing evidence and providing health advice to inform the development of the Chatbot Assessment Reporting Tool (CHART).

Evidence Review. A search of MEDLINE via Ovid, Embase via Elsevier, and Web of Science from inception to October 27, 2023, was conducted with the help of a health sciences librarian to yield 7752 articles. Two reviewers screened articles by title and abstract followed by full-text review to identify primary studies evaluating the clinical accuracy of generative AI-driven chatbots in providing health advice (chatbot health advice studies). Two reviewers then performed data extraction for 137 eligible studies.

Findings. A total of 137 studies were included. Studies examined topics in surgery (55 [40.1%]), medicine (51 [37.2%]), and primary care (13 [9.5%]). Many studies focused on treatment (91 [66.4%]), diagnosis (60 [43.8%]), or disease prevention (29 [21.2%]). Most studies (136 [99.3%]) evaluated inaccessible, closed-source LLMs and did not provide enough information to identify the version of the LLM under evaluation. All studies lacked a sufficient description of LLM characteristics, including temperature, token length, fine-tuning availability, layers, and other details. Most studies (136 [99.3%]) did not describe a prompt engineering phase in their study. The date of LLM querying was reported in 54 (39.4%) studies. Most studies (89 [65.0%]) used subjective means to define the successful performance of the chatbot, while less than one-third addressed the ethical, regulatory, and patient safety implications of the clinical integration of LLMs.

Conclusions and Relevance. In this systematic review of 137 chatbot health advice studies, the reporting quality was heterogeneous and may inform the development of the CHART reporting standards. Ethical, regulatory, and patient safety considerations are crucial as interest grows in the clinical integration of LLMs.

Full text

From PubMed Central, U.S. National Library of Medicine, as current on Oct 21, 2026. Paragraph numbers are ours, so an answer can point at one.

Introduction

1Large language models (LLMs) have expanded the potential for integrating artificial intelligence (AI) in medicine. LLMs are AI systems that are pretrained through a variety of word prediction tasks across enormous volumes of text taken from datasets, books, articles, and internet sources., With additional fine-tuning involving varying amounts of human feedback, LLMs acquire natural language processing (NLP) capabilities and can generate appropriate text outputs using layperson terminology in response to free-text prompts. Many patients and physicians use the internet to access health information, and generative AI-driven chatbots present a convenient new way to access answers to medical questions. Early data suggest that generative AI-driven chatbots can even respond to patients with expressions of empathy.

2Patients have reported interest in seeking medical advice from chatbots, leading to a rapid expansion in literature assessing the ability of chatbots to offer sound health advice to physicians and patients: chatbot health advice studies (CHAS). This research has investigated the ability of chatbots to summarize evidence and provide health advice about screening, diagnosis, treatment, and disease prevention. However, chatbots are typically not designed for medical application and currently lack regulation for their use in medicine. Furthermore, generative AI-driven chatbots present unique risks of hallucinating (ie, confident but incorrect) false results and propagating misleading information and advice. Thus, it is unsurprising that CHAS report varying levels of chatbot performance, raising concerns about their limitations and risks.,,

3CHAS also present their own unique problems. Currently, these studies are inconsistent in their design, analysis, and reporting.,, With a lack of standardized reporting, the assessment of the quality of CHAS and the interpretation of their findings remain challenging. These studies have the potential to provide useful assessment of LLM performance and insight into the patient safety implications of chatbot advice, yet the apparent limitations of the studies themselves necessitate clarifying and standardizing current approaches. In this systematic review, we investigated chatbot selection, query approach, and performance evaluation. The information gathered from this systematic review may contribute to the development of a novel chatbot assessment reporting tool (CHART) to standardize reporting methods in chatbot assessment research.

Methods

4This systematic review adheres to the reporting standards of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting guideline. In this study, we refer to LLMs, generative AI-driven chatbots, and chatbots synonymously. Our protocol was prospectively registered on Open Sciences Framework.

Search Strategy and Selection Criteria

5We set out to characterize the approach of studies that assess the performance of generative AI-driven chatbots for summarizing evidence and providing health advice. To capture all relevant studies, an academic health sciences librarian (S.S.) with expertise in systematic reviews aided the study team in designing a comprehensive search strategy. The detailed literature search syntax can be found in the eAppendix in Supplement 1. Records were retrieved from MEDLINE via Ovid, Embase via Elsevier, and Web of Science on October 27, 2023. The study team completed 2 rounds of screening, first by title and abstract and then by full text. In all cases, 2 independent researchers (W.T. and J.P.S.) conducted screening, with a third independent researcher casting a deciding vote to resolve cases of disagreement. We excluded irrelevant studies and nonprimary studies. The Box illustrates our eligibility criteria.

Inclusion Criteria

6Studies assessing the performance of chatbots for summarizing clinical evidence and providing health advice.

Exclusion Criteria

7Systematic reviews and literature/narrative reviews.

8Editorials, commentaries, opinions, studies published as correspondence articles, and studies of any type outlining earlier or potential future applications of chatbots.

9Cross-sectional studies assessing opinions of future work for chatbot research.

10Abstracts.

Inclusion and Exclusion Criteria

11Data extraction was conducted with Covidence and aimed to capture basic study details (clinical specialty, topic, intervention, and comparator) in addition to the following: (1) how LLMs are selected for use in CHAS, (2) clinical purposes for which LLMs have been evaluated, (3) methods applied, (4) approach to performance evaluation (ie, summarizing evidence alone or summarizing evidence to inform recommendation), (5) reporting practices used, and (6) the presence of ethical considerations of the clinical application of generative AI-driven chatbots.

Statistical Analysis

12The study team performed data analysis using descriptive statistics. We summarized findings pertaining to the methodologic approach of CHAS using frequencies as counts and percentages including the types of LLMs evaluated, the query strategy, and their performance evaluation. A narrative synthesis with no meta-analysis was planned, due to anticipated heterogeneity in study subjects and designs. We describe the full details related to the literature search and data extraction variables in our study protocol. The quality of the evidence was not evaluated due to the lack of high-quality, validated risk-of-bias tools available for CHAS.

Study Characteristics

13The study team reviewed 7752 articles that yielded 137 eligible CHAS (Figure). Table 1 lists all evaluated articles,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, including their relevant specialty and study aims.

14Overall, of the 137 articles reviewed, 51 (37.2%) studies queried LLMs for advice relevant for medical topics, 55 (40.1%) on surgical topics, 13 (9.5%) on primary care topics, 9 (6.6%) on radiology topics, and 7 (5.1%) on psychiatry topics (Table 1). Articles most commonly examined advice regarding treatment (91 [66.4%]), diagnosis (60 [43.8%]), and differential diagnosis (23 [16.8%]). Overall, 16 studies (11.7%) explained why the specific LLMs were selected to be studied. Almost all studies (135 [99.3%]) assessed the ability of the OpenAI Chat Generative Pretrained Transformer (ChatGPT) to provide health advice, and 136 of the LLMs studied (99.3%) were inaccessible (proprietary/closed-source) models. A single study (0.7%) fully described the version of the LLM under evaluation, and no studies described the characteristics of the LLM in comprehensive detail including temperature (a parameter to control the randomness of LLM output), token length, fine-tuning availability, penalties, add-on availability, language, and layers related to the LLM being studied. More than one-quarter (36 [26.3%]) of the studies described none of the characteristics of the LLM being examined (Table 2). Sources used to develop prompts included expert opinion (43 [31.4%]), professional society websites (28 [20.4%]), and clinical practice guidelines (24 [17.5%]).

Performance Evaluation

15Table 2 presents details of our study performance evaluation. More than one-quarter (37 [27.0%]) of the studies did not explain the source of their prompts or describe at least one aspect of their query strategy (40 [29.2%]). Few studies noted the date (54 [39.4%]) and location (5 [3.6%]) of their query. Study performance was suboptimal in noting the number of chat windows (32 [23.4%]) and the number of queries (48 [35.0%]). Studies performed poorly (20 [14.6%]) in using standardized prompts to evaluate their LLMs. Overall, 136 articles (99.3%) did not describe a prompt testing/engineering phase, while 93 (67.9%) shared the full transcript of their prompts and 65 (47.4%) included the full transcript of chatbot responses.

16With respect to performance evaluation of chatbots, few studies described a standardized evaluation process (18 [13.1%]) or blinding (16 [11.7%]), and many studies did not describe a structured approach to performance evaluation (39 [28.5%]). Rather than using clinical practice guideline recommendations as in 21 studies (15.3%), most investigators evaluated LLMs based on their opinion without a reference standard (89 [65.0%]). Less than one-third of the studies addressed the ethical (45 [32.8%]) and patient safety (44 [32.1%]) implications of the clinical integration of LLMs, while 22 (16.1%) addressed the lack of regulation of LLMs (Table 2).

Discussion

17This systematic review identified large heterogeneity in the methods of CHAS and a need for improved reporting standards. Many studies failed to adequately report the methods of chatbot performance evaluation. Moreover, most studies that described an approach to evaluation relied on anecdotal opinion rather than systematic summaries of available evidence or official guidelines. No study described the characteristics of the LLM under evaluation in sufficient detail to reproduce experiments, and many failed to describe any aspect of the LLM characteristics. Most reports did not include reasons for chatbot selection, and they seldom described a prompt engineering phase. Authors infrequently shared prompt and chatbot response transcripts. Less than half mentioned ethical issues, patient safety, or regulation of LLMs.

Reporting of LLM Characteristics and Study Method

18Generative AI-driven chatbots constitute the intervention in these studies, yet there is a lack of critical information about their characteristics. As these publications are largely prepared by clinicians, study teams may not have access to expertise in machine learning. Without full knowledge of the intervention being applied, it is difficult to interpret LLM performance.,,

19LLMs are trained to learn associations between words in large training text datasets, transform algorithm inputs to outputs, and mimic human language using NLP. This process enables LLMs to predict the next word in a sequence, and preset model characteristics, such as temperature, token length, fine-tuning availability, penalties, add-on availability, language, and layers, further impact their functionality.,, Just as sufficient detail about medical or surgical interventions must be reported in traditional studies to reproduce experiments and guide critical appraisal, LLM characteristics are similarly vital. Physicians may not be AI experts, but they should understand the principles and limitations of their LLM interventions and may benefit from involving expert LLM researchers.

20Most included studies did not report a prompt engineering phase. However, the generated output depends greatly on the initial input. LLMs have in-context learning capabilities that allow them to adapt to the given prompt, so prompts can be intentionally designed to support performance on further tasks, such as patient communication, administrative tasks, risk assessments, or clinical decision-making., Prompt engineering improves chatbot performance by tailoring LLM output to be focused and helpful for a specific task., These crucial concepts underscore the importance of including experts in machine learning, computer science, and engineering. In the push for the clinical integration of LLMs in medicine, multidisciplinary stakeholder collaboration between physicians and expert LLM and NLP researchers is essential.

21Additionally, the existing literature base emphasizes a structured approach to performance evaluation. However, most studies did not provide transcripts of chatbot prompts and responses. Inclusion of obtained responses and the details of subsequent analysis is important to maximize transparency, internal validity, and the external appraisal of chatbot assessment study findings. Authors frequently apply subjective measures of chatbot performance, such as expert opinion, in CHAS, limiting the generalizability of their methods and conclusions. A shift toward objective measures, preferably basing accuracy and relevance with respect to high-quality evidence, such as clinical practice guidelines, is necessary. Many studies describe this as defining a ground truth or evaluation dataset, and it is an essential step in LLM performance evaluation., Currently, few studies use clinical practice guidelines to define successful performance, and several chatbots have experienced inconsistent accuracy by these standards., This is especially important when considering the implications of using these LLMs on patient safety.

Ethics, Patient Safety, and LLM Regulation

22Artificial intelligence–linked medical devices are considered software as a medical device and are regulated under federal departments such as Health Canada and the US Food and Drug Administration. However, LLM applications can be excluded from this regulation if they were not designed for medical purposes or intended to replace clinical judgment.,, Although no devices that use generative AI or LLMs have been approved by the Food and Drug Administration, LLMs are being used in medical applications through this gap in regulation. Some LLMs have already been integrated into electronic medical record systems, such as the Nuance DAX Copilot, a generative AI-program embedded in an electronic health record software called Epic, which records patient interactions and automatically produces clinical documentation., To safely implement generative AI-driven chatbots in practice, there is a need for updated regulation that addresses the differences between emerging generative AI and previous forms of AI. While other regulated AI medical technologies, such as radiology image analysis or diagnosis, are trained on domain-specific medical data for targeted uses, LLMs have broader complexity, applicability, and an ability to adapt in real time, which complicates their regulation.,,

23Additionally, the specific issues of algorithmic bias, hallucinations, and risks of privacy breaches must be addressed by regulatory standards,, as these concepts have ethical implications in patient care. Several examples of bias have been reported among LLMs. These have included the propagation of biases based on race and ethnicity, gender, educational level, social class, sexuality, and age.,, Many of these biases stem from the data used to train LLMs. Additionally, LLMs have been known to produce hallucinations., Large language model performance is largely influenced by the content used for pretraining and fine-tuning, yet LLMs are generally not trained exclusively with high-quality medical literature or real-world clinical data. While clinical practice is informed by evidence from peer-reviewed research, LLMs replicate their input data and thus may propagate online health misinformation and prejudices,, compromising patient safety. Moreover, the lack of transparency surrounding the development and external validation of closed-source or proprietary LLMs presents additional challenges to their evaluation for health care purposes., Concerns have also been raised related to data privacy of patient health information among proprietary LLMs., There is a potential for misuse of patient data, medicolegal implications, and loss of physician-patient trust.,

24Taken together, these factors raise major issues as we move from the technological development of LLMs to their clinical validation for use in health care settings. The World Health Organization released guidance on ethics and governance of AI for health, which overlaps substantially with LLMs, which are unimodal models. This document outlines key messages for stakeholders involved in the health care integration of generative AI. For instance, the World Health Organization calls for developers to commit to ethical principles of inclusiveness, use higher-quality data during model development, adhere to data protection laws, be transparent about model training data, and involve key stakeholders in the design and development of LLMs, including health care professionals, patients, laypersons, and vulnerable persons. Moreover, developers may focus on the explainability of their LLMs, a concept that aims to elucidate how models arrive at outputs given a specific input. In addition, governments should develop clear regulatory policies and legislation to capture many of the concepts discussed herein, such as target product profiles, design and development standards, audits, disclosure requirements, data protection, and public infrastructure for the development of LLMs. Regulators may further look toward incentivizing researchers toward the development of standardized metrics for the clinical validation of health care applications of generative AI models such as LLMs.

Limitations

25This systematic review has limitations. First, the reporting used in CHAS may change beyond the time of this review due to the dynamic nature of this field. By engineering a comprehensive search strategy, we aimed to capture an accurate representation of the literature. Second, much of the literature evaluated in this review used hypothetical patient cases. CHAS should focus on patient-centered, prospective research using LLMs in the clinical setting. Multidisciplinary collaboration between clinicians and LLM/NLP experts may help facilitate this, while also improving the understanding and reporting of LLM characteristics in these studies. In addition, there is a paucity of literature that comments in detail on the ethical and regulatory considerations of the clinical integration of LLMs that impact patient safety. Clinicians, patients, and regulators should be involved in collective efforts to agree and enforce appropriate deployment strategies for LLM technology in medicine.

26Standardized frameworks are under development to establish detailed reporting standards to help authors design and report these studies. The CHART reporting tool may guide transparent reporting of CHAS and promote the publication of CHAS with higher methodologic rigor. Future research should address the lack of quality appraisal tools available to evaluate CHAS. Moreover, standardized metrics to validate the health care application of generative AI models are needed to facilitate their evaluation for adoption in clinical pathways. Interdisciplinary work among hospital managers, policymakers, regulatory bodies, and physicians of varying specialty backgrounds should be conducted to respond to the rapid advancement in LLM technology and the push for clinical integration by producing regulatory frameworks to support the safe clinical integration of LLMs.

Conclusions

27In this systematic review of 137 CHAS, we noted that detailed and transparent reporting of key aspects often was not included in the studies, such as LLM characteristics, prompt engineering, query strategy, and performance evaluation. An emphasis should be placed on high-quality methods to justify the deployment of novel applications of LLMs and related technologies in clinical practice.

References 193

1.
Large language models encode clinical knowledge · 2023
Corrected
2.
Thirunavukarasu AJ, Ting DSJ, Elangovan K, Gutierrez L, Tan TF, Ting DSW. Large language models in medicine. Nat Med. 2023;29(8):1930-1940
Not in our library10.1038/s41591-023-02448-8
3.
Khurana D, Koli A, Khatter K, Singh S. Natural language processing: state of the art, current trends and challenges. Multimed Tools Appl. 2023;82(3):3713-3744
Not in our library10.1007/s11042-022-13428-4
4.
Battineni G, Baldoni S, Chintalapudi N,. Factors affecting the quality and reliability of online health information. Digit Health. 2020;6:2055207620948996
Not in our library10.1177/2055207620948996
5.
Shen SA, Perez-Heydrich CA, Xie DX, Nellis JC. ChatGPT vs. web search for patient questions: what does ChatGPT do better? Eur Arch Otorhinolaryngol. 2024;281(6):3219-3225
Not in our library10.1007/s00405-024-08524-0
6.
Ayers JW, Poliak A, Dredze M,. Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum. JAMA Intern Med. 2023;183(6):589-596
Not in our library10.1001/jamainternmed.2023.1838
7.
User Intentions to Use ChatGPT for Self-Diagnosis and Health-Related Purposes: Cross-sectional Survey Study · 2023
No notice
8.
Temsah MH, Altamimi I, Jamal A, Alhasan K, Al-Eyadhy A. ChatGPT surpasses 1000 publications on PubMed: envisioning the road ahead. Cureus. 2023;15(9):e44769
Not in our library10.7759/cureus.44769
9.
Lee P, Bubeck S, Petro J. Benefits, limits, and risks of GPT-4 as an AI chatbot for medicine. N Engl J Med. 2023;388(13):1233-1239
Not in our library10.1056/nejmsr2214184
10.
Li J, Dada A, Puladi B, Kleesiek J, Egger J. ChatGPT in healthcare: a taxonomy and systematic review. Comput Methods Programs Biomed. 2024;245:108013
Not in our library10.1016/j.cmpb.2024.108013
11.
The imperative for regulatory oversight of large language models (or generative AI) in healthcare · 2023
No notice
12.
ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns · 2023
No notice
13.
Davis R, Eppler M, Ayo-Ajibola O,. Evaluating the effectiveness of artificial intelligence–powered large language models application in disseminating appropriate and readable health information in urology. J Urol. 2023;210(4):688-694
Not in our library10.1097/ju.0000000000003615
14.
How Does ChatGPT Perform on the United States Medical Licensing Examination (USMLE)? The Implications of Large Language Models for Medical Education and Knowledge Assessment · 2023
No notice
15.
Williams SC, Starup-Hansen J, Funnell JP,. Can ChatGPT outperform a neurosurgical trainee? a prospective comparative study. Br J Neurosurg. 2024:1-10
Not in our library10.1080/02688697.2024.2308222
16.
Ye C, Zweck E, Ma Z, Smith J, Katz S. Doctor versus artificial intelligence: patient and physician evaluation of large language model responses to rheumatology patient questions in a cross-sectional study. Arthritis Rheumatol. 2024;76(3):479-484
Not in our library10.1002/art.42737
17.
Guidance for Developers of Health Research Reporting Guidelines · 2010
No notice
18.
Huo B, Cacciamani GE, Collins GS, McKechnie T, Lee Y, Guyatt G. Reporting standards for the use of large language model-linked chatbots for health advice. Nat Med. 2023;29(12):2988
Not in our library10.1038/s41591-023-02656-2
19.
The PRISMA 2020 statement: an updated guideline for reporting systematic reviews · 2021
No notice
20.
Registries OSF. Protocol for a scoping review of chatbot assessment studies: guidance for the CHART tool. February 25, 2024. Accessed December 9, 2024. https://osf.io/cxsk3
Not in our library
21.
Al-Ashwal FY, Zawiah M, Gharaibeh L, Abu-Farha R, Bitar AN. Evaluating the sensitivity, specificity, and accuracy of ChatGPT-3.5, ChatGPT-4, Bing AI, and Bard against conventional drug-drug interactions clinical tools. Drug Healthc Patient Saf. 2023;15:137-147
Not in our library10.2147/dhps.s425858
22.
Alessandri-Bonetti M, Giorgino R, Naegeli M, Liu HY, Egro FM. Assessing the soft tissue infection expertise of ChatGPT and Bard compared to IDSA recommendations. Ann Biomed Eng. 2024;52(6):1551-1553
Not in our library10.1007/s10439-023-03372-1
23.
Ali MJ. ChatGPT and lacrimal drainage disorders: performance and scope of improvement. Ophthalmic Plast Reconstr Surg. 2023;39(3):221-225
Not in our library10.1097/iop.0000000000002418
24.
Ali H, Patel P, Obaitan I,. Evaluating the performance of ChatGPT in responding to questions about endoscopic procedures for patients. iGIE. 2023;2(4):553-559. https://www.igiejournal.org/article/S2949-7086(23)00120-6/fulltext
Not in our library10.1016/j.igie.2023.10.001
25.
Altamimi I, Altamimi A, Alhumimidi AS, Altamimi A, Temsah MH. Snakebite advice and counseling from artificial intelligence: an acute venomous snakebite consultation with ChatGPT. Cureus. 2023;15(6):e40351
Not in our library10.7759/cureus.40351
26.
Athavale A, Baier J, Ross E, Fukaya E. The potential of chatbots in chronic venous disease patient management. JVS Vasc Insights. 2023;1:100019
Not in our library10.1016/j.jvsvi.2023.100019
27.
Ayers JW, Zhu Z, Poliak A,. Evaluating artificial intelligence responses to public health questions. JAMA Netw Open. 2023;6(6):e2317517
Not in our library10.1001/jamanetworkopen.2023.17517
28.
Ayoub M, Ballout AA, Zayek RA, Ayoub NF. Mind + machine: ChatGPT as a basic clinical decisions support tool. Cureus. 2023;15(8):e43690
Not in our library10.7759/cureus.43690
29.
Ayoub NF, Lee YJ, Grimm D, Balakrishnan K. Comparison between ChatGPT and Google search as sources of postoperative patient instructions. JAMA Otolaryngol Head Neck Surg. 2023;149(6):556-558
Not in our library10.1001/jamaoto.2023.0704
30.
Balel Y. Can ChatGPT be used in oral and maxillofacial surgery? J Stomatol Oral Maxillofac Surg. 2023;124(5):101471
Not in our library10.1016/j.jormas.2023.101471
31.
Bellinger JR, De La Chapa JS, Kwak MW, Ramos GA, Morrison D, Kesser BW. BPPV information on Google versus AI (ChatGPT). Otolaryngol Head Neck Surg. 2024;170(6):1504-1511
Not in our library10.1002/ohn.506
32.
Benirschke RC, Wodskow J, Prasai K, Freeman A, Lee JM, Groth J. Assessment of a large language model’s utility in helping pathology professionals answer general knowledge pathology questions. Am J Clin Pathol. 2024;161(1):42-48
Not in our library10.1093/ajcp/aqad106
33.
Comparison of Ophthalmologist and Large Language Model Chatbot Responses to Online Patient Eye Care Questions · 2023
No notice
34.
Biswas S, Logan NS, Davies LN, Sheppard AL, Wolffsohn JS. Assessing the utility of ChatGPT as an artificial intelligence-based large language model for information to answer questions on myopia. Ophthalmic Physiol Opt. 2023;43(6):1562-1570
Not in our library10.1111/opo.13207
35.
Caglar U, Yildiz O, Meric A,. Evaluating the performance of ChatGPT in answering questions related to pediatric urology. J Pediatr Urol. 2024;20(1):26.e1-26.e5
Not in our library10.1016/j.jpurol.2023.08.003
36.
Cakir H, Caglar U, Yildiz O, Meric A, Ayranci A, Ozgor F. Evaluating the performance of ChatGPT in answering questions related to urolithiasis. Int Urol Nephrol. 2024;56(1):17-21
Not in our library10.1007/s11255-023-03773-0
37.
Campbell DJ, Estephan LE, Mastrolonardo EV, Amin DR, Huntley CT, Boon MS. Evaluating ChatGPT responses on obstructive sleep apnea for patient education. J Clin Sleep Med. 2023;19(12):1989-1995
Not in our library10.5664/jcsm.10728
38.
Caruccio L, Cirillo S, Polese G, Solimando G, Sundaramurthy S, Tortora G. Can ChatGPT provide intelligent diagnoses? a comparative study between predictive models and ChatGPT to define a new medical diagnostic bot. Expert Syst Appl. 2024;235(7):121186
Not in our library10.1016/j.eswa.2023.121186
39.
Chee J, Kwa ED, Goh X. “Vertigo, likely peripheral”: the dizzying rise of ChatGPT. Eur Arch Otorhinolaryngol. 2023;280(10):4687-4689
Not in our library10.1007/s00405-023-08135-1
40.
Use of Artificial Intelligence Chatbots for Cancer Treatment Information · 2023
No notice
41.
Chervenak J, Lieman H, Blanco-Breindel M, Jindal S. The promise and peril of using a large language model to obtain clinical information: ChatGPT performs strongly as a fertility counseling tool with limitations. Fertil Steril. 2023;120(3, pt 2):575-583
Not in our library10.1016/j.fertnstert.2023.05.151
42.
Chiesa-Estomba CM, Lechien JR, Vaira LA,. Exploring the potential of Chat-GPT as a supportive tool for sialendoscopy clinical decision making and patient information support. Eur Arch Otorhinolaryngol. 2024;281(4):2081-2086
Not in our library10.1007/s00405-023-08104-8
43.
Chowdhury M, Lim E, Higham A,. can large language models safely address patient questions following cataract surgery? Invest Ophthalmol Vis Sci. 2023;64(8):1214
Not in our library10.18653/v1/2023.clinicalnlp-1.17
44.
Cocci A, Pezzoli M, Lo Re M,. Quality of information and appropriateness of ChatGPT outputs for urology patients. Prostate Cancer Prostatic Dis. 2024;27(1):103-108
Not in our library10.1038/s41391-023-00705-y
45.
Coskun BN, Yagiz B, Ocakoglu G, Dalkilic E, Pehlivan Y. Assessing the accuracy and completeness of artificial intelligence language models in providing information on methotrexate use. Rheumatol Int. 2024;44(3):509-515
Not in our library10.1007/s00296-023-05473-5
46.
Coskun B, Ocakoglu G, Yetemen M, Kaygisiz O. Can ChatGPT, an artificial intelligence language model, provide accurate and high-quality patient information on prostate cancer? Urology. 2023;180:35-58
Not in our library10.1016/j.urology.2023.05.040
47.
Cox A, Seth I, Xie Y, Hunter-Smith DJ, Rozen WM. Utilizing ChatGPT-4 for providing medical information on blepharoplasties to patients. Aesthet Surg J. 2023;43(8):NP658-NP662
Not in our library10.1093/asj/sjad096
48.
Deiana G, Dettori M, Arghittu A, Azara A, Gabutti G, Castiglia P. Artificial intelligence and public health: evaluating ChatGPT responses to vaccination myths and misconceptions. Vaccines (Basel). 2023;11(7):1217
Not in our library10.3390/vaccines11071217
49.
Dubin JA, Bains SS, Chen Z,. Using a Google web search analysis to assess the utility of ChatGPT in total joint arthroplasty. J Arthroplasty. 2023;38(7):1195-1202
Not in our library10.1016/j.arth.2023.04.007
50.
Dwyer T, Hoit G, Burns D,. Use of an artificial intelligence conversational agent (chatbot) for hip arthroscopy patients following surgery. Arthrosc Sports Med Rehabil. 2023;5(2):e495-e505
Not in our library10.1016/j.asmr.2023.01.020
51.
Emile SH, Horesh N, Freund M,. How appropriate are answers of online chat-based artificial intelligence (ChatGPT) to common questions on colon cancer? Surgery. 2023;174(5):1273-1275
Not in our library10.1016/j.surg.2023.06.005
52.
Endo Y, Sasaki K, Moazzam Z,. Quality of ChatGPT responses to questions related to liver transplantation. J Gastrointest Surg. 2023;27(8):1716-1719
Not in our library10.1007/s11605-023-05714-9
53.
Farhat F. ChatGPT as a complementary mental health resource: a boon or a bane. Ann Biomed Eng. 2024;52(5):1111-1114
Not in our library10.1007/s10439-023-03326-7
54.
Franco D’Souza R, Amanullah S, Mathew M, Surapaneni KM. Appraising the performance of ChatGPT in psychiatry using 100 clinical case vignettes. Asian J Psychiatr. 2023;89:103770
Not in our library10.1016/j.ajp.2023.103770
55.
Fraser H, Crossland D, Bacher I, Ranney M, Madsen T, Hilliard R. Comparison of diagnostic and triage accuracy of ada health and WebMD symptom checkers, ChatGPT, and physicians for patients in an emergency department: clinical data analysis study. JMIR Mhealth Uhealth. 2023;11(1):e49995
Not in our library10.2196/49995
56.
Gabriel J, Shafik L, Alanbuki A, Larner T. The utility of the ChatGPT artificial intelligence tool for patient education and enquiry in robotic radical prostatectomy. Int Urol Nephrol. 2023;55(11):2717-2732
Not in our library10.1007/s11255-023-03729-4
57.
Galido PV, Butala S, Chakerian M, Agustines D. A case study demonstrating applications of ChatGPT in the clinical management of treatment-resistant schizophrenia. Cureus. 2023;15(4):e38166
Not in our library10.7759/cureus.38166
58.
Gebrael G, Sahu KK, Chigarira B,. Enhancing triage efficiency and accuracy in emergency rooms for patients with metastatic prostate cancer: a retrospective analysis of artificial intelligence-assisted triage using ChatGPT 4.0. Cancers (Basel). 2023;15(14):3717
Not in our library10.3390/cancers15143717
59.
Accuracy and Reliability of Chatbot Responses to Physician Questions · 2023
No notice
60.
Gordon EB, Towbin AJ, Wingrove P,. Enhancing patient communication with Chat-GPT in radiology: evaluating the efficacy and readability of answers to common imaging-related questions. J Am Coll Radiol. 2024;21(2):353-359
Not in our library10.1016/j.jacr.2023.09.011
61.
Gracias D, Siu A, Seth I, Dooreemeah D, Lee A. Exploring the role of an artificial intelligence chatbot on appendicitis management: an experimental study on ChatGPT. ANZ J Surg. 2024;94(3):342-352
Not in our library10.1111/ans.18736
62.
Gravel J, D’Amours-Gravel M, Osmanlliu E. Learning to fake it: limited responses and fabricated references provided by ChatGPT for medical questions. Mayo Clin Proc Digit Health. 2023;1(3):226-234
Not in our library10.1016/j.mcpdig.2023.05.004
63.
Gwak G, Hwang U, Jung S, Kim J. Search for medical information and treatment options for musculoskeletal disorders through an artificial intelligence chatbot: focusing on shoulder impingement syndrome. J Musculoskelet Sci Technol. 2023;7(1):8-16
Not in our library10.29273/jmst.2023.7.1.8
64.
Haemmerli J, Sveikata L, Nouri A,. ChatGPT in glioma adjuvant therapy decision making: ready to assume the role of a doctor in the tumour board? BMJ Health Care Inform. 2023;30(1):e100775
Not in our library10.1136/bmjhci-2023-100775
65.
Harskamp RE, De Clercq L. Performance of ChatGPT as an AI-assisted decision support tool in medicine: a proof-of-concept study for interpreting symptoms and management of common cardiac conditions (AMSTELHEART-2). Acta Cardiol. 2024;79(3):358-366
Not in our library10.1080/00015385.2024.2303528
66.
Haver HL, Lin CT, Sirajuddin A, Yi PH, Jeudy J. Evaluating ChatGPT’s accuracy in lung cancer prevention and screening recommendations. Radiol Cardiothorac Imaging. 2023;5(4):e230115
Not in our library10.1148/ryct.230115
67.
Haver HL, Ambinder EB, Bahl M, Oluyemi ET, Jeudy J, Yi PH. Appropriateness of breast cancer prevention and screening recommendations provided by ChatGPT. Radiology. 2023;307(4):e230424
Not in our library10.1148/radiol.230424
68.
Diagnostic Accuracy of Differential-Diagnosis Lists Generated by Generative Pretrained Transformer 3 Chatbot for Clinical Vignettes with Common Chief Complaints: A Pilot Study · 2023
No notice
69.
Hirosawa T, Kawamura R, Harada Y,. ChatGPT-generated differential diagnosis lists for complex case-derived clinical vignettes: diagnostic accuracy evaluation. JMIR Med Inform. 2023;11:e48808
Not in our library10.2196/48808
70.
Hirosawa T, Mizuta K, Harada Y, Shimizu T. Comparative evaluation of diagnostic accuracy between google bard and physicians. Am J Med. 2023;136(11):1119-1123.e18
Not in our library10.1016/j.amjmed.2023.08.003
71.
Hopkins AM, Logan JM, Kichenadasse G, Sorich MJ. Artificial intelligence chatbots will revolutionize how cancer patients access information: ChatGPT represents a paradigm-shift. J Natl Cancer Inst Cancer Spectr. 2023;7(2):pkad010
Not in our library10.1093/jncics/pkad010
72.
Hristidis V, Ruggiano N, Brown EL, Ganta SRR, Stewart S. ChatGPT vs Google for queries related to dementia and other cognitive decline: comparison of results. J Med Internet Res. 2023;25:e48966
Not in our library10.2196/48966
73.
Hurley ET, Crook BS, Lorentz SG,. Evaluation high-quality of information from ChatGPT (artificial intelligence-large language model) artificial intelligence on shoulder stabilization surgery. Arthroscopy. 2024;40(3):726-731.e6
Not in our library10.1016/j.arthro.2023.07.048
74.
Janopaul-Naylor JR, Koo A, Qian DC, McCall NS, Liu Y, Patel SA. Physician assessment of ChatGPT and Bing answers to American Cancer Society’s questions to Ask About Your Cancer. Am J Clin Oncol. 2024;47(1):17-21
Not in our library10.1097/coc.0000000000001050
75.
Johnson D, Goodman R, Patrinely J,. Assessing the accuracy and reliability of AI-generated medical responses: an evaluation of the Chat-GPT model. Res Sq. Preprint posted online February 28, 2023
Not in our library10.21203/rs.3.rs-2566942/v1
76.
Kao HJ, Chien TW, Wang WC, Chou W, Chow JC. Assessing ChatGPT’s capacity for clinical decision support in pediatrics: a comparative study with pediatricians using KIDMAP of Rasch analysis. Medicine (Baltimore). 2023;102(25):e34068
Not in our library10.1097/md.0000000000034068
77.
Development and Early Feasibility of Chatbots for Educating Patients With Lung Cancer and Their Caregivers in Japan: Mixed Methods Study · 2021
Not checked
78.
King RC, Samaan JS, Yeo YH, Mody B, Lombardo DM, Ghashghaei R. Appropriateness of ChatGPT in answering heart failure related questions. Heart Lung Circ. 2024;33(9):1314-1318
Not in our library10.1016/j.hlc.2024.03.005
79.
Koh SJQ, Yeo KK, Yap JJL. Leveraging ChatGPT to aid patient education on coronary angiogram. Ann Acad Med Singap. 2023;52(7):374-377
Not in our library10.47102/annals-acadmedsg.2023138
80.
Krusche M, Callhoff J, Knitza J, Ruffer N. Diagnostic accuracy of a large language model in rheumatology: comparison of physician and ChatGPT-4. Rheumatol Int. 2024;44(2):303-306
Not in our library10.1007/s00296-023-05464-6
81.
Kuroiwa T, Sarcon A, Ibara T,. The potential of ChatGPT as a self-diagnostic tool in common orthopedic diseases: exploratory study. J Med Internet Res. 2023;25(1):e47621
Not in our library10.2196/47621
82.
Kusunose K, Kashima S, Sata M. Evaluation of the accuracy of ChatGPT in answering clinical questions on the Japanese Society of Hypertension guidelines. Circ J. 2023;87(7):1030-1033
Not in our library10.1253/circj.cj-23-0308
83.
Lahat A, Shachar E, Avidan B, Glicksberg B, Klang E. Evaluating the utility of a large language model in answering common patients’ gastrointestinal health–related questions: are we there yet? Diagnostics (Basel). 2023;13(11):1950
Not in our library10.3390/diagnostics13111950
84.
Lam CS, Hua R, Koon HK,. Can ChatGPT provide quality information on integrative oncology? a brief report. J Integr Complement Med. 2024;30(2):196-205
Not in our library10.1089/jicm.2023.0290
85.
Lechien JR, Georgescu BM, Hans S, Chiesa-Estomba CM. ChatGPT performance in laryngology and head and neck surgery: a clinical case-series. Eur Arch Otorhinolaryngol. 2024;281(1):319-333
Not in our library10.1007/s00405-023-08282-5
86.
Lee TC, Staller K, Botoman V, Pathipati MP, Varma S, Kuo B. ChatGPT answers common patient questions about colonoscopy. Gastroenterology. 2023;165(2):509-511.e7
Not in our library10.1053/j.gastro.2023.04.033
87.
Levartovsky A, Ben-Horin S, Kopylov U, Klang E, Barash Y. Towards AI-augmented clinical decision-making: an examination of ChatGPT’s utility in acute ulcerative colitis presentations. Am J Gastroenterol. 2023;118(12):2283-2289
Not in our library10.14309/ajg.0000000000002483
88.
Levkovich I, Elyoseph Z. Identifying depression and its determinants upon initiating treatment: ChatGPT versus primary care physicians. Fam Med Community Health. 2023;11(4):e002391
Not in our library10.1136/fmch-2023-002391
89.
Levkovich I, Elyoseph Z. suicide risk assessments through the eyes of ChatGPT-3.5 versus ChatGPT-4: vignette study. JMIR Ment Health. 2023;10(1):e51232
Not in our library10.2196/51232
90.
Li J, Gao X, Dou T, Gao Y, Zhu W. Assessing the performance of GPT-4 in the filed of osteoarthritis and orthopaedic case consultation. Published online August 9, 2023. MedRxiv.
Not in our library
91.
Li W, Chen J, Chen F, Liang J, Yu H. Exploring the potential of ChatGPT-4 in responding to common questions about abdominoplasty: an AI-based case study of a plastic surgery consultation. Aesthetic Plast Surg. 2024;48(8):1571-1583
Not in our library10.1007/s00266-023-03660-0
92.
Benchmarking large language models’ performances for myopia care: a comparative analysis of ChatGPT-3.5, ChatGPT-4.0, and Google Bard · 2023
No notice
93.
Lim DYZ, Tan YB, Koh JTE,. ChatGPT on guidelines: providing contextual knowledge to GPT allows it to provide advice on appropriate colonoscopy intervals. J Gastroenterol Hepatol. 2024;39(1):81-106
Not in our library10.1111/jgh.16375
94.
Liu J, Zheng J, Cai X, Wu D, Yin C. A descriptive study based on the comparison of ChatGPT and evidence-based neurosurgeons. iScience. 2023;26(9):107590
Not in our library10.1016/j.isci.2023.107590
95.
Liu HY, Alessandri Bonetti M, Jeong T, Pandya S, Nguyen VT, Egro FM. Dr. ChatGPT will see you now: how do Google and ChatGPT compare in answering patient questions on breast reconstruction? J Plast Reconstr Aesthet Surg. 2023;85:488-497
Not in our library10.1016/j.bjps.2023.07.039
96.
Liu S, Wright AP, Paterson BL,. Assessing the value of ChatGPT for clinical decision support optimization. Peprint posted online February 23, 2023. MedRxiv.
Not in our library
97.
Long C, Subburam D, Lowe K,. ChatENT: augmented large language model for expert knowledge retrieval in otolaryngology-head and neck surgery. Otolaryngol Head Neck Surg. 2024;171(4):1042-1051
Not in our library10.1002/ohn.864
98.
Lower K, Seth I, Lim B, Seth N. ChatGPT-4: transforming medical education and addressing clinical exposure challenges in the post-pandemic era. Indian J Orthop. 2023;57(9):1527-1544
Not in our library10.1007/s43465-023-00967-7
99.
Luykx JJ, Gerritse F, Habets PC, Vinkers CH. The performance of ChatGPT in generating answers to clinical questions in psychiatry: a two-layer assessment. World Psychiatry. 2023;22(3):479-480
Not in our library10.1002/wps.21145
100.
Lyons RJ, Arepalli SR, Fromal O, Choi JD, Jain N. Artificial intelligence chatbot performance in triage of ophthalmic conditions. Can J Ophthalmol. 2024;59(4):e301-e308
Not in our library10.1016/j.jcjo.2023.07.016
101.
Maillard A, Micheli G, Lefevre L,. Can chatbot artificial intelligence replace infectious diseases physicians in the management of bloodstream infections? a prospective cohort study. Clin Infect Dis. 2024;78(4):825-832
Not in our library10.1093/cid/ciad632
102.
Manolitsis I, Feretzakis G, Tzelves L,. Training ChatGPT models in assisting urologists in daily practice. Stud Health Technol Inform. 2023;305:576-579
Not in our library10.3233/shti230562
103.
Mehnen L, Gruarin S, Vasileva M, Knapp B. ChatGPT as a medical doctor? a diagnostic accuracy study on common and rare diseases. Preprint posted online April 26, 2023. MedRxiv.
Not in our library
104.
Mesnier J, Suc G, Sayah N, Abtan J, Steg PG. Relevance of medical information obtained from ChatGPT: are large language models friends or foes? Arch Cardiovasc Dis. 2023;116(10):485-486
Not in our library10.1016/j.acvd.2023.07.009
105.
Mika AP, Martin JR, Engstrom SM, Polkowski GG, Wilson JM. Assessing ChatGPT responses to common patient questions regarding total hip arthroplasty. J Bone Joint Surg Am. 2023;105(19):1519-1526
Not in our library10.2106/jbjs.23.00209
106.
Mishra A, Begley SL, Chen A,. Exploring the intersection of artificial intelligence and neurosurgery: let us be cautious with ChatGPT. Neurosurgery. 2023;93(6):1366-1373
Not in our library10.1227/neu.0000000000002598
107.
Momenaei B, Wakabayashi T, Shahlaee A,. Appropriateness and readability of ChatGPT-4–generated responses for surgical treatment of retinal diseases. Ophthalmol Retina. 2023;7(10):862-868
Not in our library10.1016/j.oret.2023.05.022
108.
Moshirfar M, Altaf AW, Stoakes IM, Tuttle JJ, Hoopes PC. Artificial intelligence in ophthalmology: a comparative analysis of GPT-3.5, GPT-4, and human expertise in answering StatPearls questions. Cureus. 2023;15(6):e40822
Not in our library10.7759/cureus.40822
109.
Musheyev D, Pan A, Loeb S, Kabarriti AE. How well do artificial intelligence chatbots respond to the top search queries about urological malignancies? Eur Urol. 2024;85(1):13-16
Not in our library10.1016/j.eururo.2023.07.004
110.
Nastasi AJ, Courtright KR, Halpern SD, Weissman GE. A vignette-based evaluation of ChatGPT’s ability to provide appropriate and equitable medical advice across care contexts. Sci Rep. 2023;13(1):17885
Not in our library10.1038/s41598-023-45223-y
111.
Nielsen JPS, von Buchwald C, Grønhøj C. Validity of the large language model ChatGPT (GPT4) as a patient information source in otolaryngology by a variety of doctors in a tertiary otorhinolaryngology department. Acta Otolaryngol. 2023;143(9):779-782
Not in our library10.1080/00016489.2023.2254809
112.
O’Hagan R, Kim RH, Abittan BJ, Caldas S, Ungar J, Ungar B. Trends in accuracy and appropriateness of alopecia areata information obtained from a popular online large language model, ChatGPT. Dermatology. 2023;239(6):952-957
Not in our library10.1159/000534005
113.
Padovan M, Cosci B, Petillo A,. ChatGPT in occupational medicine: a comparative study with human experts. Bioengineering (Basel). 2024;11(1):57
Not in our library10.3390/bioengineering11010057
114.
Pan A, Musheyev D, Bockelman D, Loeb S, Kabarriti AE. Assessment of artificial intelligence chatbot responses to top searched queries about cancer. JAMA Oncol. 2023;9(10):1437-1440
Not in our library10.1001/jamaoncol.2023.2947
115.
Potapenko I, Boberg-Ans LC, Stormly Hansen M, Klefter ON, van Dijk EHC, Subhi Y. Artificial intelligence–based chatbot patient information on common retinal diseases using ChatGPT. Acta Ophthalmol. 2023;101(7):829-831
Not in our library10.1111/aos.15661
116.
Potapenko I, Malmqvist L, Subhi Y, Hamann S. Artificial intelligence-based ChatGPT responses for patient questions on optic disc drusen. Ophthalmol Ther. 2023;12(6):3109-3119
Not in our library10.1007/s40123-023-00800-2
117.
Qu RW, Qureshi U, Petersen G, Lee SC. Diagnostic and management applications of ChatGPT in structured otolaryngology clinical scenarios. OTO Open. 2023;7(3):e67
Not in our library10.1002/oto2.67
118.
Rahsepar AA, Tavakoli N, Kim GHJ, Hassani C, Abtin F, Bedayat A. How AI responds to common lung cancer questions: ChatGPT vs Google Bard. Radiology. 2023;307(5):e230922
Not in our library10.1148/radiol.230922
119.
Assessing the Utility of ChatGPT Throughout the Entire Clinical Workflow: Development and Usability Study · 2023
No notice
120.
Rao A, Kim J, Kamineni M,. Evaluating GPT as an adjunct for radiologic decision making: GPT-4 versus GPT-3.5 in a breast imaging pilot. J Am Coll Radiol. 2023;20(10):990-997
Not in our library10.1016/j.jacr.2023.05.003
121.
Rasmussen MLR, Larsen AC, Subhi Y, Potapenko I. Artificial intelligence–based ChatGPT chatbot responses for patient and parent questions on vernal keratoconjunctivitis. Graefes Arch Clin Exp Ophthalmol. 2023;261(10):3041-3043
Not in our library10.1007/s00417-023-06078-1
122.
Rau A, Rau S, Zoeller D,. A context-based chatbot surpasses trained radiologists and generic ChatGPT in following the ACR appropriateness guidelines. Radiology. 2023;308(1):e230970
Not in our library10.1148/radiol.230970
123.
Rizwan A, Sadiq T. The use of AI in diagnosing diseases and providing management plans: a consultation on cardiovascular disorders with ChatGPT. Cureus. 2023;15(8):e43106
Not in our library10.7759/cureus.43106
124.
Rogasch JMM, Metzger G, Preisler M,. ChatGPT: can you prepare my patients for [18F]FDG PET/CT and explain my reports? J Nucl Med. 2023;64(12):1876-1879
Not in our library10.2967/jnumed.123.266114
125.
Rojas-Carabali W, Cifuentes-González C, Wei X,. Evaluating the diagnostic accuracy and management recommendations of ChatGPT in uveitis. Ocul Immunol Inflamm. 2024;32(8):1526-1531
Not in our library10.1080/09273948.2023.2253471
126.
Rosen S, Saban M. Can ChatGPT assist with the initial triage? a case study of stroke in young females. Int Emerg Nurs. 2023;70:101340
Not in our library10.1016/j.ienj.2023.101340
127.
Rosen S, Saban M. Evaluating the reliability of ChatGPT as a tool for imaging test referral: a comparative study with a clinical decision support system. Eur Radiol. 2024;34(5):2826-2837
Not in our library10.1007/s00330-023-10230-0
128.
Samaan JS, Yeo YH, Ng WH,. ChatGPT’s ability to comprehend and answer cirrhosis related questions in Arabic. Arab J Gastroenterol. 2023;24(3):145-148
Not in our library10.1016/j.ajg.2023.08.001
129.
Assessing the Accuracy of Responses by the Language Model ChatGPT to Questions Regarding Bariatric Surgery · 2023
No notice
130.
Schulte B. Capacity of ChatGPT to identify guideline-based treatments for advanced solid tumors. Cureus. 2023;15(4):e37938
Not in our library10.7759/cureus.37938
131.
Seth I, Cox A, Xie Y,. Evaluating chatbot efficacy for answering frequently asked questions in plastic surgery: a ChatGPT case study focused on breast augmentation. Aesthet Surg J. 2023;43(10):1126-1135
Not in our library10.1093/asj/sjad140
132.
Seth I, Lim B, Xie Y,. Comparing the efficacy of large language models ChatGPT, BARD, and Bing AI in providing information on rhinoplasty: an observational study. Aesthet Surg J Open Forum. 2023;5:ojad084
Not in our library10.1093/asjof/ojad084
133.
Seth I, Xie Y, Rodwell A,. Exploring the role of a large language model on carpal tunnel syndrome management: an observation study of ChatGPT. J Hand Surg Am. 2023;48(10):1025-1033
Not in our library10.1016/j.jhsa.2023.07.003
134.
Sezgin E, Chekeni F, Lee J, Keim S. Clinical accuracy of large language models and Google search responses to postpartum depression questions: cross-sectional study. J Med Internet Res. 2023;25(1):e49240
Not in our library10.2196/49240
135.
Shao CY, Li H, Liu XL,. Appropriateness and comprehensiveness of Using ChatGPT for perioperative patient education in thoracic surgery in different language contexts: survey study. Interact J Med Res. 2023;12:e46900
Not in our library10.2196/46900
136.
Large language model (ChatGPT) as a support tool for breast tumor board · 2023
No notice
137.
Stevenson E, Walsh C, Hibberd L. Can artificial intelligence replace biochemists? a study comparing interpretation of thyroid function test results by ChatGPT and Google Bard to practising biochemists. Ann Clin Biochem. 2024;61(2):143-149
Not in our library10.1177/00045632231203473
138.
Stroop A, Stroop T, Zawy Alsofy S,. Large language models: are artificial intelligence–based chatbots a reliable source of patient information for spinal surgery? Eur Spine J. 2024;33(11):4135-4143
Not in our library10.1007/s00586-023-07975-z
139.
Suresh K, Rathi V, Nwosu O,. Utility of GPT-4 as an informational patient resource in otolaryngology. Published online May 16, 2023. MedRxiv.
Not in our library
140.
Szczesniewski JJ, Tellez Fouz C, Ramos Alba A, Diaz Goizueta FJ, García Tello A, Llanes González L. ChatGPT and most frequent urological diseases: analysing the quality of information and potential risks for patients. World J Urol. 2023;41(11):3149-3153
Not in our library10.1007/s00345-023-04563-0
141.
Vaira LA, Lechien JR, Abbate V,. Accuracy of ChatGPT-generated information on head and neck and oromaxillofacial surgery: a multicenter collaborative analysis. Otolaryngol Head Neck Surg. 2024;170(6):1492-1503
Not in our library10.1002/ohn.489
142.
Van Bulck L, Moons P. What if your patient switches from Dr. Google to Dr. ChatGPT? a vignette-based survey of the trustworthiness, value, and danger of ChatGPT-generated responses to health questions. Eur J Cardiovasc Nurs. 2024;23(1):95-98
Not in our library10.1093/eurjcn/zvad038
143.
Wagner MW, Ertl-Wagner BB. Accuracy of information and references using ChatGPT-3 for retrieval of clinical radiological information. Can Assoc Radiol J. 2024;75(1):69-73
Not in our library10.1177/08465371231171125
144.
Reliability of Medical Information Provided by ChatGPT: Assessment Against Clinical Guidelines and Patient Information Quality Instrument · 2023
No notice
145.
Wang Y, Visweswaran S, Kappor S, Kooragayalu S, Wu X. ChatGPT, enhanced with clinical practice guidelines, is a superior decision support tool. [published online August 13, 2023]. MedRxiv.
Not in our library
146.
Wang G, Liu Q, Chen G,. AI’s deep dive into complex pediatric inguinal hernia issues: a challenge to traditional guidelines? Hernia. 2023;27(6):1587-1599
Not in our library10.1007/s10029-023-02900-1
147.
Xie Y, Seth I, Hunter-Smith DJ, Rozen WM, Ross R, Lee M. Aesthetic surgery advice and counseling from artificial intelligence: a rhinoplasty consultation with ChatGPT. Aesthetic Plast Surg. 2023;47(5):1985-1993
Not in our library10.1007/s00266-023-03338-7
148.
Yeo YH, Samaan JS, Ng WH,. Assessing the performance of ChatGPT in answering questions regarding cirrhosis and hepatocellular carcinoma. Clin Mol Hepatol. 2023;29(3):721-732
Not in our library10.3350/cmh.2023.0089
149.
Yıldız MS, Alper A. Can ChatGPT-4 diagnose in Turkish: a comparison of ChatGPT responses to health-related questions in English and Turkish. J Consum Health Internet. 2023;27(3):294-307
Not in our library10.1080/15398285.2023.2240652
150.
Yun JY, Kim DJ, Lee N, Kim EK. A comprehensive evaluation of ChatGPT consultation quality for augmentation mammoplasty: a comparative analysis between plastic surgeons and laypersons. Int J Med Inform. 2023;179:105219
Not in our library10.1016/j.ijmedinf.2023.105219
151.
Zhou Z, Wang X, Li X, Liao L. Is ChatGPT an evidence-based doctor? Eur Urol. 2023;84(3):355-356
Not in our library10.1016/j.eururo.2023.03.037
152.
Zhou J, He X, Sun L,. SkinGPT-4: an interactive dermatology diagnostic system with visual large language model. Published online April 20, 2023.MedRxiv.
Not in our library
153.
Zhou Y, Moon C, Szatkowski J, Moore D, Stevens J. Evaluating ChatGPT responses in the context of a 53-year-old male with a femoral neck fracture: a qualitative analysis. Eur J Orthop Surg Traumatol. 2024;34(2):927-955
Not in our library10.1007/s00590-023-03742-4
154.
Zhu L, Mou W, Chen R. Can the ChatGPT and other large language models with internet-connected database solve the questions and concerns of patient with prostate cancer and help democratize medical knowledge? J Transl Med. 2023;21(1):269
Not in our library10.1186/s12967-023-04123-5
155.
Zúñiga Salazar G, Zúñiga D, Vindel CL,. Efficacy of AI chats to determine an emergency: a comparison between OpenAI’s ChatGPT, Google Bard, and Microsoft Bing AI Chat. Cureus. 2023;15(9):e45473
Not in our library10.7759/cureus.45473
156.
Glasziou P, Meats E, Heneghan C, Shepperd S. What is missing from descriptions of treatment in trials and reviews? BMJ. 2008;336(7659):1472-1474
Not in our library10.1136/bmj.39590.732037.47
157.
Wang C, Liu SX, Awadallah AH. Cost-effective hyperparameter optimization for large language model generation inference. Preprint published online August 8, 2023. arXiv
Not in our library10.48550/arxiv.2303.04673
158.
Wang PH, Hsieh SI, Chang SC,. Contextual temperature for language modeling. Published online December 25, 2020. arXiv
Not in our library10.48550/arxiv.2012.12575
159.
Wang R, Wang H, Mi F,. Enhancing Large language models against inductive instructions with dual-critique prompting. Published online March 7, 2024. arXiv
Not in our library10.48550/arxiv.2305.13733
160.
Prompt Engineering as an Important Emerging Skill for Medical Professionals: Tutorial · 2023
No notice
161.
Nguyen D, Swanson D, Newbury A, Kim YH. Evaluation of ChatGPT and Google Bard using prompt engineering in cancer screening algorithms. Acad Radiol. 2024;31(5):1799-1804
Not in our library10.1016/j.acra.2023.11.002
162.
Tian S, Jin Q, Yeganova L,. Opportunities and challenges for ChatGPT and large language models in biomedicine and health. Brief Bioinform. 2023;25(1):1-13
Not in our library10.1093/bib/bbad493
163.
Russe MF, Reisert M, Bamberg F, Rau A. Improving the use of LLMs in radiology through prompt engineering: from precision prompts to zero-shot learning. Rofo. 2024;196(11):1166-1170
Not in our library10.1055/a-2264-5631
164.
Revolutionizing healthcare: the role of artificial intelligence in clinical practice · 2023
No notice
165.
Guyatt GH, Oxman AD, Kunz R,. GRADE guidelines: 2. framing the question and deciding on important outcomes. J Clin Epidemiol. 2011;64(4):395-400
Not in our library10.1016/j.jclinepi.2010.09.012
166.
Chang Y, Wang X, Wang J,. A survey on evaluation of large language models. ACM Trans Intell Syst Technol. 2024;15(3):1-45
Not in our library10.1145/3641289
167.
Liang JJ, Tsou CH, Devarakonda MV. Ground truth creation for complex clinical NLP tasks: an iterative vetting approach and lessons learned. AMIA Jt Summits Transl Sci Proc. 2017;2017:203-212.
Not in our library
168.
Ahmed W, Saturno M, Rajjoub R,. ChatGPT versus NASS clinical guidelines for degenerative spondylolisthesis: a comparative analysis. Eur Spine J. 2024;33(11):4182-4203
Not in our library10.1007/s00586-024-08198-6
169.
Gianola S, Bargeri S, Castellini G,. Performance of ChatGPT compared to clinical practice guidelines in making informed decisions for lumbosacral radicular pain: a cross-sectional study. J Orthop Sports Phys Ther. 2024;54(3):222-228
Not in our library10.2519/jospt.2024.12151
170.
Da Silva M, Flood CM, Goldenberg A, Singh D. Regulating the safety of health–related artificial intelligence. Healthc Policy. 2022;17(4):63-77
Not in our library10.12927/hcpol.2022.26824
171.
FDA. Artificial intelligence and machine learning in software as a medical device. 2024. Accessed March 29, 2024. https://www.fda.gov/medical-devices/software-medical-device-samd/artificialintelligence-and-machine-learning-software-medical-device
Not in our library
172.
Health Canada. Guidance document: software as a medical device (SaMD): classification examples. 2022. Accessed March 29, 2024. https://www.canada.ca/en/health-canada/services/drugs-health-products/medical-devices/application-information/guidance-documents/software medical-device-guidance/examples.html#a4.3
Not in our library
173.
FDA. Artificial intelligence and machine learning (AI/ML)–enabled medical devices. Accessed March 29, 2024. https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-aiml-enabled-medical-devices
Not in our library
174.
The shaky foundations of large language models and foundation models for electronic health records · 2023
No notice
175.
Microsoft. Microsoft and Epic expand strategic collaboration with integration of Azure OpenAI Service. Accessed March 29, 2024. https://news.microsoft.com/2023/04/17/microsoft-and-epic-expand-strategic-collaboration-with-integration-of-azure-openai-service/
Not in our library
176.
Dahdah R. Microsoft makes the promise of AI in healthcare real through new collaborations with healthcare organizations and partners. Accessed March 28, 2024. https://blogs.microsoft.com/blog/2024/03/11/microsoft-makes-the-promise-of-ai-in-healthcare-real-through-new-collaborations-with-healthcare-organizations-and-partners/
Not in our library
177.
Bitkina OV, Park J, Kim HK. Application of artificial intelligence in medical technologies: a systematic review of main trends. Digit Health. 2023;9:20552076231189331
Not in our library10.1177/20552076231189331
178.
Saenz AD, Harned Z, Banerjee O, Abràmoff MD, Rajpurkar P. Autonomous AI systems in the face of liability, regulations and costs. NPJ Digit Med. 2023;6(1):185
Not in our library10.1038/s41746-023-00929-1
179.
Wu E, Wu K, Daneshjou R, Ouyang D, Ho DE, Zou J. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nat Med. 2021;27(4):582-584
Not in our library10.1038/s41591-021-01312-x
180.
Nagendran M, Chen Y, Lovejoy CA,. Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies in medical imaging. BMJ. 2020;368:m689
Not in our library10.1136/bmj.m689
181.
Ong JCL, Chang SYH, William W,. Ethical and regulatory challenges of large language models in medicine. Lancet Digit Health. 2024;6(6):e428-e432
Not in our library10.1016/s2589-7500(24)00061-x
182.
World Health Organization. Ethics and governance of artificial intelligence for health. Accessed November 4, 2024. https://www.who.int/publications/i/item/9789240029200
Not in our library
183.
Thirunavukarasu AJ. Large language models will not replace healthcare professionals: curbing popular fears and hype. J R Soc Med. 2023;116(5):181-182
Not in our library10.1177/01410768231173123
184.
Qu Y, Wang J. Performance and biases of large language models in public opinion simulation. Humanit Soc Sci Commun. 2024;11(1)
Not in our library10.1057/s41599-024-03609-x
185.
Large language models propagate race-based medicine · 2023
No notice
186.
Ayoub NF, Balakrishnan K, Ayoub MS, Barrett TF, David AP, Gray ST. Inherent bias in large language models: a random sampling analysis. Mayo Clin Proc Digit Health. 2024;2(2):186-191
Not in our library10.1016/j.mcpdig.2024.03.003
187.
Huo B, Calabrese E, Sylla P,. The performance of artificial intelligence large language model–linked chatbots in surgical decision-making for gastroesophageal reflux disease. Surg Endosc. 2024;38(5):2320-2330
Not in our library10.1007/s00464-024-10807-w
188.
Au Yeung J, Kraljevic Z, Luintel A,. AI chatbots not yet ready for clinical use. Front Digit Health. 2023;5:1161098
Not in our library10.3389/fdgth.2023.1161098
189.
Meyrowitsch DW, Jensen AK, Sørensen JB, Varga TV. AI chatbots and (mis)information in public health: impact on vulnerable communities. Front Public Health. 2023;11:1226776
Not in our library10.3389/fpubh.2023.1226776
190.
Khan B, Fatima H, Qureshi A,. Drawbacks of artificial intelligence and their potential solutions in the healthcare sector. Biomed Mater Devices. 2023;1(2):1-8
Not in our library10.1007/s44174-023-00063-2
191.
Yao Y, Duan J, Xu K, Cai Y, Sun Z, Zhang Y. A survey on large language model (LLM) security and privacy: the good, the bad, and the ugly. High-Confidence Computing. 2024;4(2)
Not in our library10.1016/j.hcc.2024.100211
192.
Ethical Considerations of Using ChatGPT in Health Care · 2023
No notice
193.
Luo H, Specia L. From understanding to utilization: a survey on explainability for large language models. arXiv. Preprint posted online January 23, 2024
Not in our library10.48550/arxiv/2401.12874

Reads

Read by people on this page

None yet.

Checked or read by AI through naulon

None yet.

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.

Are you an author?

Is this your paper?

Claim it to add your note and be paid when agents cite it. Free.