Use of Artificial Intelligence Chatbots for Cancer Treatment Information

1Large language models (LLMs) underlying chatbots can mimic human language and quickly return detailed and coherent-seeming responses. These properties can obscure that chatbots might provide inaccurate information. Because patients often use the internet for self-education, some will undoubtedly use LLM chatbots to find cancer-related medical information, which could be associated with the generation and amplification of misinformation. We evaluated an LLM chatbot’s performance to provide breast, prostate, and lung cancer treatment recommendations concordant with National Comprehensive Cancer Network (NCCN) guidelines.

2We developed 4 zero-shot prompt templates to query treatment recommendations (eMethods and eFigure in Supplement 1). These templates do not provide the model with examples of correct responses. Templates were used to create 4 prompt variations for 26 diagnosis descriptions (cancer types with or without relevant extent of disease modifiers) for a total of 104 prompts. Prompts were input to the GPT-3.5-turbo-0301 model via the ChatGPT (OpenAI) interface. In accordance with the Common Rule, institutional review board approval was not needed since human participants were not involved.

3We benchmarked the chatbot’s recommendations against 2021 NCCN guidelines because this chatbot’s knowledge cutoff was September 2021. Five scoring criteria were developed to assess guideline concordance (Table 1). The output did not have to recommend all possible regimens to be considered concordant; the recommended treatment approach needed only to be an NCCN option. Concordance of the chatbot output with NCCN guidelines was assessed by 3 of 4 board-certified oncologists, and majority rule was taken as the final score. In cases of complete disagreement, the oncologist who had not previously seen the output adjudicated. Data were analyzed between March 2 and March 14, 2023, using Excel, version 16.74 (Microsoft Corp).

4Outputs of 104 unique prompts were scored on 5 criteria for a total of 520 scores. All 3 annotators agreed for 322 of 520 (61.9%) scores. Disagreements tended to arise when the output was unclear (eg, not specifying which multiple treatments to combine). Table 1 shows agreement between the 4 prompts and the distribution of scores across cancer type and extent of disease. For 9 of 26 (34.6%) diagnosis descriptions, the 4 prompts yielded the same scores for each of the 5 scoring criteria. Table 2 shows scores across the prompts. The chatbot provided at least 1 recommendation for 102 of 104 (98%) prompts. All outputs with a recommendation included at least 1 NCCN-concordant treatment, but 35 of 102 (34.3%) of these outputs also recommended 1 or more nonconcordant treatments.

5Responses were hallucinated (ie, were not part of any recommended treatment) in 13 of 104 (12.5%) outputs. Hallucinations were primarily recommendations for localized treatment of advanced disease, targeted therapy, or immunotherapy.

6One-third of treatments recommended by the chatbot were at least partially nonconcordant with NCCN guidelines; recommendations varied based on how the question was posed. Disagreement among annotators highlighted the challenges of interpreting descriptive LLM output. Disagreements most often arose from unclear output, but differing interpretations of guidelines among annotators may have played a role. Clinicians should advise patients that LLM chatbots are not a reliable source of treatment information.

7Language learning models can pass the US Medical Licensing Examination, encode clinical knowledge, and provide diagnoses better than laypeople. However, the chatbot did not perform well at providing accurate cancer treatment recommendations. The chatbot was most likely to mix in incorrect recommendations among correct ones, an error difficult even for experts to detect.

8A study limitation is that we evaluated 1 model at a snapshot in time. Nonetheless, the findings provide insight into areas of concern and future research needs. The chatbot did not purport to be a medical device, and need not be held to such standards. However, patients will likely use such technologies in their self-education, which may affect shared decision-making and the patient-clinician relationship. Developers should have some responsibility to distribute technologies that do not cause harm, and patients and clinicians need to be aware of these technologies’ limitations.