Concordance Between ChatGPT and Real-World Adjuvant Chemotherapy Decisions in pT3N0 Colon Cancer
Bengü Dursun11Department of Medical Oncology, Ankara Etlik City Hospital, Ankara, Türkiye
Objectives: Large language models (LLMs) are increasingly explored as clinical decision-support tools, but their alignment with real-world oncology practice remains uncertain. We evaluated concordance between clinicians and ChatGPT (GPT-5.2) for adjuvant chemotherapy recommendations in pT3N0 colon cancer and examined factors influencing each recommendation pattern. Methods: This retrospective study included 186 patients with surgically resected pT3N0 colon cancer treated between 2017 and 2024. A standardized clinical vignette was generated for each case and submitted once to ChatGPT to obtain a binary recommendation (chemotherapy vs. observation). Concordance with clinicians’ documented decisions was assessed using Cohen’s kappa and McNemar’s test. Multivariable logistic regression identified factors associated with ChatGPT and clinician recommendations. Results: Overall concordance was 61.8%, with moderate agreement (?=0.278). Discordance was asymmetric, with clinicians recommending chemotherapy more often than ChatGPT (58 clinician-yes/ChatGPT-no vs. 13 ChatGPT-yes/clinician- no; p<0.001). ChatGPT recommendations were driven mainly by adverse pathological features, whereas clinician decisions more strongly reflected guideline-modifying variables and surgical quality indicators. Conclusion: ChatGPT showed only moderate alignment with real-world clinician decisions and appeared more rulebased, supporting its role as an adjunctive decision-support tool rather than an autonomous decision-maker. Keywords: Adjuvant chemotherapy, ChatGPT, Cohen’s kappa, colon cancer, concordance, large language model, McNemar, pT3N0, stage II
Cite This Article
Dursun B. Concordance Between ChatGPT and Real-World Adjuvant Chemotherapy Decisions in pT3N0 Colon Cancer. EJMI. 2026; 10(4): 508-515
Corresponding Author: