“Here’s a simple question for a simple paper:
Do you think that a chatbot can replace you in surgical plan assessment?”
That is how Stanislao Rizzo, MD, opened “Exploring AI-Chatbots’ Capability to Suggest Retinal Detachment Surgical Planning,” the first talk in the Surgical Training Expert Panel at the American Society for Retinal Specialists (ASRS) 2024 Annual Meeting. Dr. Rizzo discussed the potential use of two common large language models (LLMs) in assessing scenarios in retinal detachment and explained why his answer to the above question is “Yes.”
Strong Performances Seen for Both ChatGPT and Google Gemini
The study conducted by Dr. Rizzo, Dr. Matteo Mario Carlà, and colleagues tested the ability of ChatGPT 4 (2023) and Google Gemini (2024) to recommend vitreoretinal surgery plans. Each bot was given the same 50 real-world clinical case scenarios and asked to specify the recommended type of surgical planning and the eventual intraocular tamponade.
When their responses were compared with the common opinion of 3 expert vitreoretinal surgeons, both bots showed strong agreement with the experts, with ChatGPT achieving 84% agreement vs 70% for Google Gemini. In Dr. Rizzo’s words, “very impressive!”
Accurate Image Analysis Also Appears Possible
A second part of the study assessed ChatGPT’s ability to perform image analysis (image interpretation is outside the scope of Google Gemini’s current capabilities). When provided with 3 different retinal images and asked to make a diagnosis, ChatGPT identified all 3 correctly and was able to differentiate them as images of retinal detachment, retinal detachment with significant changes to the vitreous chamber, and retinal detachment with proliferative vitreoretinopathy.
At the conclusion of his talk, when asked about how far this AI revolution is likely to go and whether he thinks AI and technology will replace a physician, Dr. Rizzo replied that there will most likely be patients taking their own images, cloud-based large language model analyses, and robots doing surgery, joking that it would free up a lot of his time to sit at his computer or to do other things. However, his bottom line was clear: “AI will not replace physicians, but physicians who do not use AI may be replaced by those who do.”