Clinical appropriateness of ChatGPT-4.0 and ChatGPT-4.0-mini for patient-facing ophthalmic procedure information

Vollständiger Abstract

Worum geht es in dieser Arbeit?

Background: Patients increasingly turn to large language models (LLMs), such as ChatGPT, for medical guidance, yet their reliability in ophthalmology remains understudied. Prior work has addressed readability and patient handout generation, but a comprehensive evaluation of ChatGPT’s procedural accuracy is lacking. Purpose: To evaluate the accuracy, clinical appropriateness, and readability of ChatGPT-4.0 and ChatGPT-4.0-mini in responding to patient-centered questions about six common ophthalmologic procedures. Methods: In March 2025, a cross-sectional qualitative study was conducted using standardized questions across five domains (indications, alternatives, procedural steps, risks, and recovery) for six ophthalmologic procedures: intravitreal injection, selective laser trabeculoplasty, YAG capsulotomy, cataract surgery, LASIK, and vitrectomy for retinal detachment. Responses from ChatGPT-4.0 and 4.0-mini were independently reviewed by two ophthalmology residents and one board-certified attending for accuracy, relevance, and readability. Results: Both models produced generally accurate and coherent descriptions of procedural indications and recovery. However, they frequently omitted key procedural risks, lacked incidence estimates, and failed to address variability in physician technique. Outdated or inappropriate alternatives, such as topical anti-VEGF therapy for age-related macular degeneration, were identified. ChatGPT-4.0 outperformed the 4.0-mini model in content organization, clinical accuracy, and consistency with the standard of care. Conclusion: ChatGPT offers promise as a patient education tool in ophthalmology, particularly for delivering basic procedural overviews. However, current iterations lack the specificity, clinical nuance, and knowledge of treatments necessary for unsupervised patient use. Ongoing validation and physician oversight are essential before broader implementation in patient education.