Blepharoplasty FAQs: ChatGPT Has Patients Covered

Download PDF

This month, News in Review highlights selected papers from the original papers sessions at AAO 2023. Each was chosen by an Annual Meeting Program Committee subspecialty chair because it presents important news or illustrates a trend in the field. Only four subspecialties are included here; papers sessions will also be held in five other fields. For more information, see the Mobile Meeting Guide (aao.org/mobile).

Motivated by the reality that artificial intelligence (AI), including chatbots like ChatGPT, are becoming a permanent part of the ophthalmo­logical landscape, Arjun Watane, MD, at Yale School of Medicine, set out to learn more about how AI may be uti­lized by patients who are interested in undergoing upper eyelid blepharoplasty surgery. It turns out that ChatGPT-4 does an excellent job at answering patients’ questions about the procedure, Dr. Watane said.

“For many years, the internet has been a primary source of medical infor­mation, and a lot of patients turn to it for health information for themselves, their families, and their friends,” Dr. Watane said. In the last several years, he said, “a lot of health websites and search engines have incorporated or have plans to incorporate AI, like ChatGPT or ChatGPT-like language models, in their search algorithms. So, we want­ed to look at how ophthalmology is impacted.”

The research. Dr. Watane and his colleagues compared ChatGPT’s responses to six frequently asked questions (FAQs) about upper eyelid blepharoplasty to the responses of three American Society of Ophthalmic Plastic and Reconstructive Surgery (ASOPRS)–trained oculofacial plastic surgeons.

The questions were: What is an upper eyelid blepharoplasty? Why is it performed? What are the risks and complications of the procedure? Where is it performed? What kind of anesthe­sia is used? What is recovery like?

For the ChatGPT portion of the study, the researchers ran the FAQs through ChatGPT-4 three times. For the human portion, each surgeon answered the questions using an online questionnaire. Last, two additional “graders”—ASOPRS-trained oculoplas­tic surgeons—blindly evaluated all the responses based on a Likert scale where 1 = strongly disagree and 5 = strongly agree.

Results. The authors said the results from ChatGPT were noninferior to those of the oculofacial plastic sur­geons. ChatGPT achieved a mean score of 3.8 in accuracy, 3.6 in comprehen­siveness, and 3.2 in personal answer similarity (meaning the ChatGPT an­swer’s similarity to the grader’s answer), while the surgeons received a mean score of 3.6 in accuracy, 3.0 in compre­hensiveness, and 2.7 in personal answer similarity.

“Our primary finding was that ChatGPT may provide more compre­hensive answers to frequently asked questions compared to human ocu­lofacial plastic surgeons,” Dr. Watane said. “However, the graders found no significant difference between their personal answer similarities, as well as the accuracy of the answers between the two.”

The upshot. These findings have important implications for eye surgeons and patients in their clinical practic­es, primarily that ChatGPT can be beneficial for patient education. “The surgeons themselves will always need to provide the final answers, but ChatGPT does provide admissible information and may be appropriate as an adjunct but not a replacement for medical advice,” Dr. Watane said.

Ultimately, this technology may help improve physician workflow, he said. “These are very common questions, but each question needs a full and com­prehensive explanation. If patients are primed with these answers, they may already come in with a pretty broad working knowledge of the procedure. And physicians may only have to fine-tune or confirm the answers rather than offer a full explanation,” Dr. Watane said.

He said that overall, ChatGPT could improve efficiency, reduce time, and decrease the burden on an already overburdened health care system.

Caveats. Of course, there are limitations to keep in mind, he said. The current study only looked at specific questions about upper eyelid blepharoplasty, so the findings can’t be generalized to other questions or procedures. Additionally, ChatGPT and other language-learning models are limited to what that AI model is trained on, which may not always include the most current guidelines or information. “For an established procedure like blepharoplasty surgery, that may not be much of an issue,” Dr. Watane said.

—Ashley Welch

___________________________

Relevant financial disclosures: Dr. Watane—None.

For full disclosures and the disclosure key, see below.

• Magnetic Cells in Endothelial Cell Transplantation

• IIH Rose 35% in Seven Years

• Reusing Some Supplies May Trim Costs