Skip to main content

  • BERT-Based Models Improve Predictions of Glaucoma Surgery

    By Lynda Seminara
    Selected by Prem S. Subramanian, MD, PhD
    Add to My Bookmarks

    Journal Highlights

    Translational Vision Science & Technology
    2022;11(3):37

    Download PDF

    Models to predict glaucoma progression rely on OCT and visual field data, but there are other factors that affect progression, including the patient’s clinical history. Electronic health records (EHRs) make it possible to factor more data into machine-learning algorithms, but the best way to incorporate free-text notes into a glaucoma progression model is not yet known. Given that bidirectional encoder represen­tations from transformers (BERTs) have been applied successfully to clinical notes in other medical spe­cialties, Hu and Wang hypothesized that BERT-based models may be useful for ophthalmology as well. Their study of several such models showed that they can improve the ability to predict which glaucoma cases will require surgery.

    The authors began by identifying clinical notes from the EHRs of patients with glaucoma who were treated at their facility between 2009 and 2018. Among the 4,512 patients identified who met inclusion criteria, 748 (16.6%) required incisional glaucoma surgery. For each patient, the first three clinical prog­ress notes from the initial four months of follow-up (prior to surgery, where applicable) were combined into a single document and incorporated into four widely accepted BERT-based models: original BERT, BioBERT, RoBERTa, and DistilBERT. Using standard metrics, including area under the receiver operat­ing characteristic curve (AUROC) and F1 score, each model was assessed for its value in predicting the need for surgery.

    Records were allocated to training, validation, and test datasets in an approximately 8:1:1 ratio. The original BERT model, which has undergone many refinements since its inception, had the strongest predictive value, with an AUROC of 73.4% and an F1 score of 45.0%. The next-best performer was Ro­BERTa (AUROC: 72.4%, F1: 44.7%), followed by DistilBERT (AUROC: 70.2%, F1: 42.5%) and BioBERT (AUROC: 70.1%, F1: 41.7%). With respect to F1 score, all four models were superior to an ophthalmol­ogist’s assessment of clinical notes (F1: 29.9%).

    In their discussion, the authors note that while no previous studies have applied BERT to clinical notes to predict the progression of glaucoma, studies in other medical fields have used BERT with clinical notes to perform clinical prediction tasks. They add that using transfer learning with highly pretrained BERT-based models “is a natural language processing approach that can access the wealth of clinical information stored within ophthalmology clinical notes to predict the progression of glaucoma.” They recommend research to improve the performance of BERT-based models, enhance their relevance to ophthalmology, and determine how to best integrate lengthy clinical notes into the models, with the goal of optimizing clinical decisions.

    The original article can be found here.