Skip to main content

  • Accuracy of the WebMD Symptom Checker for Ophthalmic Diagnoses

    By Lynda Seminara
    Selected and Reviewed By: Neil M. Bressler, MD, and Deputy Editors
    Add to My Bookmarks

    Journal Highlights

    JAMA Ophthalmology, June 2019

    Download PDF

    As the accessibility of internet-based resources continues to grow, more patients are conducting self-guided research of symptoms, making it im­portant to determine the accuracy of online symptom checkers. In a cross-sectional study using validated ophthalmic clinical vignettes, Shen et al. looked at the accuracy of the popular WebMD symptom checker. They found that the primary diagnosis generated by the symptom checker was correct for only 26% of the clinical sce­narios. Moreover, the correct diagnosis was not on the list of possibilities for nearly half of the vignettes.

    This cross-sectional descriptive study involved generating 42 validat­ed clinical vignettes of ophthalmic symptoms and distilling them to their core presenting signs. The “cases” were entered into the WebMD symp­tom checker by two people who were masked to the diagnoses (one of whom was medically trained).

    Output from the symptom checker was documented, including triage ur­gency and the list of diagnoses ranked from most to least likely. The main outcome was diagnostic accuracy of the symptom checker.

    The mean number of symptoms entered for each case was 3.6 (range, 1-8). The median number of generated diagnoses per case was 26.8 (range, 1-99). The checker’s primary diagnosis was correct for only 11 (26%) of the 42 vignettes. The correct diagnosis was among the top three entries in 16 (38%) of cases. However, for 18 cases (43%), the correct diagnosis was not on the list.

    The triage urgency for the top-listed diagnosis was appropriate in seven (39%) of 18 emergency cases and in 21 (88%) of 24 nonurgent cases. Inter­user variability for the correct diagnosis being among the top three was at least moderate.

    The clinical vignettes generally were devoid of comorbid and distractor symptoms, which may indicate that the checker’s accuracy was overestimated. However, the vignettes devised for this study resemble those used for training physicians in pattern recognition.

    The authors emphasized that al­though the WebMD symptom checker may pinpoint an ophthalmic diagnosis, its overall accuracy is low. As a result, patients should exercise caution when using such online resources and should understand that symptom-checker output “is not a substitute for profes­sional medical advice, diagnosis, or treatment,” as stated on the main page of the WebMD tool.

    After this study was completed, the WebMD interface and output scheme were revised. (Also see related commen­tary by Rahul N. Khurana, MD, in the same issue.)

    The original article can be found here.