Skip to main content

  • Do Experts Agree on the Diagnoses Assigned to Uveitis Cases?

    By Lynda Seminara
    Selected By: Richard K. Parrish II, MD
    Add to My Bookmarks

    Journal Highlights

    American Journal of Ophthalmology, February 2018

    Download PDF

    Jabs et al. conducted an interobserver study to ascertain the level of expert agreement on diagnoses assigned to cases of uveitis. They found that independent assessment yielded only moderate agreement, which improved greatly after conference calls with colleagues.

    For their study, 5 committees (each with 9 uveitis experts) reviewed a total of 5,766 cases from a preliminary database representing 25 uveitic diseases. Initially, the experts voted online, inde­pendently, on whether each case coin­cided with its assigned diagnosis. The agreement statistic (κ) was calculated for 36 pairwise comparisons per dis­ease, and the mean κ was calculated for each disease. After independent voting, committees held consensus conference calls to discuss the cases that lacked supermajority agreement, defined as > 75%. Nominal group techniques were applied to attempt to reach the targeted level of agreement.

    The mean κ achieved from indepen­dent voting was 0.39, denoting moder­ate agreement. Disease-specific varia­tion ranged from 0.23 (for toxoplasmic retinitis) to 0.79 (for cytomegalovirus anterior uveitis). After the conference calls, supermajority agreement was at­tained for approximately 99% of cases, with disease-specific variations ranging from 96% to 100%. The remaining cases (approximately 1%) were perma­nently “tabled.” Ultimately, 71% of the cases evaluated were accepted into the final database and 28% were rejected. Acceptance rates ranged from 42% for herpes simplex anterior uveitis to 92% for serpiginous-like tuberculous cho­roiditis. Throughout the study, perfect agreement (κ = 1.00) was achieved by only 1 pair of experts. For several dis­eases, the agreement of at least 1 pair of experts was essentially “chance alone.”

    Although diagnostic agreement was only moderate early in the study, it was improved by collaborative discus­sion. Only during the conference calls did many essential disease-specific acceptance/rejection criteria begin to emerge. The obstacles to consensus that arose in this study indicate the need for clear, validated, widely accepted classi­fication criteria for uveitic conditions. With better criteria, the data derived from case series, cohort studies, and multicenter trials should become more homogeneous and thus more useful for establishing accurate diagnoses.

    The original article can be found here.