Deep learning system is equal to or better than experts at classifying optic disc abnormalities

This study compared the diagnostic performance of an artificial intelligence deep learning system with that of expert neuro-ophthalmologists in classifying optic disc appearance.

Study design

The investigators tested the abilities of a previously trained and validated artificial intelligence deep learning system (DLS) to correctly identify normal optic nerves, papilledema and other optic disc abnormalities from 800 fundus photographs. Its performance was compared with that of 2 expert neuro-ophthalmologists.

Outcomes

The DLS correctly classified 678 of 800 (84.7%) photographs, compared with 84.4% for expert 1 and 80.1% for expert 2.  The DLS had areas under the receiver operating characteristic curve of 0.97 for detecting normal discs, 0.96 for detecting papilledema and 0.89 for detecting other disc abnormalities, which was equal or better than the expert neuro-ophthalmologists. This task took 25 seconds for the DLS, 61 minutes for expert 1 and 74 minutes for expert 2. 

Limitations

In this study, the classification of the optic discs was based on fundus photographs alone. In real practice, clinical examination, ancillary testing and follow-up are important in establishing a diagnosis, which were not incorporated into the classification.

Clinical significance

This study demonstrates that the DLS was equal or better than 2 expert neuro-ophthalmologists at classifying optic disc abnormalities and took a fraction of the time. Artificial intelligence systems such as DLS could be very valuable in identifying potentially vision-threatening or neurologically dangerous disease based on fundus photographs, which will be most useful in the emergency room setting or for providers that do not have rapid access to an ophthalmologist.