Analysis of the Results
Once the results have been compiled, organizations must interpret the data correctly. For projects designed to detect important deviations from expected performance, it is important to use tests that determine statistical significance. For many measures, comparisons of mean performance are satisfactory—for example, the percentage of AAO benchmark process indicators for cataract that providers have achieved. However, for others, the best way to compare performance may be to analyze the number of patients whose care meets a given threshold. For example, investigators may want to know the percentage of patients who have at least a 90% quality score. Once that measure is obtained, researchers can compare it among providers.
In addition, when evaluating quality results investigators should consider clinical significance and difference, and factors beyond the provider’s control. Patients could refuse the cost of additional tests or refuse to be dilated because they need to be able to drive. Outcomes of chronic diseases are even more difficult to evaluate than process measures such as dilation of a patient with diabetes mellitus. For example, the rate of blindness from glaucoma over 20 years is subject to the severity at presentation and the risk for progression among the pool of patients. Quality of care for chronic diseases like glaucoma may be affected by patients’ ability to return for regular care and use their recommended treatments regularly, as well as their socioeconomic status. Thus, measuring whether the provider performs specific examination steps, such as examining the optic nerve (process), is appropriate, whereas looking at rates of blindness over 20 years (outcomes) may not be appropriate.
Organizations can use statistical analyses to evaluate potential differences in quality between providers. They can control for patients’ socioeconomic status and demographic characteristics, as well as other factors that may be related to the outcome of interest. These analyses may show that a factor that cannot be “treated” by the provider (eg, socioeconomic status) is the issue and is outside the provider’s control. Even with these caveats, quality improvement is critical to medicine. In addition, the purpose is not to be punitive but to encourage improvement in all providers and improvement in individual providers from year to year.
Excerpted from BCSC 2020-2021 series: Section 1 - Update on General Medicine. For more information and to purchase the entire series, please visit https://www.aao.org/bcsc.