Although patient-reported outcome measures (PROMs) are not new to ophthalmology, they have garnered greater visibility and interest recently. As David W. Parke II, MD, former Academy CEO, stated in his opening remarks at an AAO 2023 symposium, “This is as much the future of the practice of ophthalmology, the science of ophthalmology, as anything out there right now.”1 Similarly, the U.S. Food and Drug Administration and National Eye Institute highlighted the importance of PROMs in their September 2023 joint virtual workshop on Patient-Reported Outcomes and Vision-Related Quality of Life Questionnaires exploring the progress in and future directions for PROMs in ophthalmology.
Despite this growing interest, some barriers remain to their wider adoption. Concerns about the use of PROMs include their validity, applicability across diverse groups, and burden on patients and clinical practices in the collection of responses.2
Here is a look at some important PROMs now in use, some of the current barriers and possible solutions, and how technology can help increase the use of PROMs in research and clinical practice.
What Are PROs and PROMs?
The FDA defines patient-reported outcomes (PROs) as “any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else,” and can include information about quality of life, symptoms, function, satisfaction with care, and perceived value of treatment.3
By this definition, “PROs are a regular part of our clinical practice, anytime we ask our patients how they’re tolerating their eye drops and how they are feeling or functioning vision-wise,” said Barbara M. Wirostko, MD, at Moran Eye Center in Salt Lake City.
However, more standardized and validated outcome measures—PROMs—are necessary to capture, analyze, and incorporate patient voices into structured data that can be used in research, product development and approvals, and evidence-based clinical practice. PROMs consist of individual questions, or items, that are further grouped into categories of related concepts, called domains, and are scored on predefined multi-item scales. Although these questionnaires were historically completed on paper, they are now also administered through other modes such as the internet and mobile devices.4
PATIENT PERSPECTIVES IN DEVICE DEVELOPMENT. This schematic depicts the multiple points at which patient-reported data can inform research, development, and approval of devices.
How Are PROMs Used?
Physician-patient relationship. “Our role as physicians is to treat the patient, not just the disease,” said Emily Y. Chew, MD, at the National Eye Institute, in Bethesda, Maryland. “Improving the patient’s quality of life is one of the strategic pillars—and now part of the mission statement—of the NEI. We wanted to make eye care really patient-centric. We started that tradition 25 years ago when we developed the NEI Visual Function Questionnaire [VFQ].”
Jayne S. Weiss, MD, at the LSU Health New Orleans, LSU–School of Medicine, said, “For the clinician, PROMs are important because they give us information on what really matters to the patient—not just what the ophthalmologist sees or thinks. It also allows us to monitor the efficacy of whatever procedure we did, and the patient’s satisfaction with it.” She added that the latter is especially important for elective treatments such as LASIK or the use of multifocal lenses, where patient expectations are high.
Ultimately, “PROMs provide an opportunity to empower patients who have an ocular disease or are visually impaired,” said Jimmy T. Le, ScD, at the NEI. “How do we determine whether our treatments are working to improve patients’ quality of life? That’s the question we’re trying to answer with these instruments.”
Drug and device development. Dr. Chew noted that the European Medicines Agency requires patient-reported outcomes for marketing authorization. “So, for example, all the diabetic studies they’ve done use the NEI VFQ-25 as part of that. And in the U.K., they involve patients from the very beginning of the study design.”
Even though PROMs are not required for approval in the United States, Dr. Chew continued, “The FDA colleagues we’re working with are very interested in making the process continually more patient-centered.”
Indeed, over the past 15 years, the FDA has been issuing and updating guidance documents for industry and other stakeholders on the use of PROMs in product approval and labeling. In these documents, the agency states that the systematic collection of patient-reported data provides “valid scientific evidence to support the regulatory and health care decision-making process.”5
Ron D. Hays, PhD, at the University of California, Los Angeles, said that the only downside of working with the FDA on PROM development is that it can slow things down because the agency needs to go through its internal process. But, he said, “they provide wonderful feedback, and they’re more actively involved than most coauthors I know. So, once it gets through that process it’s usually a good measure.”
As the chief medical officer of a biotechnology company, Dr. Wirostko has extensive experience in drug and device development. She observed that the creation of PROMs can often be driven by biotech companies that have an innovation that provides a benefit in terms of treatment burden, patient satisfaction, or perceived value of treatment.
“For example, consider two hypothetical intravitreal drugs, one of which is dosed every month and the other every six months. They may have the same efficacy and safety profiles, but patients and doctors are likely to have different perceptions of these two drugs. It could even affect adherence to treatment. How can you capture that difference in the patient’s quality of life? That’s the value of PROMs.”
Clinical research and publication. PROMs are increasingly being used as secondary and even primary endpoints in randomized clinical trials (RCTs). According to one review, the subspecialties that have made greatest progress in developing PROMs and incorporating PROMs into RCTs are glaucoma, medical retina, cataract/IOL, and low vision.2
In 2013, an extension specific to the use of PROMs in RCTs was added to the CONSORT (Consolidated Standards of Reporting in Clinical Trials) guidelines in the hope of bringing the reporting of PROMs to the same level as that of other outcomes.6
A Plethora of PROMs
A 2019 review identified 160 PROMs in ophthalmology, including dozens targeted at specific ocular diseases as well as more generalized measures of eye and vision health.2 Dr. Hays has been involved in the development of many PROMs in ophthalmology and other medical specialties. He mentioned the following instruments as being of special interest:
NEI VFQ-25. The best-known PRO instrument in ophthalmology is the VFQ-25, which since its publication in 2000 has been used in dozens of ocular diseases and translated into more than 100 languages, said Dr. Chew. “In fact, it is the most frequently downloaded document from the NEI website,” she said. It was originally developed to assess patients with cataracts, age-related macular degeneration, diabetic retinopathy, primary open-angle glaucoma, cytomegalovirus retinitis, or low vision from any cause.7 Researchers have since recognized its usefulness in many other ocular conditions as well, and it has been used as a primary or secondary endpoint in dozens of clinical trials.
But this instrument has come under some criticism for not being in tune with current reality. For example, it includes questions about reading standard newsprint but does not ask anything about the use of cell phones or other mobile devices. Dr. Chew said that the next steps for the NEI VFQ-25 are not yet clear. “If we opt to revise the VFQ, it would require going through the process of focus groups and psychometric testing again—a very expensive proposition.”
Ocular Surface Disease Index (OSDI). Like the NEI VFQ, the OSDI was developed in the late 1990s and continues to appear as an endpoint in multiple studies. “It’s used very frequently, and it has stood the test of time,” said Dr. Hays. The OSDI has been used to assess not only dry eye syndrome per se but also situations including cataract and LASIK surgery and thyroid eye disease that can affect the ocular surface.
The OSDI and other PROMs for dry eye are particularly important because symptoms may be inconsistent with signs, said Michelle E. Tarver, MD, PhD, of the FDA. “Often, we’ll look at the cornea, and it is relatively clear, but the patient is having horrible dry eye symptoms.”
Patient-Reported Outcomes With LASIK Symptoms and Satisfaction (PROWL-SS). See sidebar “Close-up on PROWL.”
Assessment of IntraOcular Lens Implant Symptoms (AIOLIS). Like PROWL, the AIOLIS questionnaire is administered before and after a procedure (in this case, implantation of premium IOLs) to assess patient satisfaction and the possible emergence of postoperative symptoms.
The development of this instrument was remarkable for its collaboration among multiple stakeholders: the Academy, ophthalmologists, survey development experts from the RAND Corporation and UCLA, IOL manufacturers, and representatives of the FDA.8,9 (For further information about AIOLIS, see the feature article, “Four Refractive Challenges,” in the September 2023 EyeNet.)
NIH Toolbox Vision Survey. Dr. Hays and colleagues developed this questionnaire to be part of the larger NIH Toolbox, a comprehensive set of neurobehavioral measurements that assess cognitive, emotional, sensory, and motor functions. Although this survey covers a range of domains similar to the NEI VFQ-25, it is constructed using a different mathematical model known as item response theory (IRT).10 (Other models include classical test theory and Rasch analysis.)
In brief, IRT provides a method of developing a unified scale for responses to individual items of varying difficulty, which allows them to become part of item banks and to be used in computer-adaptive testing (CAT). CAT is an algorithm that selects appropriate successive questions for a respondent from an item bank based on their previous answers. Together, IRT and CAT hold the promise of streamlining the number of questions needed for accurate PROM assessments.11 Dr. Hays and colleagues described this approach as useful for creating “comprehensive and parsimonious” instruments.10 “With CAT, you can get really high reliability for most things you’re measuring with about five questions.”
Because of this adaptability and economy, Dr. Hays considers the NIH Toolbox Vision Survey to be the “logical next step” in vision-related PROMs, possibly as a successor to the NEI VFQ-25.
HOW IT WORKS. Incorporation of PROs into workflow.
Building Better PROMs: Challenges and Solutions
Concerns about scientific rigor. “PROMs are often criticized as being subjective,” said Dr. Tarver. “I would encourage people to understand that many of the outcome assessments we do in clinical prac-tice—such as visual acuity and visual field testing—have an element of subjectivity based on patient motivation and assessor engagement, but we still use that information to make decisions. And so, we shouldn’t discount so readily the structured assessment of patients reporting how they feel and function.”
Moreover, the development of well-designed PROMs incorporates a review of existing instruments and focus groups with patients to define the concepts of greatest importance. Dr. Tarver emphasized that “when you identify the need to develop a PROM, it’s important to include the people who are living with that condition so that they can be part of the research team … it’s essential to be intentionally inclusive of stakeholders at the outset to get to the most robust tool that will work for patients, providers, regulators, and other interested parties.”
After pilot questionnaires are developed, their psychometric properties are further tested in cognitive interviews with target populations. The developers incorporate these findings into later iterations, as necessary, and evaluate the PROM through psychometric analyses and statistical testing.3
Applicability across diverse patient groups. Differing eye diseases and levels of vision, ages, languages, cultures, health literacy, and reading skills—these are just a few of the differences among eye care patients that affect the development of appropriate PROMs. Dr. Tarver offered some suggestions for these situations:
- Poor vision. “Is the modality of administration appropriate? For example, rather than using a self-administered questionnaire on paper, you may have an interviewer help administer it or you may enable it with voice-activated text and do it on a computer so that patients can still participate and provide their insights.”
- Languages. “We encourage developers to con sider at the outset the major languages of the intended patient population and to translate questionnaires into those languages. Of course, it’s more than just translation. You check for linguistic equivalency and try to write questions that are free of idioms and other culturally nuanced language.”
- Health literacy and reading levels. “A U.S. Department of Education study in 2020 found that only 12% of Americans were considered health literate. We encourage developers to take the time to really involve individuals with diverse educational levels to ensure that the items are written in a way that people with a fifth- to eighth-grade level of education could complete those questionnaires. This can be accomplished through cognitive testing, which is a read-aloud process where people will talk about what they thought the question meant and how they are approaching their responses.”
Dr. Le added, “In designing PROMs, we should look through a health equity lens, too. Patients’ quality of life varies by social determinants of health that they experience, including their living and working environments. These need to be factored in.”
Burden on clinicians and patients. “I think that most clinicians would agree that having PROMs would be a good thing, but our clinical practices are just not set up to do that easily and effectively. Doing it routinely would take extra time and staff to capture the data,” said Dr. Wirostko. A long-term solution, she said, would be to have PROMs built into the electronic health records (EHRs) that physicians are already using. This would also allow them to be used for decision-making at the point of care.
Dr. Chew added that a major impediment is that there isn’t an incentive for the clinicians to do so. “If it were associated with quality measures, like the MIPS, they would be incentivized to make the extra effort.”
Regarding the possible burden that PROMs put on patients, Dr. Chew suggested that they could answer questionnaires on a tablet while waiting in the doctor’s office. “Patients are already used to answering questions about things like medications and family history. You could add to that.”
Also, Dr. Hays said, patients could use their cell phones or computers to access a link from the doctor’s office, log in to a patient portal, and complete questionnaires at home before coming in for an appointment. “And if the system uses CAT, the process can be highly efficient.”
NEW WAYS TO COMMUNICATE. Internet, mobile device, and wearables can aid in capturing patient-reported data.
Dominant themes that emerged from the interviewees about future directions include the following:
- Patients will have increasing options for reporting their outcomes (e.g., use of computers/tablets/ phones and even wearable devices in addition to paper questionnaires and face-to-face interviews).
- The use of item banks and CAT will continue to grow and become more important in PROMs instruments.
- EHRs and data registries will play a larger role in capturing and analyzing PROMs.
Finally, Dr. Tarver said that most clinicians may not think of themselves as researchers, but these developments may give them an opportunity “to become a part of the research enterprise. I would encourage clinicians to think of their practices as potential ‘cottage shops,’ where evidence can be generated, particularly with all that data feeding into the IRIS Registry. Such data could then be used by colleagues in ophthalmology for many different purposes,” she said. “It’s a team sport to improve patients’ lives.”
1 Patient Voice in Ophthalmology: AAO, Federal Agencies and You. Symposium presented at: Annual Meeting of the American Academy of Ophthalmology; Nov. 4, 2023; San Francisco. Sym68.
2 Braithwaite T et al. Patient Relat Outcome Meas. 2019;10:9-24.
3 Locklear T et al; NIH Collaboratory Coordinating Center. Patient-Reported Outcomes. January 2014.
4 Denniston AK et al. Eye (Lond). 2014;28(6):637-645.
5 Food and Drug Administration. Principles for Selecting, Developing, Modifying, and Adapting Patient-Reported Outcome Instruments for Use in Medical Device Evaluation. Issued Jan. 26, 2022.
6 Calvert M et al. JAMA. 2013;309(8):814-822.
7 Mangione CM et al. Arch Ophthalmol. 2001;119(7):1050-1058.
8 Masket S et al. Ophthalmology. 2023;130(7):726-734.
9 Hays RD et al. Ophthalmology. 2023;130(7):715-725.
10 Paz SH et al. Qual Life Res. 2013;22(9):2477-2487.
11 Cella D et al. J Clin Epidemiol. 2010;63(11):1179-1194.
Meet the Experts
Emily Y. Chew, MD Director of the Division of Epidemiology and Clinical Applications, National Eye Institute, Bethesda, Md. Relevant financial disclosures: None.
Ron D. Hays, PhD Distinguished Professor of Medicine and Professor of Health Policy and Management at the Fielding School of Public Health, University of California, Los Angeles. Relevant financial disclosures: None.
Jimmy T. Le, ScD Program Director for Collaborative Clinical Research, National Eye Institute, Bethesda, Md. Relevant financial disclosures: None.
Michelle E. Tarver, MD, PhD FDA Deputy Center Director, Chief Transformation Officer, Center for Devices and Radiological Health. Relevant financial disclosures: None.
Jayne S. Weiss, MD Professor and Chair of Ophthalmology, Professor of Pathology and Pharmacology, and Associate Dean for Clinical Affairs at LSU Health New Orleans School of Medicine; Chief Medical Officer of the LSU Healthcare Network; Voting Member of the Ophthalmic Devices Panel, FDA. Relevant financial disclosures: None.
Barbara M. Wirostko, MD Adjunct Professor of Ophthalmology and Biomedical Engineering at Moran Eye Center, University of Utah Health, Salt Lake City; Chief Medical Officer, Qlaris Bio, Wellesley, Mass; Founder and Medical Advisor, MyEyes, Park City, Utah. Relevant financial disclosures: None.
Full Financial Disclosures
Emily Y. Chew, MD None.
Ron D. Hays, PhD Agency for Healthcare Research and Quality: S; American Academy of Ophthalmology: S.
Jimmy T. Le, ScD None.
Michelle Tarver, MD, PhD None.
Jayne S. Weiss, MD Regeneron: C.
Barbara M. Wirostko, MD Glauconix: C, PS; Icare USA: C, L; MyEyes.net: C, EO; Qlaris Bio: P, EE, EO, PS.
|Consultant fee, paid advisory boards, or fees for attending a meeting.
|Hired to work for compensation or received a W2 from a company.
|Employee, executive role
|Hired to work in an executive role for compensation or received a W2 from a company.
|Owner of company
|Ownership or controlling interest in a company, other than stock.
|Contracted work, including contracted research.
|Lecture fees/Speakers bureau
|Lecture fees or honoraria, travel fees or reimbursements when speaking at the invitation of a commercial company.
|Beneficiary of patents and/or royalties for intellectual property.
|Equity/Stock/Stock options holder, private corporation
|Equity ownership, stock and/or stock options in privately owned firms, excluding mutual funds.
|Grant support or other financial support from all sources, including research support from government agencies (e.g., NIH), foundations, device manufacturers, and\or pharmaceutical companies. Research funding should be disclosed by the principal or named investigator even if your institution receives the grant and manages the funds.
|Stock options, public or private corporation
|Stock options in a public or private company.
|Equity/Stock holder, public corporation
|Equity ownership or stock in publicly traded firms, excluding mutual funds (listed on the stock exchange).