Lifting the Patient’s Voice in Ophthalmology

Download PDF

Although patient-reported outcome measures (PROMs) are not new to ophthalmology, they have gar­nered greater visibility and interest recently. As David W. Parke II, MD, former Academy CEO, stated in his opening remarks at an AAO 2023 symposium, “This is as much the future of the practice of ophthalmology, the science of oph­thalmology, as anything out there right now.”¹ Similarly, the U.S. Food and Drug Administration and National Eye Institute highlighted the impor­tance of PROMs in their September 2023 joint virtual workshop on Patient-Reported Outcomes and Vision-Related Quality of Life Questionnaires exploring the progress in and future directions for PROMs in ophthalmology.

Despite this growing interest, some barriers re­main to their wider adoption. Concerns about the use of PROMs include their validity, applicability across diverse groups, and burden on patients and clinical practices in the collection of responses.²

Here is a look at some important PROMs now in use, some of the current barriers and possible solutions, and how technology can help increase the use of PROMs in research and clinical practice.

What Are PROs and PROMs?

The FDA defines patient-reported outcomes (PROs) as “any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else,” and can in­clude information about quality of life, symptoms, function, satisfaction with care, and perceived value of treatment.³

By this definition, “PROs are a regular part of our clinical practice, anytime we ask our patients how they’re tolerating their eye drops and how they are feeling or functioning vision-wise,” said Barbara M. Wirostko, MD, at Moran Eye Center in Salt Lake City.

However, more standardized and validated outcome measures—PROMs—are necessary to capture, analyze, and incorporate patient voices into structured data that can be used in research, product development and approvals, and evidence-based clinical practice. PROMs consist of individ­ual questions, or items, that are further grouped into categories of related concepts, called domains, and are scored on predefined multi-item scales. Although these questionnaires were historically completed on paper, they are now also adminis­tered through other modes such as the internet and mobile devices.⁴

PATIENT PERSPECTIVES IN DEVICE DEVELOPMENT. This schematic depicts the multiple points at which patient-reported data can inform research, development, and approval of devices.

How Are PROMs Used?

Physician-patient relationship. “Our role as physicians is to treat the patient, not just the disease,” said Emily Y. Chew, MD, at the National Eye Institute, in Bethesda, Maryland. “Improving the patient’s quality of life is one of the strategic pillars—and now part of the mission statement—of the NEI. We wanted to make eye care really patient-centric. We started that tradition 25 years ago when we developed the NEI Visual Function Questionnaire [VFQ].”

Jayne S. Weiss, MD, at the LSU Health New Orleans, LSU–School of Medicine, said, “For the clinician, PROMs are important because they give us information on what really matters to the patient—not just what the ophthalmologist sees or thinks. It also allows us to monitor the efficacy of whatever procedure we did, and the patient’s satisfaction with it.” She added that the latter is especially important for elective treatments such as LASIK or the use of multifocal lenses, where patient expectations are high.

Ultimately, “PROMs provide an opportunity to empower patients who have an ocular disease or are visually impaired,” said Jimmy T. Le, ScD, at the NEI. “How do we determine whether our treatments are working to improve patients’ quality of life? That’s the question we’re trying to answer with these instruments.”

Drug and device development. Dr. Chew not­ed that the European Medicines Agency requires patient-reported outcomes for marketing autho­rization. “So, for example, all the diabetic studies they’ve done use the NEI VFQ-25 as part of that. And in the U.K., they involve patients from the very beginning of the study design.”

Even though PROMs are not required for approval in the United States, Dr. Chew contin­ued, “The FDA colleagues we’re working with are very interested in making the process continually more patient-centered.”

Indeed, over the past 15 years, the FDA has been is­suing and updating guidance documents for industry and other stakeholders on the use of PROMs in product approval and labeling. In these documents, the agency states that the systematic col­lection of patient-reported data provides “valid scientific evidence to support the regulatory and health care decision-making process.”⁵

Ron D. Hays, PhD, at the University of Califor­nia, Los Angeles, said that the only downside of working with the FDA on PROM development is that it can slow things down because the agency needs to go through its internal process. But, he said, “they provide wonderful feedback, and they’re more actively involved than most coauthors I know. So, once it gets through that process it’s usually a good measure.”

As the chief medical officer of a biotechnology company, Dr. Wirostko has extensive experience in drug and device development. She observed that the creation of PROMs can often be driven by biotech companies that have an innovation that provides a benefit in terms of treatment burden, patient satisfaction, or perceived value of treatment.

“For example, consider two hypothetical intra­vitreal drugs, one of which is dosed every month and the other every six months. They may have the same efficacy and safety profiles, but patients and doctors are likely to have different perceptions of these two drugs. It could even affect adherence to treatment. How can you capture that difference in the patient’s quality of life? That’s the value of PROMs.”

Clinical research and publication. PROMs are increasingly being used as secondary and even primary endpoints in randomized clinical trials (RCTs). According to one review, the subspecial­ties that have made greatest progress in develop­ing PROMs and incorporating PROMs into RCTs are glaucoma, medical retina, cataract/IOL, and low vision.²

In 2013, an extension specific to the use of PROMs in RCTs was added to the CONSORT (Consolidated Standards of Reporting in Clinical Trials) guidelines in the hope of bringing the reporting of PROMs to the same level as that of other outcomes.⁶

Close-up on PROWL

A unique status. The PROWL-SS questionnaire has the distinction of being the only PROMs instrument in ophthalmology that is currently qualified by the FDA. Dr. Tarver said, “It means that the FDA has looked at the evidence and accepted it as a Medical Device Development Tool [MDDT], allowing it to be used in the same manner in other trials without further validation. This builds efficiencies into research, develop­ment, and approval.” The Academy played a leading role in this process.¹

Difficult origins. This instrument had its genesis in a highly contentious environment. Dr. Weiss recounted that she was chair of the FDA Ophthalmic Devices Panel when a special meeting was called in April 2008 to respond to complaints from the public about LASIK.

“For this meeting, the FDA brought together multiple individuals who testified: patients and other members of the public, ophthalmic sur­geons, and [delegates from] various parts of organized ophthalmology. Many representa­tives of the news media were also there, and my recollection as chair of the meeting is that the atmosphere was incendiary.

“The contention among some patients and groups was that it was causing potentially severe problems that weren’t being acknowl­edged, and even some suicides that were alleged to be associated with having had LASIK.

“On the other hand, ophthalmologists and some patients testified to a very high level of satisfaction with the procedure. So, at the end of the day, which was true? Was it that most people were happy, or was it that we were not acknowledging all that was going wrong with this procedure?”

HALOS. The PROWL questionnaire uses photos to help assess the degree of halo that a LASIK patient experiences.

Next steps. In response, in 2009, the FDA and NEI launched a joint LASIK Quality of Life Collaboration Project to better understand the risk of severe problems. An essential component was to develop a tool—the PROWL questionnaire—for determining the percentage of patients who experienced difficulties after LASIK and to identify factors that might be linked with post­operative problems.²

Key features. Unlike most PROMs, “the difference with this was to have people fill out the questionnaire before and after the LASIK procedure,” said Dr. Weiss. So if a patient had post-LASIK problems with glare, dry eyes, or irritation, she added, it was important to know if these issues were present before the procedure.

The studies used web-based questionnaires. “And we learned an interesting thing,” Dr. Weiss said, “people were more than twice as likely to report their visual symptoms on the question­naire than telling their doctor.”

Another important feature, which was unusual at the time, is the use of photographs, rather than just words, to identify visual symptoms. (The recent AIOLIS instrument also incorpo­rates images.)

Outcomes. The initial study (PROWL-1), involving 262 participants, was launched at the U.S. Naval Medical Center Refractive Surgery Clinic, and the second trial (PROWL-2) was conducted among civilians (n = 312) at multiple centers.

The investigators found that the mean satis­faction with surgery rate was 93% in PROWL-1 at six months and 91% in PROWL-2 at three months, while the rates of dissatisfaction were 1% to 2% in both studies. However, up to 43% of patients in PROWL-1 and 46% of patients in PROWL-2 who had not reported visual symp­toms preoperatively reported at least a new visual symptom at three months after surgery, most often halos and starbursts.³

Interestingly, she said, there was a low cor­relation between the visual symptoms scores at three and six months with objective mea­surements such as postoperative uncorrected VA, optical aberrations, spherical equivalent in those who had preoperative myopia, and post­operative cylinder.

“Ultimately, these studies supported the overall safety and patient satisfaction with LASIK but also underscored the importance of preoperative patient counseling and informed consent,” said Dr. Weiss.

___________________________

1 Patient-Reported Outcomes With LASIK Symptoms and Satisfaction (PROWL-SS) and Scoring Guide. aao.org/education/prowl-ss. Accessed Nov. 15, 2023.

2 LASIK Quality of Life Collaboration Project. www.fda.gov/medical-devices/lasik/lasik-quality-life-collaboration-project. Accessed Nov. 15, 2023.

3 Eydelman M et al. JAMA Ophthalmol. 2017;135(1):13-22.

A Plethora of PROMs

A 2019 review identified 160 PROMs in ophthalmology, including dozens targeted at specific ocular diseases as well as more generalized measures of eye and vision health.² Dr. Hays has been involved in the development of many PROMs in ophthalmology and other medical specialties. He men­tioned the following instruments as being of special interest:

NEI VFQ-25. The best-known PRO instrument in ophthalmology is the VFQ-25, which since its publication in 2000 has been used in dozens of ocular diseases and translated into more than 100 languages, said Dr. Chew. “In fact, it is the most frequently downloaded document from the NEI website,” she said. It was originally developed to assess patients with cataracts, age-related macu­lar degeneration, diabetic retinopathy, primary open-angle glaucoma, cytomegalovirus retinitis, or low vision from any cause.⁷ Researchers have since recognized its usefulness in many other ocular conditions as well, and it has been used as a primary or secondary endpoint in dozens of clinical trials.

But this instrument has come under some criti­cism for not being in tune with current reality. For example, it includes questions about reading stan­dard newsprint but does not ask anything about the use of cell phones or other mobile devices. Dr. Chew said that the next steps for the NEI VFQ-25 are not yet clear. “If we opt to revise the VFQ, it would require going through the process of focus groups and psychometric testing again—a very expensive proposition.”

Ocular Surface Disease Index (OSDI). Like the NEI VFQ, the OSDI was developed in the late 1990s and continues to appear as an endpoint in multiple studies. “It’s used very frequently, and it has stood the test of time,” said Dr. Hays. The OSDI has been used to assess not only dry eye syndrome per se but also situations including cat­aract and LASIK surgery and thyroid eye disease that can affect the ocular surface.

The OSDI and other PROMs for dry eye are particularly important because symptoms may be inconsistent with signs, said Michelle E. Tarver, MD, PhD, of the FDA. “Often, we’ll look at the cornea, and it is relatively clear, but the patient is having horrible dry eye symptoms.”

Patient-Reported Outcomes With LASIK Symptoms and Satisfaction (PROWL-SS). See sidebar “Close-up on PROWL.”

Assessment of IntraOcular Lens Implant Symptoms (AIOLIS). Like PROWL, the AIOLIS questionnaire is administered before and after a procedure (in this case, implantation of premium IOLs) to assess patient satisfaction and the possible emergence of postoperative symptoms.

The development of this instrument was remarkable for its collaboration among multi­ple stakeholders: the Academy, ophthalmologists, survey development experts from the RAND Corporation and UCLA, IOL manufacturers, and representatives of the FDA.^8,9 (For further information about AIOLIS, see the feature article, “Four Refractive Challenges,” in the September 2023 EyeNet.)

NIH Toolbox Vision Survey. Dr. Hays and col­leagues developed this questionnaire to be part of the larger NIH Toolbox, a comprehensive set of neurobehavioral measurements that assess cog­nitive, emotional, sensory, and motor functions. Although this survey covers a range of domains similar to the NEI VFQ-25, it is constructed using a different mathematical model known as item response theory (IRT).¹⁰ (Other models include classical test theory and Rasch analysis.)

In brief, IRT provides a method of developing a unified scale for responses to individual items of varying difficulty, which allows them to become part of item banks and to be used in computer-adaptive testing (CAT). CAT is an algorithm that selects appropriate successive questions for a respondent from an item bank based on their pre­vious answers. Together, IRT and CAT hold the promise of streamlining the number of questions needed for accurate PROM assessments.¹¹ Dr. Hays and colleagues described this approach as useful for creating “comprehensive and parsi­monious” instruments.¹⁰ “With CAT, you can get really high reliability for most things you’re measuring with about five questions.”

Because of this adaptability and economy, Dr. Hays considers the NIH Toolbox Vision Survey to be the “logical next step” in vision-related PROMs, possibly as a successor to the NEI VFQ-25.

HOW IT WORKS. Incorporation of PROs into workflow.

Building Better PROMs: Challenges and Solutions

Concerns about scientific rigor. “PROMs are often criticized as being subjective,” said Dr. Tarver. “I would encourage people to understand that many of the outcome assessments we do in clinical prac-tice—such as visual acuity and visual field testing—have an element of subjectivity based on patient motivation and assessor engagement, but we still use that information to make decisions. And so, we shouldn’t discount so readily the structured assessment of patients reporting how they feel and function.”

Moreover, the development of well-designed PROMs incorporates a review of existing instru­ments and focus groups with patients to define the concepts of greatest importance. Dr. Tarver emphasized that “when you identify the need to develop a PROM, it’s important to include the people who are living with that condition so that they can be part of the research team … it’s essen­tial to be intentionally inclusive of stakeholders at the outset to get to the most robust tool that will work for patients, providers, regulators, and other interested parties.”

After pilot questionnaires are developed, their psychometric proper­ties are further tested in cognitive interviews with target populations. The developers incorporate these findings into lat­er iterations, as necessary, and evaluate the PROM through psychometric analyses and statistical testing.³

Applicability across diverse patient groups. Differing eye diseases and levels of vision, ages, languages, cultures, health literacy, and reading skills—these are just a few of the differences among eye care patients that affect the develop­ment of appropriate PROMs. Dr. Tarver offered some suggestions for these situations:

• Poor vision. “Is the modality of administra­tion appropriate? For example, rather than using a self-administered questionnaire on paper, you may have an interviewer help administer it or you may enable it with voice-activated text and do it on a computer so that patients can still participate and provide their insights.”
• Languages. “We encourage developers to con sider at the outset the major languages of the intended patient population and to translate questionnaires into those languages. Of course, it’s more than just translation. You check for linguistic equivalency and try to write questions that are free of idioms and other culturally nuanced language.”
• Health literacy and reading levels. “A U.S. Department of Education study in 2020 found that only 12% of Americans were considered health literate. We encourage developers to take the time to really involve individuals with diverse educational levels to ensure that the items are written in a way that people with a fifth- to eighth-grade level of education could complete those questionnaires. This can be accomplished through cognitive testing, which is a read-aloud process where people will talk about what they thought the question meant and how they are approaching their responses.”

Dr. Le added, “In designing PROMs, we should look through a health equity lens, too. Patients’ quality of life varies by social determinants of health that they experience, including their living and working environments. These need to be factored in.”

Burden on clinicians and patients. “I think that most clinicians would agree that having PROMs would be a good thing, but our clinical practices are just not set up to do that easily and effectively. Doing it rou­tinely would take extra time and staff to capture the data,” said Dr. Wirostko. A long-term solution, she said, would be to have PROMs built into the electronic health records (EHRs) that physicians are already using. This would also allow them to be used for decision-making at the point of care.

Dr. Chew added that a major impediment is that there isn’t an incentive for the clinicians to do so. “If it were associated with quality measures, like the MIPS, they would be incentivized to make the extra effort.”

Regarding the possible burden that PROMs put on patients, Dr. Chew suggested that they could answer questionnaires on a tablet while waiting in the doctor’s office. “Patients are already used to answering questions about things like medications and family history. You could add to that.”

Also, Dr. Hays said, patients could use their cell phones or computers to access a link from the doctor’s office, log in to a patient portal, and complete questionnaires at home before coming in for an appointment. “And if the system uses CAT, the process can be highly efficient.”

NEW WAYS TO COMMUNICATE. Internet, mobile device, and wearables can aid in capturing patient-reported data.

Looking Ahead

Dominant themes that emerged from the interview­ees about future directions include the following:

• Patients will have increasing options for report­ing their outcomes (e.g., use of computers/tablets/ phones and even wearable devices in addition to paper questionnaires and face-to-face interviews).
• The use of item banks and CAT will continue to grow and become more important in PROMs instruments.
• EHRs and data registries will play a larger role in capturing and analyzing PROMs.

Finally, Dr. Tarver said that most clinicians may not think of themselves as researchers, but these developments may give them an opportunity “to become a part of the research enterprise. I would encourage clinicians to think of their practices as potential ‘cottage shops,’ where evidence can be generated, particularly with all that data feeding into the IRIS Registry. Such data could then be used by colleagues in ophthalmology for many different purposes,” she said. “It’s a team sport to improve patients’ lives.”

___________________________

1 Patient Voice in Ophthalmology: AAO, Federal Agencies and You. Symposium presented at: Annual Meeting of the Ameri­can Academy of Ophthalmology; Nov. 4, 2023; San Francisco. Sym68.

2 Braithwaite T et al. Patient Relat Outcome Meas. 2019;10:9-24.

3 Locklear T et al; NIH Collaboratory Coordinating Center. Patient-Reported Outcomes. January 2014.

4 Denniston AK et al. Eye (Lond). 2014;28(6):637-645.

5 Food and Drug Administration. Principles for Selecting, De­veloping, Modifying, and Adapting Patient-Reported Outcome Instruments for Use in Medical Device Evaluation. Issued Jan. 26, 2022.

6 Calvert M et al. JAMA. 2013;309(8):814-822.

7 Mangione CM et al. Arch Ophthalmol. 2001;119(7):1050-1058.

8 Masket S et al. Ophthalmology. 2023;130(7):726-734.

9 Hays RD et al. Ophthalmology. 2023;130(7):715-725.

10 Paz SH et al. Qual Life Res. 2013;22(9):2477-2487.

11 Cella D et al. J Clin Epidemiol. 2010;63(11):1179-1194.

Meet the Experts

Emily Y. Chew, MD Director of the Division of Epidemiology and Clinical Applications, National Eye Institute, Bethesda, Md. Relevant financial disclosures: None.

Ron D. Hays, PhD Distinguished Professor of Medicine and Profes­sor of Health Policy and Manage­ment at the Fielding School of Public Health, University of Califor­nia, Los Angeles. Relevant financial disclosures: None.

Jimmy T. Le, ScD Program Di­rector for Collaborative Clinical Research, National Eye Institute, Bethesda, Md. Relevant financial disclosures: None.

Michelle E. Tarver, MD, PhD FDA Deputy Center Director, Chief Transformation Officer, Center for Devices and Radiological Health. Relevant financial disclosures: None.

Jayne S. Weiss, MD Professor and Chair of Ophthalmology, Professor of Pathology and Pharmacology, and Associate Dean for Clinical Affairs at LSU Health New Orleans School of Medicine; Chief Medical Officer of the LSU Healthcare Network; Voting Member of the Ophthalmic Devices Panel, FDA. Relevant financial disclosures: None.

Barbara M. Wirostko, MD Adjunct Professor of Ophthalmology and Biomedical Engineering at Moran Eye Center, University of Utah Health, Salt Lake City; Chief Medi­cal Officer, Qlaris Bio, Wellesley, Mass; Founder and Medical Advisor, MyEyes, Park City, Utah. Relevant financial disclosures: None.