A new study into how well real-world evidence could replicate clinical trials published over the course of a year highlights some of the limitations that still exist.
The study, published in the Journal of the American Medical Association, found that only 15 percent of clinical trials included in the analysis could be feasibly replicated using currently available real-world data sources. Despite opportunities to supplement or even replace some trials, “observational methods are not likely to obviate the need for traditional clinical trials,” the researchers wrote. At the same time, they wrote that the data suggest that RWE could complement clinical trials, through examining concordance between randomized and observational studies and comparing the generalizability of a trial population with the real-world population.
RWE comes from analysis of data from sources like electronic health records, patient-reported outcome measures, insurance claims, medical devices and mobile devices. Under the 21st Century Cures Act of 2016, the Food and Drug Administration has prioritized research into RWE and how it can support regulatory decision making.
To conduct the study, researchers at the Mayo Clinic, Yale University, University of California San Francisco and Columbia University searched PubMed to compile a sample of 220 trials – whether or not they were randomized – published across the seven highest-impact medical journals in 2017. They found that only 33 of the studies could be replicated using observational data. That was because for the most part, things like primary endpoints, inclusion and exclusion criteria, indication and intervention – all crucial factors in any clinical trial – could not be routinely ascertained from real-world data sources.
“I think there are lots of ways that RWE will be used for medical product evaluation, but for these data to be a suitable replacement, work is still needed to enhance the data that are available,” one of the researchers, Yale professor of medicine Dr. Joseph Ross, wrote in an email. “Working with current real-world data sources, I believe large, safety-oriented evaluations are useful and feasible. But I think more needs to be done to better discern product effectiveness using real-world data.”
Ross noted that his team is doing a lot of work with the FDA and National Evaluation System for health Technology Coordinating Center to test whether aggregating multiple real-world data sources – including EHR and pharmacy data, PROMs and data from digital devices – can be used to support pragmatic clinical trials.
A Deep-dive Into Specialty Pharma
A specialty drug is a class of prescription medications used to treat complex, chronic or rare medical conditions. Although this classification was originally intended to define the treatment of rare, also termed “orphan” diseases, affecting fewer than 200,000 people in the US, more recently, specialty drugs have emerged as the cornerstone of treatment for chronic and complex diseases such as cancer, autoimmune conditions, diabetes, hepatitis C, and HIV/AIDS.
Jeremy Rassen, president of New York-based RWE technology company Aetion, wrote in an email that rather than viewing clinical trials — especially randomized, controlled trials — and RWE as a dichotomy, he sees three different categories. In one category are cases where RCTs are clearly necessary, such as pivotal trials. The second category includes cases where trials for which RWE could potentially serve as a substitute, such as post-marketing cardiovascular outcomes studies or evidence for label expansions. And the third category consists of cases where only RWE can answer a question, such as measuring the performance of a medication in routine use outside a clinical trial.
“In the end, whether the number is 15 percent or 50 percent is a bit irrelevant, as the choice of the denominator — that is, which studies are being considered as part of ‘trials that could be reproduced’ — varies so much based on how you think about the question,” Rassen wrote.
Photo: oonal, Getty Images
