NPC, Quintiles Study Tests Tools to Evaluate Quality of Observational Research in Comparative Effectiveness Studies

WASHINGTON, Feb. 25, 2014 /PRNewswire-USNewswire/ -- With the growing demand for information about comparative effectiveness and questions about the quality of research that will support future health care decision-making, a group of researchers led by Quintiles and the National Pharmaceutical Council (NPC) have developed and tested a checklist to evaluate the quality and usefulness of observational research studies. The Good ReseArch for Comparative Effectiveness (GRACE) checklist was published as part of a peer-reviewed study in the March issue of the Journal of Managed Care & Specialty Pharmacy.

Observational studies are meant to help decision-makers understand whether a treatment will work for typical patients in routine care environments, and they can provide valuable information about patients and situations not studied in traditional randomized clinical trials. These observational studies are often performed prospectively, observing patients as they are treated and evaluating subsequent benefits and risks, or they can be retrospective analyses of existing data, such as from medical chart reviews or databases. Thousands of observational studies are undertaken each year, but the quality of these studies varies considerably because there is a lack of agreed-upon standards for the conduct and evaluation of this research in the context of a particular study purpose.  In the absence of such guidance, people often rely on studies used to bring new products to market, i.e., randomized clinical trials performed on highly selected, homogeneous patient groups who are usually treated by highly skilled clinicians, often at large academic centers.  As a result, there is a major gap in evidence for what happens when other types of patients are treated by clinicians who may be less experienced and in settings that may have fewer resources to support clinical care.

The GRACE checklist contains 11 items about data and research methods that can be used as an initial screening tool to separate observational studies that meet baseline quality criteria from those that do not. The GRACE checklist is "based on existing literature and guidance from experts with extensive experience in the conduct and utilization of observational comparative effectiveness research," and is meant to help determine which observational studies should be considered to support decision-making. This checklist is an outgrowth from the GRACE Principles, a set of high-level questions that lay out the elements of good practice for the design, conduct, analysis and reporting of observational CER studies (see www.graceprinciples.org).

"The GRACE checklist stands out because of this testing, conducted by a diverse group of collaborators from public and private sectors in North and South America, Europe, Asia and Africa. This work shows the value of the checklist for distinguishing real-world studies that are good enough for decision support," said Dr. Nancy Dreyer, Global Chief of Scientific Affairs and SVP, Quintiles, and lead author of the study. "The checklist is part of our larger efforts to promote appropriate use of real-world studies that couple good data with strong scientific methods to provide answers to questions about which treatments work best, for whom, and in what situations. The more we are able to demonstrate the value of real-world research for various purposes, the better informed decision-makers will be, which should benefit all of us."

In their study, the authors acknowledge the importance of "having an agreed-upon set of assessment elements, check lists or score cards," especially with the expected growth in funding for observational studies, and the importance of evaluating studies in the context of the purpose for which they will be used.  The authors noted that this GRACE checklist is intended to evaluate individual studies, and not as a support for meta-analysis.

"With the wide range in quality of observational studies, it will be helpful for decision-makers to be able to evaluate studies in the same way," said Robert Dubois, MD, PhD, chief science officer, National Pharmaceutical Council, and a study author. "This will give us a better idea of what 'good' looks like when it comes to quality evidence used to make treatment and coverage decisions."

About the National Pharmaceutical Council

The National Pharmaceutical Council is a health policy research organization dedicated to the advancement of good evidence and science, and to fostering an environment in the United Sates that supports medical innovation. Founded in 1953 and supported by the nation's major research-based pharmaceutical companies, NPC focuses on research development, information dissemination, and education on the critical issues of evidence, innovation and the value of medicines for patients. For more information visit www.npcnow.org and follow NPC on Twitter @npcnow.

SOURCE National Pharmaceutical Council