Oded Langer, MD, PhD
Nieli Langer, PhD
Where is the wisdom we have lost in knowledge and where is the knowledge we have lost in information?
—T.S. Eliot
Key Points
• Evidence-based medical practice (EBMP) may provide:
1. Encouragement for rigorous testing of practice-related claims regarding effectiveness
2. Means for disseminating practice-related research findings
3. Enhanced opportunities for doing more good than harm
• EBMP is a bridge between external clinical evidence and individual clinical practice.
• EBMP has many partners and entangling alliances.
INTRODUCTION
The goal of education and research in all disciplines is to develop critical thinking skills as a method for improving clinical decision making. Critical thinkers explore their own attitudes and values, investigate and analyze competing alternatives, and are motivated to articulate their point of view. The emphasis on critical thinking is nothing new and can be traced back to ancient times where Socrates believed in education by interrogating rather than by propounding. Socrates challenged his students to think about their knowledge, beliefs, and behaviors. It is widely known that Socrates would press his students until they could provide evidence to support their arguments and would dismiss those beliefs and decisions that could not be supported with proof.
Evidence-based medical practice (EBMP) originated in health care in the mid-twentieth century as an alternative to authority-based practice (i.e., basing decisions on so-called experts’ opinions). EBMP offers practitioners and administrators a foundation that is compatible with professional codes of ethics (i.e., for informed consent) and educational accreditation policies and standards. Although most people engaged in meaningful careers in health care will, in all probability, never conduct empirical research, they will be reading research articles in their professional journals that describe issues relevant to their practices. EBMP is designed to enhance practitioners’ ability to be good consumers of research. If practitioners are not familiar with up-to-date evidentiary practices and policies, they are not providing their patients with the best medical alternatives. Moreover, they cannot honor informed consent obligations to provide best possible care. To access, analyze, and apply research findings in health care, practitioners will need to understand why, by whom, and how research studies are conducted.
In light of the above, it is amazing that it took until the 1990s when a group of clinicians and epidemiologists at McMaster University in Ontario, Canada, officially coined the term “evidence-based medicine.” We cannot help but smile and believe that Socrates would look favorably on the evolution of EBMP while reminding us that we need additional evidence and the dissemination of critical thinking skills to support its use.
EBMP is a medical movement based on the application of the scientific method to medical practice, including long-established existing medical traditions not yet subjected to adequate scientific scrutiny. It originated because of gaps among evidentiary, ethical, and application concerns. From the beginning, the concept faced mixed reviews: excitement from researchers and resentment from health care practitioners who deemed it impractical in busy medical offices. Our attempts since ancient times have been to increase medical knowledge and enhance the level of medical care. The 21st century has witnessed the confluence of an accumulation of knowledge, in addition to the tools to access and deliver the fruits of this knowledge to all interested health care providers.
THE BRIDGE BETWEEN EXTERNAL CLINICAL EVIDENCE AND INDIVIDUAL CLINICAL PRACTICE
Advantages of Randomized Clinical Trials
It took until the middle of the 20th century before medical science was to help facilitate the evolution of the randomized clinical trial (RCT) that generates some of the information that becomes evidence. In medicine, since the randomized controlled trial, when conducted under the appropriate conditions, is so much more likely to inform us and so much less likely to mislead us, it has become the gold standard for judging whether a treatment does more good than harm.1 As the least biased form of medical evidence, the RCT offers many advantages. It provides the strongest evidence of causality and represents the best methodology to test the effectiveness of an intervention, that is, the extent to which an intervention, procedure, or treatment regimen produces a desired effect when deployed in the field in routine circumstances.2 When performed with an adequate sample size, randomization protects against selection bias and confounding variables.
Limitations of RCTs
There is, however, an increasing recognition of the limits of randomized controlled trials. Although RCTs can determine the effectiveness of an intervention in an experimental setting, different methods of research may be required to determine whether any harmful effects exist or to examine how patients experience any interventions they receive. In addition, randomized trials are expensive, not always feasible, and in some cases inappropriate to perform for ethical reasons.
EBMP involves tracking down the best external evidence with which to answer clinical questions. To determine the accuracy of a diagnostic test, cross-sectional studies of patients clinically suspected of harboring the relevant disorder is needed, not a randomized trial. When studying prognosis, even after a RCT, proper follow-up studies of patients assembled at a uniform early point in the clinical course of their disease is advisable. If no randomized trial has been conducted for an illness or complication, researchers and practitioners seek the next best external evidence and work from there.3
CATEGORIES OF RESEARCH DESIGNS
A basic understanding of common methods of research design is necessary to interpret the evidence presented in a research study. The case report describes an unexpected event to test whether it is a chance or regularly recurring phenomenon that needs further investigation. The report would have to address the likelihood of this phenomenon occurring by chance and if the event was predictable from any theoretical or empirical observation. The uncontrolled case series is a weightier case report since the event has occurred on numerous occasions; but, the need to address the same concerns as above applies. The rationale for conducting a case-controlled study is the potential to compare a selected endpoint in the study group to an external reference in the general population. However, it is important to evaluate the selection process to create the matched-control, that is, what biases could have influenced a person being designated a case or control and how representative were the subjects?
Studies using birth certificates and health insurance claims to generate data have become popular. Birth certificates provide data collected for civil and legal purposes, not for research. Administrative databases, that is, billing systems or state mandated record keeping structures were not created for epidemiological research.4 Misclassification is common; some procedure claims are not accurately recorded and are obtained by nonmedical personnel; some procedures that are provided are not always billed and, therefore, do not appear on the record. As a result, important reproductive health-related information, such as type of birth defects and specific hypertension drug used in treatment, may not be routinely recorded; comorbidities such as diabetes are poorly recorded; and the type of diabetes is not specified (gestational diabetes mellitus [GDM], type 1 and type 2); in addition, level of glycemia, body mass index, and diabetic treatment employed is not available in the database for extraction; the type of medication is not specified providing only a general classification, that is, oral agent. As a result, researchers extracting data from these records have automatically speculated that glyburide had been administered to all patients when in fact there are currently several oral agents routinely used that could have been pre- scribed.4,5 The veiled threat to junior faculty by their older colleagues to “publish or perish...” has often sanctioned the expedient method of obtaining data from administrative databases for epidemiologic research with the resultant “garbage in, garbage out” data obtained.6
Of the thousands of diabetes in pregnancy studies that have been published, the Cochrane Register of Clinical Trials has identified only 103 that were described as randomized trials. Of these, 28 studies were excluded. They failed to report information relevant to pregnancy compromised by diabetes and, in some cases, the publications reported on the same randomized trial. In general, the majority of proposed interventions can only achieve about 25%-35% reduction in a selected endpoint (i.e., macrosomia, shoulder dystocia). Therefore, the number of women who have to be recruited to prove that an intervention actually achieves its intended goal would have to be larger than the number currently reported in the majority of studies on diabetes in pregnancy.
The Effect of Research Methodology on Study Conclusions
The selection of the research design, the calculated sample size, and the level of glycemic control achieved in a given study are all potential confounders for study conclusions. The larger the sample size and the anticipated magnitude of the intervention, the greater the power, that is, the percent chance that the study will detect a significant difference when there is an actual difference. However, a study with a small sample size that suggests a statistical difference runs the risk of an alpha error, that is, the probability of a study showing a statistically significant difference when no real difference exists. In addition, the rate of a complication or the result of an intervention lower than expected by the acknowledged prevalence such as 50% anomalies with a small sample size raises the issue of selection bias. On the other hand, when study results do not reveal a difference in perinatal mortality, birth trauma, or shoulder dystocia, it does not mean that an important clinical difference does not exist. The failure of the study to provide evidence of a difference should not be confused with evidence of no difference. Therefore, a beta error is the probability of failing to show a statistically significant difference when a true difference exits (false negative).
Composite outcomes are those in which several individual outcomes are pooled to produce a single outcome. As the number of individual adverse outcomes decline in light of improved treatments, the use of composite outcomes can overcome this drop by combining different outcomes and enhancing the efficiency of a clinical trial. Outcome selection should obviously translate into a clinically important long-term outcome. It should be noted, however, that using composite outcomes does not necessarily lead to increased evidence of the benefit of a specified intervention. In addition, each element of the composite outcome needs to be presented as a secondary outcome so that practitioners can determine the efficacy of these outcomes in their clinical practices. When there are limited available resources for clinical trials, composite outcomes is an efficient and appropriate design solution that may also best reflect a real clinical outcome.7
It should be noted that even after an adequate sample size has been drawn, or the likelihood of making either an alpha or beta error are small, information regarding level of glycemic control throughout pregnancy, timing of diagnosis, and onset of therapy and methods of measuring levels of glycemic control can be serious confounders that alter the results of a study. See Chapters 11 and 12 in the text for appropriate examples and specific studies.
In reproductive literature, cohort, case-control and crosssectional studies are common since many research questions cannot be addressed with an RCT. These observational studies are more prone to bias than a RCT. Goodman8 suggests that, “. in identifying reasons for our scientific beliefs, we also want to know how strong a warrant they provide: how good are the reasons and how good must they be to compel us to revise our beliefs?” Therefore, once academicians and clinicians are convinced of the veracity of evidence, staying abreast of research in the field and/or one’s medical specialty becomes a moral imperative with its foundations in both the Hippocratic Oath and the Oath of Maimonides. The contribution that Archie Cochrane made to the evolution of scientific methodology in the 1970s was to make the evidence less removed or disconnected from those people who should be using it to take care of sick people. Today, the Cochrane and Campbell Collaborations provide an evolving source of database tools and ideas to facilitate this enterprise.9
With the advent of the RCT and the ascendancy of the databases for retrieval of information, the research community sought a means to develop some strategy for sifting, organizing, collating, and arranging this knowledge of variable quality or reliability. One effort to address the problem was an attempt to rank “levels of evidence” according to different aspects of clinical practice, including therapy, prognosis, diagnosis, and so forth. The Oxford Center for Evidence-Based Medicine does this by stratifying levels of evidence based on degrees of methodological power and advantage based on the original efforts of the Canadian Task Force on the Periodic Health Examination (1979).10 The US Preventive Services Task Force11 (1996) has also adopted specific criteria for the evaluation of the quality of evidence.
EBMP involves a shift in paradigms. Historically, practitioners have relied primarily on their more experienced colleagues and supervisors, expert opinions, and their own personal experiences for professional guidance—subjective information sources that too often provided inaccurate and even harmful practice guidelines.12 A charismatic spokesperson or “expert” may have tremendous influence on his peers, on policy makers, and/or the public. When all methods appear to be equally effective and those who depend on the information are not sure which direction to take, the vacuum is filled by an “expert” who has the oratorical and persuasive powers to say what is and what is not effective practice. The information, however, may be based on biased opinion and conflict of interest but not necessarily the facts. Consumers of research evidence need to ensure that the credentials of a seemingly notable scholar from a prestigious institution do not overawe them.
Advocates of EBMP explicitly reject the long-standing assumption that theory, traditional training, anecdotal experience or custom, consensus, or common sense alone provides sufficient guidance for effective decision making and professional practice. Intuition and unsystematic clinical expertise are insufficient grounds on which to make clinical decisions. On the other hand, the “value laden nature of clinical decisions” implies that we cannot rely on evidence alone. knowing the tools of evidence-based practice are necessary but not sufficient for delivering the highest quality of patient care.13
One of the origins of EBMP was the study of variations in practice and related outcomes.14 Variations in practices suggest questions such as “Are they all equally effective?” “Are some more effective than others?” “Do some result in more harm than good?” Evidence has begun to indicate that there are significant differences among hospitals or doctors in a particular specialty. What you tend to find is a bell curve: a handful of teams with very poor outcomes for their patients, a handful with incredibly good results, and a great undistinguished middle. Acknowledging this bell curve is very distressing to practitioners since it contradicts the promise that they have made to patients who become seriously ill: that they can count on the medical system to give them their very best chance at life. We used to think that a doctor’s ability depends mainly on science and skill. However, even doctors with great knowledge and technical skills can have mediocre results. What the best physicians do have, however, is a capacity to learn, whether from research data or clinical experience, and to do so faster than their average peers. What we are also learning, however, is that in addition to the above intellectual skills, the best practitioners often possess or strive to acquire more nebulous attributes such as aggressiveness, consistency, ingenuity, compassion, sensitive listening skills, and broad perspectives from the humanities and social sciences.15
A key characteristic of EBMP is to break down the division between research and practice, highlighting the importance of clinicians’ ability to critically appraise research reviews and developing a technology to help them do so. It emphasizes clinician use of their scientific training and their judgment to interpret research and individualize patient care accordingly. EBMP is a guide for thinking about how decisions should be made in light of patients’ preferences and clinicians’ recommendations. Proponents of EBMP believe that findings from the most relevant scientific studies currently available should figure prominently in the practice decisions of clinicians. Judicious use of evidence involves balancing an assessment of the individual patient’s unique characteristics, personal preferences, and life circumstances against relevant primary research findings or practice guideline recommendations for patient care.14
Misconceptions about EBMP including the criticisms that (1) it will replace or seek to replace practitioner judgment, (2) it leads to a “cookie cutter” approach to medical practice, and (3) it is too time consuming to be routinely employed in real-life practice settings also might discourage widespread adoption of EBMP.16 EBMP should never evolve into rigid practice because effective interventions require that practitioners integrate their professional understandings of patient care with recommendations derived from the best external evidence and patients’ preferences.16 The practice calls for candid descriptions of limitations of research studies and use of research methods that critically test questions addressed. It also calls for systematic research reviews rather than reviews authored solely by self-declared “experts.”
SYSTEMATIC REVIEWS: META-ANALYSIS
Meta-analysis is a statistical procedure for synthesizing research results across studies that address a common topic or issue. The term means to analyze “after or beyond” the original analysis. It is the analysis of analyses, completed on a collection of studies usually to draw general conclusions. A major achievement of EBMP has been the development of systematic reviews, methods by which researchers identify multiple studies on a topic, separate the best ones, and then critically analyze them to come up with a summary of the best available evidence. It is more than a quarter of a century since Gene Glass coined the term “meta-analysis” to refer to systematic reviews whose results from different primary studies are statistically combined into an overall estimate.17
Meta-analysis is qualitatively different from other traditional reviews. The purpose of meta-analysis is to estimate the size of treatment effects to aid clinical decision making. Another major goal is to generate hypotheses to be tested in new clinical trials. They are not always bigger, and their main aim is not simply to be comprehensive but to answer a specific question, apply stringent inclusion criteria to studies reviewed, appraise the quality of the studies included, and summarize them objectively. However, a meta-analysis is only as accurate as the data on which it is based. The reader must examine the inclusion and exclusion criteria carefully in the studies that are grouped for the meta-analysis. For example, a study that evaluated different treatment modalities in a RCT with only 22 patients would not meet the sample size or power requirements to be included in a meta-analysis.18 In two other double-blind randomized trials, the authors evaluated the efficacy of low-dose aspirin to prevent preeclampsia. The first study with 34 women found that a significantly reduced incidence of pregnancy induced hypertension and preeclampsia.19 The subsequent study recruited 471 GDM participants, 774 chronic hypertensive women, 688 patients with multifetal gestation, and 606 with preeclampsia during a previous pregnancy. The authors found that low-dose aspirin did not significantly reduce the incidence of preeclampsia or improve perinatal outcome.20 These studies demonstrate the effect of sample size on alpha and beta errors in research reporting.
Ranking different types of evidence by their level of scientific support is guided by three principles: quality, quantity, and consistency.21 Quality refers to how the individual studies collectively minimized bias; quantity addresses the number of studies, sample size, and magnitude of effect; and, consistency pertains to whether findings are similar under different study conditions using different population samples or comparable study designs.21 The strength of the evidence offered by a meta-analysis depends on how well the review is conducted. The systematic review often involves the skills of several reviewers working independently to screen thousands of abstracts and studies.
However, the high profile of meta-analysis as a method of analysis in evidence-based medicine practice has led to several misconceptions about its purpose and methods.22 Systematic reviews of nonrandomized studies are also common, and qualitative studies can be and often are included in meta-analysis as are case reports. The systematic review is a method for limiting bias. However, since the choice of which study designs to include is made by the reviewers, bias may sometimes be introduced.23-25
There is also a common myth that meta-analysis requires the adoption of a biomedical model of health. Systematic reviews do not have preferred biomedical models and that is why there are systematic reviews in such diverse disciplines as education, social work, and public policy. Reviews on the Cochrane Database of Systematic Reviews commonly include “quality of life” as an outcome variable alongside clinical indicators of the effects of interventions. The systematic review, in medicine and other disciplines, is an efficient and effective technique for testing hypotheses, summarizing results of existing studies and assessing the reliability and validity of studies.25
Many researchers as well as clinical practitioners mistakenly believe that meta-analysis always involves statistical synthesis. A major concern is the potential for combining studies that are too diverse in treatment interventions, subject selection, outcome measurements, and research design. When no single study provides the purported evidence, maybe fusing all inaccurate studies together will finally provide the elusive evidence! Some systematic reviews summarize studies by describing the methods and results while others use meta-analysis by converting the data from each study into common measurement scales and combining the studies statistically. Many reviews do not use meta-analysis since pooling studies without taking into account variations in study quality can bias the conclusions of the review.26
Finally, authors and consumers of systematic reviews need to recognize that these reviews do not necessarily produce definitive answers to health care issues. They often identify the need for additional primary studies and are the vehicle for demonstrating future directions for new research efforts.25 This methodology is useful in identifying “what works” beyond the world of EBMP and may also provide a platform for the combined knowledge and skills of the major players in health care provision today.
Clinical Guidelines
Most guidelines are a fusion of clinical experience, expert opinion, and research evidence. When the process of creating a practice guideline utilizes valid and current research evidence in systematic reviews, this has the potential to be translated into clinical decision aids for optimized health outcomes for informed policy decision makers in managed care systems and educated clinicians who in turn educate patients. It has been argued, however, that practice guidelines are too often based on the consensus of “experts” rather than actual evidence. Practice guidelines and consensus statements have sprung up under the sponsorship of groups in which the validity of the disseminated message and credibility of the distributing agent are not always positively related. When the principles of EBMP are applied to the creation of these guidelines, the potential limitations inherent in guideline development are mostly overcome.27
Who Are the “Players" in Evidence-Based Medical Practice?
Evidenced-based medical practice is as much about the knowledge and ethics of educators and researchers as it is about the ethics of practitioners and policy makers in managed care systems. The health care system faces challenges from the many players who are individually and/or collectively involved in the formulation of policy or as recipients of those decision-making processes. EBMP involves sharing responsibility among all interested players for decision making in a context of recognized uncertainty.
Patients want more effective communication with their care providers so that they can make informed choices. A striking characteristic of EBMP is the extent to which patients are involved in many different ways.28,29 There is a contemporary emphasis to compare the values and preferences of patients with recommended medical protocols and their likely consequences as well as “personalizing” the evidence to fit a specific patient’s life and health circumstances. There is also a movement to help patients develop critical appraisal skills that will facilitate more active participation in their health care. The term “evidence-based patient choice” emphasizes the importance of involving patients as autonomous participants who themselves carry out the required integration of information from diverse sources in making decisions that suit their values and needs.30
Another way in which patients are actively involved in their own care is recognizing their unique knowledge in relation to application of certain regimens. The experts in deciding whether a guideline is applicable to a given patient is the patient and providers not the researchers and academicians who critically appraise research findings. The differing expertise needed to prepare systematic reviews regarding the evidentiary base of a guideline and to identify implementation potential highlights the inappropriateness of researchers telling practitioners and patients what guidelines to use. In EBMP, patients are involved as informed participants regarding the evidentiary status of services. There is an attempt to promote candidness and clarity in place of secrecy and obscurity. EBMP requires searching for research findings related to important practice and policy decisions and sharing what is found (including nothing) with patients.
Medical educators and clinicians want scientific bases for determining “best practice” approaches in addition to the research and statistical tools to learn how to assess the results of studies to enhance patient care. However, they need to adapt a common sense approach to EBMP. This approach integrates individual clinical expertise with best available evidence (relevant studies discovered from a systematic search of the health care literature). Practicing evidence-based medicine implies not only clinical expertise (proficiency and judgment acquired through experience), but expertise in retrieving, interpreting, and applying the results of scientific studies, and in communicating the risks and benefits of different courses of action to patients. EBMP dictates that professional judgments and behavior be guided by two distinct but interdependent principles. First, whenever possible, practice should be grounded on prior findings that demonstrate empirically that certain actions performed with a particular type of patient are likely to produce predictable, beneficial, and effective results. Second, every patient over time should be individually evaluated to determine the extent to which the predicted results have been attained as a direct consequence of the practitioner’s actions. Judicious use of evidence involves balancing an assessment of the individual patient’s unique characteristics, personal preferences, and life circumstances against relevant primary research findings or practice guideline recommendations for patient care. EBMP draws on the results of systematic, rigorous, critical appraisal of research related to important practice questions such as, Is this assessment measure valid? Does this intervention do more good than harm? Efforts are made to prepare comprehensive, rigorous reviews of all research related to questions of effectiveness, prevention, screening (risk and prognosis), description and assessment, harm, and self-development.
An ultimate objective of EBMP is the practitioner’s consideration of the veracity of the findings of a given piece of research and its applicability to his patient or collective patient population. He/she will need to (1) know how to read and critique research articles and (2) assess the degree to which an intervention has been empirically tested and found promising. To access, analyze, and apply research findings in diabetic studies, practitioners will need to understand why, by whom, and how research studies are conducted. Therefore, medical school and continuing medical education will need to teach and reinforce the study of research design—the overall framework for collecting data once the problem has been formulated. In addition, these institutions will need to teach how to read and interpret the data and what they mean. The main objective of this educational strategy will be to integrate individual clinical expertise with critical evaluation of evidence discovered from a systematic literature search to solve a problem.
Understanding what kind of study has been performed is a prerequisite to thoughtful reading of research. What is now known is that physicians, under the influence of pharmaceutical advertising and promotions, are much more impressionable than was originally believed.31 Only studies with comparison groups allow investigators to assess possible causal associations, a fact often forgotten or ignored. Large amounts of poor data forestall any amount of good data. Lots of zeroes may look impressive in research findings yet they still amount to zero. Unfortunately, most physicians lack skills in evaluating studies for bias and relevancy. This can result in harmful consequences to patients and is one of the reasons the enthusiastic use of the anti-inflammatory drug Vioxx caused harm to so many patients.
Clinicians confront voluminous evidence about the clinical choices they face every day. To remedy the problem, many medical groups issue clinical practice guidelines: experts in a field sort through the reams of clinical research on a medical condition and pore over drug studies; they then publish summaries about what treatments work best so that physicians everywhere can offer the most appropriate, up-to-date care to their patients. While this sounds straightforward, the process can go awry. The recommendations issued recently by the American Association of Clinical Endocrinologists (AACE) for the treatment of diabetes elevated second- or third-line drugs to more prominent positions in the prescribing hierarchy, rivaling once uncontested go-to medications life metformin, an inexpensive generic. They also emphasized the riskiness of established treatments like insulin and glyburide, which now carry yellow warning labels in the AACE summary. Several of the now promoted drugs are expensive newcomers that lack the track records of clinical effectiveness and safety by the older, potentially displaced treatments. Physicians were perhaps given more treatment choices for their patients, but the AACE recommendations could also have been influenced by drug manufacturers who helped finance the new guidelines. What has evolved is the establishment of guidelines for guidelines, that is, guideline recommendations by various organizations are rigorously and fairly depicted (Institute of Medicine 2011 report) and not tainted by financial ties to the pharmaceutical companies that could win or lose based on their content. Overall, there is need for better study design, execution, reporting, and scientific critical appraisal skills by researchers and health care decision making as well as the drug manufacturers responsible for sales and distribution. At the end of the day, medicine, like art, is a creative process, and very much a team effort.
Excellent health care practice should be inspired by love and guided by science; both are essential. If a professional practices scientifically without compassion, he/she becomes a robot. On the other hand, if a practitioner is compassionate but unscientific, his failure to adapt EBMP methods in light of the burgeoning databases of relevant empirical findings might marginalize his medical practice and relegate his patients to substandard professional interventions.
Researchers and Peer Review
Peer review is the main apparatus that research journals use to assess the quality of the many manuscripts competing for the few places available for publication. Journal editors solicit evaluations of submitted manuscripts from outside experts who remain anonymous to the authors by the process. The results of a review can consecrate or doom the progress of a particular course of research. Often the results of clinical trials influence whether they will actually be published. Most journals want to be the first to publish positive new results. Negative results may not always be reported and are also less likely to be published in prestigious journals.32
The role of journals as gatekeepers for the scientific record dates from the 17th century when the Royal Society’s (Great Britain) council was instructed to review submissions to its Philosophical Transactions. Despite over 300 years of use, the pursuit of excellence in research has not been accompanied by a parallel pursuit in the evaluation of that excellence. Envisioned as a way to ease reviewers’ inhibitions, the practice of using anonymous reviewers diminishes accountability. Journal editors and anonymous reviewers base decisions about manuscripts on questionable criteria and standards from a largely secretive process. Medical journals often do not include clear statements about their peer review process, while reviewers are rarely informed of their role description as reviewers. In addition, because of the massive number of manuscripts in need of review, fellows in training and any other convenient reader (knowledgeable or not in the specific field) are recruited to adjudicate a manuscript’s quality for potential publication.
The system is error-prone. History has shown that great scientific discoveries have often been achieved with minimal support and despite the active hindrance by the discoverer’s “peers.” When Dr. Rose Yalow first submitted the manuscript on insulin assay for peer review, she received a resounding rejection. Needless to say, it was this work that would be recognized in the future and would be the basis for her receiving a Nobel Prize.
It is also not difficult to understand how conflict of interest and jealousy can undermine the peer review process. Researchers whose work challenges the status quo are a threat to those whose careers are entrenched in the paradigm of the day. New ideas can jeopardize special interest groups and the funding they receive to pursue traditional approaches. As a result, peer reviewers have often hindered or even sabotaged scientific breakthroughs. The flaws in the process reveal bias founded on intellectual positions, personal convictions, as well as biases related to ethnicity, nationality, gender, and status. The results of the evaluation process have produced occasionally foolish and frequently incorrect statements, a lack of accountability enhanced by anonymity, as well as often personally insulting remarks.
Opinions will differ between reformers and die-hard defenders of the current peer review system. However, if the scientific community is to enhance its credibility, the peer review process must embrace a sounder and properly validated basis, that is, oversight without imposition. It requires a priori that a potential reviewer recuse him/herself if he has a bias against the authors or minimal knowledge on the subject. The referee’s role is to read a manuscript and “... look neither for something to criticize to prove his diligence and capability as a referee nor overlook or condone omissions or errors to prove his graciousness. He should bear in mind that he is rendering a service to the editor, in the manner of an expert witness.”33
The quality and usefulness of a journal rests on the quality of the research submitted, its reviewers’ evaluations, and the editor’s critical judgment skills. To enhance the objectivity and quality of the process, the scientific community needs to make a concerted effort to select reviewers who are knowledgeable, provide constructive evaluation, and impede the natural biases inherent in the review system.
Peter Doshi,34 recently of Johns Hopkins University, is on a mission to influence and encourage the world’s largest pharmaceutical firms to open their records to outsiders in an effort to better understand the benefits and potential dangers of the drugs that billions of people take every day. He is trying to gain access to data from clinical trials and make them public. The current system is one in which the meager details of clinical trials are published in professional journals often by authors with financial affiliations to the companies whose drugs they are promoting. This is not only conflict of interest but also free commercial advertisement that may also be misleading. The efforts of Dr. Doshi and other activists have encouraged GlaxoSmithKline Pharmaceuticals to pledge to share detailed data from all global clinical trials conducted since 2000. If and when that data are eventually publicized, it would amount to more than 1000 clinical trials involving more than 90 drugs.
Another related issue to drug research arises when major drug companies export their scientific development to emerging markets such as China. Since 2006, 13 of the top 20 global pharmaceutical firms have set up research and development centers in China because it is cheaper to do research there. Auditors found that researchers did not report the results of animal studies in a drug that was already being tested in humans. Animal studies can identify safety risks and are among the main factors drug companies use to decide whether to pursue human trials. In addition, workers at the research centers had not properly monitored clinical trials and paid hospitals and participating doctors and other hospital personnel fees based on the number of people enrolled in a study. It is to the credit of Glaxo that it audited its own research facility. However, it also demonstrates what can happen when a drug company rapidly expands its clinical research programs overseas without adequate quality controls.35
Managed care providers have historically played key roles in influencing the behaviors of both practitioners and patients. They believe that EBMP is critical to the success of their plans’ clinical performance but there is concern among many that the application of evidence-based guidelines derived from systematic reviews may in some cases increase costs. How plans can incorporate evidence-based practice into medical management activities and the modification of these strategies is a current focus for managed care providers. Incentives incorporated into systems that reward more efficient health care delivery, reduce waste, and lower costs could someday resemble a system that celebrates the attributes of EBMP. Managed care appears to be evolving from its original structure and rationale in traditional medical practice approaches to utilization management to participation in an evidence-based culture. As more high-quality synthesis of information relevant to an organization and delivery of care become available, greater familiarity with the retrieval and evaluation of systematic reviews can help managers use these sources effectively. If this trend continues, the system can adapt creative ways of rewarding practitioners, hospitals, and consumer adherence to evidence-based, cost-effective performance.
THE FUTURE OF EVIDENCE-BASED MEDICAL PRACTICE
It was the best of times; it was the worst of times,
It was the age of wisdom; it was the age of foolishness,
It was the epoch of belief; it was the epoch of incredulity,
It was the spring of hope; it was the winter of despair...
—Charles Dickens
With apologies to Charles Dickens, his words suggest the circumstances we currently face in the provision of health care. When historians of the future look back on the 21st century, we have no doubt that they will be impressed by the tremendous progress that has been made in science and medicine. A majority of our population has immediate access to effective health care services of all types provided by knowledgeable health care practitioners who know how to use them. However, at a time when we have more effective therapeutic tools than ever before, there are increasing impediments to the implementation and delivery of those tools. While millions have limited access to the essential care that is basic to everyday health and well-being, others lack the capacity to pay for this level of care, even if it were available. We spend vastly more on health care than any other nation in the world, yet analysis of our health status places us at the middle to the bottom among developed countries. In addition, despite all of the emphasis to advance our health care system, the medical community and physicians have yet to meaningfully step forward to lead improvements, or to advance medicine based on science rather than tradition and anecdote. If we recognize the shortcomings of “the worst of times. ” through thoughtful, informed, compassionate, and responsible leadership and participation, we can advance “. an epoch of belief.” And “.a spring of hope.” by capitalizing on the wonderful resources and potential of our health care system.
The successful promotion of EBMP can have a profound positive collective effect on health care if each of the partners (researchers/academicians, health care practitioners, patients, pharmaceutical companies, and managed care organizations) advance the principle that scientifically proven evidence-based medicine is the standard of quality and appropriateness in health care. Anecdotes, personal testimonials, and paid advertisements cannot define the gold standard. In this regard, health consumers and their physicians need the highest level of information for making health care decisions, that is, EBMP. “What we can do is maximize quality, minimize bias, manage uncertainty, and provide adequate support for those who have the task of ensuring that as our research moves forward, generating all kinds of evidence for clinical practice and policy, we do not lose sight of human health and suffering.”8
REFERENCES
1. Sackett DL, Strauss SE, Richardson WS, et al. Evidence-Based Medicine: How to Teach and Practice EBM. 2nd ed. Edinburgh, Scotland, UK: Churchill Livingstone; 2000.
2. Last JM. A Dictionary of Epidemiology. 3rd ed. New York, NY: Oxford University Press; 1995.
3. Bennett RJ, Sackett DL, Haynes RB, et al. A controlled trial of teaching critical appraisal of the clinical literature to medical students. JAMA. 1987;257:2451-2454.
4. Grimes DA. Epidemiologic research using administrative databases: garbage in, garbage out. Obstet Gynecol. 2010;116(5):1018-1019.
5. Northam S, Knapp TR. The reliability and validity of birth certificates. J Obstet Gynecol Neonatal Nurs. 2006;35:3-12.
6. Ault MR. Combating the garbage-in, gospel-out syndrome. Radiat Protect Manag. 2004;20:26-30.
7. Ross S. Composite outcomes in randomized clinical trials: arguments for and against. Am J Obstet Gynecol. 2007;196:119.
8. Goodman KW. Ethics and Evidence-Based Medicine. Cambridge, UK: Cambridge University Press; 2003.
9. Cochrane Collaboration. http://www.cochrane.org.
10. The periodic health examination. Canadian Task Force on the Periodic Health Examination. Can Med Assoc J. 1979;121:1193-1254.
11. U.S. Preventive Services Task Force. Guide to Clinical Preventive Services. 2nd ed. Baltimore, MA: Williams and Wilkins; 1996:862.
12. Chalmers I. Scientific inquiry and authoritarianism in perinatal care and education. Birth. 1983;10(3):151-166.
13. Guyatt G, Rennie D. Users’ Guide to the Medical Literature: A Manual for Evidence-Based Clinical Practice. The Evidence - Based Medicine Working Group JAMA & Archives. Chicago, IL: American Medical Association Press; 2002.
14. Gray JAM. The origin of evidence-based practice. In: Edwards A, Elwyn G, eds. Evidence-Informed Client Choice. New York, NY: Oxford University Press; 2001:19-33.
15. Gawande A. The bell curve. The New Yorker. December 6, 2004.
16. Sackett DL, Richardson WS, Rosenberg W, et al. Evidence-Based Medicine: How to Practice and Teach EBM. New York, NY: Churchill Livingstone; 1997.
17. Glass G. Primary, secondary, and meta-analysis of research. Educ Res. 1976;10:3-8.
18. Anjalakshi C, Balajo MS, Seshiah V. A prospective study comparing insulin and glibenclamide in gestational diabetes mellitus in Asian Indian women. Diabetes Res Clin Pract. 2007;76(3):474-475.
19. Schiff E, Peleg E, Goldenberg M, et al. The use of aspirin to prevent pregnancy-induced hypertension. N Engl J Med. 1989;321(6):351-356.
20. Caritas S, Sibai B, Hauth J, et al. Low-dose aspirin to prevent preeclampsia in women at high risk. NIH/MFMU. N Engl J Med. 1998;338(11):756-757.
21. Lohr KN. Rating the strength of scientific evidence: relevance for quality improvement programs. Int J Qual Health Care. 2004;16:9-18.
22. Petticrew M. Systematic reviews from astronomy to zoology: myths and misconceptions. BMJ. 2001;322:98-101.
23. NHS Center for Reviews and Dissemination. Undertaking Systematic Reviews of Research on Effectiveness: CRD report. No. 4. York, UK: University of York; 1996.
24. Popay J, Rogers A, Williams G. Rationale and standards for the systematic review of qualitative literature in health services research. Qual Health Res. 1998;8:341-351.
25. Petticrew M, Song F, Wilson P, et al. Quality-assessed reviews of health care interventions and the database of abstracts of reviews of effectiveness (DARE). Int J Technol Assess Health Care. 1999;15:671-678.
26. Sutton A, Abrams K, Jones D, et al. Systematic reviews of trials and other studies. Health Tech Assess. 1998;2:1-276.
27. Cook DJ, Greengold NL, Ellrodt AG, et al. The relation between systematic reviews and practice guidelines. Ann Int Med. 1997;127(3):210-216.
28. Broclain D, Hill S, Oliver S, et al, eds. Cochrane Consumers & Communication Group. The Cochrane Library. Issue 3. Oxford, UK: Update Software; 2002.
29. Entwistle V, O’Donnell M. Evidence-based health care: what roles for patients? In: Edwards A, Elwyn G, eds. Evidence-Based Patient Choice: Inevitable or Impossible? New York, NY: Oxford University Press; 2001.
30. Edwards A, Elwyn G, eds. Evidence-Based Patient Choice: Inevitable or Impossible? New York, NY: Oxford University Press; 2001.
31. Wazana A. Physicians and the pharmaceutical industry: is a gift ever just a gift? JAMA. 2000;283:373-380.
32. Simes RJ. Publication bias: the case for an international registry of clinical trials. J Clin Oncol. 1986;4:1529-1541.
33. Forscher BK. Rules for referees. Science. 1965;(150):319-321.
34. Breaking the seal on drug research. New York Times. June 30, 2013.
35. Thomas K. Drug research in China falls under a cloud. New York Times. July 23, 2013.