Results of Expert Meetings
Improving cardiovascular clinical trials conduct in the United States: Recommendation from clinicians, researchers, sponsors, and regulators Javed Butler, MD, MPH, a Gregg C. Fonarow, MD, b Christopher O’Connor, MD, c Kirkwood Adams, MD, d Robert O. Bonow, MD, MS, e Robert J. Cody, MD, MBA, f Sean P. Collins, MD, MSc, g Preston Dunnmon, MD, MBA, h Wilfried Dinh, MD, i,j Mona Fiuzat, PharmD, c Vasiliki V. Georgiopoulou, MD, MPH, k Stephen Grant, MD, h So-Young Kim, MD, l Stuart Kupfer, MD, m Martin Lefkowitz, MD, n Robert J. Mentz, MD, c Frank Misselwitz, MD, PhD, l Bertram Pitt, MD, o Lothar Roessig, MD, l Erik Schelbert, MD, MS, p Monica Shah, MD, q Scott Solomon, MD, r Norman Stockbridge, MD, h Clyde Yancy, MD, MS, s and Mihai Gheorghiade, MD e Stony Brook, NY; Los Angeles, CA; Durham, NC; Chapel Hill, NC; Chicago, IL; Raritan, NJ; Nashville, TN; Silver Spring, MD; Berlin, Wuppertal, Witten, Germany; Atlanta, GA; Bayer HealthCare AG; Deerfield IL; East Hanover, NJ; Ann Arbor, MI; Pittsburgh, PA; Bethesda, MD; and Boston, MA
Advances in medical therapies leading to improved patient outcomes are in large part related to successful conduct of clinical trials that offer critical information regarding the efficacy and safety of novel interventions. The conduct of clinical trials in the United States, however, continues to face increasing challenges with recruitment and retention. These trends are paralleled by an increasing shift toward more multinational trials where most participants are enrolled in countries outside the United States, bringing into question the generalizability of the results to the American population. This manuscript presents the perspectives and recommendations from clinicians, researchers, sponsors, and regulators who attended a meeting facilitated by the Food and Drug Administration to improve upon the current clinical trial trends in the United States. (Am Heart J 2015;169:305-14.)
Clinical trials provide essential information that forms the foundation for evidence-based medicine. Significant advances have been made in the treatment of many cardiovascular diseases over the last several decades, in large part related to conduct of robust clinical trials that generated evidence for benefit, neutrality, or harm with From the aCardiology Division, Stony Brook University, Stony Brook, NY, bUniversity of California, Los Angeles, CA, cDuke Clinical Research Institute, Duke University Medical Center, Durham, NC, dUniversity of North Carolina, Chapel Hill, NC, eCenter for Cardiovascular Innovation, Northwestern University Feinberg School of Medicine, Chicago, IL, fCardiovascular & Metabolism, Janssen Pharmaceuticals, Raritan, NJ, g Vanderbilt University, Nashville, TN, hCenter for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, MD, iDepartment of Cardiology, Witten University, Witten, Germany, jGlobal Drug Discovery, Clinical Sciences, Bayer Pharma AG, Berlin, Germany, kEmory Cardiovascular Clinical Research Institute, Emory University, Atlanta, GA, lGlobal Clinical Development, Bayer HealthCare AG, Wuppertal, Germany, m Takeda Pharmaceuticals International, Deerfield IL, nNovartis Pharmaceuticals Inc., East Hanover, NJ, oDivision of Cardiology, University of Michigan School of Medicine, Ann Arbor, MI, pUniversity of Pittsburgh Medical Center Heart and Vascular Institute, Pittsburgh, PA, qDivision of Cardiovascular Sciences, National Heart, Lung, and Blood Institute, Bethesda, MD, rCardiovascular Division, Brigham and Women’s Hospital, Boston, MA, and sDivision of Cardiology, Department of Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL. James A de Lemos, MD, served as guest editor for this article. Submitted October 23, 2014; accepted December 10, 2014. Reprint requests: Javed Butler, MD, MPH, Cardiology Division, Stony Brook University, Health Sciences Center, T-16, Room 080 SUNY at Stony Brook, NY 11794. E-mail: [email protected] 0002-8703 © 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.ahj.2014.12.001
the therapies evaluated. These trials informed how to best treat patients and which therapies to avoid. Despite these discoveries over the last several decades, there remains a continued need for improvement in outcomes and, thus, for more clinical trials, for all cardiovascular diseases, especially for conditions that are highly prevalent and associated with continued poor outcomes, for example, heart failure with preserved ejection fraction or worsening heart failure requiring hospitalization. The conduct of clinical trials in the United States has become increasingly challenging with (1) suboptimal enrollment and retention, (2) exorbitant expenses, (3) declining funding, (4) lack of motives and growing clinical responsibilities for researchers, (5) negative perception of industry, and (6) distribution models for trial finances that may not be optimal. 1-7 For example, 359 sites in the Efficacy of Vasopressin Antagonism in Heart Failure: Outcome Study with Tolvaptan (EVEREST) trial enrolled 4133 patients. 8 Overall, 21% of sites in North America did not enroll any patients. Among sites that did enroll, the median enrollment over the 28-month trial period was 7 participants per site in North America, compared with more than 20 in South America and Eastern Europe. 9 Accordingly, the proportion of multinational trials continues to increase, as does the proportion non-US patients enrolled within those trials. 10 This trend is even more prominent in recent trials. In the Aliskiren Trial on Acute
American Heart Journal March 2015
306 Butler et al
Table I. Concerns about conducting clinical trials outside the United States Variant characteristic Patient risk profile Availability of therapeutic interventions Affordability of therapeutic interventions Availability of hospital infrastructure Availability of clinic infrastructure Clinical practice and standard of care Disease outcomes Genetic profile Cultural/social background
Heart Failure Outcomes (ASTRONAUT) trial, 1615 patients were randomized at 316 sites with an average enrollment rate per site of 2.9 ± 2.4 in North America, 4.1 ± 3.2 in Western Europe, 4.9 ± 4.7 in South America, 7.1 ± 6.2 in Eastern Europe, and 9.5 ± 9.8 in Asia (personal communication). These challenges are faced in the area of cardiovascular diseases in general. To facilitate understanding of the pertinent issues from the various stakeholders’ perspectives and to develop a set of actionable recommendations to improve upon the current trends in clinical trials, the Food and Drug Administration (FDA) hosted a meeting on February 21, 2014. Representatives from academia; industry; the National Heart, Lung, and Blood Institute (NHLBI); and the FDA attended this meeting, which was not industry or otherwise sponsored. This document summarizes the discussions and conclusions of this meeting.
Importance of conducting clinical trials in the United States Only a small proportion of practice guidelines are based on clinical trial data. 11 When such data are available, there continues to be a growing shift to more international trials with patients recruited throughout the globe. The United States constitutes about 5% of the world’s population and has one of the most expensive clinical trials enterprises, and over time, many trials have a limited representation of participants from the United States and some have none at all. 12 For example, the clinical trials with ivabradine were all conducted outside the United States, and the drug is already recommended for use by the European Society of Cardiology heart failure management guidelines but is not currently available in the United States. 13,14 High-quality research from outside the United States is meritorious, and the FDA accepts such data to approve drugs based on research conducted largely or solely outside the United States. However, concerns have been raised regarding the applicability of the results generated from other regions to patients in the United States (Table I) 10 considering geographic variations in (a) patients’ risk profile,
Concern Patient population is not representative Proven therapies not always provided
Appropriate follow-up and evaluation are not provided Disease progress is not representative Disease epidemiology is not representative Effectiveness of therapies is not representative Acceptance of therapies is not representative
(b) background therapy, and (c) resource availability related to standard care. 15-17 Regional differences in practice and in outcomes are well known. 18,19 Genetic and environmental dissimilarities may impact treatment effects. 20 Prevalent standards of care and baseline therapy differences between regions might affect the association of a drug with outcomes. 21 Regional differences in the magnitude of benefit from β-blockers were hypothesized to reflect differences in population, genetics, differences in disease management, or to chance finding. 22 Additional concerns have been raised regarding the human-subject protection and quality of conduct of some of the multicenter trials. The United States has long led innovation in biomedical research. However, if the current trend of declining clinical trial expertise continues and a robust network of competent sites and clinical trial investigators is not rejuvenated, then the further worsening of recruitment and retention in the United States may become insurmountable. 23 To guide the most optimal treatment recommendations for the American population, it is imperative that the clinical trials enterprise in the United States be revitalized to generate relevant, reliable, and pertinent data applicable to patients in the United States.
Difficulties in conducting clinical trials The difficulties in conducting clinical trials in the United States are multifold. Different stakeholders responsible for the successful conduct of clinical trials have varying perspectives on the barriers to optimally perform trials. The academic productivity from multicenter clinical trials has been traditionally limited to a few investigators, generally those who serve on the steering committee or the publication committee, etc. The site investigators in general have not benefitted in this respect. In the spirit of the growing consensus around the concept of team science, one may postulate that more academic opportunities to the site investigators may increase productivity. However, authorship rights based on enrolling patients may not meet the criteria for such designation, and the site investigators should be more actively involved in the academic products from the
American Heart Journal Volume 169, Issue 3
Butler et al 307
Figure 1
Distribution of funds in clinical trials. Adapted from Califf, RM, ACS and Acute Heart Failure Models – Speaker presentation at the Institute of Medicine Workshop on Transforming Clinical Research in the United States, October 7 to 8, 2009, Washington, DC. The costs are presented in US dollars (millions). There is an additional cost for publications ($0.1 million [0.04%]) that is not shown.
trials. The concept of team science is that the various scientific disciplines come together to bring varying expertise to achieve results more globally and efficiently. Whether enrolling patients at the site level meet these criteria has not been resolved.
Investigator There are usually large pools of patients with various cardiac diseases being treated at centers that participate in trials. Thus, the primary impediment remains to be investigator involvement and not patient availability. Clinicians perform trials for the large part but are facing growing pressures to increase clinical productivity. Lack of discretionary funds makes it impossible to protect time for clinicians to perform trials. Researchers are encouraged to focus either on clinical activities or on investigator-initiated grants that provide salary support. Federal funding for trials has become increasingly competitive, requiring years of effort in grant writing and submitting revisions. With changes in federal research funding, these efforts are often unsuccessful. Most trials provide no salary support for investigators, unless they are investigator initiated. Thus, clinicians choose either to not be involved in trials or to delegate the responsibility to study coordinators and, in turn, distancing themselves from active involvement. Conduct of trials for most clinicians is not part of their job description but an optional academic avocation. In addition, the patient pool at any given institution belongs to several providers beyond the site principal investigator. Unless the institutional culture dictates an emphasis on enrollment, it might prove difficult to enroll patients who are not treated primarily by the investigators themselves. The overall costs of clinical trials continue to increase. However, a large proportion of this funding is earmarked for a clinical research organization (CRO) for trial management (Figure 1). 24 From the funds that come to the site, a high proportion is used for maintaining the
regulatory and compliance infrastructure. The support to the actual investigators, despite overall high costs of trials, remains suboptimal. A large part of the effort in a clinical trial is not related to the research procedures after enrolling a patient but to pre-enrollment administrative and screening activities, discussion with patients and families, and convincing other providers; these are all activities that require time and effort by the clinicians but remain unsupported. Thus, although the cost of conducting trials has increased, these funds do not distill down to the researchers and their coordinators tasked with doing the work directly related to research. Academic research organizations play a significant role in clinical trials conduct through partnerships with academia or other stakeholders in recruiting sites through established networks and relationships and can implement all aspects associated with managing a successful study. However, costs are an issue with them as well as it is with CROs. The nonmonetary returns from participation in trials are ambiguous as well. Participation in multicenter trials as a site investigator without involvement in the publications, which is usually limited to a minority of investigators, usually does not support promotion in academia. Most investigators are not part of the steering or publication committees, and even if cited as one of the many authors on a publication, this recognition is not of major consequence for local recognition or career advancement. In addition, investigators may prefer some trials over others because of specific scientific interest or relationship with the sponsor. This can lead to enrollment of more patients in one trial at the expense of others. This competing trials dynamic may lead to poor enrollment in specific trials even in otherwise established centers and should be taken into account when sites are selected for any given trial.
National Heart, Lung, and Blood Institute The National Heart, Lung, and Blood Institute supports a wide range of fundamental clinical and population
308 Butler et al
research, including several clinical trials. Faced with the challenge of shrinking budgets and increasing demands, the institute is confronting unprecedented pressure to support more science with fewer resources. Clinical trials are a high scientific priority at NHLBI, but given the current fiscal environment, these studies are under particular scrutiny because they represent some of the largest investments at the institute. In response to these economic pressures, NHLBI is striving to optimize the use of the budget while still maximizing scientific productivity and ensuring the highest quality scientific results. To ensure optimal productivity, NHLBI is working with investigators to enhance the efficiency and operations of clinical trials and clinical trials networks. To that end, there is an increasing focus on the simplification of trial design, the use of clinical end points, the reduction of extraneous data collection, and the completion of studies on time and within budget. 25-29 Recently published data from NHLBI demonstrated that trials with clinical end points were published significantly faster than trials with surrogate end points. This type of information is helping the institute to better understand what types of trials are more likely to optimize a return on research investments. 26,27 In addition, NHLBI is encouraging investigators to use operational performance measures or “how to” metrics, such as enrollment rates, data quality measures, and time for trial start-up, to regularly track progress and enhance efficiency. 30-32 Strategies to streamline clinical trial operations, such as the use of central institutional review boards and central contracts, are also being encouraged. The National Heart, Lung, and Blood Institute funds several clinical trial networks, which provide support for operational infrastructure and dedicated investigator and study coordinator time. The purpose of these investments is to facilitate and expedite clinical studies. The use of “how to” metrics may help the institute gauge whether these investments are actually enhancing productivity. Although the fiscal crisis presents challenges, it also offers unique opportunities for both NHLBI and the scientific community to develop new ideas about clinical trials, disrupt conventional thinking, re-examine standard procedures, and create a new paradigm of efficiency and scientific productivity for the future.
Industry Industry sponsors most clinical trials, and their perspective mirrors that of the National Institutes of Health. Although sponsors’ interest in conducting clinical trials is, in part, driven by return of investment and the federal funding agencies interests are driven by its mission to improve health, lengthen life, and reduce illness and disability; they both face similar difficulties and challenges in conducting clinical trials. Industry remains concerned with both the growing costs and time related to the conduct of trials in the United States. It is apparent that a few sites enroll most
American Heart Journal March 2015
patients, and most of the remaining sites recruit few. As a result, there are large costs expended on the great number of poor-enrolling sites. Industry recognizes the importance of United States both as an engine for innovation and as a viable market, but they increasingly support trials in other regions to compensate for low US contribution to recruitment to meet timelines and cost targets. The National Institutes of Health has this dilemma in common with industry in which the time required to complete a trial in North America may be prohibitive, and under time pressures, other countries are chosen to substitute for lower than expected recruitment in the United States. 15 Industry is interested in promoting a more effective clinical trials infrastructure in the United States and realizes the high costs of CROs, but many do not have in-house capabilities to operationalize trials and fulfill regulatory requirements, necessitating the need to contract with CROs.
Food and Drug Administration With respect to the evaluation of US marketing applications for drugs and biologics, there are no FDA regulations that uniquely apply to US sites. Likewise, there are no FDA regulations prohibiting the approval of drugs in the United States based on data generated from other regions of the world. Indeed, FDA’s review process for drugs and biologics is the same regardless of whether a marketing application is based on data that are generated exclusively or partially from the United States or completely outside the United States. 33 However, for applications that are based in part or in totally on non-US data, the code of federal regulations (CFR 312.120 and CFR 314.106) requires FDA to confirm that these applications meet US criteria for marketing approval. Specifically, they must (a) show that the data are applicable to the US target patient population and to contemporary US medical practice, (b) show that the data have been generated by investigators of recognized competence, and (c) be able to be validated by FDA through on-site inspections of non-US sites of FDA’s choosing or other appropriate means. Furthermore, just as is the case with US sites, all non-US sites must be under institutional review board or similar oversight, must retain copies of informed consents signed by all subjects, and are subject to US financial disclosure rules. Thus, assuming that a non-US data set meets its prespecified efficacy end point and statistical hurdles, that FDA identifies no important trial conduct deficiencies, and that the sponsors can make a compelling benefit versus risk argument for their product, these applications face the additional hurdle of demonstrating unequivocally that their data are applicable to the US target patient population and to contemporary US medical practice, something that is assumed to be true for US-generated data. Demonstrating relevance to US patients and medical
American Heart Journal Volume 169, Issue 3
Butler et al 309
Table II. Recommendations for improving clinical trials enterprise in the United States Developing clinical trials units
Redefining cost-neutral economics of trials Changing the value equation of clinical trials
Dedicating proportional support from indirect costs Reassess CRO costs and activities
• • • • • • • • • • • • • • •
Promote pragmatic trials
• •
Regional networks
• • •
Understanding trial sites
• Training and education
• • •
Patient engagement
• •
Public awareness and perception
• • •
Dedicated physical space Streamlined regulatory and contractual processes Research services and procedures Local investigator collaboration Higher support for fewer high-potential sites Support dedicated staff, registries, screening Motives for promotion Salary support Leadership roles in trials Cultural support of trials at sites Involvement on publication committees Earmark proportion of indirect costs to develop research infrastructure Support investigator-initiated proposals with indirect funding Reassessment of the current scope of work contracted to the CRO and the associated payment structure Consideration of alternate more cost-effective means to fulfill some of the CRO activities Easier eligibility criteria standardized across studies within 1 indication Embed follow-up and outcomes ascertainment within routine care and electronic health records Disease-focused regional networks Collaboration and competition with motives Establishment of measures and methods to evaluate sites potential more effectively Evaluation of appropriate patient volume, leadership commitment, past track record, and patient flow are key factors Clinical trials rotation in residency and fellowship Human subjects training and clinical trials training Involvement in clinical events committee and data safety monitoring boards, etc. Patient advocacy efforts Focus on patient-centered questions and outcomes in alignment with advocacy involvement Involvement of national and local media Involvement of professional medical societies Reverse the perception of financial interest as the main reason for conducting clinical trials
practice is a multifaceted analysis during which the FDA evaluates US versus non-US data attributes, such as whether 1. The disease of interest was defined the same way in geographies outside the United States as within the United States 2. Efficacy events of interest were uniformly defined, ascertained, and measured 3. Safety events were uniformly ascertained, monitored, and assessed 4. Background therapies and their use were consistent with US medical standards 5. Patient factors outside the United States are not different in an important way (ie, demographics and behaviors reflect the target US patient population).
Therefore, the perception of a disproportionately heavy regulatory burden as a contributor to the decreasing participation of US sites in clinical research is highly doubtful, given that marketing applications submitted to FDA for drugs and biologics that are based in part or in total on non-US data sources are governed by the same regulations and must meet the same requirements for well-controlled, high-quality data to support approval as do applications based solely on US-generated data. Indeed, it can be argued that there is actually an additional level of complexity for sponsors using non-US data in that these sponsors assume the burden of proving the relevance of their data to the US target patient population and medical system. 33,34
310 Butler et al
Recommendations The participants acknowledge that more comprehensive reform may be needed but present the following list of practical action items as a starting point to revamp the clinical trials enterprise in the United States within the current academic and health care environment (Table II). 1. Developing clinical trial units Inefficiencies within hospitals conducting clinical trials, in part, arise from the lack of physical and functional well-defined infrastructure that results in logistic and time burden on patients, staff, and researchers. The numbers of procedures conducted as part of research in any given field are largely finite. Investment in developing physical clinical trial units (CTUs) that support the administrative needs including those related to contracts and institutional review board approval, streamlining costs structures, and performing most research procedures are likely to be more efficient and cost saving. These CTUs will need to be used on a continuous basis because the costs of their overhead and ongoing maintenance may otherwise become unsustainable. One of the most important functions of a CTU is bringing investigators together to facilitate collaboration. This model of collaboration is crucial for successful identification and enrollment, management, and follow-up of participants and often requires collaboration of investigators across specialties. 2. Redefining cost-neutral economics of clinical trials Clinical trials commonly engage a large number of sites because of the expected slow enrollment by the majority. Initiation and maintenance of low-performing sites are expensive. On the other hand, supporting front line investigators more when the overall cost of conducting trials is already a concern is unrealistic. A cost-neutral reorganization of trials is likely to yield positive results, with better support of investigators and sites with a track record, at the expense of not initiating low-performing sites. Strategies may include salary support for investigators and coordinators as well as other support such as site-based registries. This hybrid model between the traditional per-patient payment structure of trials and the salary support model of investigator initiated studies has potential benefits because it provides support for screening efforts and other trial activities between patient enrollments. If budgeted correctly, a lower overall per-patient cost can also be economically more efficient for the sponsor.
American Heart Journal March 2015
3. Changing the value equation of site-based clinical trials Clinicians are increasingly disenfranchised with trials not only because of the monetary equation but also because of other career counterincentives. Better recognition of successful trial conduct such as a metric for promotion, irrespective of authorship, is likely to motivate clinicians. The investigators should be recognized with salary support, promotion, and tenure, based on high-quality performance in site-based research, and should be granted leadership opportunities in research units locally and participation in institutional activities related to research. In addition, they should be rewarded for their expertise in study execution similar to scientific planning and data analysis. For instance, they should have the opportunity to participate not only in publications but also on clinical events committees, steering committees, and other ancillary committees involved in multicenter clinical trials. It is understood that these committees have a limited membership and may require experience and contributions beyond those characterizing a successful investigator. Nonetheless, it is important to identify and train the next generation of trial experts. 4. Dedicating proportional support of indirect dollars to trials The indirect dollars associated with clinical trial funding are largely not used for the personnel and procedures directly related to the trials. Although the regulatory and compliance support through indirect dollars is essential, dedicating a defined proportion of these indirect support resources to enhance clinical trial infrastructure and investigator-initiated studies may be considered. This proposition, although complex and requiring input from multiple representatives in many academic institutions, is necessary to maintain ongoing research support and enthusiasm. 5. Reassess CRO costs and activities The sponsor payments to CRO often greatly exceed the payments for regulatory and compliance activities at the research institution level. Because of the for-profit nature of many CROs, there remains a motive for large margins for CRO-related activities. Most sponsors for reasons of lack of infrastructure and limitations of the scope of work do not have intramural programs to offset the work currently contracted to CRO. It is recommended that the sponsors carefully reassess in granular detail the current scope of work contracted to the CRO and the associated payment structure. They should consider alternate more cost-effective means to fulfilling some of such activities, with the aim to ultimately
American Heart Journal Volume 169, Issue 3
lower the costs of the trial overall and, in turn, support an alternate payment structure to the researchers and institutions directly involved in patient-based research. 6. Promote pragmatic trials There is growing concern regarding the costs of clinical trials. In addition, the elaborate eligibility criteria along with the intense monitoring of patients have raised questions regarding the generalizability of trial results. Thus, there is an interest in designing trials that are pragmatic in eligibility criteria and embedding trial follow-up and outcomes assessment within routine health care delivery processes. 35,36 Promoting pragmatic trials holds promise to improve enrollment and timeliness of trials as well as the generalizability of the results. For new drug development, careful consideration is required as to how the more disruptive approaches would meet the current expectations for registration trials. 37,38 In this respect, the growing infrastructure and connected networks of electronic health records hold promise to identify eligible patients and embed research procedures within routine clinical care contact with the patient. Besides participation and eligibility criteria, other aspects of the conduct of clinical trials need to be simplified as well to make the logistics of the trials less burdensome for both the investigators and the sponsors. For example, a central institutional review board could save time, energy, and potentially costs from having multiple institutional boards reviewing the protocol. In addition, risk-based monitoring including reduced source document verification, targeted monitoring of data points, and triggered monitoring based on triggers to detect issues can all lessen the burden of conducting clinical trials. Using remote monitoring resources in a centralized system may help. Leveraging technology as a communication tool with sites and to manage risks in clinical trials may be implemented. 7. Regional networks A large relevant patient population in any given site is most desirable. However, the productivity of the trial site is dependent not only on the volume of patients but also on the presence of engaged investigators leading an effective team. Identification of such investigators and teams for the various disease states at each institution, while also creating regional collaborative networks, is recommended as a potential means to increase trial productivity.
Butler et al 311
8. Better understanding of the trial sites Although many CROs conduct site surveys, these are primarily based on the recall or opinions of the site investigators, which are usually unverifiable. Such attempts have failed to reliably correlate with subsequent enrollment. Moreover, many investigators at sites may not be empowered to effectively navigate within the institution to conduct trials successfully. Thus, measures and methods for better evaluation of sites are needed, such as evidence of appropriate patient volume, leadership commitment, previous track record, patient flow within the institution, and feasibility of conducting the research procedures. 9. Training and education Consideration should be made to incorporate clinical trial didactics and practical experience into residency and fellowship training. This could be a formalized rotation in which trainees gain experience with protocols, institution review boards, case reports forms, informed consent, serious adverse event, and annual reports. In doing so, the training programs will train the next generation of investigators rather than expecting them to learn this on an ad hoc basis. 10. Increase patient engagement Better participation by the patient community is important to increase the efficiency of trials for multiple reasons. Patient involvement will demand greater efficiency by the clinical trial sites. Engaged and informed patient populations promise better recruitment and retention. Involving patients also can influence steering committees to ask more patient-relevant questions. Advocacy efforts in heart failure have been relatively nonexistent when compared with other major ailments such as cancer. Thus, increasing patient engagement in clinical trials is strongly recommended. 7 11. Public awareness and perception Efforts to further involve national and local news media and professional medical societies are urgently needed to make the public aware of (a) the importance of clinical trials in advancing scientific knowledge that is directly relevant to all members of the society; (b) the existing safeguards in place to protect human subjects, conflicts of interests rules, and protection against scientific misconduct; and (c) the problems inherent with the current state of clinical trials in the United States. 36 Although examples can certainly be cited for both scientific and financial misconduct related
American Heart Journal March 2015
312 Butler et al
Figure 2
Create Clinical Trials Units Redefine CostNeutral Economics Change Value Equation Investigator Support from Indirect Costs Reassess CRO costs and activities
Industry Cardiovascular Clinical Trials
Promote Pragmatic Trials
Recommendations
National Heart, Lung, and Blood Institute
Create Regional Networks Understand Trial Sites Status
Food and Drug Administration Train and Educate
Engage Patients
Increase Public Awareness
Summary recommendations for improving the state of heart failure clinical trials in the United States.
to the conduct of clinical trials, such examples have been sensationalized and do not represent the norm, and many protective rules against misconduct are now firmly in place. The proposed initial steps will neither be easy nor comprehensive to solve all problems (Figure 2). However, these are felt to be feasible and actionable items in the short to intermediate term. It is recommended that the leadership of medical centers, industry, funders, regulators, patient advocacy groups, and investigators work in concert to implement these recommended changes.
Disclaimer The opinions and information in this article are those of the authors and do not represent the views and/or policies of the FDA or the NHLBI.
Disclosures Authors’ Conflict of Interest: Javed Butler reports research support from the National Institutes of Health, European Union, and Health Resources Service Adminis-
tration and is a consultant to Amgen, Bayer, BG Medicine, Cardiocell, Celladon, GE Healthcare, Medtronic, Novartis, Ono Pharma, Otsuka, Takeda, Trevena, and Zensun. Gregg C. Fonarow has received research funding by the National Institutes of Health, Agency for Healthcare Research and Quality, and has served as a consultant for Amgen, Bayer, Gambro, Janssen, Novartis, Medtronic, and Medicines Company. Christopher O’Connor has received research funding by ResMed corporation, Otsuka, and Roche Diagnostics and has served as a consultant for ResMed corporation. Kirkwood Adams has received research funding by Amgen, Roche Diagnostics, Novartis Critical Diagnostics, Otsuka, Cardiorentis, Cardioxyl, Sorbent, and Covis Pharmaceutical and has served as a consultant for Covis Pharmaceuticals. Robert O. Bonow has no relevant relationships. Robert J. Cody is an employee of Janssen R&D. Sean P. Collins has served as a consultant for Novartis, Radiometer, Medtronic, The Medicines Company, Trevena, and Thermo-Fisher Scientific. Preston Dunnmon has no relationships relevant to the contents of this manuscript. Wilfried Dinh is an employee of Bayer HealthCare. Mona Fiuzat has received
American Heart Journal Volume 169, Issue 3
research funding by ResMed corporation, Otsuka, and Roche Diagnostics and has served as a consultant for ResMed corporation, Novartis, and Roche Diagnostics. Vasiliki V. Georgiopoulou has no relationships relevant to the contents of this manuscript to disclose. Stephen Grant has no relationships relevant to this manuscript. So-Young Kim is an employee of Bayer HealthCare. Stuart Kupfer is an employee of Takeda. Martin Lefkowitz is an employee of Novartis. Robert J. Mentz has received honoraria from ResMed, Thoratec, Novartis, and BristolMyers Squibb and research support from Gilead Sciences. Frank Misselwitz is an employee of and owns stock for Bayer HealthCare. Bertram Pitt has served as a consultant for Pfizer, Bayer, Relypsa, Stealth Peptides, and Mesoblast. Lothar Roessig is an employee of Bayer HealthCare. Erik Schelbert has received a Prohance contrast as a gift from Bracco for research. Monica Shah has no relationships relevant to the contents of this manuscript. Scott Solomon has received research support from and has served as a consultant for Novartis and Bayer. Norman Stockbridge has no relationships relevant to this manuscript. Clyde Yancy has no relationships relevant to this manuscript. Mihai Gheorghiade reports relationships with Abbott Laboratories, Astellas, AstraZeneca, Bayer Schering Pharma AG, Cardiorentis Ltd, CorThera, Cytokinetics, CytoPherx, Inc, DebioPharm S.A., Errekappa Terapeutici, GlaxoSmithKline, Ikaria, Intersection Medical Inc, Johnson & Johnson, Medtronic, Merck, Novartis Pharma AG, Ono Parmaceuticals USA, Otsuka Pharmaceuticals, Palatin Technologies, Pericor Therapeutics, Protein Design Laboratories, Sanofi-Aventis, Sigma Tau, Solvay Pharmaceuticals, Sticares InterACT, Takeda Pharmaceuticals North America, Inc and Trevena Therapeutics.
Acknowledgements We thank Ms Fumiko Inoue for organizing the meeting.
References 1. Schroen AT, Petroni GR, Wang H, et al. Preliminary evaluation of factors associated with premature trial closure and feasibility of accrual benchmarks in phase III oncology trials. Clin Trials 2010;7:312-21. 2. Chakma J, Sun GH, Steinberg JD, et al. Asia's ascent–global trends in biomedical R&D expenditures. N Engl J Med 2014;370:3-6. 3. IOM (Institute of Medicine). Public Engagement and Clinical Trials: New Models and Disruptive Technologies: Workshop Summary. Washington, DC: The National Academies Press. 2012. [ http://www.ncbi.nlm.nih. gov/books/NBK91498/pdf/TOC.pdf 06/04/2014]. 4. Kesselheim AS, Robertson CT, Myers JA, et al. A randomized study of how physicians interpret research funding disclosures. N Engl J Med 2012;367:1119-27. 5. Yusuf S. Randomized clinical trials: slow death by a thousand unnecessary policies? CMAJ 2004;171:889-92. [discussion 892–3]. 6. Gheorghiade M, Vaduganathan M, Greene SJ, et al. Site selection in global clinical trials in patients hospitalized for heart failure: perceived problems and potential solutions. Heart Fail Rev 2014;19:135-52. 7. O'Connor C. DUCCS and SWANS: when U.S. site-based research was highly successful. JACC Heart Fail 2013;1:366-7.
Butler et al 313
8. Gheorghiade M, Konstam MA, Burnett Jr JC, et al. Short-term clinical effects of tolvaptan, an oral vasopressin antagonist, in patients hospitalized for heart failure: the EVEREST Clinical Status Trials. JAMA 2007;297:1332-43. 9. Butler J, Subacius H, Vaduganathan M, et al. Relationship between clinical trial site enrollment with participant characteristics, protocol completion, and outcomes: insights from the EVEREST (Efficacy of Vasopressin Antagonism in Heart Failure: Outcome Study with Tolvaptan) trial. J Am Coll Cardiol 2013;61:571-9. 10. Glickman SW, McHutchison JG, Peterson ED, et al. Ethical and scientific implications of the globalization of clinical research. N Engl J Med 2009;360:816-23. 11. Tricoci P, Allen JM, Kramer JM, et al. Scientific evidence underlying the ACC/AHA clinical practice guidelines. JAMA 2009;301:831-41. 12. Institute of Medicine (US). Envisioning a Transformed Clinical Trials Enterprise in the United States: Establishing An Agenda for 2020: Workshop Summary. Appendix G, Discussion Paper: Developing a Clinical Trials Infrastructure. Washington (DC): National Academies Press (US). 2012. [http://www.ncbi.nlm.nih.gov/books/ NBK92275/pdf/TOC.pdf 06/04/201]. 13. Swedberg K, Komajda M, Bohm M, et al. Ivabradine and outcomes in chronic heart failure (SHIFT): a randomised placebo-controlled study. Lancet 2010;376:875-85. 14. McMurray JJ, Adamopoulos S, Anker SD, et al. ESC Guidelines for the diagnosis and treatment of acute and chronic heart failure 2012: The Task Force for the Diagnosis and Treatment of Acute and Chronic Heart Failure 2012 of the European Society of Cardiology. Developed in collaboration with the Heart Failure Association (HFA) of the ESC. Eur Heart J 2012;33:1787-847. 15. Pitt B, Pfeffer MA, Assmann SF, et al. Spironolactone for heart failure with preserved ejection fraction. N Engl J Med 2014;370:1383-92. 16. Mentz RJ, Cotter G, Cleland JG, et al. International differences in clinical characteristics, management, and outcomes in acute heart failure patients: better short-term outcomes in patients enrolled in Eastern Europe and Russia in the PROTECT trial. Eur J Heart Fail 2014;16:614-24. 17. Mentz RJ, Kaski JC, Dan GA, et al. Implications of geographical variation on clinical outcomes of cardiovascular trials. Am Heart J 2012;164:303-12. 18. Mahaffey KW, Wojdyla DM, Carroll K, et al. Ticagrelor compared with clopidogrel by geographic region in the Platelet Inhibition and Patient Outcomes (PLATO) trial. Circulation 2011;124:544-54. 19. Pitt B, Gheorghiade M. Geographic variation in heart failure trials: time for scepticism? Eur J Heart Fail 2014;16:601-2. 20. Goldstein DB, Tate SK, Sisodiya SM. Pharmacogenetics goes genomic. Nat Rev Genet 2003;4:937-47. 21. Bhatt DL, Steg PG, Ohman EM, et al. International prevalence, recognition, and treatment of cardiovascular risk factors in outpatients with atherothrombosis. JAMA 2006;295:180-9. 22. O'Connor CM, Fiuzat M, Swedberg K, et al. Influence of global region on outcomes in heart failure beta-blocker trials. J Am Coll Cardiol 2011;58:915-22. 23. DeMets DL, Califf RM. A historical perspective on clinical trials innovation and leadership: where have the academics gone? JAMA 2011;305:713-4. 24. Institute of Medicine (US) Forum on Drug Discovery, Development, and Translation. Transforming Clinical Research in the United States: Challenges and Opportunities: Workshop Summary. Washington (DC): National Academies Press (US). 2010. [http://www.nap.edu/ catalog/12900.html 06/05/2014]. 25. Lauer MS. Thought exercises on accountability and performance measures at the National Heart, Lung, and Blood Institute (NHLBI): an invited commentary for circulation research. Circ Res 2011;108:405-9.
314 Butler et al
26. Gordon D, Taddei-Peters W, Mascette A, et al. Publication of trials funded by the National Heart, Lung, and Blood Institute. N Engl J Med 2013;369:1926-34. 27. Devereaux PJ, Yusuf S. When it comes to trials, do we get what we pay for? N Engl J Med 2013;369:1962-3. 28. Dilts DM, Cheng SK, Crites JS, et al. Phase III clinical trial development: a process of chutes and ladders. Clin Cancer Res 2010;16:5381-9. 29. Abrams JS, Mooney MM, Zwiebel JA, et al. Implementation of timeline reforms speeds initiation of National Cancer Institute-sponsored trials. J Natl Cancer Inst 2013;105:954-9. 30. Dilts DM, Sandler AB. Invisible barriers to clinical trials: the impact of structural, infrastructural, and procedural barriers to opening oncology clinical trials. J Clin Oncol 2006;24:4545-52. 31. Dorricott K. Using metrics to direct performance improvement efforts in clinical trial management. Peer Reviewed http://www.beaufortcro.com/ wp-content/uploads/2012/11/August-2012-Monitor-article.pdf. [08/20/2014]. 32. Nelson G. Implementing metrics management for improving clinical trials performance. http://www.b-eye-network.com/view/7981. [08/20/2014].
American Heart Journal March 2015
33. Department of Health and Human Services. Office of Inspector General. Challenges To FDA’S Ability to Monitor and Inspect Foreign Clinical Trials. 2010. [http://oig.hhs.gov/oei/reports/oei-01-0800510.pdf 06/05/2014]. 34. Ohman EM, Roe MT. Explaining the unexpected: insights from the PLATelet inhibition and clinical Outcomes (PLATO) trial comparing ticagrelor and clopidogrel. Editorial on Serebruany "Viewpoint: Paradoxical excess mortality in the PLATO trial should be independently verified" (Thromb Haemost 2011; 105.5). Thromb Haemost 2011;105:763-5. 35. Califf RM, Platt R. Embedding cardiovascular research into practice. JAMA 2013;310:2037-8. 36. Califf RM, Sanderson I, Miranda ML. The future of cardiovascular clinical research: informatics, clinical investigators, and community engagement. JAMA 2012;308:1747-8. 37. Lauer MS, D'Agostino Sr RB. The randomized registry trial–the next disruptive technology in clinical research? N Engl J Med 2013;369: 1579-81. 38. Eapen ZJ, Lauer MS, Temple RJ. The imperative of overcoming barriers to the conduct of large, simple trials. JAMA 2014;311: 1397-8.
Improving cardiovascular clinical trials conduct in the United States: Recommendation from clinicians, researchers, sponsors, and regulators Javed Butler, MD, MPH, a Gregg C. Fonarow, MD, b Christopher O’Connor, MD, c Kirkwood Adams, MD, d Robert O. Bonow, MD, MS, e Robert J. Cody, MD, MBA, f Sean P. Collins, MD, MSc, g Preston Dunnmon, MD, MBA, h Wilfried Dinh, MD, i,j Mona Fiuzat, PharmD, c Vasiliki V. Georgiopoulou, MD, MPH, k Stephen Grant, MD, h So-Young Kim, MD, l Stuart Kupfer, MD, m Martin Lefkowitz, MD, n Robert J. Mentz, MD, c Frank Misselwitz, MD, PhD, l Bertram Pitt, MD, o Lothar Roessig, MD, l Erik Schelbert, MD, MS, p Monica Shah, MD, q Scott Solomon, MD, r Norman Stockbridge, MD, h Clyde Yancy, MD, MS, s and Mihai Gheorghiade, MD e Stony Brook, NY; Los Angeles, CA; Durham, NC; Chapel Hill, NC; Chicago, IL; Raritan, NJ; Nashville, TN; Silver Spring, MD; Berlin, Wuppertal, Witten, Germany; Atlanta, GA; Bayer HealthCare AG; Deerfield IL; East Hanover, NJ; Ann Arbor, MI; Pittsburgh, PA; Bethesda, MD; and Boston, MA
Advances in medical therapies leading to improved patient outcomes are in large part related to successful conduct of clinical trials that offer critical information regarding the efficacy and safety of novel interventions. The conduct of clinical trials in the United States, however, continues to face increasing challenges with recruitment and retention. These trends are paralleled by an increasing shift toward more multinational trials where most participants are enrolled in countries outside the United States, bringing into question the generalizability of the results to the American population. This manuscript presents the perspectives and recommendations from clinicians, researchers, sponsors, and regulators who attended a meeting facilitated by the Food and Drug Administration to improve upon the current clinical trial trends in the United States. (Am Heart J 2015;169:305-14.)
Clinical trials provide essential information that forms the foundation for evidence-based medicine. Significant advances have been made in the treatment of many cardiovascular diseases over the last several decades, in large part related to conduct of robust clinical trials that generated evidence for benefit, neutrality, or harm with From the aCardiology Division, Stony Brook University, Stony Brook, NY, bUniversity of California, Los Angeles, CA, cDuke Clinical Research Institute, Duke University Medical Center, Durham, NC, dUniversity of North Carolina, Chapel Hill, NC, eCenter for Cardiovascular Innovation, Northwestern University Feinberg School of Medicine, Chicago, IL, fCardiovascular & Metabolism, Janssen Pharmaceuticals, Raritan, NJ, g Vanderbilt University, Nashville, TN, hCenter for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, MD, iDepartment of Cardiology, Witten University, Witten, Germany, jGlobal Drug Discovery, Clinical Sciences, Bayer Pharma AG, Berlin, Germany, kEmory Cardiovascular Clinical Research Institute, Emory University, Atlanta, GA, lGlobal Clinical Development, Bayer HealthCare AG, Wuppertal, Germany, m Takeda Pharmaceuticals International, Deerfield IL, nNovartis Pharmaceuticals Inc., East Hanover, NJ, oDivision of Cardiology, University of Michigan School of Medicine, Ann Arbor, MI, pUniversity of Pittsburgh Medical Center Heart and Vascular Institute, Pittsburgh, PA, qDivision of Cardiovascular Sciences, National Heart, Lung, and Blood Institute, Bethesda, MD, rCardiovascular Division, Brigham and Women’s Hospital, Boston, MA, and sDivision of Cardiology, Department of Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL. James A de Lemos, MD, served as guest editor for this article. Submitted October 23, 2014; accepted December 10, 2014. Reprint requests: Javed Butler, MD, MPH, Cardiology Division, Stony Brook University, Health Sciences Center, T-16, Room 080 SUNY at Stony Brook, NY 11794. E-mail: [email protected] 0002-8703 © 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.ahj.2014.12.001
the therapies evaluated. These trials informed how to best treat patients and which therapies to avoid. Despite these discoveries over the last several decades, there remains a continued need for improvement in outcomes and, thus, for more clinical trials, for all cardiovascular diseases, especially for conditions that are highly prevalent and associated with continued poor outcomes, for example, heart failure with preserved ejection fraction or worsening heart failure requiring hospitalization. The conduct of clinical trials in the United States has become increasingly challenging with (1) suboptimal enrollment and retention, (2) exorbitant expenses, (3) declining funding, (4) lack of motives and growing clinical responsibilities for researchers, (5) negative perception of industry, and (6) distribution models for trial finances that may not be optimal. 1-7 For example, 359 sites in the Efficacy of Vasopressin Antagonism in Heart Failure: Outcome Study with Tolvaptan (EVEREST) trial enrolled 4133 patients. 8 Overall, 21% of sites in North America did not enroll any patients. Among sites that did enroll, the median enrollment over the 28-month trial period was 7 participants per site in North America, compared with more than 20 in South America and Eastern Europe. 9 Accordingly, the proportion of multinational trials continues to increase, as does the proportion non-US patients enrolled within those trials. 10 This trend is even more prominent in recent trials. In the Aliskiren Trial on Acute
American Heart Journal March 2015
306 Butler et al
Table I. Concerns about conducting clinical trials outside the United States Variant characteristic Patient risk profile Availability of therapeutic interventions Affordability of therapeutic interventions Availability of hospital infrastructure Availability of clinic infrastructure Clinical practice and standard of care Disease outcomes Genetic profile Cultural/social background
Heart Failure Outcomes (ASTRONAUT) trial, 1615 patients were randomized at 316 sites with an average enrollment rate per site of 2.9 ± 2.4 in North America, 4.1 ± 3.2 in Western Europe, 4.9 ± 4.7 in South America, 7.1 ± 6.2 in Eastern Europe, and 9.5 ± 9.8 in Asia (personal communication). These challenges are faced in the area of cardiovascular diseases in general. To facilitate understanding of the pertinent issues from the various stakeholders’ perspectives and to develop a set of actionable recommendations to improve upon the current trends in clinical trials, the Food and Drug Administration (FDA) hosted a meeting on February 21, 2014. Representatives from academia; industry; the National Heart, Lung, and Blood Institute (NHLBI); and the FDA attended this meeting, which was not industry or otherwise sponsored. This document summarizes the discussions and conclusions of this meeting.
Importance of conducting clinical trials in the United States Only a small proportion of practice guidelines are based on clinical trial data. 11 When such data are available, there continues to be a growing shift to more international trials with patients recruited throughout the globe. The United States constitutes about 5% of the world’s population and has one of the most expensive clinical trials enterprises, and over time, many trials have a limited representation of participants from the United States and some have none at all. 12 For example, the clinical trials with ivabradine were all conducted outside the United States, and the drug is already recommended for use by the European Society of Cardiology heart failure management guidelines but is not currently available in the United States. 13,14 High-quality research from outside the United States is meritorious, and the FDA accepts such data to approve drugs based on research conducted largely or solely outside the United States. However, concerns have been raised regarding the applicability of the results generated from other regions to patients in the United States (Table I) 10 considering geographic variations in (a) patients’ risk profile,
Concern Patient population is not representative Proven therapies not always provided
Appropriate follow-up and evaluation are not provided Disease progress is not representative Disease epidemiology is not representative Effectiveness of therapies is not representative Acceptance of therapies is not representative
(b) background therapy, and (c) resource availability related to standard care. 15-17 Regional differences in practice and in outcomes are well known. 18,19 Genetic and environmental dissimilarities may impact treatment effects. 20 Prevalent standards of care and baseline therapy differences between regions might affect the association of a drug with outcomes. 21 Regional differences in the magnitude of benefit from β-blockers were hypothesized to reflect differences in population, genetics, differences in disease management, or to chance finding. 22 Additional concerns have been raised regarding the human-subject protection and quality of conduct of some of the multicenter trials. The United States has long led innovation in biomedical research. However, if the current trend of declining clinical trial expertise continues and a robust network of competent sites and clinical trial investigators is not rejuvenated, then the further worsening of recruitment and retention in the United States may become insurmountable. 23 To guide the most optimal treatment recommendations for the American population, it is imperative that the clinical trials enterprise in the United States be revitalized to generate relevant, reliable, and pertinent data applicable to patients in the United States.
Difficulties in conducting clinical trials The difficulties in conducting clinical trials in the United States are multifold. Different stakeholders responsible for the successful conduct of clinical trials have varying perspectives on the barriers to optimally perform trials. The academic productivity from multicenter clinical trials has been traditionally limited to a few investigators, generally those who serve on the steering committee or the publication committee, etc. The site investigators in general have not benefitted in this respect. In the spirit of the growing consensus around the concept of team science, one may postulate that more academic opportunities to the site investigators may increase productivity. However, authorship rights based on enrolling patients may not meet the criteria for such designation, and the site investigators should be more actively involved in the academic products from the
American Heart Journal Volume 169, Issue 3
Butler et al 307
Figure 1
Distribution of funds in clinical trials. Adapted from Califf, RM, ACS and Acute Heart Failure Models – Speaker presentation at the Institute of Medicine Workshop on Transforming Clinical Research in the United States, October 7 to 8, 2009, Washington, DC. The costs are presented in US dollars (millions). There is an additional cost for publications ($0.1 million [0.04%]) that is not shown.
trials. The concept of team science is that the various scientific disciplines come together to bring varying expertise to achieve results more globally and efficiently. Whether enrolling patients at the site level meet these criteria has not been resolved.
Investigator There are usually large pools of patients with various cardiac diseases being treated at centers that participate in trials. Thus, the primary impediment remains to be investigator involvement and not patient availability. Clinicians perform trials for the large part but are facing growing pressures to increase clinical productivity. Lack of discretionary funds makes it impossible to protect time for clinicians to perform trials. Researchers are encouraged to focus either on clinical activities or on investigator-initiated grants that provide salary support. Federal funding for trials has become increasingly competitive, requiring years of effort in grant writing and submitting revisions. With changes in federal research funding, these efforts are often unsuccessful. Most trials provide no salary support for investigators, unless they are investigator initiated. Thus, clinicians choose either to not be involved in trials or to delegate the responsibility to study coordinators and, in turn, distancing themselves from active involvement. Conduct of trials for most clinicians is not part of their job description but an optional academic avocation. In addition, the patient pool at any given institution belongs to several providers beyond the site principal investigator. Unless the institutional culture dictates an emphasis on enrollment, it might prove difficult to enroll patients who are not treated primarily by the investigators themselves. The overall costs of clinical trials continue to increase. However, a large proportion of this funding is earmarked for a clinical research organization (CRO) for trial management (Figure 1). 24 From the funds that come to the site, a high proportion is used for maintaining the
regulatory and compliance infrastructure. The support to the actual investigators, despite overall high costs of trials, remains suboptimal. A large part of the effort in a clinical trial is not related to the research procedures after enrolling a patient but to pre-enrollment administrative and screening activities, discussion with patients and families, and convincing other providers; these are all activities that require time and effort by the clinicians but remain unsupported. Thus, although the cost of conducting trials has increased, these funds do not distill down to the researchers and their coordinators tasked with doing the work directly related to research. Academic research organizations play a significant role in clinical trials conduct through partnerships with academia or other stakeholders in recruiting sites through established networks and relationships and can implement all aspects associated with managing a successful study. However, costs are an issue with them as well as it is with CROs. The nonmonetary returns from participation in trials are ambiguous as well. Participation in multicenter trials as a site investigator without involvement in the publications, which is usually limited to a minority of investigators, usually does not support promotion in academia. Most investigators are not part of the steering or publication committees, and even if cited as one of the many authors on a publication, this recognition is not of major consequence for local recognition or career advancement. In addition, investigators may prefer some trials over others because of specific scientific interest or relationship with the sponsor. This can lead to enrollment of more patients in one trial at the expense of others. This competing trials dynamic may lead to poor enrollment in specific trials even in otherwise established centers and should be taken into account when sites are selected for any given trial.
National Heart, Lung, and Blood Institute The National Heart, Lung, and Blood Institute supports a wide range of fundamental clinical and population
308 Butler et al
research, including several clinical trials. Faced with the challenge of shrinking budgets and increasing demands, the institute is confronting unprecedented pressure to support more science with fewer resources. Clinical trials are a high scientific priority at NHLBI, but given the current fiscal environment, these studies are under particular scrutiny because they represent some of the largest investments at the institute. In response to these economic pressures, NHLBI is striving to optimize the use of the budget while still maximizing scientific productivity and ensuring the highest quality scientific results. To ensure optimal productivity, NHLBI is working with investigators to enhance the efficiency and operations of clinical trials and clinical trials networks. To that end, there is an increasing focus on the simplification of trial design, the use of clinical end points, the reduction of extraneous data collection, and the completion of studies on time and within budget. 25-29 Recently published data from NHLBI demonstrated that trials with clinical end points were published significantly faster than trials with surrogate end points. This type of information is helping the institute to better understand what types of trials are more likely to optimize a return on research investments. 26,27 In addition, NHLBI is encouraging investigators to use operational performance measures or “how to” metrics, such as enrollment rates, data quality measures, and time for trial start-up, to regularly track progress and enhance efficiency. 30-32 Strategies to streamline clinical trial operations, such as the use of central institutional review boards and central contracts, are also being encouraged. The National Heart, Lung, and Blood Institute funds several clinical trial networks, which provide support for operational infrastructure and dedicated investigator and study coordinator time. The purpose of these investments is to facilitate and expedite clinical studies. The use of “how to” metrics may help the institute gauge whether these investments are actually enhancing productivity. Although the fiscal crisis presents challenges, it also offers unique opportunities for both NHLBI and the scientific community to develop new ideas about clinical trials, disrupt conventional thinking, re-examine standard procedures, and create a new paradigm of efficiency and scientific productivity for the future.
Industry Industry sponsors most clinical trials, and their perspective mirrors that of the National Institutes of Health. Although sponsors’ interest in conducting clinical trials is, in part, driven by return of investment and the federal funding agencies interests are driven by its mission to improve health, lengthen life, and reduce illness and disability; they both face similar difficulties and challenges in conducting clinical trials. Industry remains concerned with both the growing costs and time related to the conduct of trials in the United States. It is apparent that a few sites enroll most
American Heart Journal March 2015
patients, and most of the remaining sites recruit few. As a result, there are large costs expended on the great number of poor-enrolling sites. Industry recognizes the importance of United States both as an engine for innovation and as a viable market, but they increasingly support trials in other regions to compensate for low US contribution to recruitment to meet timelines and cost targets. The National Institutes of Health has this dilemma in common with industry in which the time required to complete a trial in North America may be prohibitive, and under time pressures, other countries are chosen to substitute for lower than expected recruitment in the United States. 15 Industry is interested in promoting a more effective clinical trials infrastructure in the United States and realizes the high costs of CROs, but many do not have in-house capabilities to operationalize trials and fulfill regulatory requirements, necessitating the need to contract with CROs.
Food and Drug Administration With respect to the evaluation of US marketing applications for drugs and biologics, there are no FDA regulations that uniquely apply to US sites. Likewise, there are no FDA regulations prohibiting the approval of drugs in the United States based on data generated from other regions of the world. Indeed, FDA’s review process for drugs and biologics is the same regardless of whether a marketing application is based on data that are generated exclusively or partially from the United States or completely outside the United States. 33 However, for applications that are based in part or in totally on non-US data, the code of federal regulations (CFR 312.120 and CFR 314.106) requires FDA to confirm that these applications meet US criteria for marketing approval. Specifically, they must (a) show that the data are applicable to the US target patient population and to contemporary US medical practice, (b) show that the data have been generated by investigators of recognized competence, and (c) be able to be validated by FDA through on-site inspections of non-US sites of FDA’s choosing or other appropriate means. Furthermore, just as is the case with US sites, all non-US sites must be under institutional review board or similar oversight, must retain copies of informed consents signed by all subjects, and are subject to US financial disclosure rules. Thus, assuming that a non-US data set meets its prespecified efficacy end point and statistical hurdles, that FDA identifies no important trial conduct deficiencies, and that the sponsors can make a compelling benefit versus risk argument for their product, these applications face the additional hurdle of demonstrating unequivocally that their data are applicable to the US target patient population and to contemporary US medical practice, something that is assumed to be true for US-generated data. Demonstrating relevance to US patients and medical
American Heart Journal Volume 169, Issue 3
Butler et al 309
Table II. Recommendations for improving clinical trials enterprise in the United States Developing clinical trials units
Redefining cost-neutral economics of trials Changing the value equation of clinical trials
Dedicating proportional support from indirect costs Reassess CRO costs and activities
• • • • • • • • • • • • • • •
Promote pragmatic trials
• •
Regional networks
• • •
Understanding trial sites
• Training and education
• • •
Patient engagement
• •
Public awareness and perception
• • •
Dedicated physical space Streamlined regulatory and contractual processes Research services and procedures Local investigator collaboration Higher support for fewer high-potential sites Support dedicated staff, registries, screening Motives for promotion Salary support Leadership roles in trials Cultural support of trials at sites Involvement on publication committees Earmark proportion of indirect costs to develop research infrastructure Support investigator-initiated proposals with indirect funding Reassessment of the current scope of work contracted to the CRO and the associated payment structure Consideration of alternate more cost-effective means to fulfill some of the CRO activities Easier eligibility criteria standardized across studies within 1 indication Embed follow-up and outcomes ascertainment within routine care and electronic health records Disease-focused regional networks Collaboration and competition with motives Establishment of measures and methods to evaluate sites potential more effectively Evaluation of appropriate patient volume, leadership commitment, past track record, and patient flow are key factors Clinical trials rotation in residency and fellowship Human subjects training and clinical trials training Involvement in clinical events committee and data safety monitoring boards, etc. Patient advocacy efforts Focus on patient-centered questions and outcomes in alignment with advocacy involvement Involvement of national and local media Involvement of professional medical societies Reverse the perception of financial interest as the main reason for conducting clinical trials
practice is a multifaceted analysis during which the FDA evaluates US versus non-US data attributes, such as whether 1. The disease of interest was defined the same way in geographies outside the United States as within the United States 2. Efficacy events of interest were uniformly defined, ascertained, and measured 3. Safety events were uniformly ascertained, monitored, and assessed 4. Background therapies and their use were consistent with US medical standards 5. Patient factors outside the United States are not different in an important way (ie, demographics and behaviors reflect the target US patient population).
Therefore, the perception of a disproportionately heavy regulatory burden as a contributor to the decreasing participation of US sites in clinical research is highly doubtful, given that marketing applications submitted to FDA for drugs and biologics that are based in part or in total on non-US data sources are governed by the same regulations and must meet the same requirements for well-controlled, high-quality data to support approval as do applications based solely on US-generated data. Indeed, it can be argued that there is actually an additional level of complexity for sponsors using non-US data in that these sponsors assume the burden of proving the relevance of their data to the US target patient population and medical system. 33,34
310 Butler et al
Recommendations The participants acknowledge that more comprehensive reform may be needed but present the following list of practical action items as a starting point to revamp the clinical trials enterprise in the United States within the current academic and health care environment (Table II). 1. Developing clinical trial units Inefficiencies within hospitals conducting clinical trials, in part, arise from the lack of physical and functional well-defined infrastructure that results in logistic and time burden on patients, staff, and researchers. The numbers of procedures conducted as part of research in any given field are largely finite. Investment in developing physical clinical trial units (CTUs) that support the administrative needs including those related to contracts and institutional review board approval, streamlining costs structures, and performing most research procedures are likely to be more efficient and cost saving. These CTUs will need to be used on a continuous basis because the costs of their overhead and ongoing maintenance may otherwise become unsustainable. One of the most important functions of a CTU is bringing investigators together to facilitate collaboration. This model of collaboration is crucial for successful identification and enrollment, management, and follow-up of participants and often requires collaboration of investigators across specialties. 2. Redefining cost-neutral economics of clinical trials Clinical trials commonly engage a large number of sites because of the expected slow enrollment by the majority. Initiation and maintenance of low-performing sites are expensive. On the other hand, supporting front line investigators more when the overall cost of conducting trials is already a concern is unrealistic. A cost-neutral reorganization of trials is likely to yield positive results, with better support of investigators and sites with a track record, at the expense of not initiating low-performing sites. Strategies may include salary support for investigators and coordinators as well as other support such as site-based registries. This hybrid model between the traditional per-patient payment structure of trials and the salary support model of investigator initiated studies has potential benefits because it provides support for screening efforts and other trial activities between patient enrollments. If budgeted correctly, a lower overall per-patient cost can also be economically more efficient for the sponsor.
American Heart Journal March 2015
3. Changing the value equation of site-based clinical trials Clinicians are increasingly disenfranchised with trials not only because of the monetary equation but also because of other career counterincentives. Better recognition of successful trial conduct such as a metric for promotion, irrespective of authorship, is likely to motivate clinicians. The investigators should be recognized with salary support, promotion, and tenure, based on high-quality performance in site-based research, and should be granted leadership opportunities in research units locally and participation in institutional activities related to research. In addition, they should be rewarded for their expertise in study execution similar to scientific planning and data analysis. For instance, they should have the opportunity to participate not only in publications but also on clinical events committees, steering committees, and other ancillary committees involved in multicenter clinical trials. It is understood that these committees have a limited membership and may require experience and contributions beyond those characterizing a successful investigator. Nonetheless, it is important to identify and train the next generation of trial experts. 4. Dedicating proportional support of indirect dollars to trials The indirect dollars associated with clinical trial funding are largely not used for the personnel and procedures directly related to the trials. Although the regulatory and compliance support through indirect dollars is essential, dedicating a defined proportion of these indirect support resources to enhance clinical trial infrastructure and investigator-initiated studies may be considered. This proposition, although complex and requiring input from multiple representatives in many academic institutions, is necessary to maintain ongoing research support and enthusiasm. 5. Reassess CRO costs and activities The sponsor payments to CRO often greatly exceed the payments for regulatory and compliance activities at the research institution level. Because of the for-profit nature of many CROs, there remains a motive for large margins for CRO-related activities. Most sponsors for reasons of lack of infrastructure and limitations of the scope of work do not have intramural programs to offset the work currently contracted to CRO. It is recommended that the sponsors carefully reassess in granular detail the current scope of work contracted to the CRO and the associated payment structure. They should consider alternate more cost-effective means to fulfilling some of such activities, with the aim to ultimately
American Heart Journal Volume 169, Issue 3
lower the costs of the trial overall and, in turn, support an alternate payment structure to the researchers and institutions directly involved in patient-based research. 6. Promote pragmatic trials There is growing concern regarding the costs of clinical trials. In addition, the elaborate eligibility criteria along with the intense monitoring of patients have raised questions regarding the generalizability of trial results. Thus, there is an interest in designing trials that are pragmatic in eligibility criteria and embedding trial follow-up and outcomes assessment within routine health care delivery processes. 35,36 Promoting pragmatic trials holds promise to improve enrollment and timeliness of trials as well as the generalizability of the results. For new drug development, careful consideration is required as to how the more disruptive approaches would meet the current expectations for registration trials. 37,38 In this respect, the growing infrastructure and connected networks of electronic health records hold promise to identify eligible patients and embed research procedures within routine clinical care contact with the patient. Besides participation and eligibility criteria, other aspects of the conduct of clinical trials need to be simplified as well to make the logistics of the trials less burdensome for both the investigators and the sponsors. For example, a central institutional review board could save time, energy, and potentially costs from having multiple institutional boards reviewing the protocol. In addition, risk-based monitoring including reduced source document verification, targeted monitoring of data points, and triggered monitoring based on triggers to detect issues can all lessen the burden of conducting clinical trials. Using remote monitoring resources in a centralized system may help. Leveraging technology as a communication tool with sites and to manage risks in clinical trials may be implemented. 7. Regional networks A large relevant patient population in any given site is most desirable. However, the productivity of the trial site is dependent not only on the volume of patients but also on the presence of engaged investigators leading an effective team. Identification of such investigators and teams for the various disease states at each institution, while also creating regional collaborative networks, is recommended as a potential means to increase trial productivity.
Butler et al 311
8. Better understanding of the trial sites Although many CROs conduct site surveys, these are primarily based on the recall or opinions of the site investigators, which are usually unverifiable. Such attempts have failed to reliably correlate with subsequent enrollment. Moreover, many investigators at sites may not be empowered to effectively navigate within the institution to conduct trials successfully. Thus, measures and methods for better evaluation of sites are needed, such as evidence of appropriate patient volume, leadership commitment, previous track record, patient flow within the institution, and feasibility of conducting the research procedures. 9. Training and education Consideration should be made to incorporate clinical trial didactics and practical experience into residency and fellowship training. This could be a formalized rotation in which trainees gain experience with protocols, institution review boards, case reports forms, informed consent, serious adverse event, and annual reports. In doing so, the training programs will train the next generation of investigators rather than expecting them to learn this on an ad hoc basis. 10. Increase patient engagement Better participation by the patient community is important to increase the efficiency of trials for multiple reasons. Patient involvement will demand greater efficiency by the clinical trial sites. Engaged and informed patient populations promise better recruitment and retention. Involving patients also can influence steering committees to ask more patient-relevant questions. Advocacy efforts in heart failure have been relatively nonexistent when compared with other major ailments such as cancer. Thus, increasing patient engagement in clinical trials is strongly recommended. 7 11. Public awareness and perception Efforts to further involve national and local news media and professional medical societies are urgently needed to make the public aware of (a) the importance of clinical trials in advancing scientific knowledge that is directly relevant to all members of the society; (b) the existing safeguards in place to protect human subjects, conflicts of interests rules, and protection against scientific misconduct; and (c) the problems inherent with the current state of clinical trials in the United States. 36 Although examples can certainly be cited for both scientific and financial misconduct related
American Heart Journal March 2015
312 Butler et al
Figure 2
Create Clinical Trials Units Redefine CostNeutral Economics Change Value Equation Investigator Support from Indirect Costs Reassess CRO costs and activities
Industry Cardiovascular Clinical Trials
Promote Pragmatic Trials
Recommendations
National Heart, Lung, and Blood Institute
Create Regional Networks Understand Trial Sites Status
Food and Drug Administration Train and Educate
Engage Patients
Increase Public Awareness
Summary recommendations for improving the state of heart failure clinical trials in the United States.
to the conduct of clinical trials, such examples have been sensationalized and do not represent the norm, and many protective rules against misconduct are now firmly in place. The proposed initial steps will neither be easy nor comprehensive to solve all problems (Figure 2). However, these are felt to be feasible and actionable items in the short to intermediate term. It is recommended that the leadership of medical centers, industry, funders, regulators, patient advocacy groups, and investigators work in concert to implement these recommended changes.
Disclaimer The opinions and information in this article are those of the authors and do not represent the views and/or policies of the FDA or the NHLBI.
Disclosures Authors’ Conflict of Interest: Javed Butler reports research support from the National Institutes of Health, European Union, and Health Resources Service Adminis-
tration and is a consultant to Amgen, Bayer, BG Medicine, Cardiocell, Celladon, GE Healthcare, Medtronic, Novartis, Ono Pharma, Otsuka, Takeda, Trevena, and Zensun. Gregg C. Fonarow has received research funding by the National Institutes of Health, Agency for Healthcare Research and Quality, and has served as a consultant for Amgen, Bayer, Gambro, Janssen, Novartis, Medtronic, and Medicines Company. Christopher O’Connor has received research funding by ResMed corporation, Otsuka, and Roche Diagnostics and has served as a consultant for ResMed corporation. Kirkwood Adams has received research funding by Amgen, Roche Diagnostics, Novartis Critical Diagnostics, Otsuka, Cardiorentis, Cardioxyl, Sorbent, and Covis Pharmaceutical and has served as a consultant for Covis Pharmaceuticals. Robert O. Bonow has no relevant relationships. Robert J. Cody is an employee of Janssen R&D. Sean P. Collins has served as a consultant for Novartis, Radiometer, Medtronic, The Medicines Company, Trevena, and Thermo-Fisher Scientific. Preston Dunnmon has no relationships relevant to the contents of this manuscript. Wilfried Dinh is an employee of Bayer HealthCare. Mona Fiuzat has received
American Heart Journal Volume 169, Issue 3
research funding by ResMed corporation, Otsuka, and Roche Diagnostics and has served as a consultant for ResMed corporation, Novartis, and Roche Diagnostics. Vasiliki V. Georgiopoulou has no relationships relevant to the contents of this manuscript to disclose. Stephen Grant has no relationships relevant to this manuscript. So-Young Kim is an employee of Bayer HealthCare. Stuart Kupfer is an employee of Takeda. Martin Lefkowitz is an employee of Novartis. Robert J. Mentz has received honoraria from ResMed, Thoratec, Novartis, and BristolMyers Squibb and research support from Gilead Sciences. Frank Misselwitz is an employee of and owns stock for Bayer HealthCare. Bertram Pitt has served as a consultant for Pfizer, Bayer, Relypsa, Stealth Peptides, and Mesoblast. Lothar Roessig is an employee of Bayer HealthCare. Erik Schelbert has received a Prohance contrast as a gift from Bracco for research. Monica Shah has no relationships relevant to the contents of this manuscript. Scott Solomon has received research support from and has served as a consultant for Novartis and Bayer. Norman Stockbridge has no relationships relevant to this manuscript. Clyde Yancy has no relationships relevant to this manuscript. Mihai Gheorghiade reports relationships with Abbott Laboratories, Astellas, AstraZeneca, Bayer Schering Pharma AG, Cardiorentis Ltd, CorThera, Cytokinetics, CytoPherx, Inc, DebioPharm S.A., Errekappa Terapeutici, GlaxoSmithKline, Ikaria, Intersection Medical Inc, Johnson & Johnson, Medtronic, Merck, Novartis Pharma AG, Ono Parmaceuticals USA, Otsuka Pharmaceuticals, Palatin Technologies, Pericor Therapeutics, Protein Design Laboratories, Sanofi-Aventis, Sigma Tau, Solvay Pharmaceuticals, Sticares InterACT, Takeda Pharmaceuticals North America, Inc and Trevena Therapeutics.
Acknowledgements We thank Ms Fumiko Inoue for organizing the meeting.
References 1. Schroen AT, Petroni GR, Wang H, et al. Preliminary evaluation of factors associated with premature trial closure and feasibility of accrual benchmarks in phase III oncology trials. Clin Trials 2010;7:312-21. 2. Chakma J, Sun GH, Steinberg JD, et al. Asia's ascent–global trends in biomedical R&D expenditures. N Engl J Med 2014;370:3-6. 3. IOM (Institute of Medicine). Public Engagement and Clinical Trials: New Models and Disruptive Technologies: Workshop Summary. Washington, DC: The National Academies Press. 2012. [ http://www.ncbi.nlm.nih. gov/books/NBK91498/pdf/TOC.pdf 06/04/2014]. 4. Kesselheim AS, Robertson CT, Myers JA, et al. A randomized study of how physicians interpret research funding disclosures. N Engl J Med 2012;367:1119-27. 5. Yusuf S. Randomized clinical trials: slow death by a thousand unnecessary policies? CMAJ 2004;171:889-92. [discussion 892–3]. 6. Gheorghiade M, Vaduganathan M, Greene SJ, et al. Site selection in global clinical trials in patients hospitalized for heart failure: perceived problems and potential solutions. Heart Fail Rev 2014;19:135-52. 7. O'Connor C. DUCCS and SWANS: when U.S. site-based research was highly successful. JACC Heart Fail 2013;1:366-7.
Butler et al 313
8. Gheorghiade M, Konstam MA, Burnett Jr JC, et al. Short-term clinical effects of tolvaptan, an oral vasopressin antagonist, in patients hospitalized for heart failure: the EVEREST Clinical Status Trials. JAMA 2007;297:1332-43. 9. Butler J, Subacius H, Vaduganathan M, et al. Relationship between clinical trial site enrollment with participant characteristics, protocol completion, and outcomes: insights from the EVEREST (Efficacy of Vasopressin Antagonism in Heart Failure: Outcome Study with Tolvaptan) trial. J Am Coll Cardiol 2013;61:571-9. 10. Glickman SW, McHutchison JG, Peterson ED, et al. Ethical and scientific implications of the globalization of clinical research. N Engl J Med 2009;360:816-23. 11. Tricoci P, Allen JM, Kramer JM, et al. Scientific evidence underlying the ACC/AHA clinical practice guidelines. JAMA 2009;301:831-41. 12. Institute of Medicine (US). Envisioning a Transformed Clinical Trials Enterprise in the United States: Establishing An Agenda for 2020: Workshop Summary. Appendix G, Discussion Paper: Developing a Clinical Trials Infrastructure. Washington (DC): National Academies Press (US). 2012. [http://www.ncbi.nlm.nih.gov/books/ NBK92275/pdf/TOC.pdf 06/04/201]. 13. Swedberg K, Komajda M, Bohm M, et al. Ivabradine and outcomes in chronic heart failure (SHIFT): a randomised placebo-controlled study. Lancet 2010;376:875-85. 14. McMurray JJ, Adamopoulos S, Anker SD, et al. ESC Guidelines for the diagnosis and treatment of acute and chronic heart failure 2012: The Task Force for the Diagnosis and Treatment of Acute and Chronic Heart Failure 2012 of the European Society of Cardiology. Developed in collaboration with the Heart Failure Association (HFA) of the ESC. Eur Heart J 2012;33:1787-847. 15. Pitt B, Pfeffer MA, Assmann SF, et al. Spironolactone for heart failure with preserved ejection fraction. N Engl J Med 2014;370:1383-92. 16. Mentz RJ, Cotter G, Cleland JG, et al. International differences in clinical characteristics, management, and outcomes in acute heart failure patients: better short-term outcomes in patients enrolled in Eastern Europe and Russia in the PROTECT trial. Eur J Heart Fail 2014;16:614-24. 17. Mentz RJ, Kaski JC, Dan GA, et al. Implications of geographical variation on clinical outcomes of cardiovascular trials. Am Heart J 2012;164:303-12. 18. Mahaffey KW, Wojdyla DM, Carroll K, et al. Ticagrelor compared with clopidogrel by geographic region in the Platelet Inhibition and Patient Outcomes (PLATO) trial. Circulation 2011;124:544-54. 19. Pitt B, Gheorghiade M. Geographic variation in heart failure trials: time for scepticism? Eur J Heart Fail 2014;16:601-2. 20. Goldstein DB, Tate SK, Sisodiya SM. Pharmacogenetics goes genomic. Nat Rev Genet 2003;4:937-47. 21. Bhatt DL, Steg PG, Ohman EM, et al. International prevalence, recognition, and treatment of cardiovascular risk factors in outpatients with atherothrombosis. JAMA 2006;295:180-9. 22. O'Connor CM, Fiuzat M, Swedberg K, et al. Influence of global region on outcomes in heart failure beta-blocker trials. J Am Coll Cardiol 2011;58:915-22. 23. DeMets DL, Califf RM. A historical perspective on clinical trials innovation and leadership: where have the academics gone? JAMA 2011;305:713-4. 24. Institute of Medicine (US) Forum on Drug Discovery, Development, and Translation. Transforming Clinical Research in the United States: Challenges and Opportunities: Workshop Summary. Washington (DC): National Academies Press (US). 2010. [http://www.nap.edu/ catalog/12900.html 06/05/2014]. 25. Lauer MS. Thought exercises on accountability and performance measures at the National Heart, Lung, and Blood Institute (NHLBI): an invited commentary for circulation research. Circ Res 2011;108:405-9.
314 Butler et al
26. Gordon D, Taddei-Peters W, Mascette A, et al. Publication of trials funded by the National Heart, Lung, and Blood Institute. N Engl J Med 2013;369:1926-34. 27. Devereaux PJ, Yusuf S. When it comes to trials, do we get what we pay for? N Engl J Med 2013;369:1962-3. 28. Dilts DM, Cheng SK, Crites JS, et al. Phase III clinical trial development: a process of chutes and ladders. Clin Cancer Res 2010;16:5381-9. 29. Abrams JS, Mooney MM, Zwiebel JA, et al. Implementation of timeline reforms speeds initiation of National Cancer Institute-sponsored trials. J Natl Cancer Inst 2013;105:954-9. 30. Dilts DM, Sandler AB. Invisible barriers to clinical trials: the impact of structural, infrastructural, and procedural barriers to opening oncology clinical trials. J Clin Oncol 2006;24:4545-52. 31. Dorricott K. Using metrics to direct performance improvement efforts in clinical trial management. Peer Reviewed http://www.beaufortcro.com/ wp-content/uploads/2012/11/August-2012-Monitor-article.pdf. [08/20/2014]. 32. Nelson G. Implementing metrics management for improving clinical trials performance. http://www.b-eye-network.com/view/7981. [08/20/2014].
American Heart Journal March 2015
33. Department of Health and Human Services. Office of Inspector General. Challenges To FDA’S Ability to Monitor and Inspect Foreign Clinical Trials. 2010. [http://oig.hhs.gov/oei/reports/oei-01-0800510.pdf 06/05/2014]. 34. Ohman EM, Roe MT. Explaining the unexpected: insights from the PLATelet inhibition and clinical Outcomes (PLATO) trial comparing ticagrelor and clopidogrel. Editorial on Serebruany "Viewpoint: Paradoxical excess mortality in the PLATO trial should be independently verified" (Thromb Haemost 2011; 105.5). Thromb Haemost 2011;105:763-5. 35. Califf RM, Platt R. Embedding cardiovascular research into practice. JAMA 2013;310:2037-8. 36. Califf RM, Sanderson I, Miranda ML. The future of cardiovascular clinical research: informatics, clinical investigators, and community engagement. JAMA 2012;308:1747-8. 37. Lauer MS, D'Agostino Sr RB. The randomized registry trial–the next disruptive technology in clinical research? N Engl J Med 2013;369: 1579-81. 38. Eapen ZJ, Lauer MS, Temple RJ. The imperative of overcoming barriers to the conduct of large, simple trials. JAMA 2014;311: 1397-8.
