Commentary
CLINICAL TRIALS
Commentary: Confidentiality of interim trial data—The emerging crisis
Clinical Trials 2015, Vol. 12(1) 15–17 Ó The Author(s) 2014 Reprints and permissions: sagepub.co.uk/journalsPermissions.nav DOI: 10.1177/1740774514561661 ctj.sagepub.com
Steven E Nissen
Randomized controlled clinical trials (RCTs) represent the pinnacle of evidence used by clinicians and regulators to determine the effectiveness of therapeutic interventions. The importance of RCTs is so universally accepted that clinical practice guidelines for many fields of medicine assign the highest rating for quality of evidence (Class 1A) to recommendations supported by data derived from multiple RCTs. Similarly, regulators have generally required evidence derived from RCTs for approval of pharmacological interventions. These policies ensure that the therapies approved by regulatory agencies and endorsed by professional societies actually deliver the benefits promised to patients. Now, after decades of application, this paradigm is threatened by a looming crisis related to the ability to conduct cardiovascular safety trials of diabetes drugs using the highest standards of scientific integrity. As described by Fleming in this edition of Clinical Trials, maintaining the confidentiality of data for interim results from ongoing clinical trials is essential to maintaining the feasibility and reliability of these crucial studies. The current threat to clinical trial integrity arises from a regulatory crisis that emerged during the past decade. The US Food and Drug Administration (FDA) and global regulatory authorities traditionally approved diabetes drugs based primarily upon the ability of these agents to lower blood sugar in the absence of any apparent toxicity. Diabetes drugs were typically evaluated based upon short-term trials (6–12 months) showing a reduction in glycohemoglobin levels, a measure of average blood glucose values over the previous 3 months. In actual practice, most sponsors, to mitigate the risk of demonstrating toxicity, conducted relatively small trials (total sample size about 3000 patients) in populations with a predicted low likelihood of adverse clinical outcomes. However, like most surrogate outcomes, the use of average blood glucose values to approve diabetes drugs carried considerable theoretical risks. A therapeutic intervention could lower blood glucose, but worsen other important outcome measures, which would go undetected in the small, short-term trials conducted with glycemic endpoints. That’s exactly what happened.
In 2005, the diabetes drug muraglitazar was presented to an FDA Advisory Panel, and a recommendation for approval received nearly unanimous support. A few weeks later, we published a meta-analysis of the clinical data presented to the FDA Panel, which showed an approximate doubling of the risk of major adverse cardiovascular events (MACE) in patients treated with muraglitazar compared with placebo or other glucoselowering therapies.1 Then, 2 years later, we published a meta-analysis of cardiovascular outcome data for an approved and widely prescribed drug, rosiglitazone, showing evidence for increased MACE.2 This latter publication resulted in considerable controversy with alarming newspaper headlines, Congressional inquiries, and public outcry. A consensus emerged that approval of diabetes drugs primarily on the basis of glucose lowering (without additional cardiovascular outcome data) was no longer tenable. Accordingly, the FDA assembled an advisory panel in 2008 to consider what changes in regulatory policy, if any, were needed. Two participants in that advisory committee meeting, Dr Fleming as a panel member and me as an invited speaker, proposed a new paradigm for diabetes drug approval. The approach we advocated was carefully designed to balance the need to approve new drugs for diabetes in a timely fashion with the compelling societal interest in making certain such drugs did not increase adverse cardiovascular outcomes. We weighed the risks and benefits of several approaches, including requiring a large-scale RCT prior to approval, but ultimately proposed a strategy that would require sponsors to rule out an upper 95% confidence interval (CI) for the hazard ratio (HR) for MACE of 1.8 prior to approval and an
Department of Cardiovascular Medicine, Cleveland Clinic, Cleveland, OH, USA Corresponding author: Steven E Nissen, Department of Cardiovascular Medicine, Cleveland Clinic, 9500 Euclid Ave., Cleveland, OH 44195, USA. Email: [email protected]
Downloaded from ctj.sagepub.com at UCSF LIBRARY & CKM on April 26, 2015
16
Clinical Trials 12(1)
upper CI\1.3 following approval. This compromise sought to allow timely drug approval for a rapidly expanding morbid-mortal disease (diabetes) while also providing robust outcome data to ensure the safety of new drugs. By the end of 2008, the FDA acted, issuing a guidance document that adopted exactly the strategy we had proposed.3 For the many reasons outlined in Dr Fleming’s current manuscript, sponsors subsequently approached this regulatory requirement by designing a single trial adequately sized to rule out an upper CI of 1.3, but with a planned interim analysis after a sufficient number of MACE events to rule out an upper CI of 1.8. This new policy has resulted in a large number of new RCTs that are providing an explosion of scientific insights into the effects of diabetes drugs on the cardiovascular system. Some studies have already revealed unexpected adverse cardiovascular effects, such as congestive heart failure.4 The large sample size needed for cardiovascular outcomes trials is also providing valuable insights into the effects of these drugs on other organ systems. The greater clarity about risks and benefits serves patients and practitioners well, allowing more intelligent use of the safest and most effective therapies for individual patients. Contrary to the predictions of some critics, the policy did not put a chill on drug development. There are now more new diabetes drugs in development that at any time in recent memory. Despite the success of this new paradigm, like any policy that emerges quickly during a crisis, there were unanticipated issues, the most salient of which, confidentiality of interim data, is described in the manuscript by Fleming. To meet regulatory responsibilities, FDA must review interim data generated from the mandated cardiovascular outcome trials. Furthermore, sponsors must prepare interim data for regulatory submission, which can only be obtained via datasets ordinarily available only to the unblinded data monitoring committee (DMC). However, knowledge of the results from an ongoing trial has the potential to seriously undermine the scientific integrity of any clinical trial. If sponsors or the academic leadership of the trial are aware of emerging data, alterations in study design, subtle or not so subtle, can affect the ultimate trial results. For this reason, the principle of restricting access to interim data to the independent DMC has represented a bedrock principle of clinical trial conduct for many years. Public access to interim data represents an equally grave risk. Inappropriate conclusions based upon incomplete data can preclude the opportunity to complete the ongoing clinical trial with a high standard of study quality. If physicians and patients are aware of trends from emerging data, whether favorable or unfavorable, the viability of the trial can be mortally compromised. For example, a point estimate with a
HR . 1.0 can discourage patients and practitioners from enrolling patients in the ongoing trial or induce them to prematurely terminate the therapy. Even more problematic, emerging trends suggesting harm, even though not statistically significant, can be over interpreted, resulting in patients withdrawing consent, resulting in a high rate of missing data. In such cases, the trial may never adequately address the risk that emerged as a non-significant trend in analysis of interim data. The converse problem is also problematic. If the trend from interim data shows evidence for benefit, the therapy will likely rule out an upper CI of 1.8 and the drug will become commercially available prior to completion of the definitive trial. Physicians and patients may over interpret the signal of benefit, resulting in crossover to the new therapy prior to completion of the ongoing trial, precluding determination if the signal of benefit is robust. For either scenario, whether a trend toward benefit or harm, the resulting impairment in trial quality could put the medical community in exactly the same predicament that resulted in the issuance of the 2008 guidance—widespread administration of diabetes drugs without solid scientific evidence of the true benefit or harm of the therapy. Compounding the current dilemma is an inescapable reality—we live in a world of instant communication where results of clinical trials are widely reported in the medical and lay press. We also live in an environment where regulatory agencies around the world independently review applications for drug approval. Importantly, there is now an emerging initiative in Europe to release clinical trial results formerly considered confidential by the European Medicines Agency.5 If interim results from a regulatory submission anywhere in the world are released, the results will instantly appear in the medical and lay media, thereby compromising confidentiality for all countries. What should thoughtful clinical trialists and industry sponsors do to stabilize this crisis? We must come together with regulators in support of a common approach. Trial data for ongoing studies must remain confidential globally. If any regulatory agency is unable to provide an assurance of confidentiality, data should not be submitted to that authority. As noted by Fleming, industries, DMCs, and trial steering committees must adopt strict policies and procedures designed to protect confidentiality during the review and submission of interim data. To mitigate the risk of premature and inappropriate access to interim data, precise planning is required, including firewalls between individuals supervising the ongoing trial and unblinded sponsor representatives who must submit the regulatory filing. Access must be limited to the smallest group of people required for regulatory activities and must never include individuals within the sponsor representing marketing or business interests. Anything less raises the
Downloaded from ctj.sagepub.com at UCSF LIBRARY & CKM on April 26, 2015
Nissen
17
specter of an outcome popularized in a World War II aphorism—‘‘loose lips sink ships.’’ 2.
Declaration of conflicting interests The author consults with many pharmaceutical companies and conducts clinical trials, some in patients with diabetes, but accepts no honoraria or consulting fees, instead requiring companies to donate any reimbursement directly to charity so that he receives neither income nor a tax deduction.
3.
Funding This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
4.
References
5.
1. Nissen SE, Wolski K and Topol EJ.Effect of muraglitazar on death and major adverse cardiovascular events in
patients with type 2 diabetes mellitus. JAMA 2005; 294: 2581–2586. Nissen SE and Wolski K.Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes. N Engl J Med 2007; 356: 2457–2471 (Erratum in N Engl J Med 2007; 357: 100). US Department of Health and Human Services, Food and Drug Administration and Center for Drug Evaluation and Research (CDER). Guidance for industry: diabetes mellitus—evaluating cardiovascular risk in new antidiabetic therapies to treat type 2 diabetes, http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm071627.pdf Scirica BM, Bhatt DL, Braunwald E, et al. Saxagliptin and cardiovascular outcomes in patients with type 2 diabetes mellitus. N Engl J Med 2013; 369: 1317–1326. European Medicines Agency. Publication and access to clinical-trial data. Draft for Public Consideration 24 June 2013, http://www.ema.europa.eu/docs/en_GB/document_library/Other/2013/06/WC500144730.pdf
Downloaded from ctj.sagepub.com at UCSF LIBRARY & CKM on April 26, 2015
CLINICAL TRIALS
Commentary: Confidentiality of interim trial data—The emerging crisis
Clinical Trials 2015, Vol. 12(1) 15–17 Ó The Author(s) 2014 Reprints and permissions: sagepub.co.uk/journalsPermissions.nav DOI: 10.1177/1740774514561661 ctj.sagepub.com
Steven E Nissen
Randomized controlled clinical trials (RCTs) represent the pinnacle of evidence used by clinicians and regulators to determine the effectiveness of therapeutic interventions. The importance of RCTs is so universally accepted that clinical practice guidelines for many fields of medicine assign the highest rating for quality of evidence (Class 1A) to recommendations supported by data derived from multiple RCTs. Similarly, regulators have generally required evidence derived from RCTs for approval of pharmacological interventions. These policies ensure that the therapies approved by regulatory agencies and endorsed by professional societies actually deliver the benefits promised to patients. Now, after decades of application, this paradigm is threatened by a looming crisis related to the ability to conduct cardiovascular safety trials of diabetes drugs using the highest standards of scientific integrity. As described by Fleming in this edition of Clinical Trials, maintaining the confidentiality of data for interim results from ongoing clinical trials is essential to maintaining the feasibility and reliability of these crucial studies. The current threat to clinical trial integrity arises from a regulatory crisis that emerged during the past decade. The US Food and Drug Administration (FDA) and global regulatory authorities traditionally approved diabetes drugs based primarily upon the ability of these agents to lower blood sugar in the absence of any apparent toxicity. Diabetes drugs were typically evaluated based upon short-term trials (6–12 months) showing a reduction in glycohemoglobin levels, a measure of average blood glucose values over the previous 3 months. In actual practice, most sponsors, to mitigate the risk of demonstrating toxicity, conducted relatively small trials (total sample size about 3000 patients) in populations with a predicted low likelihood of adverse clinical outcomes. However, like most surrogate outcomes, the use of average blood glucose values to approve diabetes drugs carried considerable theoretical risks. A therapeutic intervention could lower blood glucose, but worsen other important outcome measures, which would go undetected in the small, short-term trials conducted with glycemic endpoints. That’s exactly what happened.
In 2005, the diabetes drug muraglitazar was presented to an FDA Advisory Panel, and a recommendation for approval received nearly unanimous support. A few weeks later, we published a meta-analysis of the clinical data presented to the FDA Panel, which showed an approximate doubling of the risk of major adverse cardiovascular events (MACE) in patients treated with muraglitazar compared with placebo or other glucoselowering therapies.1 Then, 2 years later, we published a meta-analysis of cardiovascular outcome data for an approved and widely prescribed drug, rosiglitazone, showing evidence for increased MACE.2 This latter publication resulted in considerable controversy with alarming newspaper headlines, Congressional inquiries, and public outcry. A consensus emerged that approval of diabetes drugs primarily on the basis of glucose lowering (without additional cardiovascular outcome data) was no longer tenable. Accordingly, the FDA assembled an advisory panel in 2008 to consider what changes in regulatory policy, if any, were needed. Two participants in that advisory committee meeting, Dr Fleming as a panel member and me as an invited speaker, proposed a new paradigm for diabetes drug approval. The approach we advocated was carefully designed to balance the need to approve new drugs for diabetes in a timely fashion with the compelling societal interest in making certain such drugs did not increase adverse cardiovascular outcomes. We weighed the risks and benefits of several approaches, including requiring a large-scale RCT prior to approval, but ultimately proposed a strategy that would require sponsors to rule out an upper 95% confidence interval (CI) for the hazard ratio (HR) for MACE of 1.8 prior to approval and an
Department of Cardiovascular Medicine, Cleveland Clinic, Cleveland, OH, USA Corresponding author: Steven E Nissen, Department of Cardiovascular Medicine, Cleveland Clinic, 9500 Euclid Ave., Cleveland, OH 44195, USA. Email: [email protected]
Downloaded from ctj.sagepub.com at UCSF LIBRARY & CKM on April 26, 2015
16
Clinical Trials 12(1)
upper CI\1.3 following approval. This compromise sought to allow timely drug approval for a rapidly expanding morbid-mortal disease (diabetes) while also providing robust outcome data to ensure the safety of new drugs. By the end of 2008, the FDA acted, issuing a guidance document that adopted exactly the strategy we had proposed.3 For the many reasons outlined in Dr Fleming’s current manuscript, sponsors subsequently approached this regulatory requirement by designing a single trial adequately sized to rule out an upper CI of 1.3, but with a planned interim analysis after a sufficient number of MACE events to rule out an upper CI of 1.8. This new policy has resulted in a large number of new RCTs that are providing an explosion of scientific insights into the effects of diabetes drugs on the cardiovascular system. Some studies have already revealed unexpected adverse cardiovascular effects, such as congestive heart failure.4 The large sample size needed for cardiovascular outcomes trials is also providing valuable insights into the effects of these drugs on other organ systems. The greater clarity about risks and benefits serves patients and practitioners well, allowing more intelligent use of the safest and most effective therapies for individual patients. Contrary to the predictions of some critics, the policy did not put a chill on drug development. There are now more new diabetes drugs in development that at any time in recent memory. Despite the success of this new paradigm, like any policy that emerges quickly during a crisis, there were unanticipated issues, the most salient of which, confidentiality of interim data, is described in the manuscript by Fleming. To meet regulatory responsibilities, FDA must review interim data generated from the mandated cardiovascular outcome trials. Furthermore, sponsors must prepare interim data for regulatory submission, which can only be obtained via datasets ordinarily available only to the unblinded data monitoring committee (DMC). However, knowledge of the results from an ongoing trial has the potential to seriously undermine the scientific integrity of any clinical trial. If sponsors or the academic leadership of the trial are aware of emerging data, alterations in study design, subtle or not so subtle, can affect the ultimate trial results. For this reason, the principle of restricting access to interim data to the independent DMC has represented a bedrock principle of clinical trial conduct for many years. Public access to interim data represents an equally grave risk. Inappropriate conclusions based upon incomplete data can preclude the opportunity to complete the ongoing clinical trial with a high standard of study quality. If physicians and patients are aware of trends from emerging data, whether favorable or unfavorable, the viability of the trial can be mortally compromised. For example, a point estimate with a
HR . 1.0 can discourage patients and practitioners from enrolling patients in the ongoing trial or induce them to prematurely terminate the therapy. Even more problematic, emerging trends suggesting harm, even though not statistically significant, can be over interpreted, resulting in patients withdrawing consent, resulting in a high rate of missing data. In such cases, the trial may never adequately address the risk that emerged as a non-significant trend in analysis of interim data. The converse problem is also problematic. If the trend from interim data shows evidence for benefit, the therapy will likely rule out an upper CI of 1.8 and the drug will become commercially available prior to completion of the definitive trial. Physicians and patients may over interpret the signal of benefit, resulting in crossover to the new therapy prior to completion of the ongoing trial, precluding determination if the signal of benefit is robust. For either scenario, whether a trend toward benefit or harm, the resulting impairment in trial quality could put the medical community in exactly the same predicament that resulted in the issuance of the 2008 guidance—widespread administration of diabetes drugs without solid scientific evidence of the true benefit or harm of the therapy. Compounding the current dilemma is an inescapable reality—we live in a world of instant communication where results of clinical trials are widely reported in the medical and lay press. We also live in an environment where regulatory agencies around the world independently review applications for drug approval. Importantly, there is now an emerging initiative in Europe to release clinical trial results formerly considered confidential by the European Medicines Agency.5 If interim results from a regulatory submission anywhere in the world are released, the results will instantly appear in the medical and lay media, thereby compromising confidentiality for all countries. What should thoughtful clinical trialists and industry sponsors do to stabilize this crisis? We must come together with regulators in support of a common approach. Trial data for ongoing studies must remain confidential globally. If any regulatory agency is unable to provide an assurance of confidentiality, data should not be submitted to that authority. As noted by Fleming, industries, DMCs, and trial steering committees must adopt strict policies and procedures designed to protect confidentiality during the review and submission of interim data. To mitigate the risk of premature and inappropriate access to interim data, precise planning is required, including firewalls between individuals supervising the ongoing trial and unblinded sponsor representatives who must submit the regulatory filing. Access must be limited to the smallest group of people required for regulatory activities and must never include individuals within the sponsor representing marketing or business interests. Anything less raises the
Downloaded from ctj.sagepub.com at UCSF LIBRARY & CKM on April 26, 2015
Nissen
17
specter of an outcome popularized in a World War II aphorism—‘‘loose lips sink ships.’’ 2.
Declaration of conflicting interests The author consults with many pharmaceutical companies and conducts clinical trials, some in patients with diabetes, but accepts no honoraria or consulting fees, instead requiring companies to donate any reimbursement directly to charity so that he receives neither income nor a tax deduction.
3.
Funding This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
4.
References
5.
1. Nissen SE, Wolski K and Topol EJ.Effect of muraglitazar on death and major adverse cardiovascular events in
patients with type 2 diabetes mellitus. JAMA 2005; 294: 2581–2586. Nissen SE and Wolski K.Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes. N Engl J Med 2007; 356: 2457–2471 (Erratum in N Engl J Med 2007; 357: 100). US Department of Health and Human Services, Food and Drug Administration and Center for Drug Evaluation and Research (CDER). Guidance for industry: diabetes mellitus—evaluating cardiovascular risk in new antidiabetic therapies to treat type 2 diabetes, http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm071627.pdf Scirica BM, Bhatt DL, Braunwald E, et al. Saxagliptin and cardiovascular outcomes in patients with type 2 diabetes mellitus. N Engl J Med 2013; 369: 1317–1326. European Medicines Agency. Publication and access to clinical-trial data. Draft for Public Consideration 24 June 2013, http://www.ema.europa.eu/docs/en_GB/document_library/Other/2013/06/WC500144730.pdf
Downloaded from ctj.sagepub.com at UCSF LIBRARY & CKM on April 26, 2015
