New Practitioners Forum
New Practitioners Forum Simplifying and interpreting the FACTS of noninferiority trials: A stepwise approach
I
n providing evidence-based care, it is likely that new practitioners have encountered noninferiority trials in the medical literature and will continue to do so at an increasing rate. The number of noninferiority trials indexed by PubMed increased from only 1 trial published before 1999 to over 100 trials published in 2007.1 A MEDLINE search for articles published from 2013 to date (conducted using the same keywords and limits used by Suda and colleagues2 in their 20-year review of noninferiority research) yielded over 200 publications on noninferiority trials. In addition to its usefulness in determining the relative merits of established and comparator therapies, the noninferiority trial is recognized by the Food and Drug Administration as an acceptable study design for use in meeting regulatory requirements for new drug approvals.2 Noninferiority trials investigate differences in outcomes of interest between test-treatment and control groups through methodologies different from those used in superiority and equivalence studies. The goal of noninferiority trials is to determine that a treatment is no worse than a comparator treatment or treatments by a one-sided margin, whereas superiority trials examine whether one treatment is better than another, and equivalence trials aim to determine that a treatment is therapeutically similar by a two-sided margin.1 A noninferiority study design is an appropriate alternative to superiority studies in situations when it may be un-
ethical to use a placebo comparator (e.g., a chemotherapy trial), it would not be acceptable to observe any loss of efficacy compared with the standard of care (e.g., an evaluation of antiinfective agents), or it might be inappropriate to assess the risk–benefit profile of a test treatment that may offer safety advantages over the current standard of care (e.g., a study of anticoagulants).1,3 While noninferiority trials can offer an alternative research method when other study designs are not appropriate, this study methodology has notable limitations, including a lack of familiarity with noninferiority trials among researchers and journal readers, historically poor quality in the reporting of noninferiority margins, and the fact that the use of noninferiority research remains controversial among clinicians.1 Of note, noninferiority trials should be conducted only if the efficacy of the active (i.e., nonplacebo) comparator has been firmly established in previous superiority trials and the noninferiority trial will investigate similar patient populations and outcome measures.4 The intent of this article is not to provide a comprehensive review of statistical analysis and the design and conduct of noninferiority clinical trials but rather to assist new practitioners by outlining an efficient, stepwise approach to critiquing this type of trial. While readers are encouraged to review the 2010 Consolidated Standards of Reporting Trials (CONSORT) statement,1 which provides a checklist of items that should
The New Practitioners Forum column features articles that address the special professional needs of pharmacists early in their careers as they transition from students to practitioners. Authors include new practitioners or others with expertise in a topic of interest to new practitioners. AJHP readers are invited to submit topics or articles for this column to the New Practitioners Forum, c/o Jill Haug, 7272 Wisconsin Avenue, Bethesda, MD 20814 (301-664-8821 or [email protected]).
1926
Am J Health-Syst Pharm—Vol 71 Nov 15, 2014
be reported in all types of clinical studies, including noninferiority trials, here we discuss five simple steps that readers can use to critique noninferiority trials. The mnemonic “FACTS” can be helpful in remembering the five steps: 1. Formulate a null hypothesis. 2. Assess the noninferiority margin. 3. Compare the intention-to-treat (ITT) and per-protocol (PP) analyses. 4. Translate confidence intervals (CIs) graphically. 5. Summarize the clinical relevance of the results.
Formulating a null hypothesis. In noninferiority trials, researchers seek to demonstrate that a test treatment or intervention is no worse than an active comparator; therefore, researchers establish a null hypothesis asserting that an outcome or outcomes with the two treatments will differ by a predefined margin.5 Rejecting the null hypothesis involves proving that the treatments do not differ by more than the specified margin. Assessing the noninferiority margin. Also referred to as the delta margin, the predefined noninferiority margin is a key piece of information, since this is the acceptable amount of difference needed to prove that the test agent is noninferior. While it is difficult for the reader to critique the appropriateness of the margin, since both clinical judgment and statistical reasoning help determine the margin, it is important for the reader to have some baseline knowledge of its origin.6-8 The research team conducting a noninferiority trial may justify its selection of the noninferiority margin by referencing a meta-analysis of historical placebo-controlled trials or individual studies comparing a standard treatment with a placebo. These referenced studies are intended to provide perspective on the magnitude of treatment effects, as estimated via analysis of pooled data from prior studies, including the upper and Continued on page 1928
New Practitioners Forum Continued from page 1926
lower CI limits. Experts agree that the margin chosen for the noninferiority trial should be no more than 50% of the lower confidence limit of standard-treatment
effects. Although this is judged as a liberal rule, many trials specify margins that are larger than this.6,9 Although researchers should clearly explain how the selected margin was chosen, that is not always done.10 Two quick sources the reader may
Case Study 1 The EINSTEIN-PE study was a randomized, open-label, active-controlled, Phase III noninferiority study of rivaroxaban alone versus standard therapy with enoxaparin and a vitamin K antagonist (warfarin in most cases) for symptomatic pulmonary embolism with or without deep vein thrombosis.15 The primary endpoint was symptomatic recurrent venous thromboembolism (VTE).
F
The null hypothesis was that the risk of recurrent VTE during the follow-up period in the rivaroxaban group would be at least twofold higher than in the standard therapy group. Rejection of this null hypothesis would indicate that rivaroxaban was no worse than standard therapy by a hazard ratio (HR) of 2.0. A The predefined noninferiority margin, or delta (D) margin, indicated by the red dashed line in the graph, was the upper limit of the 95% confidence interval (CI) for an HR of 2.0. Although the rationale for the selected margin was not discussed by the EINSTEIN-PE investigators, a literature search identified several articles that provided justification for the use of an HR margin ranging from 2.0 to 5.0 in trials comparing treatments for VTE prevention.16-24 C After an overall mean treatment duration of about seven months, the betweentreatment difference in the risk of recurrent VTE with the use of rivaroxaban versus standard therapy corresponded to HR values of 1.12 (95% CI, 0.75– 1.68; p = 0.003 for a one-sided noninferiority margin of 2.0) in the intentionto-treat (ITT) analysis and 1.07 (95% CI, 0.70–1.63) in the per-protocol (PP) analysis.15 T As plotted on the graph, the error bars depicting the HR values and CIs derived from the ITT and PP analyses do not touch or cross the dashed line representing the noninferiority margin; therefore, the noninferiority criteria were met. S A summary of the EINSTEIN-PE study findings might include the following key points: Rivaroxaban is the first approved factor Xa inhibitor proven to be no worse than enoxaparin plus warfarin in the treatment of pulmonary embolism. Rivaroxaban offers the convenience of an oral dosing regimen, with limited drug interaction potential and no need for routine monitoring. However, no agent for reversing its anticoagulative effects is available, and there is a higher cost associated with this relatively new therapy. It should not be used in patients with a creatinine clearance of less than 30 mL/min. While the EINSTEIN-PE trial showed a decreased risk of major bleeding compared with enoxaparin, clinicians should review patients’ comorbid conditions and concurrent medications to assess the risk of bleeding.
ITT PP
0.75
1.68
0.70
1.12 1.63 1.07
Rivaroxaban Better
0
1.00
∆
2.00 Enoxaparin + Warfarin Better
Hazard Ratio for Recurrent VTE
review when justification or references for the selected margin are not provided are clinical practice guidelines and historical studies. Guidelines often contain collated results from previous studies along with an expected appropriate change as a result of treatment. For newly approved drugs, it may be more appropriate to conduct a quick literature search of previous studies with characteristics similar to those of the noninferiority trial (e.g., comparable patient populations, outcome measures, and disease severity) and using the same active control or an established standard treatment.4,11 This research not only sheds light on how the margin was chosen but also should prompt caution in extrapolating trial results when the noninferiority margin is larger than has been previously documented. Comparing the results of ITT and PP analyses. As the reader approaches the results section of the study, it is important to compare the results of both ITT and PP analyses (both should be presented).4,6,8,12 The PP analysis provides the most conservative estimate of noninferiority. Since it is easier to establish a noninferiority claim with an ITT analysis alone, verifying that the results of both the ITT and PP analyses for the primary endpoint are statistically significant reduces the risk of type I error (data leading to a false conclusion that a treatment is noninferior).4 Only if both the ITT and the PP analyses support noninferiority can it be adequately determined that noninferiority was achieved.8,12 If major differences are found between the results of the ITT and PP analyses, a closer look at the study design is warranted to determine if errors in the methods caused this to occur. If superiority testing is performed after noninferiority is established, the ITT analysis should be the primary method used because it represents the most conservative approach to assessing superiority.1 However, the results of both ITT and PP analyses should be reviewed; as with noninferiority studies, if the results of both analyses do not indicate superiority, there might be a problem with the study design. Translating CIs graphically. Creating a graph that depicts data for the primary endpoint might be useful in visually conceptualizing the study results. Three key Continued on page 1930
1928
Am J Health-Syst Pharm—Vol 71 Nov 15, 2014
New Practitioners Forum Continued from page 1928
Case Study 2 A 52-week multicenter, randomized, double-blind, active-controlled, Phase III noninferiority study was conducted to assess the use of canagliflozin 300 mg daily or sitagliptin 100 mg daily in patients with type 2 diabetes inadequately controlled with metformin and a sulfonylurea.25 The primary endpoint was the change from baseline in glycosylated hemoglobin (HbA1c) values at 52 weeks.
F The null hypothesis was that there is a difference of at least 0.3% in mean HbA
1c
reduction in canagliflozin- and sitagliptin-treated patients concurrently using metformin and a sulfonylurea. Rejection of this null hypothesis would indicate that canagliflozin was no worse than sitagliptin for reducing HbA1c values by a margin of 0.3%. A The prespecified D margin for the between-treatment difference in mean HbA1c reduction was 0.3%. A superiority evaluation was to be performed only if the noninferiority criteria were met. The Food and Drug Administration (FDA) accepts a noninferiority margin of 0.3–0.4% for trials of HbA1c-lowering therapy given that such a margin is not greater than conservative estimates of the treatment effect of the active control in previous trials.26 Moreover, dipeptidyl peptidase-4 inhibitors have been associated with decreases in HbA1c values ranging from 0.5% to 1%, which also suggests that the selected D margin of 0.3% was appropriate.27 C The study results showed that the between-treatment difference in the least squares mean (LSM) change from baseline in HbA1c was –0.37% (95% confidence interval [CI], –0.5% to –0.25%) in the intention-to-treat (ITT) analysis and –0.21% (95% CI, –0.34% to –0.08%) in the per-protocol (PP) analysis.25 T As shown on the graph, the error bars for the differences in the values for the LSM change in HbA1c (and CIs) with canagliflozin versus sitagliptin therapy, as determined in the the ITT and PP analyses, do not touch or cross the noninferiority margin of 0.3%; therefore, the noninferiority criteria were satisfied. Additionally, the CIs do not touch or cross 0, indicating superiority. S A summary of the key study findings might read as follows: Canagliflozin28 was the first FDA-approved sodium–glucose cotransporter 2 inhibitor. Canagliflozin has been shown to be not only no worse than sitagliptin but also superior to that comparator in reducing HbA1c values.25 While the relative reduction of HbA1c values with canagliflozin use was significant in the trial summarized here, it is important to assess whether the demonstrated magnitude of the reduction would be clinically meaningful in the context of helping a particular patient achieve individualized treatment goals. An analysis of other factors is also warranted in determining the place of canagliflozin in therapy. Canagliflozin offers the convenience of an oral once-a-day dosing regimen, although cardiovascular safety concerns are under investigation. In addition, preliminary reports indicate canagliflozin use is associated with mycotic genital infections and osmotic diuresis that may limit its use in certain patient populations.28
ITT
–0.5%
–0.25% –0.37%
PP
–0.34%
–0.08%
–0.21%
Canagliflozin Better
–0.3%
0
∆
0.3%
Sitagliptin Better
Difference in LSM Change From Baseline HbA1c
1930
Am J Health-Syst Pharm—Vol 71 Nov 15, 2014
pieces of information are needed: (1) the prespecified noninferiority margin, (2) the type of CIs associated with the primary endpoint, and (3) the numerical results of the primary endpoint analysis, often referred to as the between-treatment difference. If the results of the primary endpoint are expressed using CIs for general mean data, such as percentages, the intersection of the x-axis and the y-axis should be labeled “0”; if the research involved CIs for measures of association (typically used for data such as a hazard ratio, odds ratio, or relative risk), the intersection of the axes should be labeled “1” instead.13 On the x-axis, a perpendicular dashed line may be graphed to indicate the noninferiority margin (see case studies). Finally, error bars depicting the results and corresponding CIs for the primary outcome should be plotted on the graph. In order to demonstrate noninferiority, the results must lie entirely within the area of acceptable margin (i.e., to the left of the delta line if the margin is a positive number or to the right of the delta line if the margin is a negative number). Superiority can be shown if the results lie entirely on the opposite side of the margin and (depending on the type of CI) do not cross or touch either 0 or 1. If noninferiority is achieved, investigators often attempt to determine whether superiority was demonstrated as well. Testing for superiority after establishing noninferiority is generally acceptable, as long as the intent to determine superiority is clearly explained in the methods section and the ITT analysis is used.1 In contrast, occasionally researchers conducting superiority trials may attempt to prove noninferiority if superiority is not demonstrated; this sequential testing should also be explicitly discussed in the methods section.1 However, it is not appropriate for trials to assess noninferiority if superiority testing has failed and there was no inference of noninferiority testing a priori. The problem lies in the fact that bias may occur when trying to objectively set a noninferiority margin after knowing the results of the treatment differences between the groups in the superiority analysis.8,10 Summarizing the clinical relevance of the results. When assessing the clinical relevance of a noninferiority trial, it is first
New Practitioners Forum
prudent to examine the historical efficacy of the standard of care relative to a placebo or active comparator in clinical trials and meta-analyses. This will assist in putting in context the comparative efficacy of the therapies as well as the appropriateness of the interventions. While two therapies may be considered noninferior to each other, it is possible that there are additional factors, such as safety considerations, cost, and convenience, that favor one therapy over another in clinical practice.14 Additionally, analyzing the demographics of the patients investigated will aid clinicians in determining if and to what extent the results can be extrapolated to the patients they serve.14 In assessing the clinical relevance of noninferiority trials, clinicians may inquire as to whether the number needed to treat (NNT) was or can be calculated. If only noninferiority was established in a trial, the difference between the two treatments should be marginal, and the NNT would not add clinically valuable knowledge since it can only be concluded that the therapies are no worse than each other. However, if a noninferiority trial also meets the criteria for demonstration of superiority, then calculating the NNT might provide additional information to assess the risk–benefit profile of the treatment of interest. Closing notes. Two sample case studies are provided to illustrate the concepts reviewed in this article. In pursuit of providing evidencebased care, new practitioners lacking familiarity with key concepts and terminology in the conduct and reporting of noninferiority trials may be hindered in interpreting the results of such trials. The FACTS method may help simplify the process of reviewing noninferiority trials and guide decision-making with regard to applying the results in clinical practice. 1. Piaggio G, Elbourne DR, Altman DG et al., for the CONSORT Group. Reporting of noninferiority and equivalence randomized trials: an extension of the 2010 CONSORT statement. JAMA. 2012; 308:2594-604. 2. Suda KJ, Hurley AM, McKibbin T et al. Publication of noninferiority clinical trials: changes over a 20-year interval. Pharmacotherapy. 2011; 31:833-9. 3. Dasgupta A, Lawson KA, Wilson JP. Evaluating equivalence and noninferiority trials. Am J Health-Syst Pharm. 2010; 67:1337-43.
4. Piaggio G, Elbourne DR, Altman DG et al. Reporting of noninferiority and equivalence randomized trials. An extension of the CONSORT statement. JAMA. 2006; 295:1152-60. 5. Gotzsche PC. Lessons from and cautions about non-inferiority and equivalence randomized trials. JAMA. 2006; 295:1172-4. 6. Scott IA. Non-inferiority trials: determining whether alternative treatments are good enough. Med J Aust. 2009; 190:326-30. 7. Kaul S, Diamond G. Making sense of noninferiority: a clinical and statistical perspective on its application to cardiovascular clinical trials. Prog Cardiovasc Dis. 2007; 49:284-99. 8. Snappin SM. Noninferiority trials. Curr Control Trials Cardiovasc Med. 2000; 1:1921. 9. Committee for Medicinal Products for Human Use, European Medicines Agency. Guideline on the choice of the noninferiority margin (July 27, 2005). www. ema.europa.eu/docs/en_GB/document_ library/Scientific_guideline/2009/09/ WC500003636.pdf (accessed 2013 Jul 12). 10. Kaul S, Diamond G. Good enough: a primer on the analysis and interpretation of noninferiority trials. Ann Intern Med. 2006; 145:62-9. 11. Schumock GT, Pickard AS. Comparative effectiveness research: relevance and applications to pharmacy. Am J Health-Syst Pharm. 2009; 66:1278-86. 12. Jones B, Jarvis P, Lewis JA, Ebbutt AF. Trials to assess equivalence: the importance of rigorous methods. BMJ. 1996; 313:36. 13. Ferrill MJ, Brown D, Kyle JA. Clinical versus statistical significance: interpreting p values and confidence intervals related to measures of association to guide decision making. J Pharm Pract. 2010; 23:344-51. 14. Malone PM, Kier KL, Stanovich JE. Drug information: a guide for pharmacists. 4th ed. New York: McGraw-Hill; 2012:1627,198-202. 15. The EINSTEIN–PE Investigators. Oral rivaroxaban for the treatment of symptomatic pulmonary embolism. N Engl J Med. 2012; 366:1287-97. 16. Schulman S, Kearon C, Kakkar AK et al. Extended use of dabigatran, warfarin, or placebo in venous thromboembolism. N Engl J Med. 2013; 368:709-18. 17. The EINSTEIN Investigators. Oral rivaroxaban for symptomatic venous thromboembolism. N Engl J Med. 2010; 363:2499-510. 18. Büller HR, Cohen AT, Davidson B et al. Idraparinux versus standard therapy for venous thromboembolic disease. N Engl J Med. 2007; 357:1094-104. 19. Büller HR, Davidson BL, Decousus H et al. Fondaparinux or enoxaparin for the initial treatment of symptomatic deep venous thrombosis: a randomized trial. Ann Intern Med. 2004; 140:867-73. 20. Büller HR, Davidson BL, Decousus H et al. Subcutaneous fondaparinux versus intravenous unfractionated heparin in the initial treatment of pulmonary embolism. N Engl J Med. 2003; 349:1695-702. 21. Fiessinger JN, Huisman MV, Davidson BL et al. Ximelagatran vs low-molecular-
weight heparin and warfarin for the treatment of deep vein thrombosis: a randomized trial. JAMA. 2005; 293:681-9. 22. Kearon C, Ginsberg JS, Julian JA et al. Comparison of fixed-dose weight-adjusted unfractionated heparin and low-molecularweight heparin for acute treatment of venous thromboembolism. JAMA. 2006; 296:935-42. 23. Schulman S, Kearon C, Kakkar AK et al. Dabigatran versus warfarin in the treatment of acute venous thromboembolism. N Engl J Med. 2009; 361:2342-52. 24. Büller HR, Gallus AS, Pillion G et al., for the Cassiopea Investigators. Enoxaparin followed by once weekly idrabiotaparinux versus enoxaparin plus warfarin for patients with acute symptomatic pulmonary embolism: a randomised double-blind, double-dummy, non-inferiority trial. Lancet. 2012; 379:123-9. 25. Schernthaner G, Gross JL, Rosentock J et al. Canagliflozin compared with sitagliptin for patients with type 2 diabetes who do not have adequate glycemic control with metformin plus sulfonylurea: a 52-week randomized trial [published online ahead of print April 5, 2013]. Diabetes Care. 2013; 36:2508-15. [Erratum, Diabetes Care. 2013; 36:4172.] 26. Food and Drug Administration. Diabetes mellitus: developing drugs and therapeutic biologics for treatment and prevention. www.fda.gov/downloads/ Drugs/GuidanceComplianceRegulatory Information/Guidances/UCM071624.pdf (accessed 2013 Apr 21). 27. Inzucchi SE, Bergenstal RM, Buse JB et al., for the American Diabetes Association (ADA) and European Association for the Study of Diabetes (EASD). Management of hyperglycemia in type 2 diabetes: a patient-centered approach: position statement of the American Diabetes Association (ADA) and the European Association for the Study of Diabetes (EASD). Diabetes Care. 2012; 35:1364-79. [Erratum, Diabetes Care. 2013; 36:490.] 28. Food and Drug Administration. FDA approves Invokana to treat type 2 diabetes. www.fda.gov/NewsEvents/Newsroom/ PressAnnouncements/ucm345848.html (accessed 2013 Apr 21).
Krisy-Ann Thornby, Pharm.D., Assistant Professor, Pharmacy Practice, and Drug Information Co-Coordinator [email protected] Ashley Johnson, Pharm.D., BCPS, Assistant Professor, Pharmacy Practice, and Drug Information Co-Coordinator Mary J. Ferrill, Pharm.D., FASHP, Dean and Professor, Pharmacy Practice Palm Beach Atlantic University West Palm Beach, FL
The authors have declared no potential conflicts of interest. DOI 10.2146/ajhp130270
Am J Health-Syst Pharm—Vol 71 Nov 15, 2014
1931
New Practitioners Forum Simplifying and interpreting the FACTS of noninferiority trials: A stepwise approach
I
n providing evidence-based care, it is likely that new practitioners have encountered noninferiority trials in the medical literature and will continue to do so at an increasing rate. The number of noninferiority trials indexed by PubMed increased from only 1 trial published before 1999 to over 100 trials published in 2007.1 A MEDLINE search for articles published from 2013 to date (conducted using the same keywords and limits used by Suda and colleagues2 in their 20-year review of noninferiority research) yielded over 200 publications on noninferiority trials. In addition to its usefulness in determining the relative merits of established and comparator therapies, the noninferiority trial is recognized by the Food and Drug Administration as an acceptable study design for use in meeting regulatory requirements for new drug approvals.2 Noninferiority trials investigate differences in outcomes of interest between test-treatment and control groups through methodologies different from those used in superiority and equivalence studies. The goal of noninferiority trials is to determine that a treatment is no worse than a comparator treatment or treatments by a one-sided margin, whereas superiority trials examine whether one treatment is better than another, and equivalence trials aim to determine that a treatment is therapeutically similar by a two-sided margin.1 A noninferiority study design is an appropriate alternative to superiority studies in situations when it may be un-
ethical to use a placebo comparator (e.g., a chemotherapy trial), it would not be acceptable to observe any loss of efficacy compared with the standard of care (e.g., an evaluation of antiinfective agents), or it might be inappropriate to assess the risk–benefit profile of a test treatment that may offer safety advantages over the current standard of care (e.g., a study of anticoagulants).1,3 While noninferiority trials can offer an alternative research method when other study designs are not appropriate, this study methodology has notable limitations, including a lack of familiarity with noninferiority trials among researchers and journal readers, historically poor quality in the reporting of noninferiority margins, and the fact that the use of noninferiority research remains controversial among clinicians.1 Of note, noninferiority trials should be conducted only if the efficacy of the active (i.e., nonplacebo) comparator has been firmly established in previous superiority trials and the noninferiority trial will investigate similar patient populations and outcome measures.4 The intent of this article is not to provide a comprehensive review of statistical analysis and the design and conduct of noninferiority clinical trials but rather to assist new practitioners by outlining an efficient, stepwise approach to critiquing this type of trial. While readers are encouraged to review the 2010 Consolidated Standards of Reporting Trials (CONSORT) statement,1 which provides a checklist of items that should
The New Practitioners Forum column features articles that address the special professional needs of pharmacists early in their careers as they transition from students to practitioners. Authors include new practitioners or others with expertise in a topic of interest to new practitioners. AJHP readers are invited to submit topics or articles for this column to the New Practitioners Forum, c/o Jill Haug, 7272 Wisconsin Avenue, Bethesda, MD 20814 (301-664-8821 or [email protected]).
1926
Am J Health-Syst Pharm—Vol 71 Nov 15, 2014
be reported in all types of clinical studies, including noninferiority trials, here we discuss five simple steps that readers can use to critique noninferiority trials. The mnemonic “FACTS” can be helpful in remembering the five steps: 1. Formulate a null hypothesis. 2. Assess the noninferiority margin. 3. Compare the intention-to-treat (ITT) and per-protocol (PP) analyses. 4. Translate confidence intervals (CIs) graphically. 5. Summarize the clinical relevance of the results.
Formulating a null hypothesis. In noninferiority trials, researchers seek to demonstrate that a test treatment or intervention is no worse than an active comparator; therefore, researchers establish a null hypothesis asserting that an outcome or outcomes with the two treatments will differ by a predefined margin.5 Rejecting the null hypothesis involves proving that the treatments do not differ by more than the specified margin. Assessing the noninferiority margin. Also referred to as the delta margin, the predefined noninferiority margin is a key piece of information, since this is the acceptable amount of difference needed to prove that the test agent is noninferior. While it is difficult for the reader to critique the appropriateness of the margin, since both clinical judgment and statistical reasoning help determine the margin, it is important for the reader to have some baseline knowledge of its origin.6-8 The research team conducting a noninferiority trial may justify its selection of the noninferiority margin by referencing a meta-analysis of historical placebo-controlled trials or individual studies comparing a standard treatment with a placebo. These referenced studies are intended to provide perspective on the magnitude of treatment effects, as estimated via analysis of pooled data from prior studies, including the upper and Continued on page 1928
New Practitioners Forum Continued from page 1926
lower CI limits. Experts agree that the margin chosen for the noninferiority trial should be no more than 50% of the lower confidence limit of standard-treatment
effects. Although this is judged as a liberal rule, many trials specify margins that are larger than this.6,9 Although researchers should clearly explain how the selected margin was chosen, that is not always done.10 Two quick sources the reader may
Case Study 1 The EINSTEIN-PE study was a randomized, open-label, active-controlled, Phase III noninferiority study of rivaroxaban alone versus standard therapy with enoxaparin and a vitamin K antagonist (warfarin in most cases) for symptomatic pulmonary embolism with or without deep vein thrombosis.15 The primary endpoint was symptomatic recurrent venous thromboembolism (VTE).
F
The null hypothesis was that the risk of recurrent VTE during the follow-up period in the rivaroxaban group would be at least twofold higher than in the standard therapy group. Rejection of this null hypothesis would indicate that rivaroxaban was no worse than standard therapy by a hazard ratio (HR) of 2.0. A The predefined noninferiority margin, or delta (D) margin, indicated by the red dashed line in the graph, was the upper limit of the 95% confidence interval (CI) for an HR of 2.0. Although the rationale for the selected margin was not discussed by the EINSTEIN-PE investigators, a literature search identified several articles that provided justification for the use of an HR margin ranging from 2.0 to 5.0 in trials comparing treatments for VTE prevention.16-24 C After an overall mean treatment duration of about seven months, the betweentreatment difference in the risk of recurrent VTE with the use of rivaroxaban versus standard therapy corresponded to HR values of 1.12 (95% CI, 0.75– 1.68; p = 0.003 for a one-sided noninferiority margin of 2.0) in the intentionto-treat (ITT) analysis and 1.07 (95% CI, 0.70–1.63) in the per-protocol (PP) analysis.15 T As plotted on the graph, the error bars depicting the HR values and CIs derived from the ITT and PP analyses do not touch or cross the dashed line representing the noninferiority margin; therefore, the noninferiority criteria were met. S A summary of the EINSTEIN-PE study findings might include the following key points: Rivaroxaban is the first approved factor Xa inhibitor proven to be no worse than enoxaparin plus warfarin in the treatment of pulmonary embolism. Rivaroxaban offers the convenience of an oral dosing regimen, with limited drug interaction potential and no need for routine monitoring. However, no agent for reversing its anticoagulative effects is available, and there is a higher cost associated with this relatively new therapy. It should not be used in patients with a creatinine clearance of less than 30 mL/min. While the EINSTEIN-PE trial showed a decreased risk of major bleeding compared with enoxaparin, clinicians should review patients’ comorbid conditions and concurrent medications to assess the risk of bleeding.
ITT PP
0.75
1.68
0.70
1.12 1.63 1.07
Rivaroxaban Better
0
1.00
∆
2.00 Enoxaparin + Warfarin Better
Hazard Ratio for Recurrent VTE
review when justification or references for the selected margin are not provided are clinical practice guidelines and historical studies. Guidelines often contain collated results from previous studies along with an expected appropriate change as a result of treatment. For newly approved drugs, it may be more appropriate to conduct a quick literature search of previous studies with characteristics similar to those of the noninferiority trial (e.g., comparable patient populations, outcome measures, and disease severity) and using the same active control or an established standard treatment.4,11 This research not only sheds light on how the margin was chosen but also should prompt caution in extrapolating trial results when the noninferiority margin is larger than has been previously documented. Comparing the results of ITT and PP analyses. As the reader approaches the results section of the study, it is important to compare the results of both ITT and PP analyses (both should be presented).4,6,8,12 The PP analysis provides the most conservative estimate of noninferiority. Since it is easier to establish a noninferiority claim with an ITT analysis alone, verifying that the results of both the ITT and PP analyses for the primary endpoint are statistically significant reduces the risk of type I error (data leading to a false conclusion that a treatment is noninferior).4 Only if both the ITT and the PP analyses support noninferiority can it be adequately determined that noninferiority was achieved.8,12 If major differences are found between the results of the ITT and PP analyses, a closer look at the study design is warranted to determine if errors in the methods caused this to occur. If superiority testing is performed after noninferiority is established, the ITT analysis should be the primary method used because it represents the most conservative approach to assessing superiority.1 However, the results of both ITT and PP analyses should be reviewed; as with noninferiority studies, if the results of both analyses do not indicate superiority, there might be a problem with the study design. Translating CIs graphically. Creating a graph that depicts data for the primary endpoint might be useful in visually conceptualizing the study results. Three key Continued on page 1930
1928
Am J Health-Syst Pharm—Vol 71 Nov 15, 2014
New Practitioners Forum Continued from page 1928
Case Study 2 A 52-week multicenter, randomized, double-blind, active-controlled, Phase III noninferiority study was conducted to assess the use of canagliflozin 300 mg daily or sitagliptin 100 mg daily in patients with type 2 diabetes inadequately controlled with metformin and a sulfonylurea.25 The primary endpoint was the change from baseline in glycosylated hemoglobin (HbA1c) values at 52 weeks.
F The null hypothesis was that there is a difference of at least 0.3% in mean HbA
1c
reduction in canagliflozin- and sitagliptin-treated patients concurrently using metformin and a sulfonylurea. Rejection of this null hypothesis would indicate that canagliflozin was no worse than sitagliptin for reducing HbA1c values by a margin of 0.3%. A The prespecified D margin for the between-treatment difference in mean HbA1c reduction was 0.3%. A superiority evaluation was to be performed only if the noninferiority criteria were met. The Food and Drug Administration (FDA) accepts a noninferiority margin of 0.3–0.4% for trials of HbA1c-lowering therapy given that such a margin is not greater than conservative estimates of the treatment effect of the active control in previous trials.26 Moreover, dipeptidyl peptidase-4 inhibitors have been associated with decreases in HbA1c values ranging from 0.5% to 1%, which also suggests that the selected D margin of 0.3% was appropriate.27 C The study results showed that the between-treatment difference in the least squares mean (LSM) change from baseline in HbA1c was –0.37% (95% confidence interval [CI], –0.5% to –0.25%) in the intention-to-treat (ITT) analysis and –0.21% (95% CI, –0.34% to –0.08%) in the per-protocol (PP) analysis.25 T As shown on the graph, the error bars for the differences in the values for the LSM change in HbA1c (and CIs) with canagliflozin versus sitagliptin therapy, as determined in the the ITT and PP analyses, do not touch or cross the noninferiority margin of 0.3%; therefore, the noninferiority criteria were satisfied. Additionally, the CIs do not touch or cross 0, indicating superiority. S A summary of the key study findings might read as follows: Canagliflozin28 was the first FDA-approved sodium–glucose cotransporter 2 inhibitor. Canagliflozin has been shown to be not only no worse than sitagliptin but also superior to that comparator in reducing HbA1c values.25 While the relative reduction of HbA1c values with canagliflozin use was significant in the trial summarized here, it is important to assess whether the demonstrated magnitude of the reduction would be clinically meaningful in the context of helping a particular patient achieve individualized treatment goals. An analysis of other factors is also warranted in determining the place of canagliflozin in therapy. Canagliflozin offers the convenience of an oral once-a-day dosing regimen, although cardiovascular safety concerns are under investigation. In addition, preliminary reports indicate canagliflozin use is associated with mycotic genital infections and osmotic diuresis that may limit its use in certain patient populations.28
ITT
–0.5%
–0.25% –0.37%
PP
–0.34%
–0.08%
–0.21%
Canagliflozin Better
–0.3%
0
∆
0.3%
Sitagliptin Better
Difference in LSM Change From Baseline HbA1c
1930
Am J Health-Syst Pharm—Vol 71 Nov 15, 2014
pieces of information are needed: (1) the prespecified noninferiority margin, (2) the type of CIs associated with the primary endpoint, and (3) the numerical results of the primary endpoint analysis, often referred to as the between-treatment difference. If the results of the primary endpoint are expressed using CIs for general mean data, such as percentages, the intersection of the x-axis and the y-axis should be labeled “0”; if the research involved CIs for measures of association (typically used for data such as a hazard ratio, odds ratio, or relative risk), the intersection of the axes should be labeled “1” instead.13 On the x-axis, a perpendicular dashed line may be graphed to indicate the noninferiority margin (see case studies). Finally, error bars depicting the results and corresponding CIs for the primary outcome should be plotted on the graph. In order to demonstrate noninferiority, the results must lie entirely within the area of acceptable margin (i.e., to the left of the delta line if the margin is a positive number or to the right of the delta line if the margin is a negative number). Superiority can be shown if the results lie entirely on the opposite side of the margin and (depending on the type of CI) do not cross or touch either 0 or 1. If noninferiority is achieved, investigators often attempt to determine whether superiority was demonstrated as well. Testing for superiority after establishing noninferiority is generally acceptable, as long as the intent to determine superiority is clearly explained in the methods section and the ITT analysis is used.1 In contrast, occasionally researchers conducting superiority trials may attempt to prove noninferiority if superiority is not demonstrated; this sequential testing should also be explicitly discussed in the methods section.1 However, it is not appropriate for trials to assess noninferiority if superiority testing has failed and there was no inference of noninferiority testing a priori. The problem lies in the fact that bias may occur when trying to objectively set a noninferiority margin after knowing the results of the treatment differences between the groups in the superiority analysis.8,10 Summarizing the clinical relevance of the results. When assessing the clinical relevance of a noninferiority trial, it is first
New Practitioners Forum
prudent to examine the historical efficacy of the standard of care relative to a placebo or active comparator in clinical trials and meta-analyses. This will assist in putting in context the comparative efficacy of the therapies as well as the appropriateness of the interventions. While two therapies may be considered noninferior to each other, it is possible that there are additional factors, such as safety considerations, cost, and convenience, that favor one therapy over another in clinical practice.14 Additionally, analyzing the demographics of the patients investigated will aid clinicians in determining if and to what extent the results can be extrapolated to the patients they serve.14 In assessing the clinical relevance of noninferiority trials, clinicians may inquire as to whether the number needed to treat (NNT) was or can be calculated. If only noninferiority was established in a trial, the difference between the two treatments should be marginal, and the NNT would not add clinically valuable knowledge since it can only be concluded that the therapies are no worse than each other. However, if a noninferiority trial also meets the criteria for demonstration of superiority, then calculating the NNT might provide additional information to assess the risk–benefit profile of the treatment of interest. Closing notes. Two sample case studies are provided to illustrate the concepts reviewed in this article. In pursuit of providing evidencebased care, new practitioners lacking familiarity with key concepts and terminology in the conduct and reporting of noninferiority trials may be hindered in interpreting the results of such trials. The FACTS method may help simplify the process of reviewing noninferiority trials and guide decision-making with regard to applying the results in clinical practice. 1. Piaggio G, Elbourne DR, Altman DG et al., for the CONSORT Group. Reporting of noninferiority and equivalence randomized trials: an extension of the 2010 CONSORT statement. JAMA. 2012; 308:2594-604. 2. Suda KJ, Hurley AM, McKibbin T et al. Publication of noninferiority clinical trials: changes over a 20-year interval. Pharmacotherapy. 2011; 31:833-9. 3. Dasgupta A, Lawson KA, Wilson JP. Evaluating equivalence and noninferiority trials. Am J Health-Syst Pharm. 2010; 67:1337-43.
4. Piaggio G, Elbourne DR, Altman DG et al. Reporting of noninferiority and equivalence randomized trials. An extension of the CONSORT statement. JAMA. 2006; 295:1152-60. 5. Gotzsche PC. Lessons from and cautions about non-inferiority and equivalence randomized trials. JAMA. 2006; 295:1172-4. 6. Scott IA. Non-inferiority trials: determining whether alternative treatments are good enough. Med J Aust. 2009; 190:326-30. 7. Kaul S, Diamond G. Making sense of noninferiority: a clinical and statistical perspective on its application to cardiovascular clinical trials. Prog Cardiovasc Dis. 2007; 49:284-99. 8. Snappin SM. Noninferiority trials. Curr Control Trials Cardiovasc Med. 2000; 1:1921. 9. Committee for Medicinal Products for Human Use, European Medicines Agency. Guideline on the choice of the noninferiority margin (July 27, 2005). www. ema.europa.eu/docs/en_GB/document_ library/Scientific_guideline/2009/09/ WC500003636.pdf (accessed 2013 Jul 12). 10. Kaul S, Diamond G. Good enough: a primer on the analysis and interpretation of noninferiority trials. Ann Intern Med. 2006; 145:62-9. 11. Schumock GT, Pickard AS. Comparative effectiveness research: relevance and applications to pharmacy. Am J Health-Syst Pharm. 2009; 66:1278-86. 12. Jones B, Jarvis P, Lewis JA, Ebbutt AF. Trials to assess equivalence: the importance of rigorous methods. BMJ. 1996; 313:36. 13. Ferrill MJ, Brown D, Kyle JA. Clinical versus statistical significance: interpreting p values and confidence intervals related to measures of association to guide decision making. J Pharm Pract. 2010; 23:344-51. 14. Malone PM, Kier KL, Stanovich JE. Drug information: a guide for pharmacists. 4th ed. New York: McGraw-Hill; 2012:1627,198-202. 15. The EINSTEIN–PE Investigators. Oral rivaroxaban for the treatment of symptomatic pulmonary embolism. N Engl J Med. 2012; 366:1287-97. 16. Schulman S, Kearon C, Kakkar AK et al. Extended use of dabigatran, warfarin, or placebo in venous thromboembolism. N Engl J Med. 2013; 368:709-18. 17. The EINSTEIN Investigators. Oral rivaroxaban for symptomatic venous thromboembolism. N Engl J Med. 2010; 363:2499-510. 18. Büller HR, Cohen AT, Davidson B et al. Idraparinux versus standard therapy for venous thromboembolic disease. N Engl J Med. 2007; 357:1094-104. 19. Büller HR, Davidson BL, Decousus H et al. Fondaparinux or enoxaparin for the initial treatment of symptomatic deep venous thrombosis: a randomized trial. Ann Intern Med. 2004; 140:867-73. 20. Büller HR, Davidson BL, Decousus H et al. Subcutaneous fondaparinux versus intravenous unfractionated heparin in the initial treatment of pulmonary embolism. N Engl J Med. 2003; 349:1695-702. 21. Fiessinger JN, Huisman MV, Davidson BL et al. Ximelagatran vs low-molecular-
weight heparin and warfarin for the treatment of deep vein thrombosis: a randomized trial. JAMA. 2005; 293:681-9. 22. Kearon C, Ginsberg JS, Julian JA et al. Comparison of fixed-dose weight-adjusted unfractionated heparin and low-molecularweight heparin for acute treatment of venous thromboembolism. JAMA. 2006; 296:935-42. 23. Schulman S, Kearon C, Kakkar AK et al. Dabigatran versus warfarin in the treatment of acute venous thromboembolism. N Engl J Med. 2009; 361:2342-52. 24. Büller HR, Gallus AS, Pillion G et al., for the Cassiopea Investigators. Enoxaparin followed by once weekly idrabiotaparinux versus enoxaparin plus warfarin for patients with acute symptomatic pulmonary embolism: a randomised double-blind, double-dummy, non-inferiority trial. Lancet. 2012; 379:123-9. 25. Schernthaner G, Gross JL, Rosentock J et al. Canagliflozin compared with sitagliptin for patients with type 2 diabetes who do not have adequate glycemic control with metformin plus sulfonylurea: a 52-week randomized trial [published online ahead of print April 5, 2013]. Diabetes Care. 2013; 36:2508-15. [Erratum, Diabetes Care. 2013; 36:4172.] 26. Food and Drug Administration. Diabetes mellitus: developing drugs and therapeutic biologics for treatment and prevention. www.fda.gov/downloads/ Drugs/GuidanceComplianceRegulatory Information/Guidances/UCM071624.pdf (accessed 2013 Apr 21). 27. Inzucchi SE, Bergenstal RM, Buse JB et al., for the American Diabetes Association (ADA) and European Association for the Study of Diabetes (EASD). Management of hyperglycemia in type 2 diabetes: a patient-centered approach: position statement of the American Diabetes Association (ADA) and the European Association for the Study of Diabetes (EASD). Diabetes Care. 2012; 35:1364-79. [Erratum, Diabetes Care. 2013; 36:490.] 28. Food and Drug Administration. FDA approves Invokana to treat type 2 diabetes. www.fda.gov/NewsEvents/Newsroom/ PressAnnouncements/ucm345848.html (accessed 2013 Apr 21).
Krisy-Ann Thornby, Pharm.D., Assistant Professor, Pharmacy Practice, and Drug Information Co-Coordinator [email protected] Ashley Johnson, Pharm.D., BCPS, Assistant Professor, Pharmacy Practice, and Drug Information Co-Coordinator Mary J. Ferrill, Pharm.D., FASHP, Dean and Professor, Pharmacy Practice Palm Beach Atlantic University West Palm Beach, FL
The authors have declared no potential conflicts of interest. DOI 10.2146/ajhp130270
Am J Health-Syst Pharm—Vol 71 Nov 15, 2014
1931
