Clinical Trials http://ctj.sagepub.com/
Adherence reporting in randomized controlled trials Ze Zhang, Michael J Peluso, Cary P Gross, Catherine M Viscoli and Walter N Kernan Clin Trials 2014 11: 195 originally published online 19 December 2013 DOI: 10.1177/1740774513512565 The online version of this article can be found at: http://ctj.sagepub.com/content/11/2/195
Published by: http://www.sagepublications.com
On behalf of:
The Society for Clinical Trials
Additional services and information for Clinical Trials can be found at: Email Alerts: http://ctj.sagepub.com/cgi/alerts Subscriptions: http://ctj.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav
>> Version of Record - Mar 31, 2014 OnlineFirst Version of Record - Dec 19, 2013 What is This?
Downloaded from ctj.sagepub.com at TEXAS SOUTHERN UNIVERSITY on December 10, 2014
CLINICAL TRIALS
DATA MANAGEMENT AND TRIAL CONDUCT
Clinical Trials 2014; 11: 195–204
Adherence reporting in randomized controlled trials Ze Zhang, Michael J Peluso, Cary P Gross, Catherine M Viscoli and Walter N Kernan
Background Treatment adherence may influence the therapeutic effect that is observed in a randomized controlled trial (RCT). Adherence may also be an indicator of research quality and treatment acceptance by participants. Despite the importance of adherence in RCT research, little is known about current practices for its measurement and reporting. Purpose The objective of this study was to determine and evaluate adherence measurement and reporting practices in RCTs involving oral pharmacologic interventions published in high impact factor journals. Methods We conducted a systematic review of RCTs involving oral pharmacotherapy published during 2010 in 10 high-impact general medicine and subspecialty journals. Two investigators independently abstracted data regarding trial characteristics, adherence monitoring, and adherence reporting. Differences were reconciled in conference. Descriptive statistics were calculated, statistical comparisons were made using chi-square analysis, and associations assessed using Spearman’s rank correlation coefficient. Results Of 111 RCT manuscripts included in the sample, 51 (45.9%) reported study-drug adherence among participants. Studies that reported adherence results were more likely to report negative findings (i.e., no significant treatment effect in a superiority trial, non-equivalence in an equivalence trial) (p = 0.032). The most common method for adherence monitoring was pill count-back on returned bottles. Among the studies that reported adherence, the median adherence was 88.4% (range: 48%2100%), and trials with longer follow-up time reported lower adherence (r = 20.45; p = 0.0015). A minority of the 51 studies described a strategy for calculating adherence that accounted for participants who were lost to follow-up (11/51 studies; 21.6%), discontinued the study medication temporarily (6/51 studies; 11.8%), or discontinued the study drug permanently (1/51 study; 2%). Limitations This study is limited by the inclusion of a small set of journals with the highest impact factors in specific fields of clinical medicine, including general medicine. Although the analysis pertains to studies published in 2010, no new guidelines in the field since the last Consolidated Standards of Reporting Trials (CONSORT) statement have been issued that would be expected to change practices for adherence monitoring, analysis, and reporting. Conclusions Adherence measurement methodology and results are underreported in published RCTs. In the minority of RCTs that provided adherence information, there was substantial heterogeneity in how adherence was defined, analyzed, and reported. Improved reporting of adherence may enhance the interpretation of study quality and results. Clinical Trials 2014; 11: 195–204. http://ctj.sagepub.com
Department of Internal Medicine, Yale University School of Medicine, New Haven, CT, USA Author for correspondence: Walter N Kernan, IRIS Coordinating Center, Department of Internal Medicine, Yale University School of Medicine, Suite 515, 2 Church Street South, New Haven, CT 06519, USA. Email: [email protected]
Ó The Author(s), 2013 Reprints and permissions: http://www.sagepub.co.uk/journalsPermissions.nav 10.1177/1740774513512565 Downloaded from ctj.sagepub.com at TEXAS SOUTHERN UNIVERSITY on December 10, 2014
196
Z Zhang et al.
Introduction Randomized controlled trials (RCTs) are the gold standard for determining the effectiveness of medical interventions [1–3]. To safeguard accuracy and reliability, however, an RCT must be properly designed, executed, analyzed, and reported. To assure that findings are fully understood, the methods and results must be properly reported [4]. The adequacy and transparency of reporting in RCTs have come under scrutiny in the past two decades. As one result, standards have been developed to guide researchers and journal editors. The most influential of these is the Consolidated Standards of Reporting Trials (CONSORT) Statement, which was first published in 1996 [5]. Guided by the CONSORT Statement, trial researchers are now expected to report specific information about study design, methodology, conduct, analysis, and interpretation. The CONSORT Statement is an evolving document and has been revised several times. The latest version, however, still does not offer specific recommendations for reporting adherence other than indicating the number of participants who ‘did not receive allocated intervention’ [6]. In clinical care, adherence is the degree to which a patient correctly follows medical advice and represents a key determinant of therapeutic effectiveness [7–11]. In clinical trial research, adherence has a similar meaning and refers to the degree to which a study participant takes the assigned treatment as prescribed. Adherence data can influence interpretation of trial results for several reasons. Adherence will affect the magnitude of an observed treatment effect; greater adherence to an effective therapy will generally result in a larger observed effect size. Conversely, poor adherence may mask a true treatment difference between two therapies. Because of this relationship between adherence and effect size, good adherence may be critical for assuring a fair test of an experimental treatment and for preserving statistical power. Adherence may also influence the observed incidence of adverse events. In the comparison of two therapies, one toxic and the other benign, low adherence to the toxic therapy may create the deceptive appearance of equal or near-equal safety. Additionally, adherence can also provide information regarding patient acceptability of a treatment. If a large number of participants discontinued a medication in a clinical trial, knowing this can provide valuable information regarding the expected acceptability of the agent in clinical practice. For all these reasons, clinical trial researchers need to be concerned about adherence and readers need to be aware that adherence may influence the interpretation of trial results.
Despite the importance of adherence for the successful conduct and interpretation of clinical trial research, little is known about the measurement and reporting of adherence in published results. To our knowledge, only one systematic review of adherence reporting has been published, which found that 33.3% of studies published during years 2003–2004 reported adherence results [12]. This study was limited by a focus on specific disease conditions. Because of this narrow focus, the quality of adherence reporting in high-impact journals remains unclear. Moreover, this study did not assess the methods for measuring and calculating adherence, leaving a critical knowledge gap in understanding the quality of adherence data currently published. Additionally, a decade has passed since the reviewed papers were published, and thus, an examination of the current adherence reporting practices in RCTs is much needed. The objective of this study was to conduct a systematic review of adherence reporting in RCTs assessing oral pharmacologic interventions published in 10 high-impact journals, regardless of disease condition. We sought to determine the frequency with which RCTs report information about participant adherence, to describe and evaluate current practices for reporting adherence, and to analyze the characteristics of adherence reporting in RCTs.
Methods Study design We conducted a systematic review of RCTs involving oral pharmacologic therapy in humans published in 10 high-impact medical journals between 1 January 2010 and 31 December 2010. The 10 journals were selected by impact factor to include the top 4 rated journals in general medicine (New England Journal of Medicine (NEJM), Journal of the American Medical Association (JAMA), British Medical Journal (BMJ), and Lancet) as well as the top 2 rated journals in clinical cardiology (Journal of the American College of Cardiology (JACC) and Circulation), neurology (Lancet Neurology and Annals of Neurology), and cancer (Journal of Clinical Oncology and Journal of the National Cancer Institute). We limited our search to these journals because impact factor is considered a good predictor of quality in methodology and reporting of trial results [13]. We included the above subspecialty journals because of the impact that heart disease, stroke, and cancer have on the burden of disease in the United States. We included all RCTs published in these journals, regardless of disease condition addressed by the study.
Clinical Trials 2014; 11: 195–204 Downloaded from ctj.sagepub.com at TEXAS SOUTHERN UNIVERSITY on December 10, 2014
http://ctj.sagepub.com
Adherence in RCTs Literature search A reference librarian and one of the investigators (Z.Z.) conducted a MEDLINE search using the limits ‘human’ and ‘randomized clinical trials’ to identify all full-length reports of RCTs involving human oral pharmacologic therapy published in 2010 in the selected journals. We compared the results of the MEDLINE search with those of a hand-search of RCTs published in the 10 journals in January 2010 to ensure that the electronic search did not miss any manuscripts. Two investigators (Z.Z. and M.J.P.), working in parallel, assessed all titles and abstracts to identify the relevant manuscripts that met the inclusion criteria. Manuscripts were included if the RCT was identified to be a randomized, double-blinded trial involving at least three doses of oral pharmacologic therapy in human subjects and was the first fulllength publication of primary outcome results for the trial. For the publications that indicated a previously published methods article, the methods article was reviewed for adherence monitoring. If the manuscript was an interim analysis report, we included it as long as it was the first publication of the RCT reporting primary outcome data. If a trial included more than one regimen comparison, each report of primary outcome data was included as long as it was the first publication of those data. Add-on studies were included if the data for the intervention had not been previously reported. We excluded reports of subgroup or follow-up studies, observational studies, letters, editorials, reviews, and retracted publications. Data abstraction We reviewed all eligible manuscripts using a standardized data abstraction instrument developed on the basis of literature review and the CONSORT guidelines (online Appendix A). The reviewers were not blinded to the journal or author names. Each reviewer abstracted the data for all manuscripts separately, and the two sets of abstractions were compared for inter-reviewer agreement. Data were collected on general characteristics of each publication, including title, author name, trial name, journal, date of publication, medical area of trial, duration of therapy, number and type of intervention, funding, number of participants, number of centers, location of trial conduct, run-in period, type of outcome, type of analysis, and trial results. Data were also collected regarding status of adherence monitoring and results as reported in each manuscript, including method(s) for adherence
http://ctj.sagepub.com
197
monitoring, location of adherence monitoring methods within the manuscript or previous methods publication, definition of good/poor adherence, calculation of adherence, treatment of any missing participant’s adherence results, estimated adherence for clinical benefit, and other mention of adherence or compliance. If a RCT involved multiple treatment interventions, data on adherence were reviewed and abstracted for each intervention, as long as the RCT involved at least one oral medication. If a trial reported a pre-specified treatment duration range, the median duration was recorded as the midpoint of this range. For trials without a pre-specified duration, the actual median duration was used. For trials that did not specify a median duration, the mean duration was calculated from the data. Adherence results were recorded as presented in the publication and as a percentage when possible. The two reviewers met to compare their results. By protocol, any disagreements were addressed by joint examination of underlying documents to reach consensus. If consensus could not be reached, a third reviewer was available to create a binding majority opinion. Data analysis Studies were classified as having reported adherence methods if monitoring methods and calculation of adherence were described in either the main publication or the methods paper previously. Studies were classified as having reported adherence results if the adherence results appeared in the main publication. All statistical analyses were performed using SPSS version 20 (SPSS Inc, Chicago, IL) and Prism version 5.0d (GraphPad Software, San Diego, USA). The threshold for the determination of statistical significance was p \ 0.05. For categorical variables, frequency analysis was used to describe the data. Chisquare analysis was performed between reporting adherence result (yes/no), and the trial and publication characteristics. Associations between magnitude of adherence result and continuous variables were assessed using the nonparametric Spearman rank correlation coefficient. To classify adherence for each selected study, we used the author-reported adherence when adherence was reported as a percentage. If the authors did not report adherence as a percentage, we used the data provided in the manuscript to calculate a percentage. Three of the 51 studies did not report adherence as a percentage or provide results that could be converted to a percentage (e.g., denominator was not specified).
Clinical Trials 2014; 11: 195–204 Downloaded from ctj.sagepub.com at TEXAS SOUTHERN UNIVERSITY on December 10, 2014
198
Z Zhang et al.
Figure 1. Method and results of MEDLINE literature search, including title, abstract, and full-text review of the published literature. RCT: randomized controlled trial.
Results Clinical trials selected for this review Figure 1 shows the results of the MEDLINE literature review. The search generated 683 titles, 111 (16.3%) of which were full-text publications eligible for inclusion. Agreement about eligibility between reviewers was excellent for whether publications reported adherence results or methods (Cohen’s kappa . 0.9 for both). In all, 572 studies were excluded: 420 based on title information, 99 based on information obtained from the abstract, and 53 manuscripts based on the full publication itself. The reasons for exclusion were as follows: an intervention that was not an oral medication (n = 383); report of a subgroup, secondary, or follow-up analysis (n = 54); open-label or singleblind design (n = 68); observational or non-randomized design (n = 61); and intended administration of fewer than three doses of intervention medication (n = 6).
Of the 111 studies selected for review, most were phase III or IV, placebo-controlled, non-government funded, and employed an intention-to-treat analytic strategy (Table 1). Almost 40% of the studies involved cardiovascular therapies. Table 1 summarizes the broad range of subspecialty areas of these studies. Adherence reporting Of the 111 studies included in the sample, 51 (45.9%) reported adherence results (Table 1). An additional 12 (10.8%) stated that adherence was monitored, but did not report results. Studies that reported adherence results were more likely to report negative trial outcome findings (i.e., findings that do not confirm the anticipated superiority or equivalence of one treatment compared with another) (49.02 vs. 30.00, p = 0.03) and be government funded (41.17 vs. 25.00, p = 0.07). No other differences between reporting and nonreporting trial reached or approached statistical significance.
Clinical Trials 2014; 11: 195–204 Downloaded from ctj.sagepub.com at TEXAS SOUTHERN UNIVERSITY on December 10, 2014
http://ctj.sagepub.com
Adherence in RCTs
199
Table 1. Characteristics of randomized clinical trials in the review according to adherence reporting status Characteristics
Group
Phase of trial I and II III and IV Type of trial Placebo controlled Active comparison Number of compared interventions 2 3 or more Flexible dosing allowed Yes No Not specified Run-in period Yes No Pre-specified duration of treatment \1 month 1 month to 1 year .1 year Not specified Funding for conduct of triala Government Non-government Not specified Type of analysis for primary outcomeb ITT** On treatment Not specified Type of outcome Cardiovascular Neurology (non-stroke)/psychiatry Oncology Musculoskeletal Infectious disease Obstetrics and gynecology Pediatrics Lifestyle modification Gastroenterology Respiratory Renal Number of centers Single center 2–10 centers 11–50 centers 51–300 centers .300 centers Number of participants randomized \100 100–999 .1000
p-value*
Trials not reporting adherence, N = 60 n, (%)
Trials reporting adherence, N = 51 n, (%)
12 (20.00) 48 (80.00)
8 (15.69) 43 (84.31)
0.56
53 (88.33) 7 (11.67)
41 (80.39) 10 (19.61)
0.25
44 (73.33) 16 (26.67)
39 (76.47) 12 (23.53)
0.70
17 (28.33) 42 (70.00) 1 (1.67)
11 (21.57) 40 (78.43) 0 (0.00)
0.45
12 (20.00) 48 (80.00)
11 (21.57) 40 (78.43)
0.84
8 (13.33) 29 (48.33) 16 (26.67) 7 (11.67)
7 (13.73) 18 (35.29) 13 (25.49) 13 (25.49)
0.78
15 (25.00) 41 (68.33) 4 (6.67)
21 (41.17) 29 (56.86) 1 (1.96)
0.07
53 (88.33) 6 (10.00) 1 (1.67)
45 (88.24) 3 (5.88) 3 (5.88)
0.47
24 (40.00) 10 (16.67) 8 (13.33) 4 (6.67) 4 (6.67) 2 (3.33) 2 (3.33) 3 (5.00) 1 (1.67) 1 (1.67) 1 (1.67)
20 (39.22) 10 (19.61) 1 (1.96) 4 (7.84) 3 (5.88) 4 (7.84) 4 (7.84) 1 (1.96) 2 (3.92) 2 (3.92) 0 (0.00)
0.51
5 (8.33) 20 (33.33) 12 (20.00) 17 (28.33) 6 (10.00)
6 (11.76) 15 (29.41) 13 (25.49) 16 (31.37) 1 (1.96)
0.44
9 (15.00) 35 (58.33) 16 (26.67)
8 (15.69) 25 (49.02) 18 (35.29)
0.57
(continued)
http://ctj.sagepub.com
Clinical Trials 2014; 11: 195–204 Downloaded from ctj.sagepub.com at TEXAS SOUTHERN UNIVERSITY on December 10, 2014
200
Z Zhang et al.
Table 1. (Continued) Characteristics
Group
Treatment was effective or equivalentc Yes No Not specified Previously published methods Yes No
p-value*
Trials not reporting adherence, N = 60 n, (%)
Trials reporting adherence, N = 51 n, (%)
42 (70.00) 18 (30.00) 0 (0.00)
25 (49.02) 25 (49.02) 1 (1.96)
0.03
10 (16.67) 50 (83.33)
12 (23.53) 39 (76.47)
0.37
ITT: intention to treat; OT: observed treatment. a Funding: Government, if had any government funding was used; non-government, if only industry or personal or non-profit funding was used; 5 trials did not specify. b Type of analysis, 4 did not specify; if used both ITT and OT, counted as ITT. c Treatment was effective: 1 did not specify. Treatment was classified as not effective in a superiority trial if the observed difference between treatment groups did not reach a pre-defined level of statistical significance. Treatment was classified as non-equivalent in an equivalence trail if the upper bound of the 95% confidence interval around the difference in treatment effect crossed the non-inferiority margin. * p-value based on chi-square test or Fisher’s exact test if hp 5. ** Trial primary outcome results are analyzed based on the assigned treatment instead of actual treatment received.
Table 2. Methods for monitoring adherence among publications that reported adherence result Method for adherence monitoring
n (%)
Pill count-back Self-report Electronic monitoring Serum concentration Direct observation
21 (56.8) 9 (24.3) 4 (10.8) 2 (5.4) 1 (2.7)
Adherence monitoring methods Among the 51 trials that reported drug adherence results, 37 (72.5%) described the method by which adherence was measured (Table 2). Of these 37 studies, 2 measured drug serum concentration and 35 measured pill counts by some means. The most common method for measuring adherence, used in 21 studies (56.8%), was the count-back method (e.g., subtracting pills remaining in returned bottles from the dispensed quantity to calculate the number consumed). Nine (24.3%) trials used patient self-report, four (10.8%) used electronic devices for detecting access to a pill container, and one (2.7%) used direct observation by trial staff or other medical personnel. Quality of adherence reporting We examined the quality of adherence reporting among the 37 studies that described methods for
monitoring. In all 37 studies, adherence was reported as a percentage. We looked closely at each study to determine how the percentage was calculated. In all, 31 of 37 (83.8%) studies defined both the numerator (e.g., number of participants, total number of pills consumed) and denominator (e.g., total number of participants randomized, number of participants who completed the study, total number of pills prescribed by protocol). Four (10.8%) studies reported the numerator but not the denominator. Two (5.4%) studies did not specify either number. In clinical trials, it is very common for participants to stop taking the study medication temporarily or permanently before completing the research protocol and being formally exited. For this reason, we were particularly interested to know how the adherence percentage would reflect intervals when participants were off drug or when they were taking a reduced dose. For simplicity, we confined this analysis to the 35 studies that used pill counts as a basis for adherence reporting. Only 6/35 studies (17.1%) specified how periods of permanent drug discontinuation affected the adherence calculation. Four of these indicated that participants were censored at the time of permanent drug discontinuation even if this occurred before the planned end of follow-up. Therefore, intervals off drug were not included in the calculation of adherence. Two studies indicated that adherence was recorded as zero for periods of follow-up off drug. Only one study specified how periods of
Clinical Trials 2014; 11: 195–204 Downloaded from ctj.sagepub.com at TEXAS SOUTHERN UNIVERSITY on December 10, 2014
http://ctj.sagepub.com
Adherence in RCTs temporary drug discontinuation affected the adherence calculation. This study did not include the period of discontinuation in the calculation of adherence. The inclusion of subjects lost to follow-up was addressed in 11/37 (29.7%) studies, of which 9 (81.8%) excluded these subjects from the adherence calculation even if adherence data were available prior to the last contact. The other two studies indicated adherence was recorded as zero if participants were lost to follow-up. Of the 37 studies reporting adherence monitoring and results, 21 (56.8%) also reported a criterion for good adherence. These 21 studies listed criteria ranging from 50% to 80%. Three (8.1%) manuscripts estimated the adherence level required to achieve a clinically significant effect, which included achieving the therapeutic target for serum lithium in one trial and taking 80% of prescribed pills in the other two trials. Adherence results were reported over multiple time periods (e.g., 1 year, 3 years, 5 years; or 1 month, 6 months, 12 months, etc.) in 9/37 (24.3%) manuscripts. Nine of the 37 (24.3%) manuscripts indicated that adherence data were missing for a subset of trial participants. Flexible dosing was allowed in 28/111 trials (Table 1). Of the 37 trials, 2 (5.4%) specified whether participants taking a lower dose of study medication were included in the adherence analysis. In both cases, these subjects were excluded from the adherence calculation. Adherence results In the 51 studies with reported adherence, mean adherence was 85.7% (range: 48%2100%) and the median adherence was 88.4%. Figure 2 shows the range of reported adherence results as a histogram. Adherence was negatively correlated with duration of trial participation (r = 20.45, p = 0.0015). There was a non-significant negative correlation between adherence and number of subjects randomized (r = 20.24, p = 0.097). There was no relationship between the magnitude of reported adherence and number of study centers. Table 3 shows a cross-section of the language used to describe adherence results in selected publications. It reflects the variability of adherence reporting with regard to method of monitoring, details of the adherence calculation, and interpretation of the adherence result.
Discussion We found that less than half of all RCTs involving drug therapy reported adherence data in their main http://ctj.sagepub.com
201
Figure 2. Adherence results as percentages in 48/51 trials that reported adherence. If adherence was not reported as a percentage, adherence results were converted to a percentage based on provided information. Median reported adherence was 88.4% (range: 48%2100%). Note: adherence results in three studies could not be converted to a percentage.
publication. Among those that did report adherence data, a description of the methods for adherence monitoring was lacking in 27% of studies. Reported methods, furthermore, were often inadequate to allow a meaningful interpretation of adherence results. Researchers commonly failed to define the numerator and denominator for calculation of adherence from pill counts, explain how periods off drug were considered in adherence calculation, and report missing adherence data. Overall, our results indicate a striking lack of transparency in defining, monitoring, and reporting adherence. As a result of deficiencies in the reporting of methods for adherence monitoring, it is difficult to interpret the finding that average adherence among studies reporting adherence exceeded 85%. This high reported adherence may be inflated by failure to include periods of drug cessation in the adherence calculation, variable practices for accounting for periods of dose reduction, and variable practices for censoring participants during follow-up. Our findings build upon prior work by Gossec et al.’s [12] 2007 study by expanding the inclusion criteria to include all disease conditions and by performing a more detailed analysis of the quality of adherence reporting. We reviewed not only the reporting of adherence results, but also how adherence results were calculated. We paid particular attention to the transparency of adherence calculations in the publications, an issue that has been previously ignored in RCT publication standards. Thus, we believe that our results are more reflective of the general state of adherence reporting across the highimpact literature. Clinical Trials 2014; 11: 195–204
Downloaded from ctj.sagepub.com at TEXAS SOUTHERN UNIVERSITY on December 10, 2014
202
Z Zhang et al.
Table 3. Language used to describe adherence result in selected publications Stated adherence monitoring method
Reported adherence result
Reference
Pill count
‘On the basis of counts of returned pills . from 618 women, 524 (85%) took at least 50% of their tablets, 434 (70%) took 80% or more, and 237 (38%) took all their tablets; 17 (3%) did not take any .’ ‘The rate of study-drug adherence was only 85%’ ‘25 of 52 patients in the lamotrigine group were serum compliant (mean serum concentration 141 mg/L)’ ‘Regular intake of study medication (70% of study period) occurred in 93.1% of the patients in the omega group and 93.2% of the control group (P-0.95)’. ‘Among the 9969 women . the median ratio of the number of study capsules taken to the number that should have been taken, between the time of randomization and delivery, was 88% in both study groups .’ ‘The controlling of the medication blister demonstrated full compliance with the study protocol in all patients.’ ‘At 28 weeks’ gestation, 35.6% of mothers reported that they had not missed taking any capsules and an additional 34.5% reported that they had missed 3 or less per week’ ‘2193 (98%) patients in the celecoxib group and 2179 (97%) in the diclofenac plus omeprazole group had drug adherence between 80% and 120%’.
McCance et al., Lancet [14]
Self-report Serum concentration Pill count
Pill count
Pill count Self-report
Not specified
Three findings in this study warrant special attention. First, we identified a tendency to report adherence in cases where the trial outcome is negative, and to omit a discussion of adherence when the outcome is positive. The inclination to attribute a negative outcome to poor adherence is understandable, and adherence data may provide valuable support for this argument and insight into why a particular intervention might have failed. However, the utility of adherence data is not limited to trials with negative results. Transparency in adherence reporting is crucial to understanding how the results of a positive trial should be interpreted and how they might apply to a new clinical population. Consider, for example, three trials of new pharmacologic interventions for the same disease. Each trial finds a similar, statistically significant treatment effect. In the first trial, the adherence rate was extremely high. This might imply that achieving the observed treatment benefit would be difficult in realworld populations with less than ideal adherence. In the second trial, the adherence rate is very low. While a positive finding in the setting of low adherence might imply that a treatment is very effective, it might also raise concern for bias or error that could distort the observed effect. In the third trial, adherence is average. A positive result with average adherence might suggest that the intervention could be of value in real-world practice. Knowledge of treatment adherence provides insight into the meaning, reliability, and applicability of trial results.
Kaandorp et al., NEJM [15] Kapoor et al., Lancet Neurology [16] Rauch et al., Circulation [17]
Roberts et al., NEJM [18]
Sibbing et al., JACC [19] Makrides et al., JAMA [20]
Chan et al., Lancet [21]
Second, only 57% of studies that reported adherence provided a criterion for ‘adequate adherence’. These studies cited criteria ranging from 50% to 80%, which compares with a commonly used criteria for good adherence of 80% of participants taking 80% of trial medication [7,22]. On one hand, this omission is understandable in that adherence may be difficult to predict and the level of adherence required for the hypothesized treatment effect may be even more difficult to predict. On the other hand, in designing a trial, investigators should have in mind some minimal level of adherence required for success. Third, we found a lack of transparency in the calculation of adherence. At the most basic level, the denominator for calculating adherence was often not reported, making the reported adherence result difficult to interpret. The method for accounting for periods temporarily off drug was described in only one of 37 studies. For the other 36 studies, readers are left wondering if full adherence might have been imputed for intervals of temporarly non-use. The result of failing to decrement adherence for intervals off drug is to erroneously inflate the reported adherence. This is a particular problem for studies using an intention-to-treat analysis. In such studies, which are the standard for RCTs, participants continue to be followed for outcomes regardless of treatment adherence. Failure to decrement adherence may create an artificial gap between the interpretation of trial outcomes and interpretation of adherence
Clinical Trials 2014; 11: 195–204 Downloaded from ctj.sagepub.com at TEXAS SOUTHERN UNIVERSITY on December 10, 2014
http://ctj.sagepub.com
Adherence in RCTs results, rendering it more difficult to apply trial results to clinical practice. Our study has several limitations. We included a small set of journals with the highest impact factor in specific fields of clinical medicine, including general medicine. It is possible that a larger journal sample size would yield different results. However, top journals typically require the greatest methodological rigor in research publications and tend to set the standard for other journals. We chose the subspecialty journals in cardiology, neurology, and oncology because cardiovascular disease, stroke, and cancer were the three leading causes of adult death in the United States. These three fields represented some of the most prolific clinical trial research categories due to their burden of disease and thus should yield a large number of RCTs for review. Having blinded reviewers would have decreased potential bias in this study. We attempted to minimize bias by having each reviewer review the publications independently and systematically. Finally, although our analysis pertains to studies published in 2010, no new guidelines or pronouncements in the field since the last CONSORT statement have been issued that would be expected to change practices for adherence monitoring, analysis, and reporting. In summary, our findings document that published reports of RCTs using oral drug therapy commonly do not provide adequate data on study-drug adherence. The development of clear and universal standards for gathering, analyzing, and reporting adherence data may be an important next step for quality improvement within the scientific literature. These standards would need to recognize that most adherence measurements are only estimates and that best measures for an individual trial would be difficult to prescribe at this time. While these standards are considered, we believe investigators should be encouraged to take the following steps: 1. Measure adherence among all participants. 2. Include in the research protocol and, in at least one publication, the precise methods for measuring, analyzing, and reporting adherence. Provide both the numerator and the denominator for the calculation, including units of measurement. Explain how intervals off drug are factored into the adherence calculation. 3. Report adherence data in the main publication for the trial.
Acknowledgements Z.Z. and M.J.P. contributed equally to this work. We would like to thank Mark Gentry at Yale School of Medicine Cushing Medical Library for helping us design and organize the MEDLINE search for our http://ctj.sagepub.com
203
review. No compensation was provided for this contribution.
Funding This work was supported in part by a stipend from the Office of Student Affairs and the Office of Student Research at the Yale University School of Medicine to support Z.Z., and by a grant from the Doris Duke Charitable Foundation to the Yale University School of Medicine to support Clinical Research Fellow M.J.P..
Conflict of interest None.
References 1. Byar DP, Simon RM, Friedewald WT, et al. Randomized clinical trials. Perspectives on some recent ideas. N Engl J Med 1976; 295(2): 74–80. 2. Feinstein AR. Current problems and future challenges in randomized clinical trials. Circulation 1984; 70(5): 767–74. 3. Abel U, Koch A. The role of randomization in clinical studies: Myths and beliefs. J Clin Epidemiol 1999; 52(6): 487–97. 4. Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med 2000; 342(25): 1887–92. 5. Begg C, Cho M, Eastwood S, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA 1996; 276(8): 637–39. 6. Schulz KF, Altman DG, Moher D; CONSORT Group. CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials. BMJ 2010; 340: c332. 7. Haynes RB, Taylor DW, Sacket DL. Compliance in Healthcare. The Johns Hopkins University Press, Baltimore, MD, 1979. 8. Cramer JA, Spilker B. Patient Compliance in Medical Practice and Clinical Trials. Raven Press, New York, 1991. 9. Osterberg L, Blaschke T. Adherence to medication. N Engl J Med 2005; 353(5): 487–97. 10. Haynes RB, Ackloo E, Sahota N, McDonald HP, Yao X. Interventions for enhancing medication adherence. Cochrane Database Syst Rev 2008; 2: CD000011. 11. Cramer J, Rosenheck R, Kirk G, Krol W, Krystal J; VA Naltrexone Study Group 425. Medication compliance feedback and monitoring in a clinical trial: Predictors and outcomes. Value Health 2003; 6(5): 566–73. 12. Gossec L, Tubach F, Dougados M, Ravaud P. Reporting of adherence to medication in recent randomized controlled trials of 6 chronic diseases: A systematic literature review. Am J Med Sci 2007; 334(4): 248–54. 13. Lee KP, Schotland M, Bacchetti P, Bero LA. Association of journal quality indicators with methodological quality of clinical research manuscripts. JAMA 2002; 287(21): 2805–08.
Clinical Trials 2014; 11: 195–204 Downloaded from ctj.sagepub.com at TEXAS SOUTHERN UNIVERSITY on December 10, 2014
204
Z Zhang et al.
14. McCance DR, Holmes VA, Maresh MJA, et al. Vitamins C and E for prevention of pre-eclampsia in women with type 1 diabetes (DAPIT): A randomised placebocontrolled trial. Lancet 2010; 376(9737): 259–66. 15. Kaandorp SP, Goddijn M, van der Post JAM, et al. Aspirin plus heparin or aspirin alone in women with recurrent miscarriage. N Engl J Med 2010; 362(17): 1586–96. 16. Kapoor R, Furby J, Hayton T, et al. Lamotrigine for neuroprotection in secondary progressive multiple sclerosis: A randomised, double-blind, placebo-controlled, parallelgroup trial. Lancet Neurol 2010; 9(7): 681–88. 17. Rauch B, Schiele R, Schneider S, et al. OMEGA, a randomized, placebo-controlled trial to test the effect of highly purified omega-3 fatty acids on top of modern guidelineadjusted therapy after myocardial infarction. Circulation 2010; 122(21): 2152–59.
18. Roberts JM, Myatt L, Spong CY, et al. Vitamins C and E to prevent complications of pregnancy-associated hypertension. N Engl J Med 2010; 362(14): 1282–91. 19. Sibbing D, Stegherr J, Braun S, et al. A double-blind, randomized study on prevention and existence of a rebound phenomenon of platelets after cessation of clopidogrel treatment. J Am Coll Cardiol 2010; 55(6): 558–65. 20. Makrides M, Gibson RA, McPhee AJ, et al. Effect of DHA supplementation during pregnancy on maternal depression and neurodevelopment of young children: A randomized controlled trial. JAMA 2010; 304(15): 1675–83. 21. Chan FKL, Lanas A, Scheiman J, et al. Celecoxib versus omeprazole and diclofenac in patients with osteoarthritis and rheumatoid arthritis (CONDOR): A randomised trial. Lancet 2010; 376(9736): 173–79. 22. Robiner WN. Enhancing adherence in clinical research. Contemp Clin Trials 2005; 26(1): 59–77.
Clinical Trials 2014; 11: 195–204 Downloaded from ctj.sagepub.com at TEXAS SOUTHERN UNIVERSITY on December 10, 2014
http://ctj.sagepub.com
Adherence reporting in randomized controlled trials Ze Zhang, Michael J Peluso, Cary P Gross, Catherine M Viscoli and Walter N Kernan Clin Trials 2014 11: 195 originally published online 19 December 2013 DOI: 10.1177/1740774513512565 The online version of this article can be found at: http://ctj.sagepub.com/content/11/2/195
Published by: http://www.sagepublications.com
On behalf of:
The Society for Clinical Trials
Additional services and information for Clinical Trials can be found at: Email Alerts: http://ctj.sagepub.com/cgi/alerts Subscriptions: http://ctj.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav
>> Version of Record - Mar 31, 2014 OnlineFirst Version of Record - Dec 19, 2013 What is This?
Downloaded from ctj.sagepub.com at TEXAS SOUTHERN UNIVERSITY on December 10, 2014
CLINICAL TRIALS
DATA MANAGEMENT AND TRIAL CONDUCT
Clinical Trials 2014; 11: 195–204
Adherence reporting in randomized controlled trials Ze Zhang, Michael J Peluso, Cary P Gross, Catherine M Viscoli and Walter N Kernan
Background Treatment adherence may influence the therapeutic effect that is observed in a randomized controlled trial (RCT). Adherence may also be an indicator of research quality and treatment acceptance by participants. Despite the importance of adherence in RCT research, little is known about current practices for its measurement and reporting. Purpose The objective of this study was to determine and evaluate adherence measurement and reporting practices in RCTs involving oral pharmacologic interventions published in high impact factor journals. Methods We conducted a systematic review of RCTs involving oral pharmacotherapy published during 2010 in 10 high-impact general medicine and subspecialty journals. Two investigators independently abstracted data regarding trial characteristics, adherence monitoring, and adherence reporting. Differences were reconciled in conference. Descriptive statistics were calculated, statistical comparisons were made using chi-square analysis, and associations assessed using Spearman’s rank correlation coefficient. Results Of 111 RCT manuscripts included in the sample, 51 (45.9%) reported study-drug adherence among participants. Studies that reported adherence results were more likely to report negative findings (i.e., no significant treatment effect in a superiority trial, non-equivalence in an equivalence trial) (p = 0.032). The most common method for adherence monitoring was pill count-back on returned bottles. Among the studies that reported adherence, the median adherence was 88.4% (range: 48%2100%), and trials with longer follow-up time reported lower adherence (r = 20.45; p = 0.0015). A minority of the 51 studies described a strategy for calculating adherence that accounted for participants who were lost to follow-up (11/51 studies; 21.6%), discontinued the study medication temporarily (6/51 studies; 11.8%), or discontinued the study drug permanently (1/51 study; 2%). Limitations This study is limited by the inclusion of a small set of journals with the highest impact factors in specific fields of clinical medicine, including general medicine. Although the analysis pertains to studies published in 2010, no new guidelines in the field since the last Consolidated Standards of Reporting Trials (CONSORT) statement have been issued that would be expected to change practices for adherence monitoring, analysis, and reporting. Conclusions Adherence measurement methodology and results are underreported in published RCTs. In the minority of RCTs that provided adherence information, there was substantial heterogeneity in how adherence was defined, analyzed, and reported. Improved reporting of adherence may enhance the interpretation of study quality and results. Clinical Trials 2014; 11: 195–204. http://ctj.sagepub.com
Department of Internal Medicine, Yale University School of Medicine, New Haven, CT, USA Author for correspondence: Walter N Kernan, IRIS Coordinating Center, Department of Internal Medicine, Yale University School of Medicine, Suite 515, 2 Church Street South, New Haven, CT 06519, USA. Email: [email protected]
Ó The Author(s), 2013 Reprints and permissions: http://www.sagepub.co.uk/journalsPermissions.nav 10.1177/1740774513512565 Downloaded from ctj.sagepub.com at TEXAS SOUTHERN UNIVERSITY on December 10, 2014
196
Z Zhang et al.
Introduction Randomized controlled trials (RCTs) are the gold standard for determining the effectiveness of medical interventions [1–3]. To safeguard accuracy and reliability, however, an RCT must be properly designed, executed, analyzed, and reported. To assure that findings are fully understood, the methods and results must be properly reported [4]. The adequacy and transparency of reporting in RCTs have come under scrutiny in the past two decades. As one result, standards have been developed to guide researchers and journal editors. The most influential of these is the Consolidated Standards of Reporting Trials (CONSORT) Statement, which was first published in 1996 [5]. Guided by the CONSORT Statement, trial researchers are now expected to report specific information about study design, methodology, conduct, analysis, and interpretation. The CONSORT Statement is an evolving document and has been revised several times. The latest version, however, still does not offer specific recommendations for reporting adherence other than indicating the number of participants who ‘did not receive allocated intervention’ [6]. In clinical care, adherence is the degree to which a patient correctly follows medical advice and represents a key determinant of therapeutic effectiveness [7–11]. In clinical trial research, adherence has a similar meaning and refers to the degree to which a study participant takes the assigned treatment as prescribed. Adherence data can influence interpretation of trial results for several reasons. Adherence will affect the magnitude of an observed treatment effect; greater adherence to an effective therapy will generally result in a larger observed effect size. Conversely, poor adherence may mask a true treatment difference between two therapies. Because of this relationship between adherence and effect size, good adherence may be critical for assuring a fair test of an experimental treatment and for preserving statistical power. Adherence may also influence the observed incidence of adverse events. In the comparison of two therapies, one toxic and the other benign, low adherence to the toxic therapy may create the deceptive appearance of equal or near-equal safety. Additionally, adherence can also provide information regarding patient acceptability of a treatment. If a large number of participants discontinued a medication in a clinical trial, knowing this can provide valuable information regarding the expected acceptability of the agent in clinical practice. For all these reasons, clinical trial researchers need to be concerned about adherence and readers need to be aware that adherence may influence the interpretation of trial results.
Despite the importance of adherence for the successful conduct and interpretation of clinical trial research, little is known about the measurement and reporting of adherence in published results. To our knowledge, only one systematic review of adherence reporting has been published, which found that 33.3% of studies published during years 2003–2004 reported adherence results [12]. This study was limited by a focus on specific disease conditions. Because of this narrow focus, the quality of adherence reporting in high-impact journals remains unclear. Moreover, this study did not assess the methods for measuring and calculating adherence, leaving a critical knowledge gap in understanding the quality of adherence data currently published. Additionally, a decade has passed since the reviewed papers were published, and thus, an examination of the current adherence reporting practices in RCTs is much needed. The objective of this study was to conduct a systematic review of adherence reporting in RCTs assessing oral pharmacologic interventions published in 10 high-impact journals, regardless of disease condition. We sought to determine the frequency with which RCTs report information about participant adherence, to describe and evaluate current practices for reporting adherence, and to analyze the characteristics of adherence reporting in RCTs.
Methods Study design We conducted a systematic review of RCTs involving oral pharmacologic therapy in humans published in 10 high-impact medical journals between 1 January 2010 and 31 December 2010. The 10 journals were selected by impact factor to include the top 4 rated journals in general medicine (New England Journal of Medicine (NEJM), Journal of the American Medical Association (JAMA), British Medical Journal (BMJ), and Lancet) as well as the top 2 rated journals in clinical cardiology (Journal of the American College of Cardiology (JACC) and Circulation), neurology (Lancet Neurology and Annals of Neurology), and cancer (Journal of Clinical Oncology and Journal of the National Cancer Institute). We limited our search to these journals because impact factor is considered a good predictor of quality in methodology and reporting of trial results [13]. We included the above subspecialty journals because of the impact that heart disease, stroke, and cancer have on the burden of disease in the United States. We included all RCTs published in these journals, regardless of disease condition addressed by the study.
Clinical Trials 2014; 11: 195–204 Downloaded from ctj.sagepub.com at TEXAS SOUTHERN UNIVERSITY on December 10, 2014
http://ctj.sagepub.com
Adherence in RCTs Literature search A reference librarian and one of the investigators (Z.Z.) conducted a MEDLINE search using the limits ‘human’ and ‘randomized clinical trials’ to identify all full-length reports of RCTs involving human oral pharmacologic therapy published in 2010 in the selected journals. We compared the results of the MEDLINE search with those of a hand-search of RCTs published in the 10 journals in January 2010 to ensure that the electronic search did not miss any manuscripts. Two investigators (Z.Z. and M.J.P.), working in parallel, assessed all titles and abstracts to identify the relevant manuscripts that met the inclusion criteria. Manuscripts were included if the RCT was identified to be a randomized, double-blinded trial involving at least three doses of oral pharmacologic therapy in human subjects and was the first fulllength publication of primary outcome results for the trial. For the publications that indicated a previously published methods article, the methods article was reviewed for adherence monitoring. If the manuscript was an interim analysis report, we included it as long as it was the first publication of the RCT reporting primary outcome data. If a trial included more than one regimen comparison, each report of primary outcome data was included as long as it was the first publication of those data. Add-on studies were included if the data for the intervention had not been previously reported. We excluded reports of subgroup or follow-up studies, observational studies, letters, editorials, reviews, and retracted publications. Data abstraction We reviewed all eligible manuscripts using a standardized data abstraction instrument developed on the basis of literature review and the CONSORT guidelines (online Appendix A). The reviewers were not blinded to the journal or author names. Each reviewer abstracted the data for all manuscripts separately, and the two sets of abstractions were compared for inter-reviewer agreement. Data were collected on general characteristics of each publication, including title, author name, trial name, journal, date of publication, medical area of trial, duration of therapy, number and type of intervention, funding, number of participants, number of centers, location of trial conduct, run-in period, type of outcome, type of analysis, and trial results. Data were also collected regarding status of adherence monitoring and results as reported in each manuscript, including method(s) for adherence
http://ctj.sagepub.com
197
monitoring, location of adherence monitoring methods within the manuscript or previous methods publication, definition of good/poor adherence, calculation of adherence, treatment of any missing participant’s adherence results, estimated adherence for clinical benefit, and other mention of adherence or compliance. If a RCT involved multiple treatment interventions, data on adherence were reviewed and abstracted for each intervention, as long as the RCT involved at least one oral medication. If a trial reported a pre-specified treatment duration range, the median duration was recorded as the midpoint of this range. For trials without a pre-specified duration, the actual median duration was used. For trials that did not specify a median duration, the mean duration was calculated from the data. Adherence results were recorded as presented in the publication and as a percentage when possible. The two reviewers met to compare their results. By protocol, any disagreements were addressed by joint examination of underlying documents to reach consensus. If consensus could not be reached, a third reviewer was available to create a binding majority opinion. Data analysis Studies were classified as having reported adherence methods if monitoring methods and calculation of adherence were described in either the main publication or the methods paper previously. Studies were classified as having reported adherence results if the adherence results appeared in the main publication. All statistical analyses were performed using SPSS version 20 (SPSS Inc, Chicago, IL) and Prism version 5.0d (GraphPad Software, San Diego, USA). The threshold for the determination of statistical significance was p \ 0.05. For categorical variables, frequency analysis was used to describe the data. Chisquare analysis was performed between reporting adherence result (yes/no), and the trial and publication characteristics. Associations between magnitude of adherence result and continuous variables were assessed using the nonparametric Spearman rank correlation coefficient. To classify adherence for each selected study, we used the author-reported adherence when adherence was reported as a percentage. If the authors did not report adherence as a percentage, we used the data provided in the manuscript to calculate a percentage. Three of the 51 studies did not report adherence as a percentage or provide results that could be converted to a percentage (e.g., denominator was not specified).
Clinical Trials 2014; 11: 195–204 Downloaded from ctj.sagepub.com at TEXAS SOUTHERN UNIVERSITY on December 10, 2014
198
Z Zhang et al.
Figure 1. Method and results of MEDLINE literature search, including title, abstract, and full-text review of the published literature. RCT: randomized controlled trial.
Results Clinical trials selected for this review Figure 1 shows the results of the MEDLINE literature review. The search generated 683 titles, 111 (16.3%) of which were full-text publications eligible for inclusion. Agreement about eligibility between reviewers was excellent for whether publications reported adherence results or methods (Cohen’s kappa . 0.9 for both). In all, 572 studies were excluded: 420 based on title information, 99 based on information obtained from the abstract, and 53 manuscripts based on the full publication itself. The reasons for exclusion were as follows: an intervention that was not an oral medication (n = 383); report of a subgroup, secondary, or follow-up analysis (n = 54); open-label or singleblind design (n = 68); observational or non-randomized design (n = 61); and intended administration of fewer than three doses of intervention medication (n = 6).
Of the 111 studies selected for review, most were phase III or IV, placebo-controlled, non-government funded, and employed an intention-to-treat analytic strategy (Table 1). Almost 40% of the studies involved cardiovascular therapies. Table 1 summarizes the broad range of subspecialty areas of these studies. Adherence reporting Of the 111 studies included in the sample, 51 (45.9%) reported adherence results (Table 1). An additional 12 (10.8%) stated that adherence was monitored, but did not report results. Studies that reported adherence results were more likely to report negative trial outcome findings (i.e., findings that do not confirm the anticipated superiority or equivalence of one treatment compared with another) (49.02 vs. 30.00, p = 0.03) and be government funded (41.17 vs. 25.00, p = 0.07). No other differences between reporting and nonreporting trial reached or approached statistical significance.
Clinical Trials 2014; 11: 195–204 Downloaded from ctj.sagepub.com at TEXAS SOUTHERN UNIVERSITY on December 10, 2014
http://ctj.sagepub.com
Adherence in RCTs
199
Table 1. Characteristics of randomized clinical trials in the review according to adherence reporting status Characteristics
Group
Phase of trial I and II III and IV Type of trial Placebo controlled Active comparison Number of compared interventions 2 3 or more Flexible dosing allowed Yes No Not specified Run-in period Yes No Pre-specified duration of treatment \1 month 1 month to 1 year .1 year Not specified Funding for conduct of triala Government Non-government Not specified Type of analysis for primary outcomeb ITT** On treatment Not specified Type of outcome Cardiovascular Neurology (non-stroke)/psychiatry Oncology Musculoskeletal Infectious disease Obstetrics and gynecology Pediatrics Lifestyle modification Gastroenterology Respiratory Renal Number of centers Single center 2–10 centers 11–50 centers 51–300 centers .300 centers Number of participants randomized \100 100–999 .1000
p-value*
Trials not reporting adherence, N = 60 n, (%)
Trials reporting adherence, N = 51 n, (%)
12 (20.00) 48 (80.00)
8 (15.69) 43 (84.31)
0.56
53 (88.33) 7 (11.67)
41 (80.39) 10 (19.61)
0.25
44 (73.33) 16 (26.67)
39 (76.47) 12 (23.53)
0.70
17 (28.33) 42 (70.00) 1 (1.67)
11 (21.57) 40 (78.43) 0 (0.00)
0.45
12 (20.00) 48 (80.00)
11 (21.57) 40 (78.43)
0.84
8 (13.33) 29 (48.33) 16 (26.67) 7 (11.67)
7 (13.73) 18 (35.29) 13 (25.49) 13 (25.49)
0.78
15 (25.00) 41 (68.33) 4 (6.67)
21 (41.17) 29 (56.86) 1 (1.96)
0.07
53 (88.33) 6 (10.00) 1 (1.67)
45 (88.24) 3 (5.88) 3 (5.88)
0.47
24 (40.00) 10 (16.67) 8 (13.33) 4 (6.67) 4 (6.67) 2 (3.33) 2 (3.33) 3 (5.00) 1 (1.67) 1 (1.67) 1 (1.67)
20 (39.22) 10 (19.61) 1 (1.96) 4 (7.84) 3 (5.88) 4 (7.84) 4 (7.84) 1 (1.96) 2 (3.92) 2 (3.92) 0 (0.00)
0.51
5 (8.33) 20 (33.33) 12 (20.00) 17 (28.33) 6 (10.00)
6 (11.76) 15 (29.41) 13 (25.49) 16 (31.37) 1 (1.96)
0.44
9 (15.00) 35 (58.33) 16 (26.67)
8 (15.69) 25 (49.02) 18 (35.29)
0.57
(continued)
http://ctj.sagepub.com
Clinical Trials 2014; 11: 195–204 Downloaded from ctj.sagepub.com at TEXAS SOUTHERN UNIVERSITY on December 10, 2014
200
Z Zhang et al.
Table 1. (Continued) Characteristics
Group
Treatment was effective or equivalentc Yes No Not specified Previously published methods Yes No
p-value*
Trials not reporting adherence, N = 60 n, (%)
Trials reporting adherence, N = 51 n, (%)
42 (70.00) 18 (30.00) 0 (0.00)
25 (49.02) 25 (49.02) 1 (1.96)
0.03
10 (16.67) 50 (83.33)
12 (23.53) 39 (76.47)
0.37
ITT: intention to treat; OT: observed treatment. a Funding: Government, if had any government funding was used; non-government, if only industry or personal or non-profit funding was used; 5 trials did not specify. b Type of analysis, 4 did not specify; if used both ITT and OT, counted as ITT. c Treatment was effective: 1 did not specify. Treatment was classified as not effective in a superiority trial if the observed difference between treatment groups did not reach a pre-defined level of statistical significance. Treatment was classified as non-equivalent in an equivalence trail if the upper bound of the 95% confidence interval around the difference in treatment effect crossed the non-inferiority margin. * p-value based on chi-square test or Fisher’s exact test if hp 5. ** Trial primary outcome results are analyzed based on the assigned treatment instead of actual treatment received.
Table 2. Methods for monitoring adherence among publications that reported adherence result Method for adherence monitoring
n (%)
Pill count-back Self-report Electronic monitoring Serum concentration Direct observation
21 (56.8) 9 (24.3) 4 (10.8) 2 (5.4) 1 (2.7)
Adherence monitoring methods Among the 51 trials that reported drug adherence results, 37 (72.5%) described the method by which adherence was measured (Table 2). Of these 37 studies, 2 measured drug serum concentration and 35 measured pill counts by some means. The most common method for measuring adherence, used in 21 studies (56.8%), was the count-back method (e.g., subtracting pills remaining in returned bottles from the dispensed quantity to calculate the number consumed). Nine (24.3%) trials used patient self-report, four (10.8%) used electronic devices for detecting access to a pill container, and one (2.7%) used direct observation by trial staff or other medical personnel. Quality of adherence reporting We examined the quality of adherence reporting among the 37 studies that described methods for
monitoring. In all 37 studies, adherence was reported as a percentage. We looked closely at each study to determine how the percentage was calculated. In all, 31 of 37 (83.8%) studies defined both the numerator (e.g., number of participants, total number of pills consumed) and denominator (e.g., total number of participants randomized, number of participants who completed the study, total number of pills prescribed by protocol). Four (10.8%) studies reported the numerator but not the denominator. Two (5.4%) studies did not specify either number. In clinical trials, it is very common for participants to stop taking the study medication temporarily or permanently before completing the research protocol and being formally exited. For this reason, we were particularly interested to know how the adherence percentage would reflect intervals when participants were off drug or when they were taking a reduced dose. For simplicity, we confined this analysis to the 35 studies that used pill counts as a basis for adherence reporting. Only 6/35 studies (17.1%) specified how periods of permanent drug discontinuation affected the adherence calculation. Four of these indicated that participants were censored at the time of permanent drug discontinuation even if this occurred before the planned end of follow-up. Therefore, intervals off drug were not included in the calculation of adherence. Two studies indicated that adherence was recorded as zero for periods of follow-up off drug. Only one study specified how periods of
Clinical Trials 2014; 11: 195–204 Downloaded from ctj.sagepub.com at TEXAS SOUTHERN UNIVERSITY on December 10, 2014
http://ctj.sagepub.com
Adherence in RCTs temporary drug discontinuation affected the adherence calculation. This study did not include the period of discontinuation in the calculation of adherence. The inclusion of subjects lost to follow-up was addressed in 11/37 (29.7%) studies, of which 9 (81.8%) excluded these subjects from the adherence calculation even if adherence data were available prior to the last contact. The other two studies indicated adherence was recorded as zero if participants were lost to follow-up. Of the 37 studies reporting adherence monitoring and results, 21 (56.8%) also reported a criterion for good adherence. These 21 studies listed criteria ranging from 50% to 80%. Three (8.1%) manuscripts estimated the adherence level required to achieve a clinically significant effect, which included achieving the therapeutic target for serum lithium in one trial and taking 80% of prescribed pills in the other two trials. Adherence results were reported over multiple time periods (e.g., 1 year, 3 years, 5 years; or 1 month, 6 months, 12 months, etc.) in 9/37 (24.3%) manuscripts. Nine of the 37 (24.3%) manuscripts indicated that adherence data were missing for a subset of trial participants. Flexible dosing was allowed in 28/111 trials (Table 1). Of the 37 trials, 2 (5.4%) specified whether participants taking a lower dose of study medication were included in the adherence analysis. In both cases, these subjects were excluded from the adherence calculation. Adherence results In the 51 studies with reported adherence, mean adherence was 85.7% (range: 48%2100%) and the median adherence was 88.4%. Figure 2 shows the range of reported adherence results as a histogram. Adherence was negatively correlated with duration of trial participation (r = 20.45, p = 0.0015). There was a non-significant negative correlation between adherence and number of subjects randomized (r = 20.24, p = 0.097). There was no relationship between the magnitude of reported adherence and number of study centers. Table 3 shows a cross-section of the language used to describe adherence results in selected publications. It reflects the variability of adherence reporting with regard to method of monitoring, details of the adherence calculation, and interpretation of the adherence result.
Discussion We found that less than half of all RCTs involving drug therapy reported adherence data in their main http://ctj.sagepub.com
201
Figure 2. Adherence results as percentages in 48/51 trials that reported adherence. If adherence was not reported as a percentage, adherence results were converted to a percentage based on provided information. Median reported adherence was 88.4% (range: 48%2100%). Note: adherence results in three studies could not be converted to a percentage.
publication. Among those that did report adherence data, a description of the methods for adherence monitoring was lacking in 27% of studies. Reported methods, furthermore, were often inadequate to allow a meaningful interpretation of adherence results. Researchers commonly failed to define the numerator and denominator for calculation of adherence from pill counts, explain how periods off drug were considered in adherence calculation, and report missing adherence data. Overall, our results indicate a striking lack of transparency in defining, monitoring, and reporting adherence. As a result of deficiencies in the reporting of methods for adherence monitoring, it is difficult to interpret the finding that average adherence among studies reporting adherence exceeded 85%. This high reported adherence may be inflated by failure to include periods of drug cessation in the adherence calculation, variable practices for accounting for periods of dose reduction, and variable practices for censoring participants during follow-up. Our findings build upon prior work by Gossec et al.’s [12] 2007 study by expanding the inclusion criteria to include all disease conditions and by performing a more detailed analysis of the quality of adherence reporting. We reviewed not only the reporting of adherence results, but also how adherence results were calculated. We paid particular attention to the transparency of adherence calculations in the publications, an issue that has been previously ignored in RCT publication standards. Thus, we believe that our results are more reflective of the general state of adherence reporting across the highimpact literature. Clinical Trials 2014; 11: 195–204
Downloaded from ctj.sagepub.com at TEXAS SOUTHERN UNIVERSITY on December 10, 2014
202
Z Zhang et al.
Table 3. Language used to describe adherence result in selected publications Stated adherence monitoring method
Reported adherence result
Reference
Pill count
‘On the basis of counts of returned pills . from 618 women, 524 (85%) took at least 50% of their tablets, 434 (70%) took 80% or more, and 237 (38%) took all their tablets; 17 (3%) did not take any .’ ‘The rate of study-drug adherence was only 85%’ ‘25 of 52 patients in the lamotrigine group were serum compliant (mean serum concentration 141 mg/L)’ ‘Regular intake of study medication (70% of study period) occurred in 93.1% of the patients in the omega group and 93.2% of the control group (P-0.95)’. ‘Among the 9969 women . the median ratio of the number of study capsules taken to the number that should have been taken, between the time of randomization and delivery, was 88% in both study groups .’ ‘The controlling of the medication blister demonstrated full compliance with the study protocol in all patients.’ ‘At 28 weeks’ gestation, 35.6% of mothers reported that they had not missed taking any capsules and an additional 34.5% reported that they had missed 3 or less per week’ ‘2193 (98%) patients in the celecoxib group and 2179 (97%) in the diclofenac plus omeprazole group had drug adherence between 80% and 120%’.
McCance et al., Lancet [14]
Self-report Serum concentration Pill count
Pill count
Pill count Self-report
Not specified
Three findings in this study warrant special attention. First, we identified a tendency to report adherence in cases where the trial outcome is negative, and to omit a discussion of adherence when the outcome is positive. The inclination to attribute a negative outcome to poor adherence is understandable, and adherence data may provide valuable support for this argument and insight into why a particular intervention might have failed. However, the utility of adherence data is not limited to trials with negative results. Transparency in adherence reporting is crucial to understanding how the results of a positive trial should be interpreted and how they might apply to a new clinical population. Consider, for example, three trials of new pharmacologic interventions for the same disease. Each trial finds a similar, statistically significant treatment effect. In the first trial, the adherence rate was extremely high. This might imply that achieving the observed treatment benefit would be difficult in realworld populations with less than ideal adherence. In the second trial, the adherence rate is very low. While a positive finding in the setting of low adherence might imply that a treatment is very effective, it might also raise concern for bias or error that could distort the observed effect. In the third trial, adherence is average. A positive result with average adherence might suggest that the intervention could be of value in real-world practice. Knowledge of treatment adherence provides insight into the meaning, reliability, and applicability of trial results.
Kaandorp et al., NEJM [15] Kapoor et al., Lancet Neurology [16] Rauch et al., Circulation [17]
Roberts et al., NEJM [18]
Sibbing et al., JACC [19] Makrides et al., JAMA [20]
Chan et al., Lancet [21]
Second, only 57% of studies that reported adherence provided a criterion for ‘adequate adherence’. These studies cited criteria ranging from 50% to 80%, which compares with a commonly used criteria for good adherence of 80% of participants taking 80% of trial medication [7,22]. On one hand, this omission is understandable in that adherence may be difficult to predict and the level of adherence required for the hypothesized treatment effect may be even more difficult to predict. On the other hand, in designing a trial, investigators should have in mind some minimal level of adherence required for success. Third, we found a lack of transparency in the calculation of adherence. At the most basic level, the denominator for calculating adherence was often not reported, making the reported adherence result difficult to interpret. The method for accounting for periods temporarily off drug was described in only one of 37 studies. For the other 36 studies, readers are left wondering if full adherence might have been imputed for intervals of temporarly non-use. The result of failing to decrement adherence for intervals off drug is to erroneously inflate the reported adherence. This is a particular problem for studies using an intention-to-treat analysis. In such studies, which are the standard for RCTs, participants continue to be followed for outcomes regardless of treatment adherence. Failure to decrement adherence may create an artificial gap between the interpretation of trial outcomes and interpretation of adherence
Clinical Trials 2014; 11: 195–204 Downloaded from ctj.sagepub.com at TEXAS SOUTHERN UNIVERSITY on December 10, 2014
http://ctj.sagepub.com
Adherence in RCTs results, rendering it more difficult to apply trial results to clinical practice. Our study has several limitations. We included a small set of journals with the highest impact factor in specific fields of clinical medicine, including general medicine. It is possible that a larger journal sample size would yield different results. However, top journals typically require the greatest methodological rigor in research publications and tend to set the standard for other journals. We chose the subspecialty journals in cardiology, neurology, and oncology because cardiovascular disease, stroke, and cancer were the three leading causes of adult death in the United States. These three fields represented some of the most prolific clinical trial research categories due to their burden of disease and thus should yield a large number of RCTs for review. Having blinded reviewers would have decreased potential bias in this study. We attempted to minimize bias by having each reviewer review the publications independently and systematically. Finally, although our analysis pertains to studies published in 2010, no new guidelines or pronouncements in the field since the last CONSORT statement have been issued that would be expected to change practices for adherence monitoring, analysis, and reporting. In summary, our findings document that published reports of RCTs using oral drug therapy commonly do not provide adequate data on study-drug adherence. The development of clear and universal standards for gathering, analyzing, and reporting adherence data may be an important next step for quality improvement within the scientific literature. These standards would need to recognize that most adherence measurements are only estimates and that best measures for an individual trial would be difficult to prescribe at this time. While these standards are considered, we believe investigators should be encouraged to take the following steps: 1. Measure adherence among all participants. 2. Include in the research protocol and, in at least one publication, the precise methods for measuring, analyzing, and reporting adherence. Provide both the numerator and the denominator for the calculation, including units of measurement. Explain how intervals off drug are factored into the adherence calculation. 3. Report adherence data in the main publication for the trial.
Acknowledgements Z.Z. and M.J.P. contributed equally to this work. We would like to thank Mark Gentry at Yale School of Medicine Cushing Medical Library for helping us design and organize the MEDLINE search for our http://ctj.sagepub.com
203
review. No compensation was provided for this contribution.
Funding This work was supported in part by a stipend from the Office of Student Affairs and the Office of Student Research at the Yale University School of Medicine to support Z.Z., and by a grant from the Doris Duke Charitable Foundation to the Yale University School of Medicine to support Clinical Research Fellow M.J.P..
Conflict of interest None.
References 1. Byar DP, Simon RM, Friedewald WT, et al. Randomized clinical trials. Perspectives on some recent ideas. N Engl J Med 1976; 295(2): 74–80. 2. Feinstein AR. Current problems and future challenges in randomized clinical trials. Circulation 1984; 70(5): 767–74. 3. Abel U, Koch A. The role of randomization in clinical studies: Myths and beliefs. J Clin Epidemiol 1999; 52(6): 487–97. 4. Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med 2000; 342(25): 1887–92. 5. Begg C, Cho M, Eastwood S, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA 1996; 276(8): 637–39. 6. Schulz KF, Altman DG, Moher D; CONSORT Group. CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials. BMJ 2010; 340: c332. 7. Haynes RB, Taylor DW, Sacket DL. Compliance in Healthcare. The Johns Hopkins University Press, Baltimore, MD, 1979. 8. Cramer JA, Spilker B. Patient Compliance in Medical Practice and Clinical Trials. Raven Press, New York, 1991. 9. Osterberg L, Blaschke T. Adherence to medication. N Engl J Med 2005; 353(5): 487–97. 10. Haynes RB, Ackloo E, Sahota N, McDonald HP, Yao X. Interventions for enhancing medication adherence. Cochrane Database Syst Rev 2008; 2: CD000011. 11. Cramer J, Rosenheck R, Kirk G, Krol W, Krystal J; VA Naltrexone Study Group 425. Medication compliance feedback and monitoring in a clinical trial: Predictors and outcomes. Value Health 2003; 6(5): 566–73. 12. Gossec L, Tubach F, Dougados M, Ravaud P. Reporting of adherence to medication in recent randomized controlled trials of 6 chronic diseases: A systematic literature review. Am J Med Sci 2007; 334(4): 248–54. 13. Lee KP, Schotland M, Bacchetti P, Bero LA. Association of journal quality indicators with methodological quality of clinical research manuscripts. JAMA 2002; 287(21): 2805–08.
Clinical Trials 2014; 11: 195–204 Downloaded from ctj.sagepub.com at TEXAS SOUTHERN UNIVERSITY on December 10, 2014
204
Z Zhang et al.
14. McCance DR, Holmes VA, Maresh MJA, et al. Vitamins C and E for prevention of pre-eclampsia in women with type 1 diabetes (DAPIT): A randomised placebocontrolled trial. Lancet 2010; 376(9737): 259–66. 15. Kaandorp SP, Goddijn M, van der Post JAM, et al. Aspirin plus heparin or aspirin alone in women with recurrent miscarriage. N Engl J Med 2010; 362(17): 1586–96. 16. Kapoor R, Furby J, Hayton T, et al. Lamotrigine for neuroprotection in secondary progressive multiple sclerosis: A randomised, double-blind, placebo-controlled, parallelgroup trial. Lancet Neurol 2010; 9(7): 681–88. 17. Rauch B, Schiele R, Schneider S, et al. OMEGA, a randomized, placebo-controlled trial to test the effect of highly purified omega-3 fatty acids on top of modern guidelineadjusted therapy after myocardial infarction. Circulation 2010; 122(21): 2152–59.
18. Roberts JM, Myatt L, Spong CY, et al. Vitamins C and E to prevent complications of pregnancy-associated hypertension. N Engl J Med 2010; 362(14): 1282–91. 19. Sibbing D, Stegherr J, Braun S, et al. A double-blind, randomized study on prevention and existence of a rebound phenomenon of platelets after cessation of clopidogrel treatment. J Am Coll Cardiol 2010; 55(6): 558–65. 20. Makrides M, Gibson RA, McPhee AJ, et al. Effect of DHA supplementation during pregnancy on maternal depression and neurodevelopment of young children: A randomized controlled trial. JAMA 2010; 304(15): 1675–83. 21. Chan FKL, Lanas A, Scheiman J, et al. Celecoxib versus omeprazole and diclofenac in patients with osteoarthritis and rheumatoid arthritis (CONDOR): A randomised trial. Lancet 2010; 376(9736): 173–79. 22. Robiner WN. Enhancing adherence in clinical research. Contemp Clin Trials 2005; 26(1): 59–77.
Clinical Trials 2014; 11: 195–204 Downloaded from ctj.sagepub.com at TEXAS SOUTHERN UNIVERSITY on December 10, 2014
http://ctj.sagepub.com
