DOI: 10.1111/hiv.12139 HIV Medicine (2014), 15, 381–382
© 2014 British HIV Association
LETTER TO THE EDITOR
Poor citation, coding and reporting: a review of adherence-enhancing interventions for highly active antiretroviral therapy creates an inaccurate picture of the state of the field M de Bruin,1,2 J Simoni,3 KR Amico,4 JT Parsons,5 J Fisher4 and SA Safren6 Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK, 2Amsterdam School of Communication Research, University of Amsterdam, Amsterdam, the Netherlands, 3Department of Psychology, University of Washington, Seatle, WA, USA, 4Center for Health, Intervention and Prevention, University of Connecticut, Storrs, CT, USA, 5 Psychology Department, Hunter College, New York, NY, USA and 6Department of Psychiatry, Massachusetts General Hospital, Boston, MA, USA
1
scores were given because data were missing. Hence, the quality of study design and the quality of reporting have become indistinguishable, and the collapsed score is likely to offer an overly negative picture of the reviewed trials. A second example is that Mathes and colleagues rated a study’s analyses as intent-to-treat (ITT) only if this was reported for both adherence and VL/biological outcomes (and presumably at all the time-points), regardless of whether the reviewed study, a priori, identified only one outcome (or time-point) as primary (or regardless of whether ITT was applied to adherence only, which Mathes and colleagues themselves define as the primary outcome). It is inaccurate and misleading to deem a study at high risk of bias when intent-to-treat analyses have been applied to the primary (if not secondary) outcomes/time-points. Note that the specifics of this ITT coding procedure were unclear based on reading the article only; instead, the authors were contacted for clarification. The scoring procedures for the other trial quality criteria remain largely unclear, so that it is hard to further discuss their appropriateness. Finally, there appear to be errors in the coding. For example, the study by de Bruin and colleagues [7] is reported as producing significant change only in VL, even though the original report also described significant effects in ITT analyses for adherence (mean difference of 7.40%; 95% confidence interval 3.50−11.30%; P < 0.001) [7]. Unusually, the coders did not contact de Bruin or colleagues to resolve the discrepancy between their interpretation of the analysis and that of the authors, and just reported that the findings of this trial were not significant. Another example is that Parsons et al. [8] scored ‘high risk’ on generation of allocation sequence despite their use of urn randomization; an acceptable sequencing procedure [6].
The recent systematic review of interventions to promote antiretroviral therapy (ART) adherence by Mathes and colleagues [1] published in HIV Medicine contains several methodological flaws that lead to an overly pessimistic depiction of the state of the literature; indeed, it contradicts other widely cited reviews on the topic that demonstrate the efficacy of such interventions to improve both adherence and viral load (VL) [2−5]. Curiously, the only prior review cited was Simoni [5], and only then to support a lack of efficacy for adherence interventions, which is contrary to the main findings of that review. This bias in citation creates unfounded authority for the authors’ dire conclusions that ‘adherence enhancing interventions are not promising’. Mathes and colleagues report that only one of the 21 trials fulfilling their inclusion criteria was coded as indicating statistically significant effects for adherence and VL, and that the intervention trials were of poor quality (i.e., score high on risk of bias criteria). These conclusions, however, seem to at least partially arise from flawed, overly stringent or unclear quality coding of the interventions, as well as coding errors. Below are several examples of the many instances that make us question the quality of this work. First, the authors scored trials as being at high risk of bias when they could not find the relevant information in the article, failing to even attempt to contact the primary authors for clarification (using open-ended questions, as recommended by the Cochrane Bias Methods Group [6]), or to indicate in the review how many of the high risk of bias Correspondence: Dr Marijn de Bruin, Health Sciences Building, Institute of Applied Health Sciences, University of Aberdeen, Foresterhill, Aberdeen AB25 2ZD, UK. Tel: +44 (0)1224 438076 (office); +44 (0)79 1707 9584 (mobile); fax: +44 (0)1224 437971; e-mail: [email protected]
381
382 Letter to the Editor
In sum, whereas we definitely think a critical analysis of the quality of adherence trials has merit, this review ignores key literature in this domain; assumes that if trial procedures were not clear to the coders that the study had been poorly conducted; does not distinguish between primary and secondary outcomes; did not seek additional information if matters were unclear; and makes glaring errors in interpreting studies up to the level where a significant trial is reported as nonsignificant. A review of such dubious quality derails the programme of research in the area and, more ominously, threatens to misinform public health officials and providers in ways that may limit the access of patients to potentially health-promoting interventions.
References 1 Mathes T, Pieper D, Antoine SL, Eikermann M. Adherenceenhancing interventions for highly active antiretroviral therapy in HIV-infected patients–a systematic review. HIV Med 2013; 14: 583–595. 2 Amico KR, Harman JJ, Johnson BT. Efficacy of antiretroviral therapy adherence interventions; a research synthesis of trials, 1996 to 2004. J Acquir Immune Defic Syndr 2006; 41: 285–297. 3 de Bruin M, Viechtbauer W, Schaalma HP, Kok G, Abraham C, Hospers HJ. Standard care impact on effects of highly
© 2014 British HIV Association
active antiretroviral therapy adherence interventions: a meta-analysis of randomized controlled trials. Arch Intern Med 2010; 170: 240–250. 4 Rueda S, Park-Wyllie LY, Bayoumi AM et al. Patient support and education for promoting adherence to highly active antiretroviral therapy for HIV/AIDS. Cochrane Database Syst Rev 2006; (3)CD001442. 5 Simoni JM, Pearson CR, Pantalone DW, Marks G, Crepaz N. Efficacy of interventions in improving highly active antiretroviral therapy adherence and HIV-1 RNA viral load: a meta-analytic review of randomized controlled trials. J Acquir Immune Defic Syndr 2006; 43: S23–S35. 6 Higgins JPT, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available at http://handbook.cochrane.org/ (accessed 1 October 2013). 7 de Bruin M, Hospers HJ, van Breukelen GJ, Kok G, Koevoets WM, Prins JM. Electronic monitoring-based counseling to enhance adherence among HIV-infected patients: a randomized controlled trial. Health Psychol 2010; 29: 421–428. 8 Parsons JT, Golub SA, Rosof E, Holder C. Motivational interviewing and cognitive-behavioral intervention to improve HIV medication adherence among hazardous drinkers: a randomized controlled trial. J Acquir Immune Defic Syndr 2007; 46: 443–450.
HIV Medicine (2014), 15, 381–382
© 2014 British HIV Association
LETTER TO THE EDITOR
Poor citation, coding and reporting: a review of adherence-enhancing interventions for highly active antiretroviral therapy creates an inaccurate picture of the state of the field M de Bruin,1,2 J Simoni,3 KR Amico,4 JT Parsons,5 J Fisher4 and SA Safren6 Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK, 2Amsterdam School of Communication Research, University of Amsterdam, Amsterdam, the Netherlands, 3Department of Psychology, University of Washington, Seatle, WA, USA, 4Center for Health, Intervention and Prevention, University of Connecticut, Storrs, CT, USA, 5 Psychology Department, Hunter College, New York, NY, USA and 6Department of Psychiatry, Massachusetts General Hospital, Boston, MA, USA
1
scores were given because data were missing. Hence, the quality of study design and the quality of reporting have become indistinguishable, and the collapsed score is likely to offer an overly negative picture of the reviewed trials. A second example is that Mathes and colleagues rated a study’s analyses as intent-to-treat (ITT) only if this was reported for both adherence and VL/biological outcomes (and presumably at all the time-points), regardless of whether the reviewed study, a priori, identified only one outcome (or time-point) as primary (or regardless of whether ITT was applied to adherence only, which Mathes and colleagues themselves define as the primary outcome). It is inaccurate and misleading to deem a study at high risk of bias when intent-to-treat analyses have been applied to the primary (if not secondary) outcomes/time-points. Note that the specifics of this ITT coding procedure were unclear based on reading the article only; instead, the authors were contacted for clarification. The scoring procedures for the other trial quality criteria remain largely unclear, so that it is hard to further discuss their appropriateness. Finally, there appear to be errors in the coding. For example, the study by de Bruin and colleagues [7] is reported as producing significant change only in VL, even though the original report also described significant effects in ITT analyses for adherence (mean difference of 7.40%; 95% confidence interval 3.50−11.30%; P < 0.001) [7]. Unusually, the coders did not contact de Bruin or colleagues to resolve the discrepancy between their interpretation of the analysis and that of the authors, and just reported that the findings of this trial were not significant. Another example is that Parsons et al. [8] scored ‘high risk’ on generation of allocation sequence despite their use of urn randomization; an acceptable sequencing procedure [6].
The recent systematic review of interventions to promote antiretroviral therapy (ART) adherence by Mathes and colleagues [1] published in HIV Medicine contains several methodological flaws that lead to an overly pessimistic depiction of the state of the literature; indeed, it contradicts other widely cited reviews on the topic that demonstrate the efficacy of such interventions to improve both adherence and viral load (VL) [2−5]. Curiously, the only prior review cited was Simoni [5], and only then to support a lack of efficacy for adherence interventions, which is contrary to the main findings of that review. This bias in citation creates unfounded authority for the authors’ dire conclusions that ‘adherence enhancing interventions are not promising’. Mathes and colleagues report that only one of the 21 trials fulfilling their inclusion criteria was coded as indicating statistically significant effects for adherence and VL, and that the intervention trials were of poor quality (i.e., score high on risk of bias criteria). These conclusions, however, seem to at least partially arise from flawed, overly stringent or unclear quality coding of the interventions, as well as coding errors. Below are several examples of the many instances that make us question the quality of this work. First, the authors scored trials as being at high risk of bias when they could not find the relevant information in the article, failing to even attempt to contact the primary authors for clarification (using open-ended questions, as recommended by the Cochrane Bias Methods Group [6]), or to indicate in the review how many of the high risk of bias Correspondence: Dr Marijn de Bruin, Health Sciences Building, Institute of Applied Health Sciences, University of Aberdeen, Foresterhill, Aberdeen AB25 2ZD, UK. Tel: +44 (0)1224 438076 (office); +44 (0)79 1707 9584 (mobile); fax: +44 (0)1224 437971; e-mail: [email protected]
381
382 Letter to the Editor
In sum, whereas we definitely think a critical analysis of the quality of adherence trials has merit, this review ignores key literature in this domain; assumes that if trial procedures were not clear to the coders that the study had been poorly conducted; does not distinguish between primary and secondary outcomes; did not seek additional information if matters were unclear; and makes glaring errors in interpreting studies up to the level where a significant trial is reported as nonsignificant. A review of such dubious quality derails the programme of research in the area and, more ominously, threatens to misinform public health officials and providers in ways that may limit the access of patients to potentially health-promoting interventions.
References 1 Mathes T, Pieper D, Antoine SL, Eikermann M. Adherenceenhancing interventions for highly active antiretroviral therapy in HIV-infected patients–a systematic review. HIV Med 2013; 14: 583–595. 2 Amico KR, Harman JJ, Johnson BT. Efficacy of antiretroviral therapy adherence interventions; a research synthesis of trials, 1996 to 2004. J Acquir Immune Defic Syndr 2006; 41: 285–297. 3 de Bruin M, Viechtbauer W, Schaalma HP, Kok G, Abraham C, Hospers HJ. Standard care impact on effects of highly
© 2014 British HIV Association
active antiretroviral therapy adherence interventions: a meta-analysis of randomized controlled trials. Arch Intern Med 2010; 170: 240–250. 4 Rueda S, Park-Wyllie LY, Bayoumi AM et al. Patient support and education for promoting adherence to highly active antiretroviral therapy for HIV/AIDS. Cochrane Database Syst Rev 2006; (3)CD001442. 5 Simoni JM, Pearson CR, Pantalone DW, Marks G, Crepaz N. Efficacy of interventions in improving highly active antiretroviral therapy adherence and HIV-1 RNA viral load: a meta-analytic review of randomized controlled trials. J Acquir Immune Defic Syndr 2006; 43: S23–S35. 6 Higgins JPT, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available at http://handbook.cochrane.org/ (accessed 1 October 2013). 7 de Bruin M, Hospers HJ, van Breukelen GJ, Kok G, Koevoets WM, Prins JM. Electronic monitoring-based counseling to enhance adherence among HIV-infected patients: a randomized controlled trial. Health Psychol 2010; 29: 421–428. 8 Parsons JT, Golub SA, Rosof E, Holder C. Motivational interviewing and cognitive-behavioral intervention to improve HIV medication adherence among hazardous drinkers: a randomized controlled trial. J Acquir Immune Defic Syndr 2007; 46: 443–450.
HIV Medicine (2014), 15, 381–382
