pharmacoepidemiology and drug safety 2014; 23: 572–579 Published online 24 February 2014 in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/pds.3596
REVIEW
Methodological gaps in the assessment of risk minimization interventions: a systematic review Inna Gridchyna1, Anne-Marie Cloutier1,2, Lenhangmbong Nkeng1,2, Camille Craig1,2, Sarah Frise3,4 and Yola Moride1,2* 1
Faculty of Pharmacy, Université de Montreal, Montreal, Quebec, Canada Pharmacoepidemiology Unit, Research Center, University of Montreal Hospital Center (CRCHUM), Montreal, Quebec, Canada 3 Department of Patient Safety and Medical Information, AstraZeneca Canada, Mississauga, Ontario, Canada 4 Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada 2
ABSTRACT Introduction Since the introduction of therapeutic risk management regulatory guidance, an increase in the number of risk minimization interventions (RMIs) published in the literature has been observed. Methods used to evaluate their effectiveness remain, however, poorly examined. Objective This paper aimed to conduct a literature review on the methods of evaluation of effectiveness of RMIs and to identify methodological gaps. Methods The search was conducted using MEDLINE and Embase between 1 January 2000 and 31 December 2010, and updated on 1 April 2013. The following characteristics were extracted from each study: target population for the RMI, target population for the assessment of effectiveness, study design, data sources, and effectiveness outcome(s). Results A total of 188 unique RMIs were identified in the literature, of which effectiveness was evaluated in only 65 (34.6%) at the time of publication. The largest proportion of studies reviewed (n = 49, 75.4%) attempted to evaluate changes in behavior through prescribing or laboratory test practices. One quarter of studies evaluated the effect of RMIs on the occurrence of adverse events. Only a minority of studies used robust designs, such as randomized controlled trials (n = 6, 9.2%) or a quasi-experimental design with a parallel comparison group (n = 8, 12.3%). Conclusion Lack of robust methodological design used in published studies on RMI effectiveness evaluation is an important methodological gap in the evaluation of RMI effectiveness. © 2014 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons, Ltd. key words—risk minimization intervention; therapeutic risk management; REMS; effectiveness; pharmacoepidemiology Received 20 June 2013; Revised 17 January 2014; Accepted 20 January 2014
BACKGROUND In 2005 and 2006, respectively, the US Food and Drugs Administration (FDA)1 and the European Medicines Agency2 issued guidance documents on therapeutic risk management planning. The European Union recently implemented a new pharmacovigilance legislation.3,4 Risk minimization interventions (RMIs) are part of risk management plans and include all actions beyond product labeling, which aim at enhancing drug benefit–risk.1 We conducted a systematic review of the literature and of agency websites5 in order to describe the characteristics of the various RMIs that have been implemented.
*Correspondence to: Y. Moride, Université de Montréal, C.P.6128, succursale Centre-ville, Montreal, QC H3C 3J7, Canada. E-mail: [email protected]
The most frequent RMIs consisted of education material targeted to healthcare professionals and/or patients. More restrictive RMIs, such as controlled distribution programs, accounted for only 6.6% of publications and 1.2% RMIs posted on agency websites. Regulatory guidance states that the effectiveness of RMIs should be evaluated.1 Assessment of effectiveness aims at determining the performance of the RMI itself and at identifying potential deficiencies that may warrant further interventions. Despite the available guidance documents, there are no specific recommendations on the methods of assessment of effectiveness of RMIs.1,6 The objectives of our review were as follows: (1) to characterize the study designs, target populations, data sources, and measures of success (effectiveness outcomes) that have been used in RMI evaluation studies published in the literature
© 2014 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons, Ltd. This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made.
assessment of risk minimization interventions
and (2) to identify methodological gaps in methods of effectiveness evaluation.
573
METHODS
of adverse events (AEs). Natural healthcare products, devices, diagnostic chemicals, pregnancy registries without follow-up, medication errors, and products not used as therapy for illness were not retained.
Literature search strategy
Data extraction
A literature search was conducted using MEDLINE and Embase and followed the PRISMA Statement for Reporting Systematic Reviews and Meta-analyses of Studies.7 Medical Subject Heading (MeSH) terms were used where possible. The term “drug toxicity” was searched along with “patient education,” “health care professional education,” “prescriber education,” “patient alert card,” “patient registry,” “medication guide,” “drug legislation,” “informed consent,” “restricted distribution,” “physician authorisation,” “drug monitoring,” “Dear Health Care Professional Letter,” “Dear Doctor Letter,” and “black box warning.” The references of all retained articles, including review articles, were scanned for potential relevant papers (“snowballing”). The initial search was completed in March 2011 and was updated on 1 April 2013, without changing the search strategy. All abstracts identified through the electronic search were reviewed by three independent reviewers, and any disagreements were resolved with a majority (two out of three assessments) decision.
Among those studies with an evaluation component, we systematically extracted the following: type of RMI, target population, AE(s) of special interest, region, and year of publication. Methodological components of assessment of effectiveness consisted of the following: target population for the evaluation, study design, data source, effectiveness outcome(s), and main findings.
Inclusion and exclusion criteria To be included in the review, articles must have been published between 1 January 2000 and 31 March 2013 inclusively and involve drug products, use in humans, RMIs, or tools used to increase the reporting
Figure 1.
Analysis Components of evaluation were assessed using the theoretical framework adapted from Cabana et al.8 and Hudon et al.9 and depicted in Figure 1. On the basis of this framework, adherence to RMI recommendations and integration of such recommendations into medical practice are influenced by physician and patient characteristics, as well as by the healthcare setting. Factors influencing adherence to RMIs are divided into three temporal phases: (i) knowledge and awareness; (ii) attitude; and (iii) behavior. As shown in Figure 1, a large amount of information and disagreement with recommendations are both important deterrents of effectiveness. Integration of the RMI into the pathway of patient care may explain regional differences in RMI effectiveness. Finally, medical practice is also influenced by lack of healthcare resources and/or reimbursement constraints. The strength of the evidence of RMI
Theoretical framework to assess the effectiveness of risk minimization interventions. Adapted from Cabana et al. (8) and Hudon et al. (9)
© 2014 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons, Ltd.
Pharmacoepidemiology and Drug Safety, 2014; 23: 572–579 DOI: 10.1002/pds
574
i. gridchyna et al.
effectiveness was assessed using the Shadish et al.10 ranking of designs for causal inference, which ranges from robust experimental designs such as randomized controlled trials to weaker designs such as observational studies without a comparison group. RESULTS Literature search A total of 4935 sources were identified from the MEDLINE and Embase searches. Of these, 154 met the inclusion and exclusion criteria, and another 34 sources were obtained from snowballing, yielding a total of 188 sources retained for the review. Of these, 65 (34.6%) reported an effectiveness evaluation (Figure 2). The distribution of types of RMIs with an evaluation of effectiveness is shown in Figure 3, and the characteristics of the evaluation studies are described in the Supporting Information. Education material, training, black-box warning, and medication management system appeared to be most frequently evaluated in the literature.
Figure 3. Number of studies with an evaluation component, according to type of risk minimization intervention. DHCP = Dear Health Care Professional letter; SW = safety warnings; EM = education materials; CM = communication materials; MMS = medication management system; IC = informed consent; RD = restricted distribution; R = registry; General = overall knowledge on risk minimization intervention tools
Components of risk minimization The vast majority of studies with an evaluation component focused on the behavioral construct of risk minimization (n = 49, 75.4%), measured through prescribing and laboratory testing practices. Other studies evaluated knowledge or attitudes (n = 9, 13.5%). Fewer studies addressed more than one component: knowledge and attitudes,11,12 knowledge and behavior,13–15 and attitudes and behavior.16–18 Only one study, consisting of a survey, addressed all three components of risk minimization (knowledge, attitudes, and behavior).19 One quarter of studies (n = 16, 24.6%) evaluated the impact of the RMI on the occurrence of AEs. Study designs used to assess the effectiveness of risk minimization interventions
Figure 2. Results of the literature search on risk minimization interventions
© 2014 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons, Ltd.
Study designs used to assess each component of risk minimization (knowledge, attitudes, and behavior) are shown in Table 1. It should be noted that the effectiveness of a given RMI could be evaluated through more than one study. For example, the effectiveness of safety warnings on antidepressants issued by the FDA was evaluated in six studies published in the literature.17,20–24 Also, six studies evaluating label changes and Dear Health Care Professional (DHCP) letters on cisapride were found.25–30 The great majority of studies that evaluated behavior used interrupted time series analyses or pre–post intervention designs without a comparison group, often using data from administrative claims databases. Given the availability of historical data (i.e., pre-intervention), most of the RMIs involving safety warnings or DHCP letters were evaluated using time trends in prescribing or laboratory test monitoring. We have identified only six studies that used a parallel comparison group. Such Pharmacoepidemiology and Drug Safety, 2014; 23: 572–579 DOI: 10.1002/pds
575
assessment of risk minimization interventions Table 1. Distribution of study designs used to assess the various components of risk minimization interventions
Randomized controlled trials Interrupted time series analysis Pre–post intervention Cohort studies/registry
Knowledge
Attitudes
Behavior
Clinical outcome (adverse event)
0 0 0 0 1 0 0 1 4 3 9
1 0 0 0 2 0 0 0 5 1 9
1 2 17 3 10 1 5 4 5 1 49
4 1 0 4 3 1 2 1 0 0 16
With comparison Without comparison With comparison Without comparison With comparison Without comparison
Cross-sectional studies Surveys Qualitative assessment Total
comparison groups were either randomized31 or consisted of other geographical locations (states)32 or medical centers where the RMI was not implemented,33,34 another drug not targeted by the intervention,35 or young adults when children or adolescents were the target of the intervention.36 In another study27 in the absence of a reference population, all external factors, such as publicity and media coverage, which could also influence prescribing trends in addition to the DHCP letter, were qualitatively assessed. Restricted distribution programs, generally implemented at the time of product approval, tended to be evaluated through cohort studies.35,37,38 Knowledge and attitudes were most often evaluated using surveys or qualitative assessments,17,42,75,76 whereas a randomized trial39 or pre–post intervention without a comparison group12 was rarely used. As shown in Table 1, among the 16 studies that evaluated the impact of the RMI on the occurrence of AEs, robust designs such as RCTs33,34,40,41 or comparative designs (n = 6) were often used. Effectiveness outcomes Measures of effectiveness, or success, of RMIs included prescription or dispensing rate (n = 18, 27.6%), frequency of AEs (n = 16, 24.6%), including pregnancy rate during isotretinoin or thalidomide treatment, contraindicated co-prescription rate (n = 10, 15.4%), laboratory testing rate (n = 7, 10.8%), knowledge retention and attitude toward the RMI (n = 7, 10.8%), changes in medical practice (n = 6, 9.2%), rate of treatment initiation (n = 4, 6.2%), awareness of RMI (n = 4, 6.2%), rate of use of treatment alternatives (n = 3, 4.6%). Several studies evaluated the effectiveness of the RMI with more than one outcome.14,15,17,18,36,42–44 Of the 41 RMIs consisting of education, 31 used prescription rate or contraindicated co-prescription rate as a measure of effectiveness. Using the information matrix included in the Supporting Information, we © 2014 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons, Ltd.
have ascertained the aim of the intervention and the measure of effectiveness, from which we have assessed concordance. In nearly one third (n = 18) of the studies, the effectiveness measure did not correspond to the aim of the intervention. Inconsistencies originated from the following: prescription rates were used as measures of effectiveness while the RMI aimed at preventing the occurrence of AEs (n = 10),20–24,45–49 or the RMI sought to enhance laboratory testing (n = 2).50,51 In other instances, occurrence of AEs was used as the effectiveness measure while the intervention aimed at avoiding concomitant use of contraindicated medications (n = 2),52,53 at increasing laboratory testing (n = 2),54,55 or at enhancing information disclosure by healthcare professionals (n = 1).56 For studies that involved surveys, the definition of RMI effectiveness was not explicit, consisting mainly of a change in practice (change in medication dosage or drug, referral of patients to a specialist, performance of baseline assessment before initiation of a medical treatment, etc.) (n = 5),14,15,17,18,42 knowledge retention (n = 5),11,19,42,43,52 awareness of RMIs (n = 5),14,15,17,18,57 and increase of disclosure of information (n = 1).58 In cases where effectiveness was defined as knowledge retention, there was no specification on what represented adequate knowledge retention or what information should the target population have retained.19,43,52 Main data sources used to ascertain effectiveness outcomes included claims databases (n = 34, 52.3%), questionnaires (n = 19, 29.2%), medical charts or electronic health records (n = 9, 13.8%), and clinical data or laboratory findings (n = 4, 6.2%). Some studies have evaluated the effect of an RMI on the initiation of new treatments (i.e., incident drug users),20,23,46,48,59,60 whereas others focused on the prescribing rate for all users (incident and prevalent).13,21,24,36,41,45,49,53,61–65,77 Some studies defined new use as the absence of a prescription record during the preceding 4 months23,48 or 12 months.20,46 Among studies that examined the effectiveness of an Pharmacoepidemiology and Drug Safety, 2014; 23: 572–579 DOI: 10.1002/pds
576
i. gridchyna et al.
RMI on the rate of concomitant use of contraindicated medications, concomitant use was most frequently defined as two contraindicated products with active prescriptions that overlap on at least one day.15,26,27,29,30,58,66 Three surveys cited in the literature targeted different healthcare professionals, namely physicians, nurses, pharmacist, and social workers.11,12,67 Two surveys targeted only pharmacists,18,19 and two surveys only specialists, pediatricians,14 and neurologists.15 The completion rate for the surveys ranged from 50%19 to 100%67. All surveys were comprised of multiple-choice questions. One survey utilized a scale to rate the healthcare professionals’ attitude toward RMI19, and one collected doctors’ comments.14 Results of assessment of risk minimization interventions Among the studies reviewed, RMIs appeared to be successful or partly successful (n = 40, 61.5%) or 15 (23.1%) were not, and for nine studies (13.8%), it was not possible to draw a conclusion regarding their effectiveness because either the study was descriptive, had no comparison group, or had no threshold for effectiveness defined. After the implementation of the RMI, consisting mainly of warnings or DHCP letters, studies found a decrease in prescribing or co-prescribing rate of the study drug (n = 24),16,20,21,23–30,40,41,44– 46,49,51,54,62–64,66,68 a decrease in frequency of AEs (n = 10),33,34,54,55,60,61,69–71,78 a decrease in pregnancy rate while on thalidomide (n = 1),38 an increase in laboratory testing rate (n = 6),31,35,44,50,65,72 or a change in medication dosage (n = 1).65 When successive communication interventions were implemented over time,24,26–31,49,51,62,63,72 the first ones were often found unsuccessful (e.g., not reaching the expected decrease in prescription rate), whereas the latter interventions seemed more effective.26–28,30,50 Wilkinson et al. showed that multiple safety alerts are required to yield a significant decrease in the co-prescription rate of cisapride and contraindicated drugs.30 DISCUSSION Interrupted time series analyses or pre–post intervention analyses without comparison groups were the most frequent designs used to assess the effectiveness of RMIs. These are associated with well-known limitations. First, in the absence of a parallel comparison group (i.e., a group of patients unexposed to the intervention), it is difficult to ascertain if the observed outcome is attributable to the intervention or to the © 2014 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons, Ltd.
presence of external factors. Second, they are not suitable for drugs that have newly been introduced on the market because of the absence of pre-intervention data. For these reasons, they have mainly been used to assess the effect of regulatory interventions on drugs that have been marketed for a while. Only a minority of studies used a reference group, and it has been suggested that, whenever it is not possible, reference data (e.g., literature review or historical data) could be used.73 Experimental or quasi-experimental designs, although more robust, are rarely feasible because most RMIs, such as DHCP letters or safety warnings, are implemented in the entire population. Designs used to evaluate the effect of RMIs on the occurrence of AEs tended to be more robust than those that addressed the other components of risk minimization (i.e., knowledge, attitudes, and behavior). However, such studies were a minority and involved RMIs implemented at the level of the region or institution rather than in entire populations. In order to circumvent this limitation, some authors have attempted to use drugs or subpopulations not targeted by the RMI as a comparison population. Although appealing, the comparability of such reference should be assessed. For other types of RMIs, the selection of observational or descriptive studies without comparison groups could be explained by the lack of clear guidance on the design. For example, FDA guidance anticipates that RMI evaluations “would involve the analysis of observational or descriptive data.” At the same time, it is recommended, whenever feasible, to include at least two evaluation methods for each critical RMI goal.1 Most of the evaluation studies assessed behavioral changes using prescription or dispensing data. Assessing behavioral changes in a population without previously having assessed the knowledge and the attitude toward the RMI can introduce unknown confounders. Consequently, it is not possible to perform a root cause analysis of suboptimal behaviors using solely prescription data. Only one of the published studies evaluated all three components of the effectiveness framework (i.e., knowledge, attitude, and behavior). Neither European nor US guidance clearly emphasizes the assessment of all three components. Within the European dual approach, that is, implementation assessment and level of risk control assessment, the knowledge and attitudes could be a part of implementation assessment.6 The FDA guidance highlights the assessment of the effectiveness of the RMI through health outcomes, for example, complication rate, and recommends “assessments of comprehension, knowledge, attitudes, and/or desired safety behaviors Pharmacoepidemiology and Drug Safety, 2014; 23: 572–579 DOI: 10.1002/pds
assessment of risk minimization interventions
about drug safety risks” if “a health outcome cannot be particularly accurately measured”. Recently, the Office of the Inspector General determined that most RMIs, implemented in the USA, were not meeting their goals because of deficiencies in patient and prescriber awareness of drug risks.74 Among the RMIs consisting of education on key safety messages, the most utilized effectiveness outcome consisted of prescription rate or contraindicated co-prescription rate. By measuring the contraindicated co-prescribing rate, one may estimate the number of patients who are potentially at risk of AE. If the aim of the intervention is to prevent at-risk patients from receiving the drug, then such an outcome is an appropriate measure of success. However, if the intervention aims at preventing the occurrence of AEs in the target population, then occurrence of AEs would be preferable over prescribing rates. Main data sources used to ascertain effectiveness outcomes included claims databases, questionnaires, medical charts or electronic health records, and clinical data or laboratory findings. The use of claims databases to assess prescription rate is associated with information bias given that such databases record patterns of dispensing, dispensing frequency being influenced by prescribing practices, and patient acquisition behavior. Acquisition behavior is in turn a function of access and treatment compliance. In some studies, effectiveness has also been determined using the occurrence of AEs ascertained in claims databases. Although these databases are routinely used in observational studies, they are associated with well-known limitations that have been insufficiently addressed in the published studies on RMIs. Inaccuracy of diagnostic codes inherent to physician billing claims is likely to be associated with an overestimation or underestimation of the effectiveness of a given intervention. In many studies, study outcome consisted of a continuous variable, for example, prevalence of drug use, and any change over time that reached statistical significance was an indicator of success. This is a limitation, especially for studies using administrative claims databases, where statistical significance is often reached owing to large sample sizes. More guidance on a threshold of behavior change that would indicate success of the intervention would be required.
577
RMIs posted on drug agency websites or manufacturer websites, was considered. However, the amount of information disclosed on these latter resources is generally not sufficient to extract all the methodological elements required for the review. This literature review highlights some of the methodological gaps in the assessment of the effectiveness of RMIs. Only 34.6% of the RMI publications found in the literature included an evaluation component. The effectiveness evaluation of recent RMIs may be in progress or planned, and for this reason, the results are not yet available in the published sources. In studies that included an evaluation component, the measure of effectiveness did not appear consistent with the aim or the target population of the RMI for one third of the studies. These observations reveal that some evaluation studies are based on an inappropriate theoretical framework. Because of the heterogeneity of the target populations, drugs, and outcomes under study, it is difficult to rank the types of RMIs according to their effectiveness. Nevertheless, lack of robust designs is a major methodological gap and could be partly due to difficulties in finding appropriate reference population and/or absence of a clear guidance in regulatory documents. Especially for newly marketed drugs, the absence of an appropriate comparison group is a major challenge that should be addressed in future studies. CONFLICT OF INTEREST There are no conflicts of interest to declare for A. M. C., I. G., L. N., C. C. , and Y. M. S. F. is an employee of AstraZeneca Canada. KEY POINTS The study presents a literature review on the methods of evaluation of effectiveness of risk minimization interventions (RMI). • About one third of the RMI publications found in the literature included an evaluation component. • The majority of studies have evaluated prescription behavior and considerably less assessed the impact of the intervention on the occurrence of adverse events. • A major methodological gap is a lack of robust designs, owing to absence of comparison groups.
•
CONCLUSION To our knowledge, this review is the first of its kind to have been conducted. The main limitation is that only sources identified in the literature were included. A more exhaustive search of other resources, such as © 2014 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons, Ltd.
ETHICS STATEMENT The authors state that no ethical approval was needed. Pharmacoepidemiology and Drug Safety, 2014; 23: 572–579 DOI: 10.1002/pds
578
i. gridchyna et al.
ACKNOWLEDGEMENTS This review was conducted through a research grant from AstraZeneca Canada. Anne-Marie Cloutier is a recipient of a scholarship from the Canadian Institutes for Health Research (CIHR). REFERENCES 1. U.S. Food and Drug Administration. Guidance for industry development and use of risk minimization action plans. 2005; Available from: http://www.fda.gov/ downloads/RegulatoryInformation/Guidances/UCM126830.pdf.http://www.fda. gov/downloads/RegulatoryInformation/Guidances/UCM126830.pdf. 2. European Medicines Agency. Guideline on risk management systems for medical products for human use. 2008; Incorporated into: Volume 9A of The Rules Governing Medicinal Products in the European Union, Guidelines on Pharmacovigilance for Medicinal Products for Human Use. Part I-3: Requirements for Risk Management Systems; Available from: http://ec.europa.eu/ health/files/eudralex/vol-9/pdf/vol9a_09-2008_en.pdf. 3. Regulation (EU) №1235/2010 amending, as regards pharmacovigilance of medicinal products for human use, Regulation (EC) No 726/2004 laying down Community procedures for the authorisation and supervision of medicinal products for human and veterinary use and establishing a European Medicines Agency, and Regulation (EC) No 1394/2007 on advanced therapy medicinal products, (2010). 4. The European Parliment and the Council. Directive 2010/84/EU amending, as regards pharmacovigilance, Directive 2001/83/EC on the Community code relating to medicinal products for human use, (2010). 5. Nkeng L, Cloutier AM, Craig C, Lelorier J, Moride Y. Impact of regulatory guidances and drug regulation on risk minimization interventions in drug safety: a systematic review. Drug Saf 2012;35(7):535–46. Epub 2012/06/19. 6. European Medicines Agency. Guideline on good pharmacovigilance practices (GVP). Module XVI-risk monimisation measures: selection of tools and effectiveness indicators. 2013. 7. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ 2009: 339. 8. Cabana MD, Rand CS, Powe NR, et al. Why don’t physicians follow clinical practice guidelines? JAMA 1999; 282(15): 1458–1465. 9. Hudon E, Beaulieu M-D, Roberge D. Integration of the recommendations of the Canadian Task Force on Preventive Health Care Obstacles perceived by a group of family physicians. Fam Pract 2004; 21(1): 11–17. 10. Shadish WR, Cook TD, Campbell DT. Experimental and Quasi-experimental Designs for Generalized Causal Inference. 2002. 11. LaPointe NMA, Kramer JM, Weinfurt KP, Califf RM. Practitioner acceptance of the dofetilide risk-management program. Pharmacother: J Hum Pharmacol Drug Ther 2002; 22(8): 1041–1046. 12. Mayet S, Manning V, Williams A, Loaring J, Strang J. Impact of training for healthcare professionals on how to manage an opioid overdose with naloxone: effective, but dissemination is challenging. Intern J Drug Pol 2011; 22(1): 9–15. 13. Bensouda-Grimaldi L, Jonville-Béra AP, Mouret E, et al., editors. Isotrétinoïne: suivi de l’application des recommandations des prescriptions chez les femmes en âge de procréer. Annales de dermatologie et de vénéréologie; 2005: Elsevier. 14. Cheung A, Sacks D, Dewa CS, Pong J, Levitt A. Pediatric prescribing practices and the FDA black-box warning on antidepressants. J Dev Behav Pediatr 2008; 29(3): 213–215. 15. Lledo A, Dellva MA, Strombom IM, et al. Awareness of potential valvulopathy risk with pergolide and changes in clinical practice after label change: a survey among European neurologists. Eur J Neurol 2007; 14(6): 644–649. 16. Habib AS, Gan TJ. The use of droperidol before and after the Food and Drug Administration black box warning: a survey of the members of the Society of Ambulatory Anesthesia. J Clin Anesth 2008; 20(1): 35–39. 17. Richardson LP, Lewis CW, Casey-Goldstein M, McCauley E, Katon W. Pediatric primary care providers and adolescent depression: a qualitative study of barriers to treatment and the effect of the black box warning. J Adolesc Health 2007; 40(5): 433–439. 18. Schachter DC, Kleinman I. Psychiatrists’ attitudes about and informed consent practices for antipsychotics and tardive dyskinesia. Psychiatr Serv 2004; 55(6): 714–717. 19. Lee LY, Kortepeter CM, Willy ME, Nourjah P. Drug-risk communication to pharmacists: assessing the impact of risk-minimization strategies on the practice of pharmacy. J Am Pharm Assoc 2008; 48(4): 494–500. 20. Kurian BT, Ray WA, Arbogast PG, Fuchs DC, Dudley JA, Cooper WO. Effect of regulatory warnings on antidepressant prescribing for children and adolescents. Arch Pediatr Adolesc Med 2007; 161(7): 690–696.
© 2014 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons, Ltd.
21. Libby AM, Brent DA, Morrato EH, Orton HD, Allen R, Valuck RJ. Decline in treatment of pediatric depression after FDA advisory on risk of suicidality with SSRIs. Am J Psychiatry 2007;164(6):884–891. Epub 2007/06/02. 22. Valuck R, Libby A, Orton H, Morrato E, Allen R, Baldessarini R. Spillover effects on treatment of adult depression in primary care after FDA advisory on risk of pediatric suicidality with SSRIs. Am J Psych 2007; 164(8): 1198–1205. 23. Olfson M, Marcus SC, Druss BG. Effects of Food and Drug Administration warnings on antidepressant use in a national sample. Arch Gen Psychiatry 2008; 65(1): 94–101. 24. Pamer CA, Hammad TA, et al. Changes in US antidepressant and antipsychotic prescription patterns during a period of FDA actions. Pharmacoepidemiology and Drug Safety 2010; 19(2): 158–174. 25. Smalley W, Shatin D, Wysowski DK, et al. Contraindicated use of cisapride. JAMA 2000; 284(23): 3036–3039. 26. Jones JK, Fife D, Curkendall S, Goehring E, Jr, Guo JJ, Shannon M. Coprescribing and codispensing of cisapride and contraindicated drugs. JAMA 2001; 286(13): 1607–1609. 27. Weatherby LB, Walker AM, Fife D, Vervaet P, Klausner MA. Contraindicated medications dispensed with cisapride: temporal trends in relation to the sending of ‘Dear Doctor’ letters. Pharmacoepidemiol Drug Saf 2001; 10(3): 211–218. 28. Weatherby LB, Nordstrom BL, Fife D, Walker AM. The impact of wording in “Dear doctor” letters and in black box labels. Clin Pharmacol Therapeut-St Louis 2002; 72(6): 735–742. 29. Guo JJ, Curkendall S, Jones JK, Fife D, Goehring E, She D. Impact of cisapride label changes on codispensing of contraindicated medications. Pharmacoepidemiol Drug Saf 2003; 12(4): 295–301. 30. Wilkinson JJ, Force RW, Cady PS. Impact of safety warnings on drug utilization: marketplace life span of cisapride and troglitazone. Pharmacother: J Hum Pharmacol Drug Ther 2004; 24(8): 978–986. 31. Feldstein AC, Smith DH, Perrin N, et al. Improved therapeutic monitoring with several interventions: a randomized trial. Arch Intern Med 2006; 166(17): 1848–1854. 32. Paulozzi LJ, Kilbourne EM, Desai HA. Prescription drug monitoring programs and death rates from drug overdose. Pain Med 2011; 12(5): 747–754. 33. Lapane KL, Hughes CM, Daiello LA, Cameron KA, Feinberg J. Effect of a pharmacist-led multicomponent intervention focusing on the medication monitoring phase to prevent potential adverse drug events in nursing homes. J Am Geriatr Soc 2011; 59(7): 1238–1245. 34. Trivalle C, Cartier T, Verny C, et al. Identifying and preventing adverse drug events in elderly hospitalised patients: a randomised trial of a program to reduce adverse drug effects. J Nutr Health Aging 2010; 14(1): 57–61. 35. Allen LaPointe NM, Chen A, Hammill B, DeLong E, Kramer JM, Califf RM. Evaluation of the dofetilide risk-management program. Am Heart J 2003; 146(5): 894–901. 36. Katz LY, Kozyrskyj AL, Prior HJ, Enns MW, Cox BJ, Sareen J. Effect of regulatory warnings on antidepressant prescription rates, use of health services and outcomes among children, adolescents and young adults. Can Med Assoc J 2008; 178(8): 1005–1011. 37. Brinker A, Kornegay C, Nourjah P. Trends in adherence to a revised risk management program designed to decrease or eliminate isotretinoin-exposed pregnancies: evaluation of the accutane SMART program. Arch Dermatol 2005; 141(5): 563. 38. Uhl K, Cox E, Rogan R, et al. Thalidomide use in the US: experience with pregnancy testing in the S.T.E.P.S. programme. Drug Saf 2006;29(4):321–329. Epub 2006/03/30. 39. McCann L, Maguire R, Miller M, Kearney N. Patients’ perceptions and experiences of using a mobile phone-based advanced symptom management system (ASyMS©) to monitor and manage chemotherapy related toxicity. Eur J Canc Care 2009; 18(2): 156–164. 40. Kearney N, McCann L, Norrie J, et al. Evaluation of a mobile phone-based, advanced symptom management system (ASyMS©) in the management of chemotherapy-related toxicity. Support Care Canc 2009; 17(4): 437–444. 41. Murray MD, Ritchey ME, Wu J, Tu W. Effect of a pharmacist on adverse drug events and medication errors in outpatients with cardiovascular disease. Arch Intern Med 2009; 169(8): 757–763. 42. Mazor KM, Andrade SE, Auger J, Fish L, Gurwitz JH. Communicating safety information to physicians: an examination of dear doctor letters. Pharmacoepidemiol Drug Saf 2005; 14(12): 869–875. 43. Melnikow J, Paterniti D, Azari R, et al. Preferences of Women Evaluating Risks of Tamoxifen (POWER) study of preferences for tamoxifen for breast cancer risk reduction. Cancer 2005; 103(10): 1996–2005. 44. Wagner AK, Chan KA, Dashevsky I, et al. FDA drug prescribing warnings: is the black box half empty or half full? Pharmacoepidemiol Drug Saf 2006; 15(6): 369–386. 45. Dorsey E, Rabbani A, Gallagher SA, Conti RM, Alexander GC. Impact of FDA black box advisory on antipsychotic medication use. Arch Intern Med 2010; 170(1): 96–103. 46. Kurdyak PA, Juurlink DN, Mamdani MM. The effect of antidepressant warnings on prescribing trends in Ontario, Canada. J Inform 2007; 97(4): 750–754.
Pharmacoepidemiology and Drug Safety, 2014; 23: 572–579 DOI: 10.1002/pds
assessment of risk minimization interventions 47. Morrato E, Libby A, Orton H, Brent D, Allen R, Valuck R. Frequency of provider contact after FDA advisory on risk of pediatric suicidality with SSRIs. Am J Psych 2008; 165(1): 42–50. 48. Schussel K, Schulz M. Prescribing of COX-2 inhibitors in Germany after safety warnings and market withdrawals. Die Pharmazie-An Intern J Pharmaceut Sci 2006; 61(10): 878–886. 49. Valiyeva E, Herrmann N, Rochon PA, Gill SS, Anderson GM. Effect of regulatory warnings on antipsychotic prescription rates among elderly patients with dementia: a population-based time-series analysis. Can Med Assoc J 2008; 179(5): 438–446. 50. Graham DJ, Drinkard CR, Shatin D, Tsong Y, Burgess MJ. Liver enzyme monitoring in patients treated with troglitazone. JAMA 2001; 286(7): 831–833. 51. Starner CI, Schafer JA, Heaton AH, Gleason PP. Rosiglitazone and pioglitazone utilization from January 2007 through May 2008 associated with five riskwarning events. JMCP 2008; 14(6): 523–531. 52. Corry M, Bonner G, McEntee S, Dugan J, MacAuley D. Hospitals do not inform GPs about medication that should be monitored. Fam Pract 2000; 17(3): 268–271. 53. Murray ML, Thompson M, Santosh PJ, Wong ICK. Effects of the Committee on Safety of Medicines advice on antidepressant prescribing to children and adolescents in the UK. Drug Safety 2005; 28(12): 1151–1157. 54. Jennings HR, Miller EC, Williams TS, Tichenor SS, Woods EA. Reducing anticoagulant medication adverse events and avoidable patient harm. Jt Comm J Qual Patient Saf 2008; 34(4): 196–200. 55. Lederer J, Best D. Reduction in anticoagulation-related adverse drug events using a trigger-based methodology. Jt Comm J Qual Patient Saf 2005; 31(6): 313–318. 56. Morrato EH, Staffa JA. Effectiveness of risk management plans: a case study of pemoline using pharmacy claims data. Pharmacoepidemiol Drug Saf 2007; 16(1): 104–112. 57. Arana A, Allen S, Burkowitz J, et al. Infliximab paediatric Crohn’s disease educational plan. Drug Saf 2010; 33(6): 489–501. 58. De Bruin ML, Panneman MJ, Leufkens HG, Hoes AW, Herings RM. Use of cisapride with contraindicated drugs in The Netherlands. Ann Pharmacother 2002; 36(2): 338–343. 59. Morrato EH, Druss B, Hartung DM, et al. Metabolic testing rates in 3 state Medicaid programs after FDA warnings and ADA/APA recommendations for second-generation antipsychotic drugs. Arch Gen Psychiatry 2010; 67(1): 17–24. 60. Nuttall GA, Eckerman KM, Jacob KA, et al. Does low-dose droperidol administration increase the risk of drug-induced QT prolongation and torsade de pointes in the general surgical population? Anesthesiology 2007; 107(4): 531–536. 61. Hetland ML, Unkerskov J, Ravn T, et al. Routine database registration of biological therapy increases the reporting of adverse events twentyfold in clinical practice. First results from the Danish Database (DANBIO). Scand J Rheumatol 2005; 34(1): 40–44. 62. Nemeroff CB, Kalali A, Keller MB, et al. Impact of publicity concerning pediatric suicidality data on physician practice patterns in the United States. Arch Gen Psychiatry 2007; 64(4): 466–472. 63. Sanfélix-Gimeno G, Cervera-Casino P, Peiró S, López-Valcarcel BG, Blázquez A, Barbera T. Effectiveness of safety warnings in atypical antipsychotic drugs. Drug Saf 2009; 32(11): 1075–1087.
© 2014 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons, Ltd.
579
64. Trifiro G, Sini G, Sturkenboom MCJM, et al. Prescribing pattern of antipsychotic drugs in the Italian general population 2000–2005: a focus on elderly with dementia. Int Clin Psychopharmacol 2010; 25(1): 22–28. 65. Unger MM, Reese JP, Oertel WH, Eggert KM. Real-life evaluations of compliance with mandatory drug safety monitoring exemplified with tolcapone in Parkinson’s disease. Eur Neurol 2008; 60(3): 122–126. 66. Staniscia T, Romano F, Festi D, Bilker WB, Macones GA, Strom BL. Co-dispensing of contraindicated medications in patients using cisapride in Italy. Pharmacoepidemiol Drug Saf 2006; 15(7): 469–476. 67. Meagher DJ. Impact of an educational workshop upon attitudes towards pharmacotherapy for delirium. Int Psychogeriatr 2010; 22(6): 938–946. 68. Shatin D, Gardner JS, Stergachis A, Blough D, Graham D. Impact of mailed warning to prescribers on the co-prescription of tramadol and antidepressants. Pharmacoepidemiol Drug Saf 2005; 14(3): 149–154. 69. Al-Ghamdi SA, Mahmoud MA, Alammari MA, et al. The outcome of pharmacist counseling at the time of hospital discharge: an observational nonrandomized study. Ann Saudi Med 2012; 32(5): 492–497. 70. Gibbons R, Brown C, Hur K, et al. Early evidence on the effects of regulators’ suicidality warnings on SSRI prescriptions and suicide in children and adolescents. Am J Psychiatr 2007; 164(9): 1356–1363. 71. Lasser KE, Seger DL, Yu DT, et al. Adherence to black box warnings for prescription medications in outpatients. Arch Intern Med 2006; 166(3): 338–344. 72. Cluxton RJ, Li Z, Heaton PC, et al. Impact of regulatory labeling for troglitazone and rosiglitazone on hepatic enzyme monitoring compliance: findings from the state of Ohio Medicaid program. Pharmacoepidemiol Drug Saf 2005; 14(1): 1–9. 73. Prieto L, Spooner A, Hidalgo-Simon A, Rubino A, Kurz X, Arlett P. Evaluation of the effectiveness of risk minimization measures. Pharmacoepidemiol Drug Saf.2012;21(8):896–899. Epub 2012/06/26. 74. Office of Inspector General. FDA Lacks comprehensive data to determine whether risk evaluation and mitigation strategies improve drug safety. 2013. 75. Lawton K, Skjoet P. Assessment of three systems to empower the patient and decrease the risk of adverse drug events. Stud Health Technol Inform 2011; 166: 246–253. 76. Shiffman S, Gerlach KK, Sembower MA, Rohay JM. Consumer understanding of prescription drug information: an illustration using an antidepressant medication. Ann Pharmacother 2011; 45(4): 452–458. 77. Piening S, Reber KC, Wieringa JE, et al. Impact of safety-related regulatory action on drug use in ambulatory care in the Netherlands. Clin Pharmacol Therapeu 2012; 91(5): 838–845. 78. Morrison P, Meehan T, Gaskill D, Lunney P, Collings P. Enhancing case managers’ skills in the assessment and management of antipsychotic medication side-effects. Aust N Z J Psychiatry 2000;34(5):814–821. Epub 2000/10/19.
SUPPORTING INFORMATION Additional supporting information may be found in the online version of this article at the publisher’s web site.
Pharmacoepidemiology and Drug Safety, 2014; 23: 572–579 DOI: 10.1002/pds
REVIEW
Methodological gaps in the assessment of risk minimization interventions: a systematic review Inna Gridchyna1, Anne-Marie Cloutier1,2, Lenhangmbong Nkeng1,2, Camille Craig1,2, Sarah Frise3,4 and Yola Moride1,2* 1
Faculty of Pharmacy, Université de Montreal, Montreal, Quebec, Canada Pharmacoepidemiology Unit, Research Center, University of Montreal Hospital Center (CRCHUM), Montreal, Quebec, Canada 3 Department of Patient Safety and Medical Information, AstraZeneca Canada, Mississauga, Ontario, Canada 4 Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada 2
ABSTRACT Introduction Since the introduction of therapeutic risk management regulatory guidance, an increase in the number of risk minimization interventions (RMIs) published in the literature has been observed. Methods used to evaluate their effectiveness remain, however, poorly examined. Objective This paper aimed to conduct a literature review on the methods of evaluation of effectiveness of RMIs and to identify methodological gaps. Methods The search was conducted using MEDLINE and Embase between 1 January 2000 and 31 December 2010, and updated on 1 April 2013. The following characteristics were extracted from each study: target population for the RMI, target population for the assessment of effectiveness, study design, data sources, and effectiveness outcome(s). Results A total of 188 unique RMIs were identified in the literature, of which effectiveness was evaluated in only 65 (34.6%) at the time of publication. The largest proportion of studies reviewed (n = 49, 75.4%) attempted to evaluate changes in behavior through prescribing or laboratory test practices. One quarter of studies evaluated the effect of RMIs on the occurrence of adverse events. Only a minority of studies used robust designs, such as randomized controlled trials (n = 6, 9.2%) or a quasi-experimental design with a parallel comparison group (n = 8, 12.3%). Conclusion Lack of robust methodological design used in published studies on RMI effectiveness evaluation is an important methodological gap in the evaluation of RMI effectiveness. © 2014 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons, Ltd. key words—risk minimization intervention; therapeutic risk management; REMS; effectiveness; pharmacoepidemiology Received 20 June 2013; Revised 17 January 2014; Accepted 20 January 2014
BACKGROUND In 2005 and 2006, respectively, the US Food and Drugs Administration (FDA)1 and the European Medicines Agency2 issued guidance documents on therapeutic risk management planning. The European Union recently implemented a new pharmacovigilance legislation.3,4 Risk minimization interventions (RMIs) are part of risk management plans and include all actions beyond product labeling, which aim at enhancing drug benefit–risk.1 We conducted a systematic review of the literature and of agency websites5 in order to describe the characteristics of the various RMIs that have been implemented.
*Correspondence to: Y. Moride, Université de Montréal, C.P.6128, succursale Centre-ville, Montreal, QC H3C 3J7, Canada. E-mail: [email protected]
The most frequent RMIs consisted of education material targeted to healthcare professionals and/or patients. More restrictive RMIs, such as controlled distribution programs, accounted for only 6.6% of publications and 1.2% RMIs posted on agency websites. Regulatory guidance states that the effectiveness of RMIs should be evaluated.1 Assessment of effectiveness aims at determining the performance of the RMI itself and at identifying potential deficiencies that may warrant further interventions. Despite the available guidance documents, there are no specific recommendations on the methods of assessment of effectiveness of RMIs.1,6 The objectives of our review were as follows: (1) to characterize the study designs, target populations, data sources, and measures of success (effectiveness outcomes) that have been used in RMI evaluation studies published in the literature
© 2014 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons, Ltd. This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made.
assessment of risk minimization interventions
and (2) to identify methodological gaps in methods of effectiveness evaluation.
573
METHODS
of adverse events (AEs). Natural healthcare products, devices, diagnostic chemicals, pregnancy registries without follow-up, medication errors, and products not used as therapy for illness were not retained.
Literature search strategy
Data extraction
A literature search was conducted using MEDLINE and Embase and followed the PRISMA Statement for Reporting Systematic Reviews and Meta-analyses of Studies.7 Medical Subject Heading (MeSH) terms were used where possible. The term “drug toxicity” was searched along with “patient education,” “health care professional education,” “prescriber education,” “patient alert card,” “patient registry,” “medication guide,” “drug legislation,” “informed consent,” “restricted distribution,” “physician authorisation,” “drug monitoring,” “Dear Health Care Professional Letter,” “Dear Doctor Letter,” and “black box warning.” The references of all retained articles, including review articles, were scanned for potential relevant papers (“snowballing”). The initial search was completed in March 2011 and was updated on 1 April 2013, without changing the search strategy. All abstracts identified through the electronic search were reviewed by three independent reviewers, and any disagreements were resolved with a majority (two out of three assessments) decision.
Among those studies with an evaluation component, we systematically extracted the following: type of RMI, target population, AE(s) of special interest, region, and year of publication. Methodological components of assessment of effectiveness consisted of the following: target population for the evaluation, study design, data source, effectiveness outcome(s), and main findings.
Inclusion and exclusion criteria To be included in the review, articles must have been published between 1 January 2000 and 31 March 2013 inclusively and involve drug products, use in humans, RMIs, or tools used to increase the reporting
Figure 1.
Analysis Components of evaluation were assessed using the theoretical framework adapted from Cabana et al.8 and Hudon et al.9 and depicted in Figure 1. On the basis of this framework, adherence to RMI recommendations and integration of such recommendations into medical practice are influenced by physician and patient characteristics, as well as by the healthcare setting. Factors influencing adherence to RMIs are divided into three temporal phases: (i) knowledge and awareness; (ii) attitude; and (iii) behavior. As shown in Figure 1, a large amount of information and disagreement with recommendations are both important deterrents of effectiveness. Integration of the RMI into the pathway of patient care may explain regional differences in RMI effectiveness. Finally, medical practice is also influenced by lack of healthcare resources and/or reimbursement constraints. The strength of the evidence of RMI
Theoretical framework to assess the effectiveness of risk minimization interventions. Adapted from Cabana et al. (8) and Hudon et al. (9)
© 2014 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons, Ltd.
Pharmacoepidemiology and Drug Safety, 2014; 23: 572–579 DOI: 10.1002/pds
574
i. gridchyna et al.
effectiveness was assessed using the Shadish et al.10 ranking of designs for causal inference, which ranges from robust experimental designs such as randomized controlled trials to weaker designs such as observational studies without a comparison group. RESULTS Literature search A total of 4935 sources were identified from the MEDLINE and Embase searches. Of these, 154 met the inclusion and exclusion criteria, and another 34 sources were obtained from snowballing, yielding a total of 188 sources retained for the review. Of these, 65 (34.6%) reported an effectiveness evaluation (Figure 2). The distribution of types of RMIs with an evaluation of effectiveness is shown in Figure 3, and the characteristics of the evaluation studies are described in the Supporting Information. Education material, training, black-box warning, and medication management system appeared to be most frequently evaluated in the literature.
Figure 3. Number of studies with an evaluation component, according to type of risk minimization intervention. DHCP = Dear Health Care Professional letter; SW = safety warnings; EM = education materials; CM = communication materials; MMS = medication management system; IC = informed consent; RD = restricted distribution; R = registry; General = overall knowledge on risk minimization intervention tools
Components of risk minimization The vast majority of studies with an evaluation component focused on the behavioral construct of risk minimization (n = 49, 75.4%), measured through prescribing and laboratory testing practices. Other studies evaluated knowledge or attitudes (n = 9, 13.5%). Fewer studies addressed more than one component: knowledge and attitudes,11,12 knowledge and behavior,13–15 and attitudes and behavior.16–18 Only one study, consisting of a survey, addressed all three components of risk minimization (knowledge, attitudes, and behavior).19 One quarter of studies (n = 16, 24.6%) evaluated the impact of the RMI on the occurrence of AEs. Study designs used to assess the effectiveness of risk minimization interventions
Figure 2. Results of the literature search on risk minimization interventions
© 2014 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons, Ltd.
Study designs used to assess each component of risk minimization (knowledge, attitudes, and behavior) are shown in Table 1. It should be noted that the effectiveness of a given RMI could be evaluated through more than one study. For example, the effectiveness of safety warnings on antidepressants issued by the FDA was evaluated in six studies published in the literature.17,20–24 Also, six studies evaluating label changes and Dear Health Care Professional (DHCP) letters on cisapride were found.25–30 The great majority of studies that evaluated behavior used interrupted time series analyses or pre–post intervention designs without a comparison group, often using data from administrative claims databases. Given the availability of historical data (i.e., pre-intervention), most of the RMIs involving safety warnings or DHCP letters were evaluated using time trends in prescribing or laboratory test monitoring. We have identified only six studies that used a parallel comparison group. Such Pharmacoepidemiology and Drug Safety, 2014; 23: 572–579 DOI: 10.1002/pds
575
assessment of risk minimization interventions Table 1. Distribution of study designs used to assess the various components of risk minimization interventions
Randomized controlled trials Interrupted time series analysis Pre–post intervention Cohort studies/registry
Knowledge
Attitudes
Behavior
Clinical outcome (adverse event)
0 0 0 0 1 0 0 1 4 3 9
1 0 0 0 2 0 0 0 5 1 9
1 2 17 3 10 1 5 4 5 1 49
4 1 0 4 3 1 2 1 0 0 16
With comparison Without comparison With comparison Without comparison With comparison Without comparison
Cross-sectional studies Surveys Qualitative assessment Total
comparison groups were either randomized31 or consisted of other geographical locations (states)32 or medical centers where the RMI was not implemented,33,34 another drug not targeted by the intervention,35 or young adults when children or adolescents were the target of the intervention.36 In another study27 in the absence of a reference population, all external factors, such as publicity and media coverage, which could also influence prescribing trends in addition to the DHCP letter, were qualitatively assessed. Restricted distribution programs, generally implemented at the time of product approval, tended to be evaluated through cohort studies.35,37,38 Knowledge and attitudes were most often evaluated using surveys or qualitative assessments,17,42,75,76 whereas a randomized trial39 or pre–post intervention without a comparison group12 was rarely used. As shown in Table 1, among the 16 studies that evaluated the impact of the RMI on the occurrence of AEs, robust designs such as RCTs33,34,40,41 or comparative designs (n = 6) were often used. Effectiveness outcomes Measures of effectiveness, or success, of RMIs included prescription or dispensing rate (n = 18, 27.6%), frequency of AEs (n = 16, 24.6%), including pregnancy rate during isotretinoin or thalidomide treatment, contraindicated co-prescription rate (n = 10, 15.4%), laboratory testing rate (n = 7, 10.8%), knowledge retention and attitude toward the RMI (n = 7, 10.8%), changes in medical practice (n = 6, 9.2%), rate of treatment initiation (n = 4, 6.2%), awareness of RMI (n = 4, 6.2%), rate of use of treatment alternatives (n = 3, 4.6%). Several studies evaluated the effectiveness of the RMI with more than one outcome.14,15,17,18,36,42–44 Of the 41 RMIs consisting of education, 31 used prescription rate or contraindicated co-prescription rate as a measure of effectiveness. Using the information matrix included in the Supporting Information, we © 2014 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons, Ltd.
have ascertained the aim of the intervention and the measure of effectiveness, from which we have assessed concordance. In nearly one third (n = 18) of the studies, the effectiveness measure did not correspond to the aim of the intervention. Inconsistencies originated from the following: prescription rates were used as measures of effectiveness while the RMI aimed at preventing the occurrence of AEs (n = 10),20–24,45–49 or the RMI sought to enhance laboratory testing (n = 2).50,51 In other instances, occurrence of AEs was used as the effectiveness measure while the intervention aimed at avoiding concomitant use of contraindicated medications (n = 2),52,53 at increasing laboratory testing (n = 2),54,55 or at enhancing information disclosure by healthcare professionals (n = 1).56 For studies that involved surveys, the definition of RMI effectiveness was not explicit, consisting mainly of a change in practice (change in medication dosage or drug, referral of patients to a specialist, performance of baseline assessment before initiation of a medical treatment, etc.) (n = 5),14,15,17,18,42 knowledge retention (n = 5),11,19,42,43,52 awareness of RMIs (n = 5),14,15,17,18,57 and increase of disclosure of information (n = 1).58 In cases where effectiveness was defined as knowledge retention, there was no specification on what represented adequate knowledge retention or what information should the target population have retained.19,43,52 Main data sources used to ascertain effectiveness outcomes included claims databases (n = 34, 52.3%), questionnaires (n = 19, 29.2%), medical charts or electronic health records (n = 9, 13.8%), and clinical data or laboratory findings (n = 4, 6.2%). Some studies have evaluated the effect of an RMI on the initiation of new treatments (i.e., incident drug users),20,23,46,48,59,60 whereas others focused on the prescribing rate for all users (incident and prevalent).13,21,24,36,41,45,49,53,61–65,77 Some studies defined new use as the absence of a prescription record during the preceding 4 months23,48 or 12 months.20,46 Among studies that examined the effectiveness of an Pharmacoepidemiology and Drug Safety, 2014; 23: 572–579 DOI: 10.1002/pds
576
i. gridchyna et al.
RMI on the rate of concomitant use of contraindicated medications, concomitant use was most frequently defined as two contraindicated products with active prescriptions that overlap on at least one day.15,26,27,29,30,58,66 Three surveys cited in the literature targeted different healthcare professionals, namely physicians, nurses, pharmacist, and social workers.11,12,67 Two surveys targeted only pharmacists,18,19 and two surveys only specialists, pediatricians,14 and neurologists.15 The completion rate for the surveys ranged from 50%19 to 100%67. All surveys were comprised of multiple-choice questions. One survey utilized a scale to rate the healthcare professionals’ attitude toward RMI19, and one collected doctors’ comments.14 Results of assessment of risk minimization interventions Among the studies reviewed, RMIs appeared to be successful or partly successful (n = 40, 61.5%) or 15 (23.1%) were not, and for nine studies (13.8%), it was not possible to draw a conclusion regarding their effectiveness because either the study was descriptive, had no comparison group, or had no threshold for effectiveness defined. After the implementation of the RMI, consisting mainly of warnings or DHCP letters, studies found a decrease in prescribing or co-prescribing rate of the study drug (n = 24),16,20,21,23–30,40,41,44– 46,49,51,54,62–64,66,68 a decrease in frequency of AEs (n = 10),33,34,54,55,60,61,69–71,78 a decrease in pregnancy rate while on thalidomide (n = 1),38 an increase in laboratory testing rate (n = 6),31,35,44,50,65,72 or a change in medication dosage (n = 1).65 When successive communication interventions were implemented over time,24,26–31,49,51,62,63,72 the first ones were often found unsuccessful (e.g., not reaching the expected decrease in prescription rate), whereas the latter interventions seemed more effective.26–28,30,50 Wilkinson et al. showed that multiple safety alerts are required to yield a significant decrease in the co-prescription rate of cisapride and contraindicated drugs.30 DISCUSSION Interrupted time series analyses or pre–post intervention analyses without comparison groups were the most frequent designs used to assess the effectiveness of RMIs. These are associated with well-known limitations. First, in the absence of a parallel comparison group (i.e., a group of patients unexposed to the intervention), it is difficult to ascertain if the observed outcome is attributable to the intervention or to the © 2014 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons, Ltd.
presence of external factors. Second, they are not suitable for drugs that have newly been introduced on the market because of the absence of pre-intervention data. For these reasons, they have mainly been used to assess the effect of regulatory interventions on drugs that have been marketed for a while. Only a minority of studies used a reference group, and it has been suggested that, whenever it is not possible, reference data (e.g., literature review or historical data) could be used.73 Experimental or quasi-experimental designs, although more robust, are rarely feasible because most RMIs, such as DHCP letters or safety warnings, are implemented in the entire population. Designs used to evaluate the effect of RMIs on the occurrence of AEs tended to be more robust than those that addressed the other components of risk minimization (i.e., knowledge, attitudes, and behavior). However, such studies were a minority and involved RMIs implemented at the level of the region or institution rather than in entire populations. In order to circumvent this limitation, some authors have attempted to use drugs or subpopulations not targeted by the RMI as a comparison population. Although appealing, the comparability of such reference should be assessed. For other types of RMIs, the selection of observational or descriptive studies without comparison groups could be explained by the lack of clear guidance on the design. For example, FDA guidance anticipates that RMI evaluations “would involve the analysis of observational or descriptive data.” At the same time, it is recommended, whenever feasible, to include at least two evaluation methods for each critical RMI goal.1 Most of the evaluation studies assessed behavioral changes using prescription or dispensing data. Assessing behavioral changes in a population without previously having assessed the knowledge and the attitude toward the RMI can introduce unknown confounders. Consequently, it is not possible to perform a root cause analysis of suboptimal behaviors using solely prescription data. Only one of the published studies evaluated all three components of the effectiveness framework (i.e., knowledge, attitude, and behavior). Neither European nor US guidance clearly emphasizes the assessment of all three components. Within the European dual approach, that is, implementation assessment and level of risk control assessment, the knowledge and attitudes could be a part of implementation assessment.6 The FDA guidance highlights the assessment of the effectiveness of the RMI through health outcomes, for example, complication rate, and recommends “assessments of comprehension, knowledge, attitudes, and/or desired safety behaviors Pharmacoepidemiology and Drug Safety, 2014; 23: 572–579 DOI: 10.1002/pds
assessment of risk minimization interventions
about drug safety risks” if “a health outcome cannot be particularly accurately measured”. Recently, the Office of the Inspector General determined that most RMIs, implemented in the USA, were not meeting their goals because of deficiencies in patient and prescriber awareness of drug risks.74 Among the RMIs consisting of education on key safety messages, the most utilized effectiveness outcome consisted of prescription rate or contraindicated co-prescription rate. By measuring the contraindicated co-prescribing rate, one may estimate the number of patients who are potentially at risk of AE. If the aim of the intervention is to prevent at-risk patients from receiving the drug, then such an outcome is an appropriate measure of success. However, if the intervention aims at preventing the occurrence of AEs in the target population, then occurrence of AEs would be preferable over prescribing rates. Main data sources used to ascertain effectiveness outcomes included claims databases, questionnaires, medical charts or electronic health records, and clinical data or laboratory findings. The use of claims databases to assess prescription rate is associated with information bias given that such databases record patterns of dispensing, dispensing frequency being influenced by prescribing practices, and patient acquisition behavior. Acquisition behavior is in turn a function of access and treatment compliance. In some studies, effectiveness has also been determined using the occurrence of AEs ascertained in claims databases. Although these databases are routinely used in observational studies, they are associated with well-known limitations that have been insufficiently addressed in the published studies on RMIs. Inaccuracy of diagnostic codes inherent to physician billing claims is likely to be associated with an overestimation or underestimation of the effectiveness of a given intervention. In many studies, study outcome consisted of a continuous variable, for example, prevalence of drug use, and any change over time that reached statistical significance was an indicator of success. This is a limitation, especially for studies using administrative claims databases, where statistical significance is often reached owing to large sample sizes. More guidance on a threshold of behavior change that would indicate success of the intervention would be required.
577
RMIs posted on drug agency websites or manufacturer websites, was considered. However, the amount of information disclosed on these latter resources is generally not sufficient to extract all the methodological elements required for the review. This literature review highlights some of the methodological gaps in the assessment of the effectiveness of RMIs. Only 34.6% of the RMI publications found in the literature included an evaluation component. The effectiveness evaluation of recent RMIs may be in progress or planned, and for this reason, the results are not yet available in the published sources. In studies that included an evaluation component, the measure of effectiveness did not appear consistent with the aim or the target population of the RMI for one third of the studies. These observations reveal that some evaluation studies are based on an inappropriate theoretical framework. Because of the heterogeneity of the target populations, drugs, and outcomes under study, it is difficult to rank the types of RMIs according to their effectiveness. Nevertheless, lack of robust designs is a major methodological gap and could be partly due to difficulties in finding appropriate reference population and/or absence of a clear guidance in regulatory documents. Especially for newly marketed drugs, the absence of an appropriate comparison group is a major challenge that should be addressed in future studies. CONFLICT OF INTEREST There are no conflicts of interest to declare for A. M. C., I. G., L. N., C. C. , and Y. M. S. F. is an employee of AstraZeneca Canada. KEY POINTS The study presents a literature review on the methods of evaluation of effectiveness of risk minimization interventions (RMI). • About one third of the RMI publications found in the literature included an evaluation component. • The majority of studies have evaluated prescription behavior and considerably less assessed the impact of the intervention on the occurrence of adverse events. • A major methodological gap is a lack of robust designs, owing to absence of comparison groups.
•
CONCLUSION To our knowledge, this review is the first of its kind to have been conducted. The main limitation is that only sources identified in the literature were included. A more exhaustive search of other resources, such as © 2014 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons, Ltd.
ETHICS STATEMENT The authors state that no ethical approval was needed. Pharmacoepidemiology and Drug Safety, 2014; 23: 572–579 DOI: 10.1002/pds
578
i. gridchyna et al.
ACKNOWLEDGEMENTS This review was conducted through a research grant from AstraZeneca Canada. Anne-Marie Cloutier is a recipient of a scholarship from the Canadian Institutes for Health Research (CIHR). REFERENCES 1. U.S. Food and Drug Administration. Guidance for industry development and use of risk minimization action plans. 2005; Available from: http://www.fda.gov/ downloads/RegulatoryInformation/Guidances/UCM126830.pdf.http://www.fda. gov/downloads/RegulatoryInformation/Guidances/UCM126830.pdf. 2. European Medicines Agency. Guideline on risk management systems for medical products for human use. 2008; Incorporated into: Volume 9A of The Rules Governing Medicinal Products in the European Union, Guidelines on Pharmacovigilance for Medicinal Products for Human Use. Part I-3: Requirements for Risk Management Systems; Available from: http://ec.europa.eu/ health/files/eudralex/vol-9/pdf/vol9a_09-2008_en.pdf. 3. Regulation (EU) №1235/2010 amending, as regards pharmacovigilance of medicinal products for human use, Regulation (EC) No 726/2004 laying down Community procedures for the authorisation and supervision of medicinal products for human and veterinary use and establishing a European Medicines Agency, and Regulation (EC) No 1394/2007 on advanced therapy medicinal products, (2010). 4. The European Parliment and the Council. Directive 2010/84/EU amending, as regards pharmacovigilance, Directive 2001/83/EC on the Community code relating to medicinal products for human use, (2010). 5. Nkeng L, Cloutier AM, Craig C, Lelorier J, Moride Y. Impact of regulatory guidances and drug regulation on risk minimization interventions in drug safety: a systematic review. Drug Saf 2012;35(7):535–46. Epub 2012/06/19. 6. European Medicines Agency. Guideline on good pharmacovigilance practices (GVP). Module XVI-risk monimisation measures: selection of tools and effectiveness indicators. 2013. 7. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ 2009: 339. 8. Cabana MD, Rand CS, Powe NR, et al. Why don’t physicians follow clinical practice guidelines? JAMA 1999; 282(15): 1458–1465. 9. Hudon E, Beaulieu M-D, Roberge D. Integration of the recommendations of the Canadian Task Force on Preventive Health Care Obstacles perceived by a group of family physicians. Fam Pract 2004; 21(1): 11–17. 10. Shadish WR, Cook TD, Campbell DT. Experimental and Quasi-experimental Designs for Generalized Causal Inference. 2002. 11. LaPointe NMA, Kramer JM, Weinfurt KP, Califf RM. Practitioner acceptance of the dofetilide risk-management program. Pharmacother: J Hum Pharmacol Drug Ther 2002; 22(8): 1041–1046. 12. Mayet S, Manning V, Williams A, Loaring J, Strang J. Impact of training for healthcare professionals on how to manage an opioid overdose with naloxone: effective, but dissemination is challenging. Intern J Drug Pol 2011; 22(1): 9–15. 13. Bensouda-Grimaldi L, Jonville-Béra AP, Mouret E, et al., editors. Isotrétinoïne: suivi de l’application des recommandations des prescriptions chez les femmes en âge de procréer. Annales de dermatologie et de vénéréologie; 2005: Elsevier. 14. Cheung A, Sacks D, Dewa CS, Pong J, Levitt A. Pediatric prescribing practices and the FDA black-box warning on antidepressants. J Dev Behav Pediatr 2008; 29(3): 213–215. 15. Lledo A, Dellva MA, Strombom IM, et al. Awareness of potential valvulopathy risk with pergolide and changes in clinical practice after label change: a survey among European neurologists. Eur J Neurol 2007; 14(6): 644–649. 16. Habib AS, Gan TJ. The use of droperidol before and after the Food and Drug Administration black box warning: a survey of the members of the Society of Ambulatory Anesthesia. J Clin Anesth 2008; 20(1): 35–39. 17. Richardson LP, Lewis CW, Casey-Goldstein M, McCauley E, Katon W. Pediatric primary care providers and adolescent depression: a qualitative study of barriers to treatment and the effect of the black box warning. J Adolesc Health 2007; 40(5): 433–439. 18. Schachter DC, Kleinman I. Psychiatrists’ attitudes about and informed consent practices for antipsychotics and tardive dyskinesia. Psychiatr Serv 2004; 55(6): 714–717. 19. Lee LY, Kortepeter CM, Willy ME, Nourjah P. Drug-risk communication to pharmacists: assessing the impact of risk-minimization strategies on the practice of pharmacy. J Am Pharm Assoc 2008; 48(4): 494–500. 20. Kurian BT, Ray WA, Arbogast PG, Fuchs DC, Dudley JA, Cooper WO. Effect of regulatory warnings on antidepressant prescribing for children and adolescents. Arch Pediatr Adolesc Med 2007; 161(7): 690–696.
© 2014 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons, Ltd.
21. Libby AM, Brent DA, Morrato EH, Orton HD, Allen R, Valuck RJ. Decline in treatment of pediatric depression after FDA advisory on risk of suicidality with SSRIs. Am J Psychiatry 2007;164(6):884–891. Epub 2007/06/02. 22. Valuck R, Libby A, Orton H, Morrato E, Allen R, Baldessarini R. Spillover effects on treatment of adult depression in primary care after FDA advisory on risk of pediatric suicidality with SSRIs. Am J Psych 2007; 164(8): 1198–1205. 23. Olfson M, Marcus SC, Druss BG. Effects of Food and Drug Administration warnings on antidepressant use in a national sample. Arch Gen Psychiatry 2008; 65(1): 94–101. 24. Pamer CA, Hammad TA, et al. Changes in US antidepressant and antipsychotic prescription patterns during a period of FDA actions. Pharmacoepidemiology and Drug Safety 2010; 19(2): 158–174. 25. Smalley W, Shatin D, Wysowski DK, et al. Contraindicated use of cisapride. JAMA 2000; 284(23): 3036–3039. 26. Jones JK, Fife D, Curkendall S, Goehring E, Jr, Guo JJ, Shannon M. Coprescribing and codispensing of cisapride and contraindicated drugs. JAMA 2001; 286(13): 1607–1609. 27. Weatherby LB, Walker AM, Fife D, Vervaet P, Klausner MA. Contraindicated medications dispensed with cisapride: temporal trends in relation to the sending of ‘Dear Doctor’ letters. Pharmacoepidemiol Drug Saf 2001; 10(3): 211–218. 28. Weatherby LB, Nordstrom BL, Fife D, Walker AM. The impact of wording in “Dear doctor” letters and in black box labels. Clin Pharmacol Therapeut-St Louis 2002; 72(6): 735–742. 29. Guo JJ, Curkendall S, Jones JK, Fife D, Goehring E, She D. Impact of cisapride label changes on codispensing of contraindicated medications. Pharmacoepidemiol Drug Saf 2003; 12(4): 295–301. 30. Wilkinson JJ, Force RW, Cady PS. Impact of safety warnings on drug utilization: marketplace life span of cisapride and troglitazone. Pharmacother: J Hum Pharmacol Drug Ther 2004; 24(8): 978–986. 31. Feldstein AC, Smith DH, Perrin N, et al. Improved therapeutic monitoring with several interventions: a randomized trial. Arch Intern Med 2006; 166(17): 1848–1854. 32. Paulozzi LJ, Kilbourne EM, Desai HA. Prescription drug monitoring programs and death rates from drug overdose. Pain Med 2011; 12(5): 747–754. 33. Lapane KL, Hughes CM, Daiello LA, Cameron KA, Feinberg J. Effect of a pharmacist-led multicomponent intervention focusing on the medication monitoring phase to prevent potential adverse drug events in nursing homes. J Am Geriatr Soc 2011; 59(7): 1238–1245. 34. Trivalle C, Cartier T, Verny C, et al. Identifying and preventing adverse drug events in elderly hospitalised patients: a randomised trial of a program to reduce adverse drug effects. J Nutr Health Aging 2010; 14(1): 57–61. 35. Allen LaPointe NM, Chen A, Hammill B, DeLong E, Kramer JM, Califf RM. Evaluation of the dofetilide risk-management program. Am Heart J 2003; 146(5): 894–901. 36. Katz LY, Kozyrskyj AL, Prior HJ, Enns MW, Cox BJ, Sareen J. Effect of regulatory warnings on antidepressant prescription rates, use of health services and outcomes among children, adolescents and young adults. Can Med Assoc J 2008; 178(8): 1005–1011. 37. Brinker A, Kornegay C, Nourjah P. Trends in adherence to a revised risk management program designed to decrease or eliminate isotretinoin-exposed pregnancies: evaluation of the accutane SMART program. Arch Dermatol 2005; 141(5): 563. 38. Uhl K, Cox E, Rogan R, et al. Thalidomide use in the US: experience with pregnancy testing in the S.T.E.P.S. programme. Drug Saf 2006;29(4):321–329. Epub 2006/03/30. 39. McCann L, Maguire R, Miller M, Kearney N. Patients’ perceptions and experiences of using a mobile phone-based advanced symptom management system (ASyMS©) to monitor and manage chemotherapy related toxicity. Eur J Canc Care 2009; 18(2): 156–164. 40. Kearney N, McCann L, Norrie J, et al. Evaluation of a mobile phone-based, advanced symptom management system (ASyMS©) in the management of chemotherapy-related toxicity. Support Care Canc 2009; 17(4): 437–444. 41. Murray MD, Ritchey ME, Wu J, Tu W. Effect of a pharmacist on adverse drug events and medication errors in outpatients with cardiovascular disease. Arch Intern Med 2009; 169(8): 757–763. 42. Mazor KM, Andrade SE, Auger J, Fish L, Gurwitz JH. Communicating safety information to physicians: an examination of dear doctor letters. Pharmacoepidemiol Drug Saf 2005; 14(12): 869–875. 43. Melnikow J, Paterniti D, Azari R, et al. Preferences of Women Evaluating Risks of Tamoxifen (POWER) study of preferences for tamoxifen for breast cancer risk reduction. Cancer 2005; 103(10): 1996–2005. 44. Wagner AK, Chan KA, Dashevsky I, et al. FDA drug prescribing warnings: is the black box half empty or half full? Pharmacoepidemiol Drug Saf 2006; 15(6): 369–386. 45. Dorsey E, Rabbani A, Gallagher SA, Conti RM, Alexander GC. Impact of FDA black box advisory on antipsychotic medication use. Arch Intern Med 2010; 170(1): 96–103. 46. Kurdyak PA, Juurlink DN, Mamdani MM. The effect of antidepressant warnings on prescribing trends in Ontario, Canada. J Inform 2007; 97(4): 750–754.
Pharmacoepidemiology and Drug Safety, 2014; 23: 572–579 DOI: 10.1002/pds
assessment of risk minimization interventions 47. Morrato E, Libby A, Orton H, Brent D, Allen R, Valuck R. Frequency of provider contact after FDA advisory on risk of pediatric suicidality with SSRIs. Am J Psych 2008; 165(1): 42–50. 48. Schussel K, Schulz M. Prescribing of COX-2 inhibitors in Germany after safety warnings and market withdrawals. Die Pharmazie-An Intern J Pharmaceut Sci 2006; 61(10): 878–886. 49. Valiyeva E, Herrmann N, Rochon PA, Gill SS, Anderson GM. Effect of regulatory warnings on antipsychotic prescription rates among elderly patients with dementia: a population-based time-series analysis. Can Med Assoc J 2008; 179(5): 438–446. 50. Graham DJ, Drinkard CR, Shatin D, Tsong Y, Burgess MJ. Liver enzyme monitoring in patients treated with troglitazone. JAMA 2001; 286(7): 831–833. 51. Starner CI, Schafer JA, Heaton AH, Gleason PP. Rosiglitazone and pioglitazone utilization from January 2007 through May 2008 associated with five riskwarning events. JMCP 2008; 14(6): 523–531. 52. Corry M, Bonner G, McEntee S, Dugan J, MacAuley D. Hospitals do not inform GPs about medication that should be monitored. Fam Pract 2000; 17(3): 268–271. 53. Murray ML, Thompson M, Santosh PJ, Wong ICK. Effects of the Committee on Safety of Medicines advice on antidepressant prescribing to children and adolescents in the UK. Drug Safety 2005; 28(12): 1151–1157. 54. Jennings HR, Miller EC, Williams TS, Tichenor SS, Woods EA. Reducing anticoagulant medication adverse events and avoidable patient harm. Jt Comm J Qual Patient Saf 2008; 34(4): 196–200. 55. Lederer J, Best D. Reduction in anticoagulation-related adverse drug events using a trigger-based methodology. Jt Comm J Qual Patient Saf 2005; 31(6): 313–318. 56. Morrato EH, Staffa JA. Effectiveness of risk management plans: a case study of pemoline using pharmacy claims data. Pharmacoepidemiol Drug Saf 2007; 16(1): 104–112. 57. Arana A, Allen S, Burkowitz J, et al. Infliximab paediatric Crohn’s disease educational plan. Drug Saf 2010; 33(6): 489–501. 58. De Bruin ML, Panneman MJ, Leufkens HG, Hoes AW, Herings RM. Use of cisapride with contraindicated drugs in The Netherlands. Ann Pharmacother 2002; 36(2): 338–343. 59. Morrato EH, Druss B, Hartung DM, et al. Metabolic testing rates in 3 state Medicaid programs after FDA warnings and ADA/APA recommendations for second-generation antipsychotic drugs. Arch Gen Psychiatry 2010; 67(1): 17–24. 60. Nuttall GA, Eckerman KM, Jacob KA, et al. Does low-dose droperidol administration increase the risk of drug-induced QT prolongation and torsade de pointes in the general surgical population? Anesthesiology 2007; 107(4): 531–536. 61. Hetland ML, Unkerskov J, Ravn T, et al. Routine database registration of biological therapy increases the reporting of adverse events twentyfold in clinical practice. First results from the Danish Database (DANBIO). Scand J Rheumatol 2005; 34(1): 40–44. 62. Nemeroff CB, Kalali A, Keller MB, et al. Impact of publicity concerning pediatric suicidality data on physician practice patterns in the United States. Arch Gen Psychiatry 2007; 64(4): 466–472. 63. Sanfélix-Gimeno G, Cervera-Casino P, Peiró S, López-Valcarcel BG, Blázquez A, Barbera T. Effectiveness of safety warnings in atypical antipsychotic drugs. Drug Saf 2009; 32(11): 1075–1087.
© 2014 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons, Ltd.
579
64. Trifiro G, Sini G, Sturkenboom MCJM, et al. Prescribing pattern of antipsychotic drugs in the Italian general population 2000–2005: a focus on elderly with dementia. Int Clin Psychopharmacol 2010; 25(1): 22–28. 65. Unger MM, Reese JP, Oertel WH, Eggert KM. Real-life evaluations of compliance with mandatory drug safety monitoring exemplified with tolcapone in Parkinson’s disease. Eur Neurol 2008; 60(3): 122–126. 66. Staniscia T, Romano F, Festi D, Bilker WB, Macones GA, Strom BL. Co-dispensing of contraindicated medications in patients using cisapride in Italy. Pharmacoepidemiol Drug Saf 2006; 15(7): 469–476. 67. Meagher DJ. Impact of an educational workshop upon attitudes towards pharmacotherapy for delirium. Int Psychogeriatr 2010; 22(6): 938–946. 68. Shatin D, Gardner JS, Stergachis A, Blough D, Graham D. Impact of mailed warning to prescribers on the co-prescription of tramadol and antidepressants. Pharmacoepidemiol Drug Saf 2005; 14(3): 149–154. 69. Al-Ghamdi SA, Mahmoud MA, Alammari MA, et al. The outcome of pharmacist counseling at the time of hospital discharge: an observational nonrandomized study. Ann Saudi Med 2012; 32(5): 492–497. 70. Gibbons R, Brown C, Hur K, et al. Early evidence on the effects of regulators’ suicidality warnings on SSRI prescriptions and suicide in children and adolescents. Am J Psychiatr 2007; 164(9): 1356–1363. 71. Lasser KE, Seger DL, Yu DT, et al. Adherence to black box warnings for prescription medications in outpatients. Arch Intern Med 2006; 166(3): 338–344. 72. Cluxton RJ, Li Z, Heaton PC, et al. Impact of regulatory labeling for troglitazone and rosiglitazone on hepatic enzyme monitoring compliance: findings from the state of Ohio Medicaid program. Pharmacoepidemiol Drug Saf 2005; 14(1): 1–9. 73. Prieto L, Spooner A, Hidalgo-Simon A, Rubino A, Kurz X, Arlett P. Evaluation of the effectiveness of risk minimization measures. Pharmacoepidemiol Drug Saf.2012;21(8):896–899. Epub 2012/06/26. 74. Office of Inspector General. FDA Lacks comprehensive data to determine whether risk evaluation and mitigation strategies improve drug safety. 2013. 75. Lawton K, Skjoet P. Assessment of three systems to empower the patient and decrease the risk of adverse drug events. Stud Health Technol Inform 2011; 166: 246–253. 76. Shiffman S, Gerlach KK, Sembower MA, Rohay JM. Consumer understanding of prescription drug information: an illustration using an antidepressant medication. Ann Pharmacother 2011; 45(4): 452–458. 77. Piening S, Reber KC, Wieringa JE, et al. Impact of safety-related regulatory action on drug use in ambulatory care in the Netherlands. Clin Pharmacol Therapeu 2012; 91(5): 838–845. 78. Morrison P, Meehan T, Gaskill D, Lunney P, Collings P. Enhancing case managers’ skills in the assessment and management of antipsychotic medication side-effects. Aust N Z J Psychiatry 2000;34(5):814–821. Epub 2000/10/19.
SUPPORTING INFORMATION Additional supporting information may be found in the online version of this article at the publisher’s web site.
Pharmacoepidemiology and Drug Safety, 2014; 23: 572–579 DOI: 10.1002/pds
