pharmacoepidemiology and drug safety 2014; 23: 107–108
Published online in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/pds.3518
LETTER TO THE EDITOR
Drug risk assessment and data reuse To the Editor We read with interest the commentary by Toh and colleagues1 discussing issues around whether data from the Mini Sentinel (MS) database used to identify safety signals of potential adverse outcomes can be further re-used in formal epidemiologic studies of the identified signals to draw causal inferences. Although we appreciate the thoughtful recommendations, we do not believe that the framework and recommendations as written in the commentary capture the complexity of regulatory drug risk assessment and decision making. Therefore, we thought a timely response to the commentary would provide a useful context for the reader. Data reuse for both hypothesis generation and confirmation in the same database is generally recognized as less than ideal. In theory, using a single dataset, such as the entire MS database, to generate and confirm hypotheses is problematic because the Type I and Type II errors are not easily quantified. Using partitioned data within MS to independently generate and confirm hypotheses or using a database external to MS to confirm would resolve the MS data reuse issue. However, this may not always be a pragmatic option, especially with situations involving low product uptake or extremely rare events as well as in situations where there are no accessible external databases, independent of the one that identified the signal. In these situations, the findings might best be considered as an extension of the initial signal identification/refinement process or as an exploratory analysis. However, in some circumstances, findings from even an exploratory activity, in the context of the totality of available relevant clinical and non-clinical data, may have regulatory implications.2 Regulatory decisions must often be made to provide a timely but carefully calibrated response to safety concerns, taking into consideration all the available data on benefits, risks, and residual uncertainties. Under the framework presented in the commentary, MS activities are divided into three categories based on the strength of prior knowledge or evidence about the suspected exposure–outcome Published 2013. This article is a U.S. Government work and is in the public domain in the USA.
association. The commentary recommends that the initial step in any safety activity should be to determine to which category the activity belongs. We believe it would be challenging to adopt the framework as described. The Sentinel Initiative is designed to augment, but not replace, existing drug risk evaluation process, which is complex, dynamic, and iterative.2 We believe a purpose-driven approach to determine how various streams of evidence are viewed and analyzed should be followed in regulatory decision making. The evidence should be re-evaluated on an ongoing basis as new evidence becomes available. Because of the dynamic nature of the evidence, developing standard definitions for the strength of prior evidence would prove challenging. In addition, relying on the pre-test probability as the main determinant of how and when to use MS seems to diminish the role of other factors in the decision making, such as the seriousness of the adverse event, the number of people potentially affected, and other public health considerations. It is conceivable that there might be scenarios where a confirmatory epidemiological study is undertaken in the absence of “strong” prior evidence that an adverse effect is related to the use of a particular medical product if the public health importance warrants it. CONFLICT OF INTEREST The author declare no conflict of interest. REFERENCES 1. Sengwee Toh, Jerry Avorn, Ralph B. D’Agostino, Jerry H. Gurwitz, Bruce M. Psaty, Kenneth J. Rothman, Kenneth G. Saag, Miriam CJM Sturkenboom, Jan P. Vandenbroucke, Almut G. Winterstein, Brian L. Strom. “Re-using Mini-Sentinel data following rapid assessments of potential safety signals via modular analytic programs”. Pharmacoepidemiology and drug safety 2013. Published online in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/pds.3478. 2. Tarek A. Hammad, George A. Neyarapally, Solomon Iyasu, Judy A. Staffa, Gerald Dal Pan. “The Future of Population-Based Postmarket Drug Risk Assessment: A Regulator’s Perspective”. Clinical Pharmacology & Therapeutics, advance online publication 10 July 2013. doi:10.1038/ clpt.2013.118.
108
letter to the editor
SOLOMON IYASU1* JUDY STAFFA2 DAVE GRAHAM1 AZADEH SHOAIBI3 MARK LEVENSON4 ALOKA CHAKRAVARTY4 TAREK A. HAMMAD5†
and Research, Food and Drug Administration, Silver Spring, MD 20993 4
Division of Biometrics VII, Office of Biostatistics, Office of Translational Science, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, MD 20993
1
Office of Pharmacovigilance and Epidemiology, Office of Surveillance and Epidemiology, Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, MD 20993 2
Division of Epidemiology-2, Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, Food and Drug Administration Silver Spring, MD 20993 3
Office of Medical Policy, Center for Drug Evaluation
Published 2013. This article is a U.S. Government work and is in the public domain in the USA.
5
Epidemiology, Merck Research Laboratories, Merck & Co, Inc. One Merck Drive Whitehouse Station, New Jersey 08889 *
Solomon Iyasu, Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, MD 20993 E-mail: [email protected] †
Dr. Hammad was a Food and Drug Administration employee when the letter was submitted
Pharmacoepidemiology and Drug Safety, 2014; 23: 107–108 DOI: 10.1002/pds
Published online in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/pds.3518
LETTER TO THE EDITOR
Drug risk assessment and data reuse To the Editor We read with interest the commentary by Toh and colleagues1 discussing issues around whether data from the Mini Sentinel (MS) database used to identify safety signals of potential adverse outcomes can be further re-used in formal epidemiologic studies of the identified signals to draw causal inferences. Although we appreciate the thoughtful recommendations, we do not believe that the framework and recommendations as written in the commentary capture the complexity of regulatory drug risk assessment and decision making. Therefore, we thought a timely response to the commentary would provide a useful context for the reader. Data reuse for both hypothesis generation and confirmation in the same database is generally recognized as less than ideal. In theory, using a single dataset, such as the entire MS database, to generate and confirm hypotheses is problematic because the Type I and Type II errors are not easily quantified. Using partitioned data within MS to independently generate and confirm hypotheses or using a database external to MS to confirm would resolve the MS data reuse issue. However, this may not always be a pragmatic option, especially with situations involving low product uptake or extremely rare events as well as in situations where there are no accessible external databases, independent of the one that identified the signal. In these situations, the findings might best be considered as an extension of the initial signal identification/refinement process or as an exploratory analysis. However, in some circumstances, findings from even an exploratory activity, in the context of the totality of available relevant clinical and non-clinical data, may have regulatory implications.2 Regulatory decisions must often be made to provide a timely but carefully calibrated response to safety concerns, taking into consideration all the available data on benefits, risks, and residual uncertainties. Under the framework presented in the commentary, MS activities are divided into three categories based on the strength of prior knowledge or evidence about the suspected exposure–outcome Published 2013. This article is a U.S. Government work and is in the public domain in the USA.
association. The commentary recommends that the initial step in any safety activity should be to determine to which category the activity belongs. We believe it would be challenging to adopt the framework as described. The Sentinel Initiative is designed to augment, but not replace, existing drug risk evaluation process, which is complex, dynamic, and iterative.2 We believe a purpose-driven approach to determine how various streams of evidence are viewed and analyzed should be followed in regulatory decision making. The evidence should be re-evaluated on an ongoing basis as new evidence becomes available. Because of the dynamic nature of the evidence, developing standard definitions for the strength of prior evidence would prove challenging. In addition, relying on the pre-test probability as the main determinant of how and when to use MS seems to diminish the role of other factors in the decision making, such as the seriousness of the adverse event, the number of people potentially affected, and other public health considerations. It is conceivable that there might be scenarios where a confirmatory epidemiological study is undertaken in the absence of “strong” prior evidence that an adverse effect is related to the use of a particular medical product if the public health importance warrants it. CONFLICT OF INTEREST The author declare no conflict of interest. REFERENCES 1. Sengwee Toh, Jerry Avorn, Ralph B. D’Agostino, Jerry H. Gurwitz, Bruce M. Psaty, Kenneth J. Rothman, Kenneth G. Saag, Miriam CJM Sturkenboom, Jan P. Vandenbroucke, Almut G. Winterstein, Brian L. Strom. “Re-using Mini-Sentinel data following rapid assessments of potential safety signals via modular analytic programs”. Pharmacoepidemiology and drug safety 2013. Published online in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/pds.3478. 2. Tarek A. Hammad, George A. Neyarapally, Solomon Iyasu, Judy A. Staffa, Gerald Dal Pan. “The Future of Population-Based Postmarket Drug Risk Assessment: A Regulator’s Perspective”. Clinical Pharmacology & Therapeutics, advance online publication 10 July 2013. doi:10.1038/ clpt.2013.118.
108
letter to the editor
SOLOMON IYASU1* JUDY STAFFA2 DAVE GRAHAM1 AZADEH SHOAIBI3 MARK LEVENSON4 ALOKA CHAKRAVARTY4 TAREK A. HAMMAD5†
and Research, Food and Drug Administration, Silver Spring, MD 20993 4
Division of Biometrics VII, Office of Biostatistics, Office of Translational Science, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, MD 20993
1
Office of Pharmacovigilance and Epidemiology, Office of Surveillance and Epidemiology, Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, MD 20993 2
Division of Epidemiology-2, Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, Food and Drug Administration Silver Spring, MD 20993 3
Office of Medical Policy, Center for Drug Evaluation
Published 2013. This article is a U.S. Government work and is in the public domain in the USA.
5
Epidemiology, Merck Research Laboratories, Merck & Co, Inc. One Merck Drive Whitehouse Station, New Jersey 08889 *
Solomon Iyasu, Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, MD 20993 E-mail: [email protected] †
Dr. Hammad was a Food and Drug Administration employee when the letter was submitted
Pharmacoepidemiology and Drug Safety, 2014; 23: 107–108 DOI: 10.1002/pds
