Epidemiology  •  Volume 25, Number 5, September 2014 Letters

bias; higher matching ratios had marginally greater residual bias ­(Figure). When the simulated probability of both exposure and outcome was 10.0% reduction in standard error could be observed; with more common exposures and outcomes, improvements in precision with greater matching ratios were negligible. Small improvements in precision with variable ratio matching in extreme circumstances must be weighed against the practical advantages of 1:1 matching, such as transparency, ease of distributed analyses, and ease of assessing covariate balance. Analysis of 1:1 matched cohorts is as straightforward as it gets. Data can be displayed with simple 2 × 2 contingency tables, from which readers can instantly reproduce the multivariable-adjusted analysis. In contrast, a variable-matched cohort must account for the matching ratio “behind the scenes” through conditioning on matched set or number of matches. For a 1:1 matched cohort, a “Table 1” displaying patient characteristics stratified by exposure status is intuitive and readily understandable. With variable matching, the need to account for different ratios of patients within each matched set makes displaying and assessing covariate balance in a concise table more complex. In active monitoring, initially small numbers of exposed and few outcomes are expected. Variable ratio matching results will likely improve precision of estimates over 1:1 matched results. However, after a new product enters the market, there is great potential for unmeasured patient factors channeling treatment choice. The initial small numbers of exposed patients are early adopters, among whom confounding may be particularly strong.1 In this context, it may be prudent to focus on transparent “Table 1’s” from 1:1 matching to assess and compare the characteristics of early adopters to users of an established comparator, allowing confidence intervals to reflect uncertainty of initial estimates, given small numbers of exposed events, and to wait for accrual of additional exposed patients and outcomes rather than attempt to maximize precision of estimates as early as possible. © 2014 Lippincott Williams & Wilkins

Shirley V. Wang Sebastian Schneeweiss Jeremy A. Rassen Division of Pharmacoepidemiology and Pharmacoeconomics Department of Medicine Harvard Medical School Brigham and Women’s Hospital Boston, MA [email protected]

REFERENCES 1. Schneeweiss S, Gagne JJ, Glynn RJ, Ruhl M, Rassen JA. Assessing the comparative effectiveness of newly marketed medications: methodological challenges and implications for drug development. Clin Pharmacol Ther. 2011;90:777–790. 2. Rassen JA, Shelat AA, Myers J, Glynn RJ, Rothman KJ, Schneeweiss S. One-to-many propensity score matching in cohort studies. Pharmacoepidemiol Drug Saf. 2012;21(suppl 2):69–80. 3. Schneeweiss S. A basic study design for expedited safety signal evaluation based on electronic healthcare data. Pharmacoepidemiol Drug Saf. 2010;19:858–868. 4. Austin PC. Statistical criteria for selecting the optimal number of untreated subjects matched to each treated subject when using many-toone matching on the propensity score. Am J Epidemiol. 2010;172:1092–1097. 5. Gagne JJ, Glynn RJ, Rassen JA, et al. Active safety monitoring of newly marketed medications in a distributed data network: application of a semi-automated monitoring system. Clin Pharmacol Ther. 2012;92:80–86. 6. Austin PC. Some methods of propensity-score matching had superior performance to others: results of an empirical investigation and Monte Carlo simulations. Biom J. 2009;51:171–184.

Validating Assisted Reproductive Technology Self-Report To the Editor: he use of fertility-enhancing therapies, including assisted reproductive technologies (ART), has more than

T

R.F.L. and M.A. work in a department that has received funding from the Centers for Disease Control and Prevention to participate in the National Birth Defects Prevention Study. J.R. works at the Centers for Disease Control and Prevention and is a co-investigator for the Copyright © 2014 by Lippincott Williams & Wilkins ISSN: 1044-3983/14/2505-0773 DOI: 10.1097/EDE.0000000000000153

doubled in the United States between 1996 and 2005.1 Approximately 1.4% of US live births and 4.3% of Massachusetts live births are conceived using ART.2 Several studies have noted an increased risk of birth defects in ART pregnancies compared with spontaneous pregnancies.3,4 Previous attempts to validate maternal self-reported ART have shown mixed results. A Danish study demonstrated sensitivity of 83% for assisted conception.5 However, maternal selfreport in the Pregnancy Risk Assessment Monitoring System in the United States overestimated ART use when compared with clinic-reported data.6 We compared self-reported ART use among Massachusetts participants in the National Birth Defects Prevention Study (NBDPS) with data from ART clinics. The NBDPS is a populationbased, multicenter case-control study of birth defects, with exposure information collected by maternal interview.3,7 The Society for Assisted Reproductive Technology (SART) collects ART procedure information from clinics, providing a gold standard for comparison with self-reported information. The SART database contains information from over 91% of ART clinics in the United States, with validation conducted annually.2 All Massachusetts ART clinics report to SART. SART records have been linked to vital records for Massachusetts deliveries as part of a project to evaluate ART outcomes.8 Massachusetts NBDPS participants with in-state deliveries between September 2004 and December 2008 were matched to SART records by delivery date, birth/fetal death certificate number, and birth outcome. Sensitivity and specificity were calculated to measure the accuracy of self-reported ART use among NBDPS participants National Birth Defects Prevention Study. B.L. is a consultant to the Society for Assisted Reproductive Technology, and M.A., B.L., and J.E.S. have received grant funding from the National Institutes of Health. This work was supported, in part, by the Centers for Disease Control and Prevention, grant no. 1U01DD000493.

www.epidem.com  |  773

Epidemiology  •  Volume 25, Number 5, September 2014

Letters

TABLE.  Sensitivity and Specificity of Maternal Self-Reported ART Use Among Massachusetts NBDPS Participants Compared with Clinic Data from SARTa SART

ART Use

NBDPS

IVF ICSIb Donor eggc Frozen cycle

c

Multiple fetuses

Yes No Yes No Yes No Yes No Yes No

Yes (No.)

No (No.)

69 7 15 6 4 2 4 0 41 0

0 0 3 20 0 70 1 71 2 34

Sensitivity (95% CI) (%)

Specificity (95% CI) (%)

91 (82–96)

N/A

71 (48–89)

87 (66–97)

67 (22–96)

100 (95–100)

100 (40–100)

99 (92–100)

100 (91–100)

94 (81–99)

Note: One NBDPS subject who reported IVF and 3 subjects who reported ICSI did not match to SART. If all 4 of these unmatched subjects had IVF, sensitivity would be 91% (95% CI = 83–96%). If none of these subjects had IVF, sensitivity would be 90% (95% CI = 81–96%). If all 3 unmatched subjects who reported ICSI actually had the procedure, ICSI sensitivity would be 75% (95% CI = 53–90%). If none of these 3 subjects had the procedure, ICSI sensitivity would be 67% (95% CI = 41–87%). a Among 77 NBDPS subjects who matched to SART, 1 subject reported “Don’t Know” for fertility procedure and was excluded. This subject was recorded in SART as having had IVF and ICSI. b

Based on 46 NBDPS subjects who delivered between 2006 and 2008 and matched to SART, 2 subjects were missing ICSI information in SART and were therefore excluded from this analysis. c One NBDPS subject reported “Don’t Know” for fertility procedure and was excluded. This subject was recorded in SART as having had no donor egg or frozen cycle.

774  |  www.epidem.com

ACKNOWLEDGMENTS We acknowledge Daksha Gopal for her assistance with this project. Rebecca F. Liberman Massachusetts Department of Public Health Boston, MA [email protected]

Judy E. Stern Geisel School of Medicine at Dartmouth Lebanon, NH

Barbara Luke Michigan State University East Lansing, MI

CI denotes confidence interval.

compared with ART use recorded in SART. Variables compared include in vitro fertilization (IVF), intracytoplasmic sperm injection (ICSI), use of donor eggs, frozen eggs or embryos (frozen cycle), and presence of multiple fetuses on ultrasound. The NBDPS did not collect ICSI information until 2006, so these analyses are restricted to 2006– 2008 deliveries. Of 1452 Massachusetts NBDPS participants with in-state deliveries during the study period, 77 (5.3%) matched to a record in the SART database. Four NBDPS subjects who reported IVF or ICSI did not match to a SART record, possibly because of misreporting or because ART had been performed out-of-state. Among NBDPS subjects with matching SART records, specificity was 87% or greater for all procedures and outcomes examined (Table). Sensitivity was 91% for IVF use and did not differ by case-control status. Sensitivity was 100% for use of frozen cycle and

Among deliveries to NBDPS subjects with matching SART records, self-reported use of IVF and several other ART procedures and outcomes demonstrated good agreement with clinic data, while use of ICSI and donor eggs were underreported. Future studies need to validate ART exposure with larger sample sizes, including subjects from other states. These results increase confidence in observed associations between ART and birth defects3 in the NBDPS.

presence of multiple fetuses. Sensitivity was lower for ICSI (71%) and donor egg use (67%), although the number of donor egg cycles was small. Sensitivity for ICSI use was 82% among 20 subjects interviewed at