Wo m e n ’s I m a g i n g • O r i g i n a l R e s e a r c h Henderson et al. Diagnostic Mammography Interpretative Performance

Downloaded from www.ajronline.org by NYU Langone Med Ctr-Sch of Med on 05/31/15 from IP address 128.122.253.212. Copyright ARRS. For personal use only; all rights reserved

Women’s Imaging Original Research

Do Mammographic Technologists Affect Radiologists’ Diagnostic Mammography Interpretative Performance? Louise M. Henderson1 Thad Benefield1 J. Michael Bowling2 Danielle D. Durham 3 Mary W. Marsh1 Bruce F. Schroeder 1,4,5 Bonnie C. Yankaskas1 Henderson LM, Benefield T, Bowling JM, et al.

Keywords: diagnostic mammography, false-positive rate, positive predictive value of biopsy, sensitivity, variability DOI:10.2214/AJR.14.12903 Received March 24, 2014; accepted after revision August 12, 2014. Based on a presentation at the AcademyHealth 2014 annual meeting, San Diego, CA. Supported by grants R01CA155342 and U01CA70040 from the National Cancer Institute (National Institutes of Health). 1 Department of Radiology, The University of North Carolina, CB 7515, Chapel Hill, NC 27599-7515. Address correspondence to L. M. Henderson ([email protected]). 2 Department of Health Behavior, The University of North Carolina, Chapel Hill, NC. 3 Department of Epidemiology, The University of North Carolina, Chapel Hill, NC. 4

Carolina Breast Imaging Specialists, Greenville, NC.

5

Departments of Radiology and Oncology, The Brody School of Medicine at East Carolina University, Greenville, NC. AJR 2015; 204:903–908 0361–803X/15/2044–903 © American Roentgen Ray Society

OBJECTIVE. The purpose of this study was to determine whether the technologist has an effect on the radiologists’ interpretative performance of diagnostic mammography. MATERIALS AND METHODS. Using data from a community-based mammography registry from 1994 to 2009, we identified 162,755 diagnostic mammograms interpreted by 286 radiologists and performed by 303 mammographic technologists. We calculated sensitivity, false-positive rate, and positive predictive value (PPV) of the recommendation for biopsy from mammography for examinations performed (i.e., images acquired) by each mammographic technologist, separately for conventional (film-screen) and digital modalities. We assessed the variability of these performance measures among mammographic technologists, using mixed effects logistic regression and taking into account the clustering of examinations within women, radiologists, and radiology practices. RESULTS. Among the 291 technologists performing conventional examinations, mean sensitivity of the examinations performed was 83.0% (95% CI, 80.8–85.2%), mean falsepositive rate was 8.5% (95% CI, 8.0–9.0%), and mean PPV of the recommendation for biopsy from mammography was 27.1% (95% CI, 24.8–29.4%). For the 45 technologists performing digital examinations, mean sensitivity of the examinations they performed was 79.6% (95% CI, 73.1–86.2%), mean false-positive rate was 8.8% (95% CI, 7.5–10.0%), and mean PPV of the recommendation for biopsy from mammography was 23.6% (95% CI, 18.8–28.4%). We found significant variation by technologist in the sensitivity, false-positive rate, and PPV of the recommendation for biopsy from mammography for conventional but not digital mammography (p < 0.0001 for all three interpretive performance measures). CONCLUSION. Our results suggest that the technologist has an influence on radiologists’ interpretive performance for diagnostic conventional but not digital mammography. Future studies should examine why this difference between modalities exists and determine if similar patterns are observed for screening mammography.

D

iagnostic mammography is defined as those examinations performed for indications other than the screening of asymptomatic women [1, 2]. Understanding the variability in the performance of diagnostic mammography is vital because breast cancer is approximately 10 times more prevalent among women receiving diagnostic as opposed to screening mammography [3, 4]. Prior studies report significant variation in the interpretative performance of diagnostic mammography across radiologists [3]. In particular, higher radiologist performance has been reported for radiologists specializing in breast imaging and those working in a facility with at least one radiologist with

high interpretative volume [5, 6]. Prior studies have also found that radiologists’ performance improves with the availability of clinical symptom information and prior images for comparison [7]. Patient characteristics have not been shown to explain the radiologists’ performance variability [8]. In addition to radiologist and patient factors, variability may derive from factors associated with the radiologic technologists performing the diagnostic mammogram. The radiologists’ interpretive ability may be influenced by the quality of the image, the accuracy of positioning, and the interaction between the radiologist and the technologist. There is scant information examining the influence of technologists on the variability in

AJR:204, April 2015 903

the radiologists’ interpretative performance of diagnostic mammography. The purpose of this study was to determine whether radiologists’ interpretation of diagnostic mammography differs according to the technologist involved in performing the examination, for both conventional (film-screen) and digital modalities. We hypothesize that there is technologist variability associated with performance characteristics, after controlling for the effect of the radiologist. Materials and Methods The data used in this study were collected after a waiver of patient consent. The study was reviewed and approved by the institutional review board at the University of North Carolina at Chapel Hill.

Study Sample We identified all diagnostic mammograms from the Carolina Mammography Registry (CMR) from January 1994 to December 2009 among women 18 years old and older. Diagnostic mammograms were excluded if a woman had a personal history of breast cancer or had breast implants. In addition, we required that each technologist perform an average of 20 diagnostic mammograms per year.

Data Sources and Definitions Prospective data collected from participating CMR mammography practices included characteristics of women, reason for the breast imaging visit, breast cancer risk factors, imaging procedures performed, radiologist findings, assessment and management recommendations, and de-identified codes indicating the radiologist interpreting the mammogram and the mammographic technologist performing the examination. These data

were linked with the North Carolina Central Cancer Registry (NCCCR) and pathology data to allow the calculation of standard performance measures including recall rate, sensitivity, false-positive rate, and positive predictive value (PPV) of the recommendation for biopsy from mammography. Patient data in the descriptive analyses included patient age, race, breast density (based on the BI-RADS density classifications of almost entirely fat, scattered fibroglandular densities, heterogeneously dense, and extremely dense [9]), family history of breast cancer, history of a breast biopsy or surgery, time since last mammogram, and year of the examination. Lack of a response and responses of no were grouped for family history of breast cancer and history of a breast biopsy or surgery. Adhering to convention, we defined diagnostic mammograms as those performed for workup of a problem or symptom. Each mammogram was classified as positive or negative on the basis of the BI-RADS category. Positive mammograms were BI-RADS categories 0, 4, and 5, as well as BI-RADS category 3 when the recommendation was biopsy, fine-needle aspiration (FNA), or surgical consultation. Negative mammograms were BI-RADS categories 1 and 2, as well as 3 when no recommendation was made for biopsy, FNA, or surgical consultation [10]. Among conventional examinations, 26,835 were BI-RADS category 3, and of these, 1398 (5.2%) were recommended to be followed up by biopsy, FNA, or surgical consult and were thus classified as positive. Among the digital examinations, 4165 were BI-RADS category 3, and of these, 53 (1.3%) were recommended to be followed up by biopsy, FNA, or surgical consult and were thus classified as positive. This classification for BI-RADS category 3 is the same as published in studies by the Breast Cancer

50

Data Analysis We computed descriptive statistics for the patient characteristics associated with conventional and digital mammograms. We described the number of diagnostic mammograms performed by each technologist. We calculated the means and 95% CIs for the performance measures of sensitivity, false-positive rate, and PPVof the recommendation for biopsy from mammography. To assess variability in performance measures across technologists, we employed a set of mixed effects logistic regressions, using PROC GLIMMIX (SAS version 9.3, SAS Institute). We modeled the

10 No. of Technologists

40 35 30 25 20

8 6 4 2

15 10

0

5

1 12 24 5– 1 15 49 0– 1 17 74 5– 1 20 99 0– 2 25 49 0– 2 30 99 0– 3 40 99 0– 49 9 ≥ 50 0

–9 9

< 25

25–49 50–99 100–199 200–299 300–399 Mean No. of Digital Diagnostic Examinations Performed per Year

≥ 400

B

10

0–

–7 4

75

50

25

Do mammographic technologists affect radiologists' diagnostic mammography interpretative performance?

The purpose of this study was to determine whether the technologist has an effect on the radiologists' interpretative performance of diagnostic mammog...
816KB Sizes 2 Downloads 12 Views