Correspondence

Verification Bias in Diagnostic Accuracy Studies for Fine-Needle Aspiration Cytology

Reply to Verification Bias in Diagnostic Accuracy Studies for Fine-Needle Aspiration Cytology

I wish to comment on the study by Karunamurthy et al, which was recently published.1 In this study, there were 420 satisfactory fine-needle aspiration (FNA) specimens, 97 of which received histological follow-up. The authors reported the sensitivity and specificity of endobronchial ultrasound-guided FNA based on the 97 samples with histological verification. This type of study design is common in diagnostic accuracy studies for FNA but it leads to a type of bias known as verification bias.2,3 This type of bias typically causes overestimates of sensitivity and underestimates of specificity (predictive values are not affected). Statistical methods have been developed to estimate the bias; however, these methods rest on assumptions that are difficult to verify.4 The majority of published FNA studies suffer from verification bias.5 Authors and reviewers need to be aware of verification bias and its impact on accuracy estimates.

We would like to thank Dr. Schmidt for highlighting the importance of verification bias. We agree that this is a frequent drawback of studies examining sensitivity and specificity based on histological follow-up. The use of histological follow-up usually lowers the number of true-positive cases in endobronchial ultrasoundguided fine-needle aspiration (EBUS-FNA) specimens given that EBUS-FNAs with a positive diagnosis usually do not go on for confirmatory histological sampling. This is in contrast to other types of diagnostic accuracy studies in which a positive screening cytology finding will usually result in a gold-standard procedure, such as surgical resection, and thus more positive cytology cases will usually have histological follow-up than negative cases. For ethical reasons and good clinical practice, verification bias therefore cannot be truly eliminated in this clinical setting because it is often not feasible to perform an invasive mediastinoscopy after a positive EBUS diagnosis in which sufficient cellular material has been collected. In our study,1 there were 420 satisfactory EBUSFNAs, 138 of which (33%) had a negative cytological diagnosis and 216 of which (51%) had a positive cytological diagnosis, but among the 97 histologically verified samples with a positive or negative EBUS-FNA diagnosis, 81 (84%) were initially negative and 16 (16%) were initially positive. Therefore, overall, more negative samples were followed with biopsy, and this verification bias may actually underestimate the study’s sensitivity using histological follow-up.1 In prior, smaller studies that examined the sensitivity and specificity of EBUS-FNA based on cases with histological follow-up, the results were found to be similar, with sensitivity ranging from approximately 69% to 89%.2–5 In addition, the few EBUS studies incorporating clinical follow-up have demonstrated that the sensitivity increases to approximately  95%, given that patients with EBUS-FNAs that were positive for malignancy and never underwent surgery were included as having true-positive results based on clinical and radiological parameters.3,4 We would like to thank Dr. Schmidt for his valuable comments and for highlighting the importance of verification bias.

FUNDING SUPPORT No specific funding was disclosed.

CONFLICT OF INTEREST DISCLOSURES The authors made no disclosures.

REFERENCES 1. Karunamurthy A, Cai G, Dacic S, Khalbuss WE, Pantanowitz L, Monaco SE. Evaluation of endobronchial ultrasound-guided fineneedle aspirations (EBUS-FNA): correlation with adequacy and histologic follow-up. Cancer (Cancer Cytopathol). 2014;122:23-32. 2. Schmidt RL, Jedrzkiewicz JD, Allred RJ, Masuoka S, Witt BL. Verification bias in diagnostic accuracy studies for fine- and core needle biopsy of salivary gland lesions in otolaryngology journals: a systematic review and analysis [published online ahead of print September 30, 2013]. Head Neck. doi: 10.1002/hed.23495. 3. Zhou X-H, Obuchowski N, McLish D. Statistical Methods in Diagnostic Medicine. 2nd ed. Hoboken, NJ: John Wiley and Sons; 2011. 4. Zhou XH. Correcting for verification bias in studies of a diagnostic test’s accuracy. Stat Methods Med Res. 1998;7:337-353. 5. Schmidt RL, Factor RE, Witt BL, Layfield LJ. Quality appraisal of diagnostic accuracy studies in fine-needle aspiration cytology: a survey of risk of bias and comparability. Arch Pathol Lab Med. 2013;137:566-575.

Robert L. Schmidt, MD Department of Pathology University of Utah Salt Lake City, Utah DOI: 10.1002/cncy.21433 Published online April 18, 2014 in Wiley Online Library (wileyonlinelibrary.com)

474

Cancer Cytopathology

June 2014

Correspondence

FUNDING SUPPORT

Arivarasan Karunamurthy, MD

No specific funding was disclosed.

Department of Pathology University of Pittsburgh Medical Center Pittsburgh, Pennsylvania

CONFLICT OF INTEREST DISCLOSURES

Guoping Cai, MD Department of Pathology Yale University School of Medicine New Haven, Connecticut

The authors made no disclosures.

Sanja Dacic, MD, PhD

REFERENCES 1. Karunamurthy A, Cai G, Dacic S, Khalbuss WE, Pantanowitz L, Monaco SE. Evaluation of endobronchial ultrasound-guided fineneedle aspirations (EBUS-FNA): correlation with adequacy and histologic follow-up. Cancer (Cancer Cytopathol). 2014;122:23-32. 2. Alsharif M, Andrade RS, Groth SS, Stelow EB, Pambuccian SE. Endobronchial ultrasound-guided transbronchial fine-needle aspiration: the University of Minnesota experience, with emphasis on usefulness, adequacy assessment, and diagnostic difficulties. Am J Clin Pathol. 2008;130:434-443. 3. Feller-Kopman D, Yung RC, Burroughs F, Li QK. Cytology of endobronchial ultrasound-guided transbronchial needle aspiration: a retrospective study with histology correlation. Cancer. 2009;117:482490. 4. Khazai L, Kundu UR, Jacob B, et al. Endobronchial ultrasound-guided transbronchial needle aspiration biopsy is useful evaluating mediastinal lymphadenopathy in a cancer center. Cytojournal. 2011;8:10. 5. Sun W, Song K, Zervos M, et al. The diagnostic value of endobronchial ultrasound-guided needle biopsy in lung cancer and mediastinal adenopathy. Diagn Cytopathol. 2010;38:337-342.

Cancer Cytopathology

June 2014

Department of Pathology University of Pittsburgh Medical Center Pittsburgh, Pennsylvania

Walid E. Khalbuss, MD, PhD Department of Pathology University of Pittsburgh Medical Center Pittsburgh, Pennsylvania

Liron Pantanowitz, MD Department of Pathology University of Pittsburgh Medical Center Pittsburgh, Pennsylvania

Sara E. Monaco, MD Department of Pathology University of Pittsburgh Medical Center Pittsburgh, Pennsylvania DOI: 10.1002/cncy.21432 Published online April 18, 2014 in Wiley Online Library (wileyonlinelibrary.com)

475

Reply to verification bias in diagnostic accuracy studies for fine-needle aspiration cytology.

Reply to verification bias in diagnostic accuracy studies for fine-needle aspiration cytology. - PDF Download Free
49KB Sizes 0 Downloads 0 Views