Original article
Computer-assisted interpretation of planar whole-body bone scintigraphy in patients with newly diagnosed prostate cancer Lars J. Petersena,b, Jesper C. Mortensenc, Henrik Bertelsena,d and Helle D. Zachoa,e Purpose The aim of this study was to compare the diagnostic properties of EXINI BoneBSI in newly diagnosed prostate cancer in comparison with expert reading. Materials and methods Bone scintigraphy was performed in consecutive patients referred for staging at three clinics (342 patients with DICOM file format, 272 with Interfile format). Images were reported by three independent readers on a four-point scale (class 1–4) and by using a dichotomous outcome (M1 or M0). The software analyzed data in balanced mode, as well as using ‘patient-specific’ settings (based on tumor characteristics), and classified outcome as normal (N), probably normal (pN), probably abnormal (pA), and abnormal (A). Results Classification of bone metastasis using the software (pA + A) versus experts (class 3 + 4) showed a sensitivity of 93.3%, specificity of 89.3%, positive predictive value of 57.5%, and negative predictive value of 98.9% with DICOM files. The diagnostic properties of the software were notably different with Interfile format. For example, expert M1 versus software A showed a sensitivity of 90.0%, specificity of 98.9%, positive predictive value of 88.2%, and negative predictive value of 98.3% with DICOM files, versus 69.2, 88.2, 38.3, and 96.4% with Interfile format, respectively. Generally, patient-specific settings did not influence the
Introduction Prostate cancer is the most common cancer in European men [1]. Despite the introduction of novel imaging modalities such as single-photon emission computerized tomography/computed tomography (SPECT/CT) [2], 18 F-fluoride PET/CT, and 18F-choline or 11C-choline PET/CT [3], the most recent American [4] and European [5] urology guidelines still recommend planar bone scintigraphy for the detection of osseous metastatic disease in the staging of prostate cancer. The interpretation of imaging is a subjective task, and it has been described as the weakest aspect of clinical imaging [6]. Nevertheless, studies on observer agreement of bone scans are few. In a nationwide survey from Sweden, Sadik et al. [7] reported moderate agreement among readers using a four-point outcome scale. In contrast, Ore et al. [8] reported good to excellent Presented as an abstract (P482) at the EAMN 2014 (http://eanm14.eanm.org/ abstracts/abstract_sea) 0143-3636 Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
diagnostic characteristics of the software versus balanced setting with expert reading as reference. Conclusion EXINI BoneBSI showed high sensitivity and specificity for bone metastasis in patients with newly diagnosed prostate cancer. The software ruled out metastasis with confidence, whereas the positive predictive value was modest. The diagnostic properties were different for DICOM and Interfile file formats. Nucl Med Commun 36:679–685 Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved. Nuclear Medicine Communications 2015, 36:679–685 Keywords: bone metastasis, computer-aided analysis, diagnostic properties, predictive values, prostate cancer, sensitivity, specificity a Department of Nuclear Medicine, Clinical Cancer Research Center, Aalborg University Hospital, bDepartment of Clinical Medicine, Aalborg University, Aalborg, c Department of Nuclear Medicine, Regional Hospital West Jutland, Herning, d Department of Clinical Physiology and Nuclear Medicine, Randers Hospital, Randers and eDepartment of Clinical Physiology, Viborg Hospital, Viborg, Denmark
Correspondence to Lars J. Petersen, MD, DMSc, Department of Nuclear Medicine, Aalborg University Hospital, Hobrovej 18-22, DK-9000 Aalborg, Denmark Tel: + 45 61 60 59 51; fax: + 45 97 66 55 01; e-mail: [email protected] Received 30 October 2014 Revised 28 January 2015 Accepted 31 January 2015
agreement for a dichotomous reporting of metastasis or benign findings. Interpretation of bone scans may be influenced by the observer’s knowledge of prostate specific antigen (PSA)-value, Gleason or T-stage, as well as clinical expertise. Computer-aided assessment and diagnosis of bone scans may be valuable to improve reading speed, as well as reading accuracy, in clinical practice. EXINI BoneBSI is one such system with solid documentation [9–12]. The validation of computer-assisted software has mostly been performed in mixed patient populations with different types of cancer, in patients with a high prevalence for metastatic disease, in retrospective settings, and with limited sample sizes. We have recently published a large series of nearly 700 patients with newly diagnosed prostate cancer enrolled in a prospective study [13]. The primary aim of the present study was to determine the diagnostic properties of computer-assisted interpretation of planar bone scans versus reading by experienced DOI: 10.1097/MNM.0000000000000307
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
680
Nuclear Medicine Communications 2015, Vol 36 No 7
readers in a large population of consecutive patients with newly diagnosed prostate cancer.
Materials and methods Study design and patients
The study was performed at three hospital units in Central Region Jutland, Denmark. The study included all patients with newly diagnosed prostate cancer referred routinely for bone imaging from March 2008 to October 2009. The patients were included in a noninterventional study [13]. Eligibility criteria were as follows: (a) prostate cancer diagnosed within 3 months from referral; (b) no prior or concurrent cancers except nonmelanoma skin cancer; and (c) initiation of any anticancer therapies for more than 1 week before the time of bone scintigraphy. Bone scintigraphy
Whole-body planar 99mTc-bone scintigraphy was acquired according to institutional practice [13]. The participating institutions all complied with the guidelines on bone scintigraphy by the European Society of Nuclear Medicine [14]. Expert reading of bone scintigraphy
Once all patients had been recruited, the images were read under standardized conditions. All bone scans were read simultaneously and independently by three boardcertified nuclear medicine physicians (L.J.P., J.C.M., H. C.B.) at the three centers. The readers have 10–15 years of experience with bone scans. An operator loaded the images on high-quality monitors and adjusted the display if required by the readers. The readings were performed without any clinical information. The readers graded each bone scan into one of four categories (hereafter entitled expert class): (a) normal bone scan or findings most likely to represent benign conditions; (b) equivocal findings; (c) changes most likely to be malignant; and (d) multiple metastases. In addition, each reader graded each bone scintigraphy on a dichotomous scale as positive (M1) or negative (M0) for bone metastasis. The median values for both types of categories were used in the analyses. Computer-assisted analysis of bone scintigraphy
Commercially available software (EXINI BoneBSI, version 1.6.2; EXINI Diagnostics AB, Lund, Sweden) was used. The program has full CE approval and FDA 510k clearance. A study contract was signed providing all rights of the data to the investigators, including the right to publish the findings. In comparison with most other publications with this software, no company representatives were coauthors. However, the company has the right to review and comment on all publications and presentations from the project. There was no payment between the parts. Medical/technical representatives from the company assisted with the installation and confirmed correct use with a test sample. The import file
format was Interfile 3.3 at one site (West Jutland Hospital, Herning, Denmark) and DICOM format at the other sites. EXINI BoneBSI is computer-assisted software which analyses bone scans by means of a patented algorithm for the presence and numbers of pathological lesions, characterization of lesions as malignant or benign, and finally providing diagnostic advice and quantification of results. The software then transforms lesion data into a numeric value from 0 to 1 (‘Patient ANN Outcome’, ANN meaning artificial neural network). These values were then categorized into classes of values from 0–0.25, 0.25–0.50, 0.50–0.75, and 0.75–1.0, and then converted automatically by the software into four separate patient categories: (a) normal (N); (b) probably normal (pN); (c) probably abnormal (pA); and (d) abnormal (A). The latter classification was defined prior to this study as the primary endpoint in the statistical analysis plan. The analysis was made using ‘balanced mode’ (see below).
Patient-specific settings
A feature of EXINI BoneBSI was the ability to select settings in terms of probability of bone metastasis – that is, ‘balanced’, ‘high specificity’, and ‘high sensitivity’. Diagnostic outcome was calculated based on individual risk in accordance with the European Association of Urology (EAU) risk classification [5]. Clinical T-stage subgroups were not available. Thus, patients were classified into (a) low-risk prostate cancer (if T1–T2, Gleason 2–6, and PSA < 10 ng/ml), (b) intermediate-risk prostate cancer (if T1–T2, Gleason 7 or PSA 10–20 ng/ml), (c) high-risk prostate cancer (if T3 or Gleason score 8–10 or PSA > 20 ng/ml), or (d) very high risk prostate cancer (if T4 or any T, N1). Patients with low-risk prostate cancer were analyzed using the setting ‘high specificity’, whereas patients with high or very high risk were analyzed using the setting ‘high sensitivity’. Patients with intermediate risk were analyzed in the ‘balanced’ setting.
Statistical analysis
Descriptive statistics were reported as mean ± SEM in the case of normal distributed data and median with total range for nonparametric datasets. Sensitivity, specificity, and positive and negative predictive values (PPV, NPV) were calculated. Analyses were performed using Stata 13 (StataCorp LP, College Station, Texas, USA).
Approvals
Because of the noninterventional nature, the study did not require ethical approval in accordance with national legislation [13]. This was confirmed by the local ethics committee. The Danish National Board of Health provided a waiver for informed consent for access to medical information from patient files. The study was approved by the Danish Data Protection Agency.
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
Computer-aided diagnosis of bone scans Petersen et al. 681
Results Clinical data
A total of 673 Caucasian patients were recruited. A total of 36 patients did not meet the eligibility criteria [13]. In addition, whole-body files were not available from 23 patients. Thus, a total of 614 patients were included in the final analysis. Their median PSA was 15.6 ng/ml (range 1.5–15 357), median Gleason score was 7, and the vast majority of patients had limited local disease (T1 or T2). Expert reading
The proportion of expert class 1–4 bone scintigraphy was 56.2, 33.0, 3.7, and 7.0%, respectively. The proportion of likely or evident bone metastasis (expert class 3 and 4) was 10.7% and varied among centers from 7.7 to 17.9%. For the dichotomous classification, bone metastasis (M1) was present in 12.4% of the patients. The agreement in classes among the three readers, within the margin of one expert class, was 99% (with unanimous agreement in 67% of the cases). Unanimous agreement in the dichotomous outcome was observed in 95% of the cases. Removal of artifacts
The software showed nonlesional activity in 113 of 614 cases (18.4%). Most artifacts were anatomical structures classified as lesions, such as bladder and renal pelvis and urinary devices. Removal of artifacts did not change patient category in most patients (76/113 cases, 67.3%) and down-staged all remaining patients but one. After removal of artifacts, the distribution of patient categories was as follows: N 67.6%, pN 8.3%, pA 8.1%, and A 16.0%. File format
The cumulated proportion of patients reported as expert class 1 and software N, class 2 and pN, class 3 and pA, and expert and A was 57–60% at the two sites with DICOM files and 49% at the site using Interfile format. The proportion of software-reported likely or definitive bone metastasis (pA + A) varied from 20.4 to 23.1% at the two DICOM sites and 27.6% at the Interfile site. In contrast, the corresponding values of likely or definitive bone metastasis by expert reading (class 3 + 4) were 10.7–19.7% and 7.7%. The discrepancy between software and expert reporting was most evident at the center using Interfile file format. The software upgraded patients compared with expert class readings in 12.8–18.6% of the cases at the two centers using DICOM format versus 21.3% at the site using Interfile format. Data-driven analysis showed a significant difference in the proportion of upgraded patients from the Interfile site versus the two DICOM sites (Fisher’s exact test, P = 0.0064 and 0.0005), but there were no significant differences among the two DICOM sites (P = 0.22).
The comparison of EXINI BoneBSI patient categories versus the dichotomous expert classification (M1 vs. M0) is shown in Fig. 1. It appeared that software confirmed the absence (N) or presence (A) of bone metastasis (M0 or M1), as reported by experts in 84–90% of cases with DICOM files. The corresponding figures were 67–69% with Interfile format. Apparent disagreement was rare with DICOM files: expert M0 versus software A was observed in 2.0–2.1% of patients and M1 versus N in 3.6–4.7% of patients. In comparison, such disagreement was evident in 11.8% and 11.5% of cases at the site using Interfile format. Because of clear differences in the diagnostic performance of EXINI BoneBSI with data imported by DICOM and Interfile 3.3 formats, data were reported separately by file format in the subsequent sections. Data from the two sites using DICOM format are cumulated for such analyses. Diagnostic characteristics
The diagnostic properties of the four EXINI BoneBSI outcome classes versus expert reading by four classes or dichotomous outcome are shown in Table 1. Since the four classes by software and readers were not directly comparable, comparison of expert class 4 versus software A, expert class 3 + 4 versus A, and expert class 3 + 4 versus pA + A was made. Similarly, M1 by readers were compared with EXINI BoneBSI class A as well as pA + A. The sensitivity of the four class patient outcome by EXINI BoneBSI (DICOM data) was 91–100%, specificity was 90–97%, PPV was 55–80%, and NPV was 97–100%. Using expert reading as M1 or M0 as reference, the sensitivity was 90–92%, specificity was 91–98%, PPV was 63–88%, and NPP was 98–99%. Thus, the diagnostic performance varied with the prevalence of disease, as well as the expert reading outcome classification. The diagnostic performance of EXINI BoneBSI based on Interfile 3.3 data showed somewhat different outcome (Table 1). The sensitivity was notably lower than that with DICOM files, the specificity was marginally lower, the NPVs were very similar to data obtained with DICOM files, but the PPV was notably lower. Patient-specific settings
A total of 604 of 614 patients had data for PSA, Gleason, and T-stage and were included in the analysis based on the EAU risk classification. The use of patient-specific outcome upstaged in 34 (9.9%) and 30 (11.0%) patients with DICOM and Interfile formats, as well as downstaged in two (0.6%) and six (2.2%) patients. The distribution of patient outcome in balanced mode versus using patient-specific outcome for all patients is shown in Fig. 2. The use of individual settings did not influence patient outcome in the N and A classes (95–100% consistency with results with balanced mode) but showed
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
682 Nuclear Medicine Communications 2015, Vol 36 No 7
Fig. 1
100
(b) 100
80
80
60
60
40
40
20
20
%
(a)
0
0 N
pN
pA
A
%
(c) 100
(d) 100
80
80
60
60
40
40
20
20
N
pN
pA
A
N
pN
pA
A
0
0 N
pN
pA
A
Distribution of EXINI BoneBSI patient categories versus expert reading. The left panels (a, c) represent the relative distribution of software categories among patients classified as nonmalignant (M0) by experts (open bars), whereas the right panels (b, d) represent data for patients reported to have metastatic disease (M1) by expert readers (closed bars). The upper panels (a, b) represent data from patients analyzed with DICOM files (n = 342), and the lower panels (c, d) data with Interfile files (n = 272). The abbreviations ‘N’,‘pN’,‘pA’, and ‘A’ denote normal, probably normal, probably abnormal, and abnormal, respectively.
Table 1
Diagnostic properties of EXINI BoneBSI (using balanced settings) versus expert reading class and dichotomous reading EXINI boneBSI Bone metastasis
Expert reading Bone metastasis Present DICOM format Class 4 Class 3 + 4 Class 3 + 4 M1 M1 Interfile format Class 4 Class 3 + 4 Class 3 + 4 M1 M1
Absent
Present
Absent
Sensitivity (%)
Specificity (%)
PPV (%)
NPV (%)
Prevalence (%)a
Class 1–3 Class 1 + 2 Class 1 + 2 M0 M0
A A A + pA A A + pA
N, pN, pA N, pA, pN N + pN N, pN, pA N + pN
100.0 91.1 93.3 90.0 92.0
92.7 96.6 89.6 98.0 90.8
54.9 80.4 57.5 88.2 63.0
100.0 98.6 98.9 98.3 98.5
8.2 13.2 13.2 14.6 14.6
Class 1–3 Class 1 + 2 Class 1 + 2 M0 M0
A A A + pA A A + pA
N, pN, pA N, pA, pN N + pN N, pN, pA N + pN
80.0 71.4 81.0 69.2 80.8
86.4 87.3 76.9 88.2 78.0
25.5 31.9 22.7 38.3 28.0
98.7 97.3 98.0 96.4 97.5
5.5 7.7 7.7 9.6 9.6
A, abnormal; DICOM, Digital Imaging and Communications in Medicine; EXINI categories: N, normal; pA, probably abnormal; pN, probably normal. a Proportion of patients with bone metastasis by expert reading using the specified criteria as a percentage of the total study population.
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
Computer-aided diagnosis of bone scans Petersen et al. 683
Discussion
Fig. 2
100 Patient-specific
%
N
pN
pA
A
50
0 N
pN
pA
A
Balanced setting classification Relative change in the distribution of EXINI BoneBSI patient categories by patient-specific settings versus balanced settings. The majority of normal reports (N, filled bars), as well as abnormal reports (A, open bars), using balanced settings remained unchanged by the use of patient-specific settings. In comparison, a notable proportion of patients were upstaged in the intermediate categories with probably normal repots (pN, dark gray bars) and those with probably abnormal reports (pA, light gray bars). A total of 604 patients were analyzed.
notable changes in the pN and pA classes with upstaging in a large proportion of patients. The diagnostic properties of patient-specific settings are shown in Table 2. The use of individualized settings versus balanced settings generally improved sensitivity (0–4.6% improvement) but decreased specificity (reduction − 0.5 to 5.6%). There was no major difference among file formats for the change of sensitivity and specificity by settings. In comparison, a change from balanced to individual settings reduced the PPV by 7.0–21.0% with DICOM format and by 1.3–9.0% with Interfile format. There were no changes in the values of NPV with specific settings, irrespective of file format (all changes less than 1% change). Table 2
Planar bone scintigraphy remains the guidelinerecommended method for staging of prostate cancer patients. In addition, bone scans are routinely used for the determination of bone metastasis in recurrent disease and for monitoring of treatment control in castrationresistant metastatic disease. Thus, bone scintigraphy remains one of the pivotal but also time-consuming nuclear medicine examinations in daily practice. Implementation of software that could assist the interpretation of bone scintigraphy could be of great help to both untrained and trained observers. Our results showed that EXINI BoneBSI has a high sensitivity and specificity for identification of bone metastasis in a large cohort of newly diagnosed prostate cancer patients. The software ruled out bone metastasis with a high predictive power. The PPVs were moderate to good, suggesting that a positive EXINI BoneBSI scan should be handled with caution. Reading of the initial EXINI BoneBSI scan reports indicated an issue with false identification of bone lesions, lesions which trained observers classified as nonbony lesions. The software identified nonbony lesions such as dilated pelvis, bladder activity, or urinary devices as pathological findings in nearly 20% of the patients. The proportion of artifacts in our study is far greater than the 1–2% reported in the initial study by Sadik et al. [9]. The occurrence and handling of artifacts have not been described in most recent studies. However, such artifacts were easily identified and removed without change of the software classification. The user manual of the EXINI BoneBSI software claimed the use of both Interfile and DICOM formats. However, our results strongly indicated differential diagnostic properties of data captured by the two file formats. To ensure full compliance with the software, medical and technical representatives from EXINI Diagnostics assisted with installation and sample verification of data at
Diagnostic properties of EXINI BoneBSI versus expert readings for 604 patients using patient-specific settings EXINI boneBSI Bone metastasis
Expert reading Bone metastasis Present DICOM format Class 4 Class 3 + 4 Class 3 + 4 M1 M1 Interfile format Class 4 Class 3 + 4 Class 3 + 4 M1 M1
Absent
Present
Absent
Sensitivity (%)
Specificity (%)
PPV (%)
NPV (%
Prevalence (%)a
Class 1–3 Class 1 + 2 Class 1 + 2 M0 M0
A A A + pA A A + pA
N, pN, pA N, pA, pN N + pN N, pN, pA N + pN
100.0 93.2 97.7 91.8 95.9
97.1 91.1 86.0 92.3 87.1
40.3 61.2 51.2 67.2 56.0
100.0 98.9 99.6 98.5 99.2
8.0 13.1 13.1 14.6 14.6
Class 1–3 Class 1 + 2 Class 1 + 2 M0 M0
A A A + pA A A + pA
N, pN, pA N, pA, pN N + pN N, pN, pA N + pN
84.6 73.7 84.2 70.8 83.3
81.6 82.3 76.3 83.2 77.5
19.0 24.1 21.3 29.3 26.7
99.0 97.6 98.4 96.7 97.9
4.9 7.1 7.1 9.0 9.0
A, abnormal; DICOM, Digital Imaging and Communications in Medicine; EXINI categories: N, normal; pA, probably abnormal; pN, probably normal. a Proportion of patients with disease present by expert reading as a percentage of the total study population.
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
684
Nuclear Medicine Communications 2015, Vol 36 No 7
each test site. A statistically significant larger proportion of patients were classified with bone metastasis with Interfile format with the software versus expert reading, compared with data from DICOM files. This was evident with both four-class expert class reporting and dichotomous expert reading. The analysis of the diagnostic properties showed much lower PPV of EXINI BoneBSI with Interfile format than with DICOM format, having lower sensitivity and specificity but comparable NPV. The importance of file formats on the diagnostic outcome with the computer-aided software has not previously been reported. We have identified similar findings of file format on lesion analysis (e.g. the number of malignant lesions and the bone scan index, including the association between these variables) from the same data (Petersen et al., unpublished observations). However, it has been shown that a different reference population must be used for the Caucasian and Asian populations [15,16]. The company is aware of the file issue, and the user manual has been updated with instructions only to use images in DICOM format (EXINI AB, personal communication, January 2015). The file format issue is relevant when comparing data among sites and in serial scans if different file formats are used. The diagnostic properties of the computer-aided analysis program showed sensitivities and specificities of 90–100% with DICOM format files and 69–88% with Interfile formats. The high sensitivities and specificities are in line with previous reports [9,10,12,17]. Most studies with EXINI BoneBSI have been performed in patients with metastatic disease in whom the prevalence of metastasis varied from 15 to 40% [10,12,15]. In clinical practice, the true reference is seldom known and the PPV and NPVs are of major importance. The PPVs were generally modest with DICOM format data and poor with Interfile format data. The NPVs were excellent, irrespective of file format. In line with conventions, the predictive values, but not sensitivity and specificity, are sensitive to the prevalence of disease in the study population. Thus, the diagnostic properties therefore likely differ in newly diagnosed patients, especially when PSA screening is used, versus in patients with recurrent disease. Most prior studies have not reported predictive values [9,10,15]. Tokuda and colleagues published NPV very similar to our data (98–100% in prostate cancer). However, our PPV of 55–89% with DICOM files was notably higher than the value of 51% observed by Tokuda and colleagues in prostate cancer patients with a prevalence of metastasis of nearly 40%. They presented great differences in PPV in different anatomical regions in prostate cancer and much lower PPV (∼10%) in all other types of cancer in men and women than in prostate cancer. The tested version of the software had options to select settings from high sensitivity over balanced to high specificity. On the basis of patient-specific information, we
performed an analysis of patient-specific settings. Patients with a high risk for bone metastasis were analyzed with high sensitivity and patients with low risk with high specificity. The results changed patient outcome class in ∼ 10% of the patients, mainly upstaging, and predominantly in patients with indeterminate software results (pN and pA). The use of patient-specific settings slightly increased sensitivity and decreased specificity. However, this setting notably decreased PPV, with no effect on NPV versus balanced settings. We have not been able to identify any prior studies investigating the impact on patient settings on the diagnostic outcome with this software. Risk allocation was performed using the EAU risk classification. It can be debated if other methods for selection of patient-specific settings are superior to EAU criteria. However, these criteria are quite similar to risk classifications by other main cancer or urological societies or organizations. In conclusion, the results indicated no major diagnostic improvement of using patient-specific settings versus balanced settings in a large cohort of consecutive patients. This study examined the diagnostic performance of EXINI BoneBSI in an unselected population of patients with newly diagnosed prostate cancer. The software showed high sensitivity and specificity and very high NPVs. As such, the software may be used in clinical practice to identify patients with a low probability for metastatic disease, so emphasis can be dedicated to patients with potential metastatic disease. However, the PPVs were modest and indicated that a positive EXINI BoneBSI scan result should be examined with care before classifying a patient as having metastatic disease. The overall findings and recommendations for use of the software were similar with Interfile 3.3 and DICOM files. However, users should be aware of the file issue when comparing data analyzed with different file formats (e.g. between institutions or over time).
Acknowledgements EXINI Diagnostics AB, Lund, Sweden, kindly provided the software for the purpose of this study. Conflicts of interest
The authors declare no conflict of interest with regards to the present work. Lars J. Petersen has received a speaker's honorarium from Janssen Pharma and AstraZeneca and consulting fees from LJP Medical and the KLIFO Drug Developing Council.
References 1 2
3
Damber JE, Aus G. Prostate cancer. Lancet 2008; 371:1710–1721. Palmedo H, Marx C, Ebert A, Kreft B, Ko Y, Turler A, et al. Whole-body SPECT/CT for bone scintigraphy: diagnostic value and effect on patient management in oncological patients. Eur J Nucl Med Mol Imaging 2014; 41:59–67. Wondergem M, van der Zant FM, van der Ploeg T, Knol RJ. A literature review of 18F-fluoride PET/CT and 18F-choline or 11C-choline PET/CT for detection
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
Computer-aided diagnosis of bone scans Petersen et al. 685
4
5
6 7
8 9
10
11
of bone metastases in patients with prostate cancer. Nucl Med Commun 2013; 34:935–945. Mohler JL, Kantoff PW, Armstrong AJ, Bahnson RR, Cohen M, D’Amico AV, et al. Prostate cancer, version 2.2014. J Natl Compr Canc Netw 2014; 12:686–718. Heidenreich A, Bellmunt J, Bolla M, Joniau S, Mason M, Matveev V, et al. EAU guidelines on prostate cancer. Part 1: screening, diagnosis, and treatment of clinically localised disease. Eur Urol 2011; 59:61–71. Robinson PJ. Radiology’s Achilles’ heel: error and variation in the interpretation of the Rontgen image. Br J Radiol 1997; 70:1085–1098. Sadik M, Suurkula M, Hoglund P, Jarund A, Edenbrandt L. Quality of planar whole-body bone scan interpretations – a nationwide survey. Eur J Nucl Med Mol Imaging 2008; 35:1464–1472. Ore L, Hardoff R, Gips S, Tamir A, Epstein L. Observer variation in the interpretation of bone scintigraphy. J Clin Epidemiol 1996; 49:67–71. Sadik M, Jakobsson D, Olofsson F, Ohlsson M, Suurkula M, Edenbrandt L. A new computer-based decision-support system for the interpretation of bone scans. Nucl Med Commun 2006; 27:417–423. Sadik M, Hamadeh I, Nordblom P, Suurkula M, Hoglund P, Ohlsson M, et al. Computer-assisted interpretation of planar whole-body bone scans. J Nucl Med 2008; 49:1958–1965. Sadik M, Suurkula M, Hoglund P, Jarund A, Edenbrandt L. Improved classifications of planar whole-body bone scans using a computer-assisted
12
13
14
15
16
17
diagnosis system: a multicenter, multiple-reader, multiple-case study. J Nucl Med 2009; 50:368–375. Tokuda O, Harada Y, Ohishi Y, Matsunaga N, Edenbrandt L. Investigation of computer-aided diagnosis system for bone scans: a retrospective analysis in 406 patients. Ann Nucl Med 2014; 28:329–339. Zacho HD, Barsi T, Mortensen JC, Mogensen MK, Bertelsen H, Josephsen N, et al. Prospective multicenter study of bone scintigraphy in consecutive patients with newly diagnosed prostate cancer. Clin Nucl Med 2014; 39:26–31. Bombardieri E, Aktolun C, Baum RP, Bishof-Delaloye A, Buscombe J, Chatal JF, et al. Bone scintigraphy: procedure guidelines for tumour imaging. Eur J Nucl Med Mol Imaging 2003; 30:BP99–106. Horikoshi H, Kikuchi A, Onoguchi M, Sjostrand K, Edenbrandt L. Computeraided diagnosis system for bone scintigrams from Japanese patients: importance of training database. Ann Nucl Med 2012; 26:622–626. Kikuchi A, Onoguchi M, Horikoshi H, Sjostrand K, Edenbrandt L. Automated segmentation of the skeleton in whole-body bone scans: influence of difference in atlas. Nucl Med Commun 2012; 33:947–953. Kaboteh R, Gjertsson P, Leek H, Lomsky M, Ohlsson M, Sjostrand K, et al. Progression of bone metastases in patients with prostate cancer – automated detection of new lesions and calculation of bone scan index. EJNMMI Res 2013; 3:64.
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
Computer-assisted interpretation of planar whole-body bone scintigraphy in patients with newly diagnosed prostate cancer Lars J. Petersena,b, Jesper C. Mortensenc, Henrik Bertelsena,d and Helle D. Zachoa,e Purpose The aim of this study was to compare the diagnostic properties of EXINI BoneBSI in newly diagnosed prostate cancer in comparison with expert reading. Materials and methods Bone scintigraphy was performed in consecutive patients referred for staging at three clinics (342 patients with DICOM file format, 272 with Interfile format). Images were reported by three independent readers on a four-point scale (class 1–4) and by using a dichotomous outcome (M1 or M0). The software analyzed data in balanced mode, as well as using ‘patient-specific’ settings (based on tumor characteristics), and classified outcome as normal (N), probably normal (pN), probably abnormal (pA), and abnormal (A). Results Classification of bone metastasis using the software (pA + A) versus experts (class 3 + 4) showed a sensitivity of 93.3%, specificity of 89.3%, positive predictive value of 57.5%, and negative predictive value of 98.9% with DICOM files. The diagnostic properties of the software were notably different with Interfile format. For example, expert M1 versus software A showed a sensitivity of 90.0%, specificity of 98.9%, positive predictive value of 88.2%, and negative predictive value of 98.3% with DICOM files, versus 69.2, 88.2, 38.3, and 96.4% with Interfile format, respectively. Generally, patient-specific settings did not influence the
Introduction Prostate cancer is the most common cancer in European men [1]. Despite the introduction of novel imaging modalities such as single-photon emission computerized tomography/computed tomography (SPECT/CT) [2], 18 F-fluoride PET/CT, and 18F-choline or 11C-choline PET/CT [3], the most recent American [4] and European [5] urology guidelines still recommend planar bone scintigraphy for the detection of osseous metastatic disease in the staging of prostate cancer. The interpretation of imaging is a subjective task, and it has been described as the weakest aspect of clinical imaging [6]. Nevertheless, studies on observer agreement of bone scans are few. In a nationwide survey from Sweden, Sadik et al. [7] reported moderate agreement among readers using a four-point outcome scale. In contrast, Ore et al. [8] reported good to excellent Presented as an abstract (P482) at the EAMN 2014 (http://eanm14.eanm.org/ abstracts/abstract_sea) 0143-3636 Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
diagnostic characteristics of the software versus balanced setting with expert reading as reference. Conclusion EXINI BoneBSI showed high sensitivity and specificity for bone metastasis in patients with newly diagnosed prostate cancer. The software ruled out metastasis with confidence, whereas the positive predictive value was modest. The diagnostic properties were different for DICOM and Interfile file formats. Nucl Med Commun 36:679–685 Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved. Nuclear Medicine Communications 2015, 36:679–685 Keywords: bone metastasis, computer-aided analysis, diagnostic properties, predictive values, prostate cancer, sensitivity, specificity a Department of Nuclear Medicine, Clinical Cancer Research Center, Aalborg University Hospital, bDepartment of Clinical Medicine, Aalborg University, Aalborg, c Department of Nuclear Medicine, Regional Hospital West Jutland, Herning, d Department of Clinical Physiology and Nuclear Medicine, Randers Hospital, Randers and eDepartment of Clinical Physiology, Viborg Hospital, Viborg, Denmark
Correspondence to Lars J. Petersen, MD, DMSc, Department of Nuclear Medicine, Aalborg University Hospital, Hobrovej 18-22, DK-9000 Aalborg, Denmark Tel: + 45 61 60 59 51; fax: + 45 97 66 55 01; e-mail: [email protected] Received 30 October 2014 Revised 28 January 2015 Accepted 31 January 2015
agreement for a dichotomous reporting of metastasis or benign findings. Interpretation of bone scans may be influenced by the observer’s knowledge of prostate specific antigen (PSA)-value, Gleason or T-stage, as well as clinical expertise. Computer-aided assessment and diagnosis of bone scans may be valuable to improve reading speed, as well as reading accuracy, in clinical practice. EXINI BoneBSI is one such system with solid documentation [9–12]. The validation of computer-assisted software has mostly been performed in mixed patient populations with different types of cancer, in patients with a high prevalence for metastatic disease, in retrospective settings, and with limited sample sizes. We have recently published a large series of nearly 700 patients with newly diagnosed prostate cancer enrolled in a prospective study [13]. The primary aim of the present study was to determine the diagnostic properties of computer-assisted interpretation of planar bone scans versus reading by experienced DOI: 10.1097/MNM.0000000000000307
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
680
Nuclear Medicine Communications 2015, Vol 36 No 7
readers in a large population of consecutive patients with newly diagnosed prostate cancer.
Materials and methods Study design and patients
The study was performed at three hospital units in Central Region Jutland, Denmark. The study included all patients with newly diagnosed prostate cancer referred routinely for bone imaging from March 2008 to October 2009. The patients were included in a noninterventional study [13]. Eligibility criteria were as follows: (a) prostate cancer diagnosed within 3 months from referral; (b) no prior or concurrent cancers except nonmelanoma skin cancer; and (c) initiation of any anticancer therapies for more than 1 week before the time of bone scintigraphy. Bone scintigraphy
Whole-body planar 99mTc-bone scintigraphy was acquired according to institutional practice [13]. The participating institutions all complied with the guidelines on bone scintigraphy by the European Society of Nuclear Medicine [14]. Expert reading of bone scintigraphy
Once all patients had been recruited, the images were read under standardized conditions. All bone scans were read simultaneously and independently by three boardcertified nuclear medicine physicians (L.J.P., J.C.M., H. C.B.) at the three centers. The readers have 10–15 years of experience with bone scans. An operator loaded the images on high-quality monitors and adjusted the display if required by the readers. The readings were performed without any clinical information. The readers graded each bone scan into one of four categories (hereafter entitled expert class): (a) normal bone scan or findings most likely to represent benign conditions; (b) equivocal findings; (c) changes most likely to be malignant; and (d) multiple metastases. In addition, each reader graded each bone scintigraphy on a dichotomous scale as positive (M1) or negative (M0) for bone metastasis. The median values for both types of categories were used in the analyses. Computer-assisted analysis of bone scintigraphy
Commercially available software (EXINI BoneBSI, version 1.6.2; EXINI Diagnostics AB, Lund, Sweden) was used. The program has full CE approval and FDA 510k clearance. A study contract was signed providing all rights of the data to the investigators, including the right to publish the findings. In comparison with most other publications with this software, no company representatives were coauthors. However, the company has the right to review and comment on all publications and presentations from the project. There was no payment between the parts. Medical/technical representatives from the company assisted with the installation and confirmed correct use with a test sample. The import file
format was Interfile 3.3 at one site (West Jutland Hospital, Herning, Denmark) and DICOM format at the other sites. EXINI BoneBSI is computer-assisted software which analyses bone scans by means of a patented algorithm for the presence and numbers of pathological lesions, characterization of lesions as malignant or benign, and finally providing diagnostic advice and quantification of results. The software then transforms lesion data into a numeric value from 0 to 1 (‘Patient ANN Outcome’, ANN meaning artificial neural network). These values were then categorized into classes of values from 0–0.25, 0.25–0.50, 0.50–0.75, and 0.75–1.0, and then converted automatically by the software into four separate patient categories: (a) normal (N); (b) probably normal (pN); (c) probably abnormal (pA); and (d) abnormal (A). The latter classification was defined prior to this study as the primary endpoint in the statistical analysis plan. The analysis was made using ‘balanced mode’ (see below).
Patient-specific settings
A feature of EXINI BoneBSI was the ability to select settings in terms of probability of bone metastasis – that is, ‘balanced’, ‘high specificity’, and ‘high sensitivity’. Diagnostic outcome was calculated based on individual risk in accordance with the European Association of Urology (EAU) risk classification [5]. Clinical T-stage subgroups were not available. Thus, patients were classified into (a) low-risk prostate cancer (if T1–T2, Gleason 2–6, and PSA < 10 ng/ml), (b) intermediate-risk prostate cancer (if T1–T2, Gleason 7 or PSA 10–20 ng/ml), (c) high-risk prostate cancer (if T3 or Gleason score 8–10 or PSA > 20 ng/ml), or (d) very high risk prostate cancer (if T4 or any T, N1). Patients with low-risk prostate cancer were analyzed using the setting ‘high specificity’, whereas patients with high or very high risk were analyzed using the setting ‘high sensitivity’. Patients with intermediate risk were analyzed in the ‘balanced’ setting.
Statistical analysis
Descriptive statistics were reported as mean ± SEM in the case of normal distributed data and median with total range for nonparametric datasets. Sensitivity, specificity, and positive and negative predictive values (PPV, NPV) were calculated. Analyses were performed using Stata 13 (StataCorp LP, College Station, Texas, USA).
Approvals
Because of the noninterventional nature, the study did not require ethical approval in accordance with national legislation [13]. This was confirmed by the local ethics committee. The Danish National Board of Health provided a waiver for informed consent for access to medical information from patient files. The study was approved by the Danish Data Protection Agency.
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
Computer-aided diagnosis of bone scans Petersen et al. 681
Results Clinical data
A total of 673 Caucasian patients were recruited. A total of 36 patients did not meet the eligibility criteria [13]. In addition, whole-body files were not available from 23 patients. Thus, a total of 614 patients were included in the final analysis. Their median PSA was 15.6 ng/ml (range 1.5–15 357), median Gleason score was 7, and the vast majority of patients had limited local disease (T1 or T2). Expert reading
The proportion of expert class 1–4 bone scintigraphy was 56.2, 33.0, 3.7, and 7.0%, respectively. The proportion of likely or evident bone metastasis (expert class 3 and 4) was 10.7% and varied among centers from 7.7 to 17.9%. For the dichotomous classification, bone metastasis (M1) was present in 12.4% of the patients. The agreement in classes among the three readers, within the margin of one expert class, was 99% (with unanimous agreement in 67% of the cases). Unanimous agreement in the dichotomous outcome was observed in 95% of the cases. Removal of artifacts
The software showed nonlesional activity in 113 of 614 cases (18.4%). Most artifacts were anatomical structures classified as lesions, such as bladder and renal pelvis and urinary devices. Removal of artifacts did not change patient category in most patients (76/113 cases, 67.3%) and down-staged all remaining patients but one. After removal of artifacts, the distribution of patient categories was as follows: N 67.6%, pN 8.3%, pA 8.1%, and A 16.0%. File format
The cumulated proportion of patients reported as expert class 1 and software N, class 2 and pN, class 3 and pA, and expert and A was 57–60% at the two sites with DICOM files and 49% at the site using Interfile format. The proportion of software-reported likely or definitive bone metastasis (pA + A) varied from 20.4 to 23.1% at the two DICOM sites and 27.6% at the Interfile site. In contrast, the corresponding values of likely or definitive bone metastasis by expert reading (class 3 + 4) were 10.7–19.7% and 7.7%. The discrepancy between software and expert reporting was most evident at the center using Interfile file format. The software upgraded patients compared with expert class readings in 12.8–18.6% of the cases at the two centers using DICOM format versus 21.3% at the site using Interfile format. Data-driven analysis showed a significant difference in the proportion of upgraded patients from the Interfile site versus the two DICOM sites (Fisher’s exact test, P = 0.0064 and 0.0005), but there were no significant differences among the two DICOM sites (P = 0.22).
The comparison of EXINI BoneBSI patient categories versus the dichotomous expert classification (M1 vs. M0) is shown in Fig. 1. It appeared that software confirmed the absence (N) or presence (A) of bone metastasis (M0 or M1), as reported by experts in 84–90% of cases with DICOM files. The corresponding figures were 67–69% with Interfile format. Apparent disagreement was rare with DICOM files: expert M0 versus software A was observed in 2.0–2.1% of patients and M1 versus N in 3.6–4.7% of patients. In comparison, such disagreement was evident in 11.8% and 11.5% of cases at the site using Interfile format. Because of clear differences in the diagnostic performance of EXINI BoneBSI with data imported by DICOM and Interfile 3.3 formats, data were reported separately by file format in the subsequent sections. Data from the two sites using DICOM format are cumulated for such analyses. Diagnostic characteristics
The diagnostic properties of the four EXINI BoneBSI outcome classes versus expert reading by four classes or dichotomous outcome are shown in Table 1. Since the four classes by software and readers were not directly comparable, comparison of expert class 4 versus software A, expert class 3 + 4 versus A, and expert class 3 + 4 versus pA + A was made. Similarly, M1 by readers were compared with EXINI BoneBSI class A as well as pA + A. The sensitivity of the four class patient outcome by EXINI BoneBSI (DICOM data) was 91–100%, specificity was 90–97%, PPV was 55–80%, and NPV was 97–100%. Using expert reading as M1 or M0 as reference, the sensitivity was 90–92%, specificity was 91–98%, PPV was 63–88%, and NPP was 98–99%. Thus, the diagnostic performance varied with the prevalence of disease, as well as the expert reading outcome classification. The diagnostic performance of EXINI BoneBSI based on Interfile 3.3 data showed somewhat different outcome (Table 1). The sensitivity was notably lower than that with DICOM files, the specificity was marginally lower, the NPVs were very similar to data obtained with DICOM files, but the PPV was notably lower. Patient-specific settings
A total of 604 of 614 patients had data for PSA, Gleason, and T-stage and were included in the analysis based on the EAU risk classification. The use of patient-specific outcome upstaged in 34 (9.9%) and 30 (11.0%) patients with DICOM and Interfile formats, as well as downstaged in two (0.6%) and six (2.2%) patients. The distribution of patient outcome in balanced mode versus using patient-specific outcome for all patients is shown in Fig. 2. The use of individual settings did not influence patient outcome in the N and A classes (95–100% consistency with results with balanced mode) but showed
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
682 Nuclear Medicine Communications 2015, Vol 36 No 7
Fig. 1
100
(b) 100
80
80
60
60
40
40
20
20
%
(a)
0
0 N
pN
pA
A
%
(c) 100
(d) 100
80
80
60
60
40
40
20
20
N
pN
pA
A
N
pN
pA
A
0
0 N
pN
pA
A
Distribution of EXINI BoneBSI patient categories versus expert reading. The left panels (a, c) represent the relative distribution of software categories among patients classified as nonmalignant (M0) by experts (open bars), whereas the right panels (b, d) represent data for patients reported to have metastatic disease (M1) by expert readers (closed bars). The upper panels (a, b) represent data from patients analyzed with DICOM files (n = 342), and the lower panels (c, d) data with Interfile files (n = 272). The abbreviations ‘N’,‘pN’,‘pA’, and ‘A’ denote normal, probably normal, probably abnormal, and abnormal, respectively.
Table 1
Diagnostic properties of EXINI BoneBSI (using balanced settings) versus expert reading class and dichotomous reading EXINI boneBSI Bone metastasis
Expert reading Bone metastasis Present DICOM format Class 4 Class 3 + 4 Class 3 + 4 M1 M1 Interfile format Class 4 Class 3 + 4 Class 3 + 4 M1 M1
Absent
Present
Absent
Sensitivity (%)
Specificity (%)
PPV (%)
NPV (%)
Prevalence (%)a
Class 1–3 Class 1 + 2 Class 1 + 2 M0 M0
A A A + pA A A + pA
N, pN, pA N, pA, pN N + pN N, pN, pA N + pN
100.0 91.1 93.3 90.0 92.0
92.7 96.6 89.6 98.0 90.8
54.9 80.4 57.5 88.2 63.0
100.0 98.6 98.9 98.3 98.5
8.2 13.2 13.2 14.6 14.6
Class 1–3 Class 1 + 2 Class 1 + 2 M0 M0
A A A + pA A A + pA
N, pN, pA N, pA, pN N + pN N, pN, pA N + pN
80.0 71.4 81.0 69.2 80.8
86.4 87.3 76.9 88.2 78.0
25.5 31.9 22.7 38.3 28.0
98.7 97.3 98.0 96.4 97.5
5.5 7.7 7.7 9.6 9.6
A, abnormal; DICOM, Digital Imaging and Communications in Medicine; EXINI categories: N, normal; pA, probably abnormal; pN, probably normal. a Proportion of patients with bone metastasis by expert reading using the specified criteria as a percentage of the total study population.
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
Computer-aided diagnosis of bone scans Petersen et al. 683
Discussion
Fig. 2
100 Patient-specific
%
N
pN
pA
A
50
0 N
pN
pA
A
Balanced setting classification Relative change in the distribution of EXINI BoneBSI patient categories by patient-specific settings versus balanced settings. The majority of normal reports (N, filled bars), as well as abnormal reports (A, open bars), using balanced settings remained unchanged by the use of patient-specific settings. In comparison, a notable proportion of patients were upstaged in the intermediate categories with probably normal repots (pN, dark gray bars) and those with probably abnormal reports (pA, light gray bars). A total of 604 patients were analyzed.
notable changes in the pN and pA classes with upstaging in a large proportion of patients. The diagnostic properties of patient-specific settings are shown in Table 2. The use of individualized settings versus balanced settings generally improved sensitivity (0–4.6% improvement) but decreased specificity (reduction − 0.5 to 5.6%). There was no major difference among file formats for the change of sensitivity and specificity by settings. In comparison, a change from balanced to individual settings reduced the PPV by 7.0–21.0% with DICOM format and by 1.3–9.0% with Interfile format. There were no changes in the values of NPV with specific settings, irrespective of file format (all changes less than 1% change). Table 2
Planar bone scintigraphy remains the guidelinerecommended method for staging of prostate cancer patients. In addition, bone scans are routinely used for the determination of bone metastasis in recurrent disease and for monitoring of treatment control in castrationresistant metastatic disease. Thus, bone scintigraphy remains one of the pivotal but also time-consuming nuclear medicine examinations in daily practice. Implementation of software that could assist the interpretation of bone scintigraphy could be of great help to both untrained and trained observers. Our results showed that EXINI BoneBSI has a high sensitivity and specificity for identification of bone metastasis in a large cohort of newly diagnosed prostate cancer patients. The software ruled out bone metastasis with a high predictive power. The PPVs were moderate to good, suggesting that a positive EXINI BoneBSI scan should be handled with caution. Reading of the initial EXINI BoneBSI scan reports indicated an issue with false identification of bone lesions, lesions which trained observers classified as nonbony lesions. The software identified nonbony lesions such as dilated pelvis, bladder activity, or urinary devices as pathological findings in nearly 20% of the patients. The proportion of artifacts in our study is far greater than the 1–2% reported in the initial study by Sadik et al. [9]. The occurrence and handling of artifacts have not been described in most recent studies. However, such artifacts were easily identified and removed without change of the software classification. The user manual of the EXINI BoneBSI software claimed the use of both Interfile and DICOM formats. However, our results strongly indicated differential diagnostic properties of data captured by the two file formats. To ensure full compliance with the software, medical and technical representatives from EXINI Diagnostics assisted with installation and sample verification of data at
Diagnostic properties of EXINI BoneBSI versus expert readings for 604 patients using patient-specific settings EXINI boneBSI Bone metastasis
Expert reading Bone metastasis Present DICOM format Class 4 Class 3 + 4 Class 3 + 4 M1 M1 Interfile format Class 4 Class 3 + 4 Class 3 + 4 M1 M1
Absent
Present
Absent
Sensitivity (%)
Specificity (%)
PPV (%)
NPV (%
Prevalence (%)a
Class 1–3 Class 1 + 2 Class 1 + 2 M0 M0
A A A + pA A A + pA
N, pN, pA N, pA, pN N + pN N, pN, pA N + pN
100.0 93.2 97.7 91.8 95.9
97.1 91.1 86.0 92.3 87.1
40.3 61.2 51.2 67.2 56.0
100.0 98.9 99.6 98.5 99.2
8.0 13.1 13.1 14.6 14.6
Class 1–3 Class 1 + 2 Class 1 + 2 M0 M0
A A A + pA A A + pA
N, pN, pA N, pA, pN N + pN N, pN, pA N + pN
84.6 73.7 84.2 70.8 83.3
81.6 82.3 76.3 83.2 77.5
19.0 24.1 21.3 29.3 26.7
99.0 97.6 98.4 96.7 97.9
4.9 7.1 7.1 9.0 9.0
A, abnormal; DICOM, Digital Imaging and Communications in Medicine; EXINI categories: N, normal; pA, probably abnormal; pN, probably normal. a Proportion of patients with disease present by expert reading as a percentage of the total study population.
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
684
Nuclear Medicine Communications 2015, Vol 36 No 7
each test site. A statistically significant larger proportion of patients were classified with bone metastasis with Interfile format with the software versus expert reading, compared with data from DICOM files. This was evident with both four-class expert class reporting and dichotomous expert reading. The analysis of the diagnostic properties showed much lower PPV of EXINI BoneBSI with Interfile format than with DICOM format, having lower sensitivity and specificity but comparable NPV. The importance of file formats on the diagnostic outcome with the computer-aided software has not previously been reported. We have identified similar findings of file format on lesion analysis (e.g. the number of malignant lesions and the bone scan index, including the association between these variables) from the same data (Petersen et al., unpublished observations). However, it has been shown that a different reference population must be used for the Caucasian and Asian populations [15,16]. The company is aware of the file issue, and the user manual has been updated with instructions only to use images in DICOM format (EXINI AB, personal communication, January 2015). The file format issue is relevant when comparing data among sites and in serial scans if different file formats are used. The diagnostic properties of the computer-aided analysis program showed sensitivities and specificities of 90–100% with DICOM format files and 69–88% with Interfile formats. The high sensitivities and specificities are in line with previous reports [9,10,12,17]. Most studies with EXINI BoneBSI have been performed in patients with metastatic disease in whom the prevalence of metastasis varied from 15 to 40% [10,12,15]. In clinical practice, the true reference is seldom known and the PPV and NPVs are of major importance. The PPVs were generally modest with DICOM format data and poor with Interfile format data. The NPVs were excellent, irrespective of file format. In line with conventions, the predictive values, but not sensitivity and specificity, are sensitive to the prevalence of disease in the study population. Thus, the diagnostic properties therefore likely differ in newly diagnosed patients, especially when PSA screening is used, versus in patients with recurrent disease. Most prior studies have not reported predictive values [9,10,15]. Tokuda and colleagues published NPV very similar to our data (98–100% in prostate cancer). However, our PPV of 55–89% with DICOM files was notably higher than the value of 51% observed by Tokuda and colleagues in prostate cancer patients with a prevalence of metastasis of nearly 40%. They presented great differences in PPV in different anatomical regions in prostate cancer and much lower PPV (∼10%) in all other types of cancer in men and women than in prostate cancer. The tested version of the software had options to select settings from high sensitivity over balanced to high specificity. On the basis of patient-specific information, we
performed an analysis of patient-specific settings. Patients with a high risk for bone metastasis were analyzed with high sensitivity and patients with low risk with high specificity. The results changed patient outcome class in ∼ 10% of the patients, mainly upstaging, and predominantly in patients with indeterminate software results (pN and pA). The use of patient-specific settings slightly increased sensitivity and decreased specificity. However, this setting notably decreased PPV, with no effect on NPV versus balanced settings. We have not been able to identify any prior studies investigating the impact on patient settings on the diagnostic outcome with this software. Risk allocation was performed using the EAU risk classification. It can be debated if other methods for selection of patient-specific settings are superior to EAU criteria. However, these criteria are quite similar to risk classifications by other main cancer or urological societies or organizations. In conclusion, the results indicated no major diagnostic improvement of using patient-specific settings versus balanced settings in a large cohort of consecutive patients. This study examined the diagnostic performance of EXINI BoneBSI in an unselected population of patients with newly diagnosed prostate cancer. The software showed high sensitivity and specificity and very high NPVs. As such, the software may be used in clinical practice to identify patients with a low probability for metastatic disease, so emphasis can be dedicated to patients with potential metastatic disease. However, the PPVs were modest and indicated that a positive EXINI BoneBSI scan result should be examined with care before classifying a patient as having metastatic disease. The overall findings and recommendations for use of the software were similar with Interfile 3.3 and DICOM files. However, users should be aware of the file issue when comparing data analyzed with different file formats (e.g. between institutions or over time).
Acknowledgements EXINI Diagnostics AB, Lund, Sweden, kindly provided the software for the purpose of this study. Conflicts of interest
The authors declare no conflict of interest with regards to the present work. Lars J. Petersen has received a speaker's honorarium from Janssen Pharma and AstraZeneca and consulting fees from LJP Medical and the KLIFO Drug Developing Council.
References 1 2
3
Damber JE, Aus G. Prostate cancer. Lancet 2008; 371:1710–1721. Palmedo H, Marx C, Ebert A, Kreft B, Ko Y, Turler A, et al. Whole-body SPECT/CT for bone scintigraphy: diagnostic value and effect on patient management in oncological patients. Eur J Nucl Med Mol Imaging 2014; 41:59–67. Wondergem M, van der Zant FM, van der Ploeg T, Knol RJ. A literature review of 18F-fluoride PET/CT and 18F-choline or 11C-choline PET/CT for detection
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
Computer-aided diagnosis of bone scans Petersen et al. 685
4
5
6 7
8 9
10
11
of bone metastases in patients with prostate cancer. Nucl Med Commun 2013; 34:935–945. Mohler JL, Kantoff PW, Armstrong AJ, Bahnson RR, Cohen M, D’Amico AV, et al. Prostate cancer, version 2.2014. J Natl Compr Canc Netw 2014; 12:686–718. Heidenreich A, Bellmunt J, Bolla M, Joniau S, Mason M, Matveev V, et al. EAU guidelines on prostate cancer. Part 1: screening, diagnosis, and treatment of clinically localised disease. Eur Urol 2011; 59:61–71. Robinson PJ. Radiology’s Achilles’ heel: error and variation in the interpretation of the Rontgen image. Br J Radiol 1997; 70:1085–1098. Sadik M, Suurkula M, Hoglund P, Jarund A, Edenbrandt L. Quality of planar whole-body bone scan interpretations – a nationwide survey. Eur J Nucl Med Mol Imaging 2008; 35:1464–1472. Ore L, Hardoff R, Gips S, Tamir A, Epstein L. Observer variation in the interpretation of bone scintigraphy. J Clin Epidemiol 1996; 49:67–71. Sadik M, Jakobsson D, Olofsson F, Ohlsson M, Suurkula M, Edenbrandt L. A new computer-based decision-support system for the interpretation of bone scans. Nucl Med Commun 2006; 27:417–423. Sadik M, Hamadeh I, Nordblom P, Suurkula M, Hoglund P, Ohlsson M, et al. Computer-assisted interpretation of planar whole-body bone scans. J Nucl Med 2008; 49:1958–1965. Sadik M, Suurkula M, Hoglund P, Jarund A, Edenbrandt L. Improved classifications of planar whole-body bone scans using a computer-assisted
12
13
14
15
16
17
diagnosis system: a multicenter, multiple-reader, multiple-case study. J Nucl Med 2009; 50:368–375. Tokuda O, Harada Y, Ohishi Y, Matsunaga N, Edenbrandt L. Investigation of computer-aided diagnosis system for bone scans: a retrospective analysis in 406 patients. Ann Nucl Med 2014; 28:329–339. Zacho HD, Barsi T, Mortensen JC, Mogensen MK, Bertelsen H, Josephsen N, et al. Prospective multicenter study of bone scintigraphy in consecutive patients with newly diagnosed prostate cancer. Clin Nucl Med 2014; 39:26–31. Bombardieri E, Aktolun C, Baum RP, Bishof-Delaloye A, Buscombe J, Chatal JF, et al. Bone scintigraphy: procedure guidelines for tumour imaging. Eur J Nucl Med Mol Imaging 2003; 30:BP99–106. Horikoshi H, Kikuchi A, Onoguchi M, Sjostrand K, Edenbrandt L. Computeraided diagnosis system for bone scintigrams from Japanese patients: importance of training database. Ann Nucl Med 2012; 26:622–626. Kikuchi A, Onoguchi M, Horikoshi H, Sjostrand K, Edenbrandt L. Automated segmentation of the skeleton in whole-body bone scans: influence of difference in atlas. Nucl Med Commun 2012; 33:947–953. Kaboteh R, Gjertsson P, Leek H, Lomsky M, Ohlsson M, Sjostrand K, et al. Progression of bone metastases in patients with prostate cancer – automated detection of new lesions and calculation of bone scan index. EJNMMI Res 2013; 3:64.
Copyright © 2015 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
