Detection
Downloaded from www.ajronline.org by 117.253.233.243 on 10/06/15 from IP address 117.253.233.243. Copyright ARRS. For personal use only; all rights reserved
JOEL
The
detection
important little work
accuracy
in everyday
Accuracy
E. GRAY,”2
KENNETH
of the diagnostic
medical
decision
in Chest W. TAYLOR,’
radiologist
making.
is
disagreement
increase
with increasing
RMS
noise.
It is
also shown that the nonradiologist responds to changes noise in exactly the same way as the radiologist.
in
Introduction
It is often
assumed that any degradation in image quality, in terms of noise or image fidelity (resolution or modulation transfer function [MTF]), will significantly decrease the detection accuracy of the radiologist. This is an important assumption because of the added cost required to produce imaging equipment with improved resolution and because of the additional dose required to produce low noise radiographs. Feddema and Botden [1] concluded that the radiologist can detect the pertinent pathology on images of extremely low resolution but, given the choice, would normally select the most aesthetically pleasing high resolution image. Most studies associating image quality with diagnostic accuracy have had serious limitations in the types of simulated radiographs used Several authors [2-5] limited their studies to relatively simple stimuli, such as uniformly exposed radiographs containing circular Icsions. Kundel and Revesz [6] studied this problem by superimposing simulated lesions on conventional radiographs using a video system. This approach is limited by the restricted response of video systems. More recently, Brogdon et al. [7] used radiographs produced by superimposing normal radiographs and radiographs of simulated pathology. However, the image quality of the duplicate of the superimposed films was not established. Studies using clinical radiographs are usually not conclusive due to variations in the patient and the position of the lesion. In such studies it is difficult to determine if changes in detectability are due to differences in image quality or to lesion location. In an attempt to overcome these problems and to determine the effect of image quality on detection accuracy, we carried out a study using four screen-film
Received
November
9, 1977;
accepted
I
S
Present
address:
Am J Roentgnol 0 1978 American
Diagnostic
after revision March 16, 1978. the James Picker Foundation University
Radiology,
131 :247-253, August Roentgen Ray Society
1978
Mayo
of Toronto,
Clinic
and
Toronto,
Mayo
BARRY
B. HOBBS’
Materials Experimental
and
instances
Methods
Films
Films were produced using 2.0 mm) and four screen-film the
modulation
three focal spots (0.3, 1 .0, and combinations (table 1). In all
transfer
function
(fig.
1) was
deter-
mined by means of edge-gradient analysis [8]. Only one MTF is shown for Alpha 4 (fig. 1A) since the MTF for all three combinations using that screen were identical. The effect of phase was disregarded since it was found to be negligible for the focal spots used [9]. The Alpha 4/RP system is a mismatched system because the film is not spectrally sensitive to the green emission of the rare earth phosphors. It was used in the study so that three systems with different noise levels and the same MTF could be studied. The radiographs for this study (fig. 2) were produced on 28 x 35.5 cm film using a thorax phantom (3M Co.) consisting of human bone encased in Plexiglas. The phantom lungs were dog lungs inflated with formalin fumes while the vascular system was perfused with latex. The dried, preserved lungs simulated the vasculature of the human lung. No attempt was made to simulate the mediastinal, diaphragmatic, and abdominal regions. The lesions consisted of Lucite spheres 6.4 mm in diameter (fig. 3). Several radiologists were shown chest radiographs containing 6.4 and 4.8 mm lesions in identical locations. They detected only one or two of the nine smaller lesions, whereas they detected six to eight of the larger lesions. Consequently, the lesions of larger size were used. All radiographs were produced with the geometry shown (fig. 4) using an anteroposterior projection. The phantom was positioned identically for each radiograph, assuring that the lesions and superimposed structures were in the same positions on the radiographs. The radiographs were made at 90 kVp using stationary grids (10:1 ratio, 40 lines/cm with a 183 cm source-
.
This work was supported by funds from Radiological Research Laboratories,
AND
combinations (Alpha 4/RP, Hi Plus/RP, Alpha 4/XD, and Alpha 4/XM) and three focal spots (0.3, 1 .0, and 2.0 mm nominal sizes). Though rare earth screen-film systems are not conventionally used for chest radiography, the potential for patient dose reduction is significant. However, it must be assured that the detection accuracy of the radiologist is not impaired by the noisier, lower dose image’s. The major variables studied were image quality (modulation transfer function, MTF) and noise (mottle) with differences in responses between radiologists and nonradiologists also being considered. Images of a thorax phantom with lungs and simulated spherical lesions were produced. So that the radiologists would not become familiar with the distributions, 14 lesion patterns were used.
However,
has been done relating the detection accuracy of the radiologist to the quality of the image. This study, using a thorax and lung phantom, simulated tissue-equivalent 6.4 mm lesions, and a 183 cm source-to-image distance, shows that the detection accuracy is not dependent on the focal spot size (over a range of 0.3-2.0 mm). However, the false positive rate increases when using small focal spots. In addition, the detection accuracy decreases with increasing root-meansquare (RMS) noise (a measure of the amount of quantum mottle in the image), while the false positive rate and intraobserver
Radiography
and the Edward Ontario,
Foundation,
247
Christie
Stevens
Foundation.
Canada.
Rochester,
Minnesota
55901
.
Address
reprint requests to J. E. Gray.
0361 -803X/78/08-0247
$00.00
GRAY
248 TABLE Characteristics
Downloaded from www.ajronline.org by 117.253.233.243 on 10/06/15 from IP address 117.253.233.243. Copyright ARRS. For personal use only; all rights reserved
Screen-Film Combination
DuPont
1
of Screen-Film
Screen Type
Combinations RootMeansquare Noise
Relative Expo sure
Film Type
(x
10’)
Hi PIus/
Blue-sensitive
0.90
0.56
Blue-sensitive
1 .65
0.57
Green-sensitive Green-sensitive
1 .00 0.30
0.61 0.73
Kodak RP. ‘CaWO4 3M Alpha 4: With Kodak RP Rare earth With With
ET AL.
3M XD. 3M XM.
Rare earth Rare earth
A
B
1.0 0.8 LI.
I-
0.6 0.4 High
0.2
Plus -i
0
2
4
6
8
0
10 Frequency
Fig.
(MTF).
1.-Screen-film
A, MTF
and
for
screen-film
for three focal spots as function
focal
I
I
I
I
4
6
8
10
Ic/mm)
spot
system
I
2
modulation
as function
transfer
of frequency.
density
difference
between
two
points
of
phantom
are indicated
with
nine
by circles;
lesions.
Obvious
remaining
seven
lesions,
lesions
are
by arrows.
B, MTF
of frequency.
A Kodak M6AN X-Omat, using DuPont Cronex Multi-Process chemistry at 32#{176}C, was controlled sensitometrically throughout the experiment. A medium density of 1 .0 above the base-plusfog (B + F) level was maintained to better than ±0.10 in optical the
in analysis,
function
ing.
while
2.-Radiograph
indicated
to-image distance). Though higher kVp techniques are conventionally used, it was necessary to limit the study to 90 kVp due to generator timing-circuit limitations. That is, the high speed rare earth screen-film combinations, when used at higher kilovoltages, require exposure times much shorter than many older generators are capable of accurately and consistently produc-
density,
F.,j.
deleted
about
0.25 and 2.0 above B + F was maintained to better than ± 0.10 in optical density. The density in the clear, circled region of the radiograph (fig. 2) was maintained at 1 .0 ± 0.05 (1 SD) above the B + F level of the film. This density was selected on the basis of the radiologists’ choice of a “diagnostic film” from radiographs made at various density levels. All lesions were placed over the lung areas and were located on the anterior surface of the phantom (with an anteroposterior projection). Each radiograph contained nine lesions; three were located in the intercostal spaces, three on the ribs, and three on the edge of the ribs so that about half of the lesion was on the rib and half was in the intercostal space. In order to study the effect of noise, six lesion patterns (P1P6) were used with each of the four screen-film combinations, two patterns for each of the three focal spots (0.3 mm, P1 and P2; 1 .0, P3 and P4; 2.0, P5 and P6). The determination of the inconsistency of the participants in reading identical films was possible, since a second radiograph was made for each of the 24 screen-film-focal spot combinations, making a total of 48 films.
In order to study the effect of focal spot degradation, an additional eight lesion patterns (P7-P14) were used with each of the three focal spots, two patterns for each of the four screen-film combinations (Alpha 4/RP; P7 and PB; Hi Plus/RP,
Pg and PlO; Alpha
4/XD,
P11 and P12; and Alpha
4/XM,
and P14). Only one subject (radiologist there seemed to be one or two patterns
no. 8) mentioned that were duplicated
it seems
sufficient
that
14
sets
of
patterns
Not all of the radiologists Film
Reading
The
with
read all of the 72 films
for
the
in each
,so
study.
session.
Conditions
participants
room
were
P13 that
in the study
subdued
light
and
were with
shown
no
the films
interruptions.
in a quiet Each
heard
the same tape-recorded instructions (see Appendix). Several of the participants viewed the radiographs multiple times with 2 days to several weeks between sessions. The participants included radiologists, residents, and nonradiologists (table 2). All participants had visual acuity of 20/20 at 30 and 100 cm. One radiologist (no. 1) and three nonradiologists (A, B, and C) knew that
a limited
there
were
number
nine
lesions
A conventional 35.5
cm
(Im/m2).
was The
lesion
viewbox,
used
participants
of
room were
.
patterns
in each
masked
The viewbox was asked
dimly to
Analysis
the
and
that
illuminated mark
the
28 x
was 9 x lOl apostilbs at 100
correct
lx (lm/m2).
lesion
The
locations
on
radiograph. The viewing by means of a television could be found between
and the MTF or noise of the radiographs.
of Data
Several further
used
to a film size just under
brightness
an 11 x 14 cm reproduction of the distances and times were measured tape recorder. However, no correlation
these parameters
were
radiograph.
lesions analysis,
detection
were since
accuracy
very
obvious
changes
for
in image
such
lesions.
and
were
quality
This
deleted would
merely
from
not
affect
reduced
Downloaded from www.ajronline.org by 117.253.233.243 on 10/06/15 from IP address 117.253.233.243. Copyright ARRS. For personal use only; all rights reserved
DETECTION
ACCURACY
IN CHEST
RADIOGRAPHY
249
Fig. 3.-Array of Lucite spheres: 9.5, 7.9, 6.4, 4.8, 4.0, 3.2, and 2.4 mm in diameter. Lucite spheres chosen to simulate lesions were 6.4 mm. A, Kodak type M film without intensifying screens and with no scatter or focal spot degradation. B, Alpha 4-xM screen-film combination with sufficient scatter to reduce lesion contrast to level similar to that encountered in phantom images and with
no focal
spot
degradation.
TABLE Summary 5ubject ldentificalion
2
of Subjects 5ubject ldentificalion
Background
Background
1
Radiologist
8
2
Radiologist
9
Resident
3
Resident (third year) Radiologist Radiologist Resident (second year) Resident (first year)
A B C D
Secretary Graduate student Medical physicist Radiologic technologist
4
5 6 7
.
Radiologist
.
(fourth
year)
for deletion from the focal spot series were located correctly in eight or nine of a possible nine responses and were clearly visible on the films made with the 2.0 mm focal spot. Of the 126 lesions,
50
were
deleted;
bone, 25 were lesions edges.
of
these
oft bone,
50,
seven
were
To study the inconsistency, or intraobserver a given observer, two identical radiographs
each screen-film-focal
lesions
and 18 were lesions
spot combination.
disagreement, were made
(All lesions,
on
on bone of with
including
those deleted for the detection accuracy analysis, were included in the intraobserver disagreement analysis). For each lesion that
the participant the
same
three
possible
lesion
Fig. 4.-Geometry
for x-ray imaging
spot
both
and
responses:
correctly
correctly
system.
saw twice in the same session
focal
either times;
screen-film (1)
time; and
(2) (3)
the the the
on films made with
combination, there were subject did not locate the subject subject
located located
the
lesion
the
lesion
correctly only once. The ratio of the number of the responses locating the lesion correctly only once to the total number of lesion pairs is the intraobserver disagreement. The relative
the overall detection then defined as:
accuracy.
The detection
accuracy
was
intraobserver vidual’s dition
A_C
-O -
(1)
where N is the number of lesions, 0 is the number of obvious lesions, and C is the number of correct responses. The lesions deleted from the screen-film series were those that were correctly identified in 22-24 of 24 possible responses and were clearly visible in the radiograph made with the Alpha 4/XM combination (the noisiest image). Likewise, the lesions selected
tion
disagreement
intraobserver by his
average
due to differences
is then found
disagreement disagreement,
between
for
thereby
sessions
by dividing
a specific
the mdi-
imaging
reducing
the
convaria-
and observers.
Much of the analysis used data from the three films in combination with Alpha 4 screens. This assured that the changes noted were due to changes in the noise only and not to differences in the MTF of the intensifying screen. A count of the total number of false positive responses was made, and the average number per reading session was determined.
250
GRAY
ET AL.
Results 0.8
Radiologists
3
3
3
as Group
Downloaded from www.ajronline.org by 117.253.233.243 on 10/06/15 from IP address 117.253.233.243. Copyright ARRS. For personal use only; all rights reserved
0.7
An attempt was made to determine the effect of changes in image quality on the detection accuracy of the radiologists using the responses from one session for each of nine radiologists (fig. 5). Though trends may be apparent, it is not possible to determine from figure 5 if the changes in detection accuracy were significant, due to the wide spread in the overall accuracies of the individual radiologists. The wide range of individual accuracies has not been considered in many previous studies. It is clear that changes in detection accuracy must be considered for each radiologist in order to determine whether some improve and some get worse with changes in image quality. Radiologists
as Individuals
Five radiologists were asked to read the films several times. Only the change in the criterion level of the individual radiologist and a possible learning effect could produce changes in the detection accuracy from session to session. The results of the multiple sessions for the five radiologists evaluated are shown in figure 6. Although their mean accuracies vary from 0.45 to 0.85, all radiologists respond in a similar fashion to changes in image quality. The detection accuracy of the individual radiologist is not affected significantly by the change in focal spot size for the imaging geometry used. The false positive rate tends to decrease with increasing focal spot size (fig. 7A). Since the radiographs made with the smallest focal spot tend to reproduce the fine detail in the phantom clearly (i .e. , the amount of structured noise is increased), it would be expected that the radiologist might falsely indicate the presence of a lesion in the noisy background. In addition, the radiologist would be expected to have a more difficult task in reading the radiographs made with the small focal spot because he is not accustomed to viewing chest radiographs with considerable fine detail. Since it has been shown that the root-mean-square (RMS) noise will interfere with detection [10], we used the RMS noise as a relatively simple way to characterize fluctuations in the density of a radiograph. The RMS noise was determined by uniformly exposing a radiograph and measuring the density at 750 independent locations on a film (with fixed aperture of 1 mm). The average and the square of the differences (deviations) between each measurement were determined. The RMS noise is then the square root of the mean of the squared deviations or the standard deviation of the density measurements. The RMS noise has been discussed in detail in the literature [11] and recently applied in the radiographic field [12]. It is evident (figs. 6C and 6D) that the detection accuracy of the individual radiologist decreases with increasing RMS noise. Most radiologists tend to report more false positives when the noise is increased (fig. Also, the radiologist tends to disagree with himself 7B).
8
Fig.
5.-Detection
as function
accuracy
of nominal
size for radiologists Averages for nine areindicatedbydots.
focal spot
Downloaded from www.ajronline.org by 117.253.233.243 on 10/06/15 from IP address 117.253.233.243. Copyright ARRS. For personal use only; all rights reserved
JOEL
The
detection
important little work
accuracy
in everyday
Accuracy
E. GRAY,”2
KENNETH
of the diagnostic
medical
decision
in Chest W. TAYLOR,’
radiologist
making.
is
disagreement
increase
with increasing
RMS
noise.
It is
also shown that the nonradiologist responds to changes noise in exactly the same way as the radiologist.
in
Introduction
It is often
assumed that any degradation in image quality, in terms of noise or image fidelity (resolution or modulation transfer function [MTF]), will significantly decrease the detection accuracy of the radiologist. This is an important assumption because of the added cost required to produce imaging equipment with improved resolution and because of the additional dose required to produce low noise radiographs. Feddema and Botden [1] concluded that the radiologist can detect the pertinent pathology on images of extremely low resolution but, given the choice, would normally select the most aesthetically pleasing high resolution image. Most studies associating image quality with diagnostic accuracy have had serious limitations in the types of simulated radiographs used Several authors [2-5] limited their studies to relatively simple stimuli, such as uniformly exposed radiographs containing circular Icsions. Kundel and Revesz [6] studied this problem by superimposing simulated lesions on conventional radiographs using a video system. This approach is limited by the restricted response of video systems. More recently, Brogdon et al. [7] used radiographs produced by superimposing normal radiographs and radiographs of simulated pathology. However, the image quality of the duplicate of the superimposed films was not established. Studies using clinical radiographs are usually not conclusive due to variations in the patient and the position of the lesion. In such studies it is difficult to determine if changes in detectability are due to differences in image quality or to lesion location. In an attempt to overcome these problems and to determine the effect of image quality on detection accuracy, we carried out a study using four screen-film
Received
November
9, 1977;
accepted
I
S
Present
address:
Am J Roentgnol 0 1978 American
Diagnostic
after revision March 16, 1978. the James Picker Foundation University
Radiology,
131 :247-253, August Roentgen Ray Society
1978
Mayo
of Toronto,
Clinic
and
Toronto,
Mayo
BARRY
B. HOBBS’
Materials Experimental
and
instances
Methods
Films
Films were produced using 2.0 mm) and four screen-film the
modulation
three focal spots (0.3, 1 .0, and combinations (table 1). In all
transfer
function
(fig.
1) was
deter-
mined by means of edge-gradient analysis [8]. Only one MTF is shown for Alpha 4 (fig. 1A) since the MTF for all three combinations using that screen were identical. The effect of phase was disregarded since it was found to be negligible for the focal spots used [9]. The Alpha 4/RP system is a mismatched system because the film is not spectrally sensitive to the green emission of the rare earth phosphors. It was used in the study so that three systems with different noise levels and the same MTF could be studied. The radiographs for this study (fig. 2) were produced on 28 x 35.5 cm film using a thorax phantom (3M Co.) consisting of human bone encased in Plexiglas. The phantom lungs were dog lungs inflated with formalin fumes while the vascular system was perfused with latex. The dried, preserved lungs simulated the vasculature of the human lung. No attempt was made to simulate the mediastinal, diaphragmatic, and abdominal regions. The lesions consisted of Lucite spheres 6.4 mm in diameter (fig. 3). Several radiologists were shown chest radiographs containing 6.4 and 4.8 mm lesions in identical locations. They detected only one or two of the nine smaller lesions, whereas they detected six to eight of the larger lesions. Consequently, the lesions of larger size were used. All radiographs were produced with the geometry shown (fig. 4) using an anteroposterior projection. The phantom was positioned identically for each radiograph, assuring that the lesions and superimposed structures were in the same positions on the radiographs. The radiographs were made at 90 kVp using stationary grids (10:1 ratio, 40 lines/cm with a 183 cm source-
.
This work was supported by funds from Radiological Research Laboratories,
AND
combinations (Alpha 4/RP, Hi Plus/RP, Alpha 4/XD, and Alpha 4/XM) and three focal spots (0.3, 1 .0, and 2.0 mm nominal sizes). Though rare earth screen-film systems are not conventionally used for chest radiography, the potential for patient dose reduction is significant. However, it must be assured that the detection accuracy of the radiologist is not impaired by the noisier, lower dose image’s. The major variables studied were image quality (modulation transfer function, MTF) and noise (mottle) with differences in responses between radiologists and nonradiologists also being considered. Images of a thorax phantom with lungs and simulated spherical lesions were produced. So that the radiologists would not become familiar with the distributions, 14 lesion patterns were used.
However,
has been done relating the detection accuracy of the radiologist to the quality of the image. This study, using a thorax and lung phantom, simulated tissue-equivalent 6.4 mm lesions, and a 183 cm source-to-image distance, shows that the detection accuracy is not dependent on the focal spot size (over a range of 0.3-2.0 mm). However, the false positive rate increases when using small focal spots. In addition, the detection accuracy decreases with increasing root-meansquare (RMS) noise (a measure of the amount of quantum mottle in the image), while the false positive rate and intraobserver
Radiography
and the Edward Ontario,
Foundation,
247
Christie
Stevens
Foundation.
Canada.
Rochester,
Minnesota
55901
.
Address
reprint requests to J. E. Gray.
0361 -803X/78/08-0247
$00.00
GRAY
248 TABLE Characteristics
Downloaded from www.ajronline.org by 117.253.233.243 on 10/06/15 from IP address 117.253.233.243. Copyright ARRS. For personal use only; all rights reserved
Screen-Film Combination
DuPont
1
of Screen-Film
Screen Type
Combinations RootMeansquare Noise
Relative Expo sure
Film Type
(x
10’)
Hi PIus/
Blue-sensitive
0.90
0.56
Blue-sensitive
1 .65
0.57
Green-sensitive Green-sensitive
1 .00 0.30
0.61 0.73
Kodak RP. ‘CaWO4 3M Alpha 4: With Kodak RP Rare earth With With
ET AL.
3M XD. 3M XM.
Rare earth Rare earth
A
B
1.0 0.8 LI.
I-
0.6 0.4 High
0.2
Plus -i
0
2
4
6
8
0
10 Frequency
Fig.
(MTF).
1.-Screen-film
A, MTF
and
for
screen-film
for three focal spots as function
focal
I
I
I
I
4
6
8
10
Ic/mm)
spot
system
I
2
modulation
as function
transfer
of frequency.
density
difference
between
two
points
of
phantom
are indicated
with
nine
by circles;
lesions.
Obvious
remaining
seven
lesions,
lesions
are
by arrows.
B, MTF
of frequency.
A Kodak M6AN X-Omat, using DuPont Cronex Multi-Process chemistry at 32#{176}C, was controlled sensitometrically throughout the experiment. A medium density of 1 .0 above the base-plusfog (B + F) level was maintained to better than ±0.10 in optical the
in analysis,
function
ing.
while
2.-Radiograph
indicated
to-image distance). Though higher kVp techniques are conventionally used, it was necessary to limit the study to 90 kVp due to generator timing-circuit limitations. That is, the high speed rare earth screen-film combinations, when used at higher kilovoltages, require exposure times much shorter than many older generators are capable of accurately and consistently produc-
density,
F.,j.
deleted
about
0.25 and 2.0 above B + F was maintained to better than ± 0.10 in optical density. The density in the clear, circled region of the radiograph (fig. 2) was maintained at 1 .0 ± 0.05 (1 SD) above the B + F level of the film. This density was selected on the basis of the radiologists’ choice of a “diagnostic film” from radiographs made at various density levels. All lesions were placed over the lung areas and were located on the anterior surface of the phantom (with an anteroposterior projection). Each radiograph contained nine lesions; three were located in the intercostal spaces, three on the ribs, and three on the edge of the ribs so that about half of the lesion was on the rib and half was in the intercostal space. In order to study the effect of noise, six lesion patterns (P1P6) were used with each of the four screen-film combinations, two patterns for each of the three focal spots (0.3 mm, P1 and P2; 1 .0, P3 and P4; 2.0, P5 and P6). The determination of the inconsistency of the participants in reading identical films was possible, since a second radiograph was made for each of the 24 screen-film-focal spot combinations, making a total of 48 films.
In order to study the effect of focal spot degradation, an additional eight lesion patterns (P7-P14) were used with each of the three focal spots, two patterns for each of the four screen-film combinations (Alpha 4/RP; P7 and PB; Hi Plus/RP,
Pg and PlO; Alpha
4/XD,
P11 and P12; and Alpha
4/XM,
and P14). Only one subject (radiologist there seemed to be one or two patterns
no. 8) mentioned that were duplicated
it seems
sufficient
that
14
sets
of
patterns
Not all of the radiologists Film
Reading
The
with
read all of the 72 films
for
the
in each
,so
study.
session.
Conditions
participants
room
were
P13 that
in the study
subdued
light
and
were with
shown
no
the films
interruptions.
in a quiet Each
heard
the same tape-recorded instructions (see Appendix). Several of the participants viewed the radiographs multiple times with 2 days to several weeks between sessions. The participants included radiologists, residents, and nonradiologists (table 2). All participants had visual acuity of 20/20 at 30 and 100 cm. One radiologist (no. 1) and three nonradiologists (A, B, and C) knew that
a limited
there
were
number
nine
lesions
A conventional 35.5
cm
(Im/m2).
was The
lesion
viewbox,
used
participants
of
room were
.
patterns
in each
masked
The viewbox was asked
dimly to
Analysis
the
and
that
illuminated mark
the
28 x
was 9 x lOl apostilbs at 100
correct
lx (lm/m2).
lesion
The
locations
on
radiograph. The viewing by means of a television could be found between
and the MTF or noise of the radiographs.
of Data
Several further
used
to a film size just under
brightness
an 11 x 14 cm reproduction of the distances and times were measured tape recorder. However, no correlation
these parameters
were
radiograph.
lesions analysis,
detection
were since
accuracy
very
obvious
changes
for
in image
such
lesions.
and
were
quality
This
deleted would
merely
from
not
affect
reduced
Downloaded from www.ajronline.org by 117.253.233.243 on 10/06/15 from IP address 117.253.233.243. Copyright ARRS. For personal use only; all rights reserved
DETECTION
ACCURACY
IN CHEST
RADIOGRAPHY
249
Fig. 3.-Array of Lucite spheres: 9.5, 7.9, 6.4, 4.8, 4.0, 3.2, and 2.4 mm in diameter. Lucite spheres chosen to simulate lesions were 6.4 mm. A, Kodak type M film without intensifying screens and with no scatter or focal spot degradation. B, Alpha 4-xM screen-film combination with sufficient scatter to reduce lesion contrast to level similar to that encountered in phantom images and with
no focal
spot
degradation.
TABLE Summary 5ubject ldentificalion
2
of Subjects 5ubject ldentificalion
Background
Background
1
Radiologist
8
2
Radiologist
9
Resident
3
Resident (third year) Radiologist Radiologist Resident (second year) Resident (first year)
A B C D
Secretary Graduate student Medical physicist Radiologic technologist
4
5 6 7
.
Radiologist
.
(fourth
year)
for deletion from the focal spot series were located correctly in eight or nine of a possible nine responses and were clearly visible on the films made with the 2.0 mm focal spot. Of the 126 lesions,
50
were
deleted;
bone, 25 were lesions edges.
of
these
oft bone,
50,
seven
were
To study the inconsistency, or intraobserver a given observer, two identical radiographs
each screen-film-focal
lesions
and 18 were lesions
spot combination.
disagreement, were made
(All lesions,
on
on bone of with
including
those deleted for the detection accuracy analysis, were included in the intraobserver disagreement analysis). For each lesion that
the participant the
same
three
possible
lesion
Fig. 4.-Geometry
for x-ray imaging
spot
both
and
responses:
correctly
correctly
system.
saw twice in the same session
focal
either times;
screen-film (1)
time; and
(2) (3)
the the the
on films made with
combination, there were subject did not locate the subject subject
located located
the
lesion
the
lesion
correctly only once. The ratio of the number of the responses locating the lesion correctly only once to the total number of lesion pairs is the intraobserver disagreement. The relative
the overall detection then defined as:
accuracy.
The detection
accuracy
was
intraobserver vidual’s dition
A_C
-O -
(1)
where N is the number of lesions, 0 is the number of obvious lesions, and C is the number of correct responses. The lesions deleted from the screen-film series were those that were correctly identified in 22-24 of 24 possible responses and were clearly visible in the radiograph made with the Alpha 4/XM combination (the noisiest image). Likewise, the lesions selected
tion
disagreement
intraobserver by his
average
due to differences
is then found
disagreement disagreement,
between
for
thereby
sessions
by dividing
a specific
the mdi-
imaging
reducing
the
convaria-
and observers.
Much of the analysis used data from the three films in combination with Alpha 4 screens. This assured that the changes noted were due to changes in the noise only and not to differences in the MTF of the intensifying screen. A count of the total number of false positive responses was made, and the average number per reading session was determined.
250
GRAY
ET AL.
Results 0.8
Radiologists
3
3
3
as Group
Downloaded from www.ajronline.org by 117.253.233.243 on 10/06/15 from IP address 117.253.233.243. Copyright ARRS. For personal use only; all rights reserved
0.7
An attempt was made to determine the effect of changes in image quality on the detection accuracy of the radiologists using the responses from one session for each of nine radiologists (fig. 5). Though trends may be apparent, it is not possible to determine from figure 5 if the changes in detection accuracy were significant, due to the wide spread in the overall accuracies of the individual radiologists. The wide range of individual accuracies has not been considered in many previous studies. It is clear that changes in detection accuracy must be considered for each radiologist in order to determine whether some improve and some get worse with changes in image quality. Radiologists
as Individuals
Five radiologists were asked to read the films several times. Only the change in the criterion level of the individual radiologist and a possible learning effect could produce changes in the detection accuracy from session to session. The results of the multiple sessions for the five radiologists evaluated are shown in figure 6. Although their mean accuracies vary from 0.45 to 0.85, all radiologists respond in a similar fashion to changes in image quality. The detection accuracy of the individual radiologist is not affected significantly by the change in focal spot size for the imaging geometry used. The false positive rate tends to decrease with increasing focal spot size (fig. 7A). Since the radiographs made with the smallest focal spot tend to reproduce the fine detail in the phantom clearly (i .e. , the amount of structured noise is increased), it would be expected that the radiologist might falsely indicate the presence of a lesion in the noisy background. In addition, the radiologist would be expected to have a more difficult task in reading the radiographs made with the small focal spot because he is not accustomed to viewing chest radiographs with considerable fine detail. Since it has been shown that the root-mean-square (RMS) noise will interfere with detection [10], we used the RMS noise as a relatively simple way to characterize fluctuations in the density of a radiograph. The RMS noise was determined by uniformly exposing a radiograph and measuring the density at 750 independent locations on a film (with fixed aperture of 1 mm). The average and the square of the differences (deviations) between each measurement were determined. The RMS noise is then the square root of the mean of the squared deviations or the standard deviation of the density measurements. The RMS noise has been discussed in detail in the literature [11] and recently applied in the radiographic field [12]. It is evident (figs. 6C and 6D) that the detection accuracy of the individual radiologist decreases with increasing RMS noise. Most radiologists tend to report more false positives when the noise is increased (fig. Also, the radiologist tends to disagree with himself 7B).
8
Fig.
5.-Detection
as function
accuracy
of nominal
size for radiologists Averages for nine areindicatedbydots.
focal spot
