Downloaded from www.ajronline.org by 179.181.160.140 on 06/22/16 from IP address 179.181.160.140. Copyright ARRS. For personal use only; all rights reserved
533
Photostimulable Radiography
Phosphor Digital of the Extremities:
Diagnostic Accuracy Compared Conventional Radiography
Anthony
J. Wilson1 F. A. Mann William A. Murphy, Jr. Barbara S. Monsees Margaret R. Linn
A direct
comparison
was
made
between
digital
and
with
conventional
radiographs
to
assess the relative accuracy of a photostimulable phosphor digital imaging system in detecting and localizing minor trauma in the extremities. Matched sets of images were obtained on 103 patients who came to the emergency the hand, wrist, foot, or ankle. One set was obtained
system.
The other set was obtained
department for radiographs of with a conventional screen/film
with photostimulable
phosphor
digital cassettes.
The two sets of images radiologists in a blinded
of each patient were independently interpreted by three fashion. The findings of each of these three readers were compared with the consensus opinion of two different radiologists. Receiver-operatingcharacteristic (ROC) curves were plotted for each of the three readers, areas under the curves were calculated, and true-positive fractions were determined at false-positive fractions of 0.1. Although no significant differences in the areas under the ROC curves for the two imaging systems were detected, conventional radiography showed a slight
advantage.
However,
when true-positive
fractions
for fracture detection
were compared
at false-positive fractions of 0.1 a statistically significant difference was shown, with conventional screen/film radiography being more sensitive. This study raises questions about the use of currently available photostimulable phosphor systems for imaging trauma of the extremities and suggests that those systems should not be used exclusively. AJR
157:533-538,
September
1991
Photostimulable phosphor digital [1 ] and has been the subject
Japan [7-18]
studies.
These reports
radiography (PPDR) was of a number of technical
have compared
radiography, describing a much greater spatial resolution than with conventional limited use in a variety of settings including several authors have studied the efficacy
[1 3, 14], breast results
have
been
[1 5, 16], and abdominal published
regarding
[1 3, 19, 20]. We were concerned Received March 11, 1991; accepted sion April29, 1991.
after revi-
Presented at the annual meeting of the American Ray Society, Washington, DC, May 1990. 1 All authors: Mallinckrodt Institute of Radiology, Roentgen
Washington University School of Medicine, St. Louis, MO 63110. Address reprint requests to A. J. Wilson.
0361-803X/91/1573-0533 © American Roentgen Ray Society
first developed in [1 -7] and clinical
PPDR with conventional
screen/film
dynamic range with PPDR, but lower radiography. The technique has been in emergency departments [7, 8]. Although of this technique for chest [9-1 2], spine
radiography
musculoskeletal
that a system,
[1 7, 18], few investigative applications
promoted
of the system
to replace
the gold
standard of conventional radiography, is being used in clinical settings for which it has not been adequately validated. Before using PPDR for extremity imaging in our own emergency department, we thought that a study was necessary to compare this technique directly with the established standard of conventional
radiography. The purpose of this study was to compare directly the diagnostic accuracy of the PPDR system with our existing conventional screen/film combination for extremity imaging in an emergency department setting. As a wide range of extremity disorders are seen in the emergency department, it seemed appropriate to use cases from the full spectrum of these. The question to be answered was: Is PPDR a satisfactory alternative to conventional radiography, or does the known loss in
534
WILSON
spatial resolution with PPDR affect on diagnostic efficacy?
have
a measurable
ET AL.
adverse
type
AJR:157,
of examination
used, as excessive tectability
Downloaded from www.ajronline.org by 179.181.160.140 on 06/22/16 from IP address 179.181.160.140. Copyright ARRS. For personal use only; all rights reserved
Materials Study
performance
with
paired
photostimulable
phosphor
High-definition
im-
[5]. The 8 x 1 0 cassettes
standard.
life-threatening
disorders,
impaired
mental
was approved
pregnant
status
were
by our institutional
patients, not
children,
studied.
The
and patients study
fine
protocol
Reference
screens
nations included three examinations included The PCR system printed
onto
and
T-Mat
“truth”
G film (Eastman Kodak Company,
no
views: frontal, lateral, and oblique. four views: posteroanterior, ulnar installed
video
display
film by using
supplied
Fig. 1.-Computed laterally. densest
images
14,
per film, one of which
resolution
of 5 line pairs per
in our institution capability.
standardized
The
images produced
postprocessing
by the manufacturer
(A, B) and conventional Although fractures are readily visible bony areas easier to see.
(C, D) radiographs on both
sets
established
from
the images
alone.
To establish
truth,
and digital
image
sets were studied
together,
and finally
a consensus was achieved between the two radiologists by discussion of differences of opinion. Although focusing on minor trauma, this truth included 14 separate categories of positive findings (Table 1). Arthritic alterations were further subdivided into cartilage
loss, erosions,
osteophytes,
and other arthritic changes.
on are
Image
parameters
and were optimized
was
conventional
The wrist deviation,
produces images
Standard
two experienced musculoskeletal radiologists performed a pilot study. Initially, each set of images was studied alone, then the
selected before imaging. These parameters were changed from those originally
reduced-size
fullx
As the study group consisted of patients in the emergency department for whom reliable follow-up was difficult to obtain, and because there was no other logical gold standard for diagnosis, the radiologic
lateral, and oblique. has
with a single
review board.
Rochester, NY). The examination was then repeated by the same technologist by using Philips Computed Radiography (PCR) 8 x 10 in. (20 x 25 cm) cassettes (Philips Medical Systems, Shelton, CT) with the same X-ray tube and generator, the same exposure factors, and the same projections (Fig. 1). The hand, foot, and ankle exami-
and
the viewer
millimeter (Ip/mm) [5]. This is clearly less than the resolution of 1 4 Ip/ mm claimed by the manufacturer for the screen/film system used in this study [Kodak, personal communication].
Matched pairs ofconventional and PPDR images of the extremities of 103 patients were obtained. After informed consent was obtained, the examination requested by the referring physician was performed
film
because
the larger sizes (1 0 x 12, 14
a pixel size of 0.1 mm and spatial
Acquisition
Lanex
selected
the 8 x 1 0 cassettes produce images much closer in appearance to conventional film than do the other cassette sizes. PPDR images obtained on 8 x 1 0 cassettes have a 251 0 x 2000 matrix, producing
Cases for this study were recruited from our urban emergency department (65,000 annual visits). Patients having hand, wrist, foot, and ankle examinations, on weekdays between 8 am. and 4 p.m., were offered the opportunity to participate in the study. Patients with
with
two
were
usually has a high degree of digital edge enhancement. The option of two images on the film is not available for 8 x 10 cassettes. Thus
Group
Image
was
than the larger PCR cassettes
also present
sized image on film, whereas 14 x 17) produce
with
enhancement
radiographs as closely as on a laser disk for archival
8 x 1 0 in. PCR cassettes
these have higher spatial resolution
ages and with conventional radiographs was compared by using data from three readers. A consensus panel was used to establish the
Study
edge
purposes and additional postprocessing, but this does not occur automatically. All of the images in this study were stored on disk.
Design
reference
Minimal
1991
enhancement has been shown to decrease dodefects [20]. The factors used were designed to
make the final images mimic conventional possible. The PCR image may be stored
and Methods
Observer
of cortical
in the study.
September
Three
for each
without
of an ankle
of radiographs,
Evaluation
showing increased
other
access
lateral dynamic
radiologists
to clinical
then
interpreted
information.
each
set
The experience
of images,
level and
and posterior malleolar fractures with soft-tissue swelling range of computed images makes both soft tissues and
DIGITAL
AJR:157,
September
1991
TABLE
1: Range
and Number
of Abnormalities
Type of Abnormality
Downloaded from www.ajronline.org by 179.181.160.140 on 06/22/16 from IP address 179.181.160.140. Copyright ARRS. For personal use only; all rights reserved
RADIOGRAPHY
OF THE
EXTREMITIES
535
:#{149}#{149}
Recorded
No. of Cases
Fractures Arthritic changes Soft-tissue swelling Nonarticular bony spurs
31 28 28 14
Generalized
12
0.40
Dislocations
6
0.30
Vascular calcifications
5
Lacerations/ulcerations
5 4
0.20
osteopenia
Foreign bodies Tumors Arthrodeses
0.10
2 2
Amputations Medullary infarct Total
0.0c 0.00
.
,
,
,
,
0.10
0.20
0.30
0.40
0’1
0.2
0.3
0.4
,
0.50
,
0.60
,
0.70
“
0.80
-
0.90
1.00
A
2 1 140
0.9 0.8 0.7 0.6
subspecialty experienced
expertise
radiologist, completion
of the three
readers
varied:
One was an
0.5
musculoskeletal radiologist, one an experienced general and one a musculoskeletal fellow less than 6 months after of residency. A separate data sheet was provided for
0.4
each reader for each set of images (206 sheets per reader). The data sheet required a positive response to each of the radiographic features (Table 1) using a five-point normal, 2 = probably normal, 3 abnormal, 5 = definitely abnormal).
recorded with free text. The digital and conventional
confidence score possibly abnormal,
=
The
location
(1 4
of each
=
definitely
=
probably
finding
was
0.3 0.2 0.1 ‘0
0.5
examinations
0.6 0.5 0.4 0.3 02 0.1
C
Statistical
Fig. 2.-Receiver-operating-characteristic reader 1 (A), reader 2 (B), and reader black crosses = digital radiograph. were
generated
and comparisons
made,
curves
(Figs.
2 and 3) for the performance
of each reader
with conventional radiography and PPDR were generated from the confidence scores by using a personal computer-based program (CORROC 2, Metz et al., University of Chicago, Chicago, IL). Comparisons of true-positive fractions (TPF) at a false-positive fraction (FPF) of 0.1 0 for each reader were tested for significance, using critical
ratios
FPF because settings [22].
(Table 2) [21]. The value it reflects the reported
Co
0.2
0.3
04
of 0.1 0 was chosen for the utility in routine diagnostic
the readers.
readers.
Results
These
curves
were
The ROC curves
a lack of any statistical tional
radiography
the
compared 1 03 matched
25 hands,
50 women
and 53 men, from
age of 40 years.
pairs
of images,
14 fingers,
there
and 13 wrists.
1 9 to 85 years
were
07
0.8
3 (C).
curves for combined data for Shaded boxes = plain film,
26 feet,
There were
old, with a mean
closely
derived
matched
for fractures
for all three
alone (Fig. 3)
yielded A values as listed in Table 3. Again no significant difference was found between the paired A values. Despite
data. Using
Of the
06
The ROC curves plotted for the combined categories (Fig. 2) yielded an area under each curve (As) as shown in Table 3. No significant difference in A was shown for any of
difference,
to have
and
last
methods and
bution.
exists
Similarly,
and
we found
of either comparison
almost
between
than
and combined [23,
readings
reduction
learning
show conven-
performance
McNeil
each observer’s
readers 1 and 2, but a moderate no evidence
overall
for both fractures
of Hanley
As between 20 cases,
the curves
a better
PPDR had for all readers
25 ankles,
0.5 FPF
for each
reader, for fractures (n = 31) and for the combined data of all of the 14 findings listed in Table 1 (n = 1 40). Receiver-operating-characteristic (ROC)
0.9
0.7
was asked to look at 30 of the examinations a second time, several weeks after the initial readings. These 30 examinations were randomly selected; 15 were conventional radiographs and 15 were PCR images.
data
0.8
0.8
window level and width or to perform any other postprocessing. Finally, to estimate intrareader variance, each of the three radiologists
Methods
0.7
0.9
of each case were presented at four separate sittings in a 2-month period, with digital and conventional examinations mixed in equal quantities during each reading session. The digital and conventional examinations of each case were presented at different times. All conventional and digital images were presented on film. Each reader had access to only one copy of each digital image and did not have the opportunity to alter
Statistical
0.6
B
24],
we
of the first
no difference
for
for reader 3. Thus,
effect the
or case first
and
maldistrisecond
536
WILSON
ET AL.
AJR:i57,
TABLE
3: Areas
Under
September1991
Receiver-Operating-Characteristic
Curves
0.90 0.80 0.70
Reader No.1
Reader No.2
Reader No.3
0.9374 0.9281
0.9534 0.9382
0.9517 0.9504
Downloaded from www.ajronline.org by 179.181.160.140 on 06/22/16 from IP address 179.181.160.140. Copyright ARRS. For personal use only; all rights reserved
0.60
Combined
0.50 0.40
data
Film/screen
radiographs
Computed
radiographs (p
0.30 0.20 0.10 0.0c 0 DO
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.
=
(p
.6079)
=
.51 21)
Fractures Film/screen
radiographs
0.9004
0.881 0
Computed
radiographs
0.8842
0.8280
(p
)0
=
(p
.8085)
=
(p
=
.9352)
0.9618
0.8788 (p
.6063)
=
.2062)
A was greater
0.90
than their intertechnique
ens our ability to recognize small, tween techniques in this study.
0.80 0.70
differences. but real,
This weak-
differences
be-
As the shapes of the ROC curves for PPDR and SFR differ
0.60
(i.e., all but one of the ROC thought a direct comparison
0.50 0.40
curves cross; Figs. 2 and 3), we of TPFs at FPF 0.1 0 (Table 2)
would be a more meaningful method of assessment than As [21]. Although these showed no significant difference for the
0.30 0.20
combined
0.10
shown for fractures
0 DO
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
interval,
categories TPF
=
(p >
.5), a significant
for all readers
0.722
(p
=
.04,
difference was confidence
95%
to 0.772).
B Discussion Our study
shows
PPDR in acute minor trauma
to be less
accurate than is conventional screen/film radiography. Although this difference is statistically significant, we doubt that it has major clinical significance. Close cooperation between clinician and radiologist, careful clinical examination, and the
selective
use of conventional
radiographs
in appropriate
pa-
tients should result in reliable detection of subtle fractures when PPDR is used as the primary imaging technique. We failed to show a statistically significant difference between the two techniques, using ROC curves and comparing
C Fig.
3.-Receiver-operating-characteristic
curves
for
reader I (A), reader 2 (B), and reader 3 (C). Shaded boxes black crosses = digital radiograph.
fractures = plain
for film,
A values. However, when looking only at FPFs of 0.1 0, we see a significant difference for fracture detection, with conventional radiography being superior. We believe that the latter method is the better method when applied to these data
[21]. TABLE 0.10
2: True-Positive
Combined Film/screen
Computed Fractures
Fractions
at False-Positive
Fraction of
Reader No.1
Reader No.2
Reader No.3
0.8145 0.8084
0.861 0 0.81 77
0.8560 0.8503
0.8256 0.7457
0.8330 0.71 23
0.8899 0.7829
data radiographs
radiographs
Film/screen
radiographs
Computed
radiographs
This study has several strengths: The study cases were all pained, which is an efficient experimental method to detect small, but real, differences in diagnostic accuracy. The range of abnormalities seen represents an important and common
disease spectrum. The readers, while all competent nadiologists, had a range of skill levels representative of those found among subspecialists and general radiologists. Finally, the image parameters used for the PPDR images were optimized for the different anatomic areas studied. On the other hand, the study has at least two weaknesses: We demonstrated a large intrareader variance and we used the images themselves to establish “radiographic truth,” in
readings
the absence
of any better
difference
shortcomings
should
of the 30 randomly selected cases showed no for reader 1 , a moderate difference for reader 2, and a marked difference for reader 3. It should be noted that reader 3 was also the least experienced radiologist. The magnitude of the intrareader differences for readers 2 and 3
small
differences
decrease
between
gold the
the two
standard.
Both
sensitivity
of this study to techniques. As
imaging
of these
we believe that the clinical impact of our demonstrated difference between PPDR and conventional radiography is small,
AJR:157,
September
and that
DIGITAL
1991
it can be compensated
for during
RADIOGRAPHY
clinical
practice,
Downloaded from www.ajronline.org by 179.181.160.140 on 06/22/16 from IP address 179.181.160.140. Copyright ARRS. For personal use only; all rights reserved
we do not believe that these weaknesses are critical. The potential advantages of PPDR over conventional nadiography have been reported by others [1 -1 8] and include: greater dynamic range than film, ability to correct for exposure errors with a resultant decreased rate of repeated examinations, ability to apply postpnocessing algorithms (such as edge enhancement), and ability to transmit images almost instantaneously to remote viewing areas. The major disadvantage of PPDR is a measurable loss of spatial resolution as com-
pared with standard tion of the spatial
radiography
resolution
extremity
systems.
requirements
by using conventional
radiognaphs
optically
digitized
that some
that have been fine detail
may be
lost at matrix sizes similar to the size used by the current PPDR systems (251 0 x 2000) [25-27]. PPDR images have a potential advantage over digitized conventional nadiographs. The latter cannot present any data that were not on the original conventional film. On the other hand, an image that was initially acquired by a digital system
may indeed
contain
data that could
not be captured
on a
single exposure by the lower dynamic range of the screen/ film system or data that may have been lost through incorrect exposure.
For
example,
two
conventional
radiographs
result
the digital
of the decreased
data manipulation Previous
[25-27],
image
may contain
spatial
resolution,
electronic
digitized
conventional
films
have suggested
that a pixel size smaller
than 0.2
using
for detection
537
nadiographs. The PCR 8 x 1 0 cassettes achieve a of 0.1 mm, but this resolution is still inferior to that with conventional screen/film systems in current intent was to test the diagnostic efficacy of a method digital acquisition in a demanding clinical setting and mine whether or not the lessen spatial resolution system, as compared with conventional screen/film naphy, was an obvious handicap. We believe that
mix obtained
in this study
is representative
cases in our emergency department and should therefore have provided
difference
pixel size achieved use. Our of direct to deterof the radiogthe case
of the extremity
during an average a realistic test.
in fracture
detection
week,
was shown
for all readers at 0.1 0 FPF, there may indeed be a clinically significant difference between PPDR and conventional radiography. Further, a semiquantitative review of the literature has revealed that, although statistically significant differences have not been shown between the two imaging methods, conventional radiography generally outperforms PPDR. We have failed to show that PPDR is equal to traditional film/ screen systems and feel that if PPDR is used as the primary imaging method for acute trauma to the extremities, conventional radiography should still be available.
testing
with larger groups
of patients
with
more focused ranges of examinations and abnormalities is necessary to evaluate more completely the effectiveness of this technique for extremity imaging. Particular attention must
be paid to the detection
of subtle fractures.
We are concerned
that the spatial resolution of currently available photostimulable phosphor systems may be insufficient for such systems to completely replace state-of-the-art film/screen radiography.
as a
noise,
or
after imaging.
studies,
mm is necessary
less data,
EXTREMITIES
More extensive
of a
hand may be necessary to reveal soft-tissue detail in the fingertips and bony detail in the distal radius, whereas both anatomic areas can be displayed on the same digital image (Fig. 4). It is therefore possible that a PPDR image may present data to the reader that are simply not present on a conventional radiograph obtained under similar conditions.
Conversely,
THE
As a significant
for musculoskeletal
imaging
has suggested
Evalua-
OF
REFERENCES
of fine bone
detail
on digital
1 . Sonoda
M, Takano M, Miyahara J, Kato H. Computed radiography utilizing laser stimulated luminescence. Radiology 1983; 148:833-838
scanning
2. Kato H, Miyahara J, Takano M. New computed radiography using scanning laser stimulated luminescence. Neurosurg Rev 1985;8:53-62 3. Higashida V. Moribe N, Hirata V. et al. Computed radiography utilizing laser-stimulated
luminescence: detectability of simulated low-contrast raComput Med Imaging Graph 1988:12: 137-1 45 4. Fuhrman CR, Gur D, Schaetzing R. High-resolution digital imaging with storage phosphors. J Thorac Imaging 1990:5:21-30 5. Blume H. PCR technical review. Shelton CT: Philips Medical Systems,
diographic objects.
Issue #1; April 1989:1-14 6. Sherrier RH, Chotas GH, Johnson GA, Chiles C, Ravin CE. Image optimization in a computed-radiography/photostimulable-phosphor system. J DigitalImaging 1989:2:212-219 7. Hillman BJ, Fajardo LL. Clinical assessment radiography: equipment strategy and
of phosphor-plate computed methods. J Digital Imaging
1989:2:220-227 8. Kangarloo H, Capek P. Barbaric Z, et al. Two-year clinical experience with a computed radiography system. AJR 1988:151:605-608 9. Kogutt MS, Jones JP, Perkins DP. Low-dose digital computed radiography in pediatric chest imaging. AJR 1988:151:775-779 10. Margin SI, Rowberg AH, Godwin JD. Preliminary experience with portable digital
imaging
for
intensive
care
49-54 1 1 . Schaefer CM, Greene R, Oestmann
A
imaging disease.
B
Fig. 4.-A and B, Computed (A) and conventional (B) radiographs enchondroma in a proximal phalanx. Although both images show well, soft tissues are more visible on computed radiograph.
radiography.
J Thorac
tion
with
1990:5:
JW, et al. Digital storage phosphor in CT-documented chest
versus conventional film radiography Radiology 1990:174:207-210
12. Schaeffer CM, Greene R, Hall DA, et al. Mediastinal of an tumor
Imag
storage
phosphor
digital
radiography.
abnormalities:
detec-
Radiology
1991:178:
N. Digital
radiography
169-1 73 1 3. Pettersson
H, Aspelin
P. Bousen
E, Herrlin
K, Egund
WILSON
538
of the spine, large bones and joints using stimulable phosphor. Acta Radiol 1988;29:267-27i 14. Yang PJ, Seeley GW, Carmody RF, Seeger JF, Yoshino MT. Mockbee B. Conventional vs computed 1988;9: 165-1 68
radiography:
evaluation of myelography.
AJNR
Downloaded from www.ajronline.org by 179.181.160.140 on 06/22/16 from IP address 179.181.160.140. Copyright ARRS. For personal use only; all rights reserved
15. Chan HP, vytomy
CJ, MacMahon H, Metz CE, Dci K, Sickles EA. Digitel of the effects of pixel size and unsharp-mask on the detection of subtle microcalcifications. Invest Radiol
mammography
filtering
ROC studies
1987;22:581-589
i6. Oestmann JW, Kopans D, Hail DA, McCarthy KA, Rubens JR, Greene R. A comparison of digitized storage phosphors and conventional mammography
in the
detection
of
malignant
microcalcifications.
Invest
ET AL.
1991
radiography: effect of varying exposure factors and unsharp mask filtering on the detectability of cortical bone defects. Radiology 1990;177:109-1i3 21 . Swets JA, Picket RM. Evaluation of diagnostic systems. New York: Acedemic Press, 1982:38-39 22. Swets JA, Swets JB. ROC approach to cost-benefit analysis. Proceedings ofsixth conference on computerapplications in radiology. New York: IEEE, 1979: 203-206
23. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143:29-36
Radiol
24. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology
stimu-
25. Murphey MD. Digital skeletal radiography: spatial resolution requirements
1983;148:839-843 H, Bousen
E. Digital radiography using for evaluation of urinary calcifications.
laser
Acta Radiol
26.
1988;29:277-284
18. Lee KR, Chang J, Murphey MD, et al. Computed radiographic imaging in pediatric examinations. AppI Radiol 1990;19:27-34 19. Nakano Y, Hiraoka T, Togashi K, et al. Direct radiographic magnification with computed radiography. AiR 1987;148:569-573 20. Prokop M, Galanski M, Oestmann JW, et al. Storage phosphor versus
The Radiology help disadvantaged
27.
for detection of subperiosteal resorption. AJR 1989;152:541-546 Murphey MD, Bramble JM, Cook LT, Martin NL, Dwyer SJ. Nondisplaced fractures: spatial resolution requirements for detection with digital skeletal imaging. Radiology 1990;174:865-870 Wegryn SA Piraino DW, Richmond BJ, et al. Comparison of digital and conventional musculoskeletal radiography: an observer performance study. Radiology 1990;175:225-228
(ROF) is a nonprofit corporation whose goal is to their health care by providing radiology equipment, books, consultation, education, and training to their practitioners. This assistance is on an application basis that is independent of political, ethnic, or religious orientation of the grantee. It depends on the need of the people and the ability of the ROF to meet that need. The ROF is approved by the U.S. Internal Revenue Service as a tax-exempt organization. It is endorsed by the following radiologic societies: American Association of Women Radiologists, American College of Radiology, American Roentgen Ray Soci-
Outreach Foundation Countries improve
ety, Association Society
September
screen-film
1988;23:725-728 17. Aspelin P, Petterson lated luminescence
AJR:i57,
of
of Chairmen
ogy, European
Society
University of Academic of Podiatric
Radiologists,
Radiological
Society
Radiology
Departments,
Society
of for
North Pediatric
America, Radiol-
Radiology.
All donations to the ROF are tax deductible. Persons who would like to contribute financially to the ROF, would be interested in being a visiting professor, would like to send books or journals to any of the institutions supported by the ROF, or would like further information about the ROF should write to Charles A. Gooding, M.D., President, Radiology Outreach Foundation, 341 5 Sacramento St., San Francisco, CA 941 1 8 USA.
533
Photostimulable Radiography
Phosphor Digital of the Extremities:
Diagnostic Accuracy Compared Conventional Radiography
Anthony
J. Wilson1 F. A. Mann William A. Murphy, Jr. Barbara S. Monsees Margaret R. Linn
A direct
comparison
was
made
between
digital
and
with
conventional
radiographs
to
assess the relative accuracy of a photostimulable phosphor digital imaging system in detecting and localizing minor trauma in the extremities. Matched sets of images were obtained on 103 patients who came to the emergency the hand, wrist, foot, or ankle. One set was obtained
system.
The other set was obtained
department for radiographs of with a conventional screen/film
with photostimulable
phosphor
digital cassettes.
The two sets of images radiologists in a blinded
of each patient were independently interpreted by three fashion. The findings of each of these three readers were compared with the consensus opinion of two different radiologists. Receiver-operatingcharacteristic (ROC) curves were plotted for each of the three readers, areas under the curves were calculated, and true-positive fractions were determined at false-positive fractions of 0.1. Although no significant differences in the areas under the ROC curves for the two imaging systems were detected, conventional radiography showed a slight
advantage.
However,
when true-positive
fractions
for fracture detection
were compared
at false-positive fractions of 0.1 a statistically significant difference was shown, with conventional screen/film radiography being more sensitive. This study raises questions about the use of currently available photostimulable phosphor systems for imaging trauma of the extremities and suggests that those systems should not be used exclusively. AJR
157:533-538,
September
1991
Photostimulable phosphor digital [1 ] and has been the subject
Japan [7-18]
studies.
These reports
radiography (PPDR) was of a number of technical
have compared
radiography, describing a much greater spatial resolution than with conventional limited use in a variety of settings including several authors have studied the efficacy
[1 3, 14], breast results
have
been
[1 5, 16], and abdominal published
regarding
[1 3, 19, 20]. We were concerned Received March 11, 1991; accepted sion April29, 1991.
after revi-
Presented at the annual meeting of the American Ray Society, Washington, DC, May 1990. 1 All authors: Mallinckrodt Institute of Radiology, Roentgen
Washington University School of Medicine, St. Louis, MO 63110. Address reprint requests to A. J. Wilson.
0361-803X/91/1573-0533 © American Roentgen Ray Society
first developed in [1 -7] and clinical
PPDR with conventional
screen/film
dynamic range with PPDR, but lower radiography. The technique has been in emergency departments [7, 8]. Although of this technique for chest [9-1 2], spine
radiography
musculoskeletal
that a system,
[1 7, 18], few investigative applications
promoted
of the system
to replace
the gold
standard of conventional radiography, is being used in clinical settings for which it has not been adequately validated. Before using PPDR for extremity imaging in our own emergency department, we thought that a study was necessary to compare this technique directly with the established standard of conventional
radiography. The purpose of this study was to compare directly the diagnostic accuracy of the PPDR system with our existing conventional screen/film combination for extremity imaging in an emergency department setting. As a wide range of extremity disorders are seen in the emergency department, it seemed appropriate to use cases from the full spectrum of these. The question to be answered was: Is PPDR a satisfactory alternative to conventional radiography, or does the known loss in
534
WILSON
spatial resolution with PPDR affect on diagnostic efficacy?
have
a measurable
ET AL.
adverse
type
AJR:157,
of examination
used, as excessive tectability
Downloaded from www.ajronline.org by 179.181.160.140 on 06/22/16 from IP address 179.181.160.140. Copyright ARRS. For personal use only; all rights reserved
Materials Study
performance
with
paired
photostimulable
phosphor
High-definition
im-
[5]. The 8 x 1 0 cassettes
standard.
life-threatening
disorders,
impaired
mental
was approved
pregnant
status
were
by our institutional
patients, not
children,
studied.
The
and patients study
fine
protocol
Reference
screens
nations included three examinations included The PCR system printed
onto
and
T-Mat
“truth”
G film (Eastman Kodak Company,
no
views: frontal, lateral, and oblique. four views: posteroanterior, ulnar installed
video
display
film by using
supplied
Fig. 1.-Computed laterally. densest
images
14,
per film, one of which
resolution
of 5 line pairs per
in our institution capability.
standardized
The
images produced
postprocessing
by the manufacturer
(A, B) and conventional Although fractures are readily visible bony areas easier to see.
(C, D) radiographs on both
sets
established
from
the images
alone.
To establish
truth,
and digital
image
sets were studied
together,
and finally
a consensus was achieved between the two radiologists by discussion of differences of opinion. Although focusing on minor trauma, this truth included 14 separate categories of positive findings (Table 1). Arthritic alterations were further subdivided into cartilage
loss, erosions,
osteophytes,
and other arthritic changes.
on are
Image
parameters
and were optimized
was
conventional
The wrist deviation,
produces images
Standard
two experienced musculoskeletal radiologists performed a pilot study. Initially, each set of images was studied alone, then the
selected before imaging. These parameters were changed from those originally
reduced-size
fullx
As the study group consisted of patients in the emergency department for whom reliable follow-up was difficult to obtain, and because there was no other logical gold standard for diagnosis, the radiologic
lateral, and oblique. has
with a single
review board.
Rochester, NY). The examination was then repeated by the same technologist by using Philips Computed Radiography (PCR) 8 x 10 in. (20 x 25 cm) cassettes (Philips Medical Systems, Shelton, CT) with the same X-ray tube and generator, the same exposure factors, and the same projections (Fig. 1). The hand, foot, and ankle exami-
and
the viewer
millimeter (Ip/mm) [5]. This is clearly less than the resolution of 1 4 Ip/ mm claimed by the manufacturer for the screen/film system used in this study [Kodak, personal communication].
Matched pairs ofconventional and PPDR images of the extremities of 103 patients were obtained. After informed consent was obtained, the examination requested by the referring physician was performed
film
because
the larger sizes (1 0 x 12, 14
a pixel size of 0.1 mm and spatial
Acquisition
Lanex
selected
the 8 x 1 0 cassettes produce images much closer in appearance to conventional film than do the other cassette sizes. PPDR images obtained on 8 x 1 0 cassettes have a 251 0 x 2000 matrix, producing
Cases for this study were recruited from our urban emergency department (65,000 annual visits). Patients having hand, wrist, foot, and ankle examinations, on weekdays between 8 am. and 4 p.m., were offered the opportunity to participate in the study. Patients with
with
two
were
usually has a high degree of digital edge enhancement. The option of two images on the film is not available for 8 x 10 cassettes. Thus
Group
Image
was
than the larger PCR cassettes
also present
sized image on film, whereas 14 x 17) produce
with
enhancement
radiographs as closely as on a laser disk for archival
8 x 1 0 in. PCR cassettes
these have higher spatial resolution
ages and with conventional radiographs was compared by using data from three readers. A consensus panel was used to establish the
Study
edge
purposes and additional postprocessing, but this does not occur automatically. All of the images in this study were stored on disk.
Design
reference
Minimal
1991
enhancement has been shown to decrease dodefects [20]. The factors used were designed to
make the final images mimic conventional possible. The PCR image may be stored
and Methods
Observer
of cortical
in the study.
September
Three
for each
without
of an ankle
of radiographs,
Evaluation
showing increased
other
access
lateral dynamic
radiologists
to clinical
then
interpreted
information.
each
set
The experience
of images,
level and
and posterior malleolar fractures with soft-tissue swelling range of computed images makes both soft tissues and
DIGITAL
AJR:157,
September
1991
TABLE
1: Range
and Number
of Abnormalities
Type of Abnormality
Downloaded from www.ajronline.org by 179.181.160.140 on 06/22/16 from IP address 179.181.160.140. Copyright ARRS. For personal use only; all rights reserved
RADIOGRAPHY
OF THE
EXTREMITIES
535
:#{149}#{149}
Recorded
No. of Cases
Fractures Arthritic changes Soft-tissue swelling Nonarticular bony spurs
31 28 28 14
Generalized
12
0.40
Dislocations
6
0.30
Vascular calcifications
5
Lacerations/ulcerations
5 4
0.20
osteopenia
Foreign bodies Tumors Arthrodeses
0.10
2 2
Amputations Medullary infarct Total
0.0c 0.00
.
,
,
,
,
0.10
0.20
0.30
0.40
0’1
0.2
0.3
0.4
,
0.50
,
0.60
,
0.70
“
0.80
-
0.90
1.00
A
2 1 140
0.9 0.8 0.7 0.6
subspecialty experienced
expertise
radiologist, completion
of the three
readers
varied:
One was an
0.5
musculoskeletal radiologist, one an experienced general and one a musculoskeletal fellow less than 6 months after of residency. A separate data sheet was provided for
0.4
each reader for each set of images (206 sheets per reader). The data sheet required a positive response to each of the radiographic features (Table 1) using a five-point normal, 2 = probably normal, 3 abnormal, 5 = definitely abnormal).
recorded with free text. The digital and conventional
confidence score possibly abnormal,
=
The
location
(1 4
of each
=
definitely
=
probably
finding
was
0.3 0.2 0.1 ‘0
0.5
examinations
0.6 0.5 0.4 0.3 02 0.1
C
Statistical
Fig. 2.-Receiver-operating-characteristic reader 1 (A), reader 2 (B), and reader black crosses = digital radiograph. were
generated
and comparisons
made,
curves
(Figs.
2 and 3) for the performance
of each reader
with conventional radiography and PPDR were generated from the confidence scores by using a personal computer-based program (CORROC 2, Metz et al., University of Chicago, Chicago, IL). Comparisons of true-positive fractions (TPF) at a false-positive fraction (FPF) of 0.1 0 for each reader were tested for significance, using critical
ratios
FPF because settings [22].
(Table 2) [21]. The value it reflects the reported
Co
0.2
0.3
04
of 0.1 0 was chosen for the utility in routine diagnostic
the readers.
readers.
Results
These
curves
were
The ROC curves
a lack of any statistical tional
radiography
the
compared 1 03 matched
25 hands,
50 women
and 53 men, from
age of 40 years.
pairs
of images,
14 fingers,
there
and 13 wrists.
1 9 to 85 years
were
07
0.8
3 (C).
curves for combined data for Shaded boxes = plain film,
26 feet,
There were
old, with a mean
closely
derived
matched
for fractures
for all three
alone (Fig. 3)
yielded A values as listed in Table 3. Again no significant difference was found between the paired A values. Despite
data. Using
Of the
06
The ROC curves plotted for the combined categories (Fig. 2) yielded an area under each curve (As) as shown in Table 3. No significant difference in A was shown for any of
difference,
to have
and
last
methods and
bution.
exists
Similarly,
and
we found
of either comparison
almost
between
than
and combined [23,
readings
reduction
learning
show conven-
performance
McNeil
each observer’s
readers 1 and 2, but a moderate no evidence
overall
for both fractures
of Hanley
As between 20 cases,
the curves
a better
PPDR had for all readers
25 ankles,
0.5 FPF
for each
reader, for fractures (n = 31) and for the combined data of all of the 14 findings listed in Table 1 (n = 1 40). Receiver-operating-characteristic (ROC)
0.9
0.7
was asked to look at 30 of the examinations a second time, several weeks after the initial readings. These 30 examinations were randomly selected; 15 were conventional radiographs and 15 were PCR images.
data
0.8
0.8
window level and width or to perform any other postprocessing. Finally, to estimate intrareader variance, each of the three radiologists
Methods
0.7
0.9
of each case were presented at four separate sittings in a 2-month period, with digital and conventional examinations mixed in equal quantities during each reading session. The digital and conventional examinations of each case were presented at different times. All conventional and digital images were presented on film. Each reader had access to only one copy of each digital image and did not have the opportunity to alter
Statistical
0.6
B
24],
we
of the first
no difference
for
for reader 3. Thus,
effect the
or case first
and
maldistrisecond
536
WILSON
ET AL.
AJR:i57,
TABLE
3: Areas
Under
September1991
Receiver-Operating-Characteristic
Curves
0.90 0.80 0.70
Reader No.1
Reader No.2
Reader No.3
0.9374 0.9281
0.9534 0.9382
0.9517 0.9504
Downloaded from www.ajronline.org by 179.181.160.140 on 06/22/16 from IP address 179.181.160.140. Copyright ARRS. For personal use only; all rights reserved
0.60
Combined
0.50 0.40
data
Film/screen
radiographs
Computed
radiographs (p
0.30 0.20 0.10 0.0c 0 DO
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.
=
(p
.6079)
=
.51 21)
Fractures Film/screen
radiographs
0.9004
0.881 0
Computed
radiographs
0.8842
0.8280
(p
)0
=
(p
.8085)
=
(p
=
.9352)
0.9618
0.8788 (p
.6063)
=
.2062)
A was greater
0.90
than their intertechnique
ens our ability to recognize small, tween techniques in this study.
0.80 0.70
differences. but real,
This weak-
differences
be-
As the shapes of the ROC curves for PPDR and SFR differ
0.60
(i.e., all but one of the ROC thought a direct comparison
0.50 0.40
curves cross; Figs. 2 and 3), we of TPFs at FPF 0.1 0 (Table 2)
would be a more meaningful method of assessment than As [21]. Although these showed no significant difference for the
0.30 0.20
combined
0.10
shown for fractures
0 DO
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
interval,
categories TPF
=
(p >
.5), a significant
for all readers
0.722
(p
=
.04,
difference was confidence
95%
to 0.772).
B Discussion Our study
shows
PPDR in acute minor trauma
to be less
accurate than is conventional screen/film radiography. Although this difference is statistically significant, we doubt that it has major clinical significance. Close cooperation between clinician and radiologist, careful clinical examination, and the
selective
use of conventional
radiographs
in appropriate
pa-
tients should result in reliable detection of subtle fractures when PPDR is used as the primary imaging technique. We failed to show a statistically significant difference between the two techniques, using ROC curves and comparing
C Fig.
3.-Receiver-operating-characteristic
curves
for
reader I (A), reader 2 (B), and reader 3 (C). Shaded boxes black crosses = digital radiograph.
fractures = plain
for film,
A values. However, when looking only at FPFs of 0.1 0, we see a significant difference for fracture detection, with conventional radiography being superior. We believe that the latter method is the better method when applied to these data
[21]. TABLE 0.10
2: True-Positive
Combined Film/screen
Computed Fractures
Fractions
at False-Positive
Fraction of
Reader No.1
Reader No.2
Reader No.3
0.8145 0.8084
0.861 0 0.81 77
0.8560 0.8503
0.8256 0.7457
0.8330 0.71 23
0.8899 0.7829
data radiographs
radiographs
Film/screen
radiographs
Computed
radiographs
This study has several strengths: The study cases were all pained, which is an efficient experimental method to detect small, but real, differences in diagnostic accuracy. The range of abnormalities seen represents an important and common
disease spectrum. The readers, while all competent nadiologists, had a range of skill levels representative of those found among subspecialists and general radiologists. Finally, the image parameters used for the PPDR images were optimized for the different anatomic areas studied. On the other hand, the study has at least two weaknesses: We demonstrated a large intrareader variance and we used the images themselves to establish “radiographic truth,” in
readings
the absence
of any better
difference
shortcomings
should
of the 30 randomly selected cases showed no for reader 1 , a moderate difference for reader 2, and a marked difference for reader 3. It should be noted that reader 3 was also the least experienced radiologist. The magnitude of the intrareader differences for readers 2 and 3
small
differences
decrease
between
gold the
the two
standard.
Both
sensitivity
of this study to techniques. As
imaging
of these
we believe that the clinical impact of our demonstrated difference between PPDR and conventional radiography is small,
AJR:157,
September
and that
DIGITAL
1991
it can be compensated
for during
RADIOGRAPHY
clinical
practice,
Downloaded from www.ajronline.org by 179.181.160.140 on 06/22/16 from IP address 179.181.160.140. Copyright ARRS. For personal use only; all rights reserved
we do not believe that these weaknesses are critical. The potential advantages of PPDR over conventional nadiography have been reported by others [1 -1 8] and include: greater dynamic range than film, ability to correct for exposure errors with a resultant decreased rate of repeated examinations, ability to apply postpnocessing algorithms (such as edge enhancement), and ability to transmit images almost instantaneously to remote viewing areas. The major disadvantage of PPDR is a measurable loss of spatial resolution as com-
pared with standard tion of the spatial
radiography
resolution
extremity
systems.
requirements
by using conventional
radiognaphs
optically
digitized
that some
that have been fine detail
may be
lost at matrix sizes similar to the size used by the current PPDR systems (251 0 x 2000) [25-27]. PPDR images have a potential advantage over digitized conventional nadiographs. The latter cannot present any data that were not on the original conventional film. On the other hand, an image that was initially acquired by a digital system
may indeed
contain
data that could
not be captured
on a
single exposure by the lower dynamic range of the screen/ film system or data that may have been lost through incorrect exposure.
For
example,
two
conventional
radiographs
result
the digital
of the decreased
data manipulation Previous
[25-27],
image
may contain
spatial
resolution,
electronic
digitized
conventional
films
have suggested
that a pixel size smaller
than 0.2
using
for detection
537
nadiographs. The PCR 8 x 1 0 cassettes achieve a of 0.1 mm, but this resolution is still inferior to that with conventional screen/film systems in current intent was to test the diagnostic efficacy of a method digital acquisition in a demanding clinical setting and mine whether or not the lessen spatial resolution system, as compared with conventional screen/film naphy, was an obvious handicap. We believe that
mix obtained
in this study
is representative
cases in our emergency department and should therefore have provided
difference
pixel size achieved use. Our of direct to deterof the radiogthe case
of the extremity
during an average a realistic test.
in fracture
detection
week,
was shown
for all readers at 0.1 0 FPF, there may indeed be a clinically significant difference between PPDR and conventional radiography. Further, a semiquantitative review of the literature has revealed that, although statistically significant differences have not been shown between the two imaging methods, conventional radiography generally outperforms PPDR. We have failed to show that PPDR is equal to traditional film/ screen systems and feel that if PPDR is used as the primary imaging method for acute trauma to the extremities, conventional radiography should still be available.
testing
with larger groups
of patients
with
more focused ranges of examinations and abnormalities is necessary to evaluate more completely the effectiveness of this technique for extremity imaging. Particular attention must
be paid to the detection
of subtle fractures.
We are concerned
that the spatial resolution of currently available photostimulable phosphor systems may be insufficient for such systems to completely replace state-of-the-art film/screen radiography.
as a
noise,
or
after imaging.
studies,
mm is necessary
less data,
EXTREMITIES
More extensive
of a
hand may be necessary to reveal soft-tissue detail in the fingertips and bony detail in the distal radius, whereas both anatomic areas can be displayed on the same digital image (Fig. 4). It is therefore possible that a PPDR image may present data to the reader that are simply not present on a conventional radiograph obtained under similar conditions.
Conversely,
THE
As a significant
for musculoskeletal
imaging
has suggested
Evalua-
OF
REFERENCES
of fine bone
detail
on digital
1 . Sonoda
M, Takano M, Miyahara J, Kato H. Computed radiography utilizing laser stimulated luminescence. Radiology 1983; 148:833-838
scanning
2. Kato H, Miyahara J, Takano M. New computed radiography using scanning laser stimulated luminescence. Neurosurg Rev 1985;8:53-62 3. Higashida V. Moribe N, Hirata V. et al. Computed radiography utilizing laser-stimulated
luminescence: detectability of simulated low-contrast raComput Med Imaging Graph 1988:12: 137-1 45 4. Fuhrman CR, Gur D, Schaetzing R. High-resolution digital imaging with storage phosphors. J Thorac Imaging 1990:5:21-30 5. Blume H. PCR technical review. Shelton CT: Philips Medical Systems,
diographic objects.
Issue #1; April 1989:1-14 6. Sherrier RH, Chotas GH, Johnson GA, Chiles C, Ravin CE. Image optimization in a computed-radiography/photostimulable-phosphor system. J DigitalImaging 1989:2:212-219 7. Hillman BJ, Fajardo LL. Clinical assessment radiography: equipment strategy and
of phosphor-plate computed methods. J Digital Imaging
1989:2:220-227 8. Kangarloo H, Capek P. Barbaric Z, et al. Two-year clinical experience with a computed radiography system. AJR 1988:151:605-608 9. Kogutt MS, Jones JP, Perkins DP. Low-dose digital computed radiography in pediatric chest imaging. AJR 1988:151:775-779 10. Margin SI, Rowberg AH, Godwin JD. Preliminary experience with portable digital
imaging
for
intensive
care
49-54 1 1 . Schaefer CM, Greene R, Oestmann
A
imaging disease.
B
Fig. 4.-A and B, Computed (A) and conventional (B) radiographs enchondroma in a proximal phalanx. Although both images show well, soft tissues are more visible on computed radiograph.
radiography.
J Thorac
tion
with
1990:5:
JW, et al. Digital storage phosphor in CT-documented chest
versus conventional film radiography Radiology 1990:174:207-210
12. Schaeffer CM, Greene R, Hall DA, et al. Mediastinal of an tumor
Imag
storage
phosphor
digital
radiography.
abnormalities:
detec-
Radiology
1991:178:
N. Digital
radiography
169-1 73 1 3. Pettersson
H, Aspelin
P. Bousen
E, Herrlin
K, Egund
WILSON
538
of the spine, large bones and joints using stimulable phosphor. Acta Radiol 1988;29:267-27i 14. Yang PJ, Seeley GW, Carmody RF, Seeger JF, Yoshino MT. Mockbee B. Conventional vs computed 1988;9: 165-1 68
radiography:
evaluation of myelography.
AJNR
Downloaded from www.ajronline.org by 179.181.160.140 on 06/22/16 from IP address 179.181.160.140. Copyright ARRS. For personal use only; all rights reserved
15. Chan HP, vytomy
CJ, MacMahon H, Metz CE, Dci K, Sickles EA. Digitel of the effects of pixel size and unsharp-mask on the detection of subtle microcalcifications. Invest Radiol
mammography
filtering
ROC studies
1987;22:581-589
i6. Oestmann JW, Kopans D, Hail DA, McCarthy KA, Rubens JR, Greene R. A comparison of digitized storage phosphors and conventional mammography
in the
detection
of
malignant
microcalcifications.
Invest
ET AL.
1991
radiography: effect of varying exposure factors and unsharp mask filtering on the detectability of cortical bone defects. Radiology 1990;177:109-1i3 21 . Swets JA, Picket RM. Evaluation of diagnostic systems. New York: Acedemic Press, 1982:38-39 22. Swets JA, Swets JB. ROC approach to cost-benefit analysis. Proceedings ofsixth conference on computerapplications in radiology. New York: IEEE, 1979: 203-206
23. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143:29-36
Radiol
24. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology
stimu-
25. Murphey MD. Digital skeletal radiography: spatial resolution requirements
1983;148:839-843 H, Bousen
E. Digital radiography using for evaluation of urinary calcifications.
laser
Acta Radiol
26.
1988;29:277-284
18. Lee KR, Chang J, Murphey MD, et al. Computed radiographic imaging in pediatric examinations. AppI Radiol 1990;19:27-34 19. Nakano Y, Hiraoka T, Togashi K, et al. Direct radiographic magnification with computed radiography. AiR 1987;148:569-573 20. Prokop M, Galanski M, Oestmann JW, et al. Storage phosphor versus
The Radiology help disadvantaged
27.
for detection of subperiosteal resorption. AJR 1989;152:541-546 Murphey MD, Bramble JM, Cook LT, Martin NL, Dwyer SJ. Nondisplaced fractures: spatial resolution requirements for detection with digital skeletal imaging. Radiology 1990;174:865-870 Wegryn SA Piraino DW, Richmond BJ, et al. Comparison of digital and conventional musculoskeletal radiography: an observer performance study. Radiology 1990;175:225-228
(ROF) is a nonprofit corporation whose goal is to their health care by providing radiology equipment, books, consultation, education, and training to their practitioners. This assistance is on an application basis that is independent of political, ethnic, or religious orientation of the grantee. It depends on the need of the people and the ability of the ROF to meet that need. The ROF is approved by the U.S. Internal Revenue Service as a tax-exempt organization. It is endorsed by the following radiologic societies: American Association of Women Radiologists, American College of Radiology, American Roentgen Ray Soci-
Outreach Foundation Countries improve
ety, Association Society
September
screen-film
1988;23:725-728 17. Aspelin P, Petterson lated luminescence
AJR:i57,
of
of Chairmen
ogy, European
Society
University of Academic of Podiatric
Radiologists,
Radiological
Society
Radiology
Departments,
Society
of for
North Pediatric
America, Radiol-
Radiology.
All donations to the ROF are tax deductible. Persons who would like to contribute financially to the ROF, would be interested in being a visiting professor, would like to send books or journals to any of the institutions supported by the ROF, or would like further information about the ROF should write to Charles A. Gooding, M.D., President, Radiology Outreach Foundation, 341 5 Sacramento St., San Francisco, CA 941 1 8 USA.
