0730-725X/92 $S.O!l + .OO Copyright 0 1992 Pergamon Press plc
Magnetic Resonance Imaging, Vol. 10. pp. 143-154, 1992 Printed in the USA. All rights reserved.
0 Original Contribution COMPOSITE AND CLASSIFIED COLOR DISPLAY IN MR IMAGING OF THE FEMALE PELVIS H. KEITH BROWN,*? TODD R. HA~ELTON,~ JAMES V. FIORICA,§ ANNA K. PARSONS,~ LAURENCE P. CLARKE,? AND MARTIN L. SILBIGER~ Departments of *Anatomy, tRadiology, and #Obstetrics and Gynecology, University of South Florida College of Medicine, Tampa, Florida 33612-4799,USA Because of its superior soft-tissue-imaging capabilities, MRI has proved to be an excellent modality for visualizing the contents of the female pelvis. In an effort to potentially improve gynecological MRI studies, we have applied color composite techniques to sets of spin-echo and gradient-echo gray-tone MR images obtained from various individuals. For composite generation, based on tissue region of interest calculated mean pixel intensity values, various colors were applied to spatially aligned images using a DEC MicroVAX II computer with interactive digital language (IDL) so that tissue contrast patterns could be optimized in the final image. The IDL procedures, which are similar to those used in NASA’s LANDSAT image processing system, allowed the generation of single composite images displaying the combined information present in a series of spatially aligned images acquired using different pulse sequences. With our composite generation techniques, it was possible to generate seminatural-appearing color images of the female pelvis that possessed enhanced conspicuity of specific tissues and fluids. For comparison with color composites, classified images were also generated based on computer recognition and statistical separation of distinct tissue intensity patterns in an image set using the maximum likelihood processing algorithm. Keywords:
Magnetic resonance imaging; Pelvis; MRI studies; Pelvis, female.
substantial amount of data, studies for assessing the diagnostic utility of MRI for the evaluation of various pathologic conditions within the female pelvis have yet to characterize the full clinical potential of this noninvasive imaging modality. Because the intensity characteristics of individual tissues in different types of MR images are dependent on the pulse sequence parameters selected for image acquisition, various investigators have endeavored to develop both gray-tone and color display methods for tissue characterization, based on pattern recognition techniques using multispectral MR image sets.3-8 In addition to the generation of classified images using pattern recognition techniques, various methods for
INTRODUCTION
Because of its unique ability to provide high contrast resolution and excellent soft-tissue contrast, magnetic resonance imaging (MRI) is well suited for evaluating the contents of the female pelvis. Moreover, MRI is capable of demonstrating gynecological anatomy not visible with other imaging techniques and has the additional benefit of being able to scan in multiple imaging planes.’ The advantages that these distinctive properties provide for the enhanced imaging of various gynecological pathologic conditions are evidenced by the sizable number of MRI studies of the abnormal female pelvis that have been reported.2 Even with this 2/8/9 1; ACCEPTED 6/28/91. $Medical Student, University of South Florida College of Medicine. Funding and resource support for this project were provided by H. Lee Moffitt Cancer Center and Research Institute, University Diagnostic Institute, University of South Florida President’s Council, and NASA-American Cancer Society (#89-25).Portions of this work were done during an
American Cancer Society R.G. Thompson Summer Research Fellowship (T.R.H.) and during the tenure of a Med-
RECEIVED
ical Student Research Fellowship of the American Heart Association (T.R.H.). Address correspondence and reprint requests to H. Keith Brown, Ph.D., Department of Anatomy, University of South Florida College of Medicine, Box 6, 12901 N. 30th Street, Tampa, FL 33612-4799, USA.
143
144
Magnetic Resonance Imaging 0 Volume 10, Number 1, 1992
producing hybrid, or composite, color images have been described.7v9-‘3 A method for simultaneously presenting the Ti and T2relaxation times of selected tissues in a single color image has also been reported. I4 Although the color display methods previously described in the literature have yielded images with enhanced differentiation of various tissues and fluids, most of the images that have been generated possess unnatural color combinations that often require special legends in order to facilitate their interpretation by the viewer. In contrast, methods for generating composite images that exhibit seminatural color assignments, in addition to enhanced tissue conspicuity, have been described. i5,i6 Advantages of color image display, in contrast to achromatic or monochromatic presentation, include the potential for a more realistic appearance, enhanced information processing, and increased ability of the viewer to discriminate and interpret related and unrelated data. I7 Moreover, while each display point in a gray-tone image possesses intensity as its only variable, each display point in a color image has three variable attributes (hue, saturation, and intensity), which allows a greater information capacity within the display.18 With the relatively high incidence rate and often difficult noninvasive assessment of various significant gynecological pathologic conditions, the advantages afforded by color display should not be overlooked with regard to MRI of the female pelvis. Presented in this report are applications of two methods of color image display that may potentially improve the diagnostic accuracy and efficiency of MRI studies of the abnormal female pelvis. METHODS The purpose of this investigation was to assess the feasibility of applying color composite and pattern recognition methods to various sets of MR images of the female pelvis, acquired using both routine and specialized pulse sequences. Images utilized in this study were obtained from patients and volunteers in accordance with the rules and regulations established for human studies by the Institutional Review Board of the University of South Florida Health Sciences Center. Imaging studies were performed on a 1S-tesla (T) General Electric Signa imager (General Electric Corporation, Medical Systems Group, Milwaukee, WI) and on 0.35, 1.O-, and 1.5-T Siemens Magnatom imagers (Siemens Medical Systems, Inc., Iselin, NJ). Spin-echo (SE) images were obtained using the following pulse sequences: T,-weighted images had repetition time and echo time (TRITE) values of 600/l 5 and 600/20; T,-weighted images had TR/TE values of 2000/60, 2500/80, 2500190, 2600190, and 2700/70;
and proton-density-weighted images had TR/TE values of 2000/30,2500/20,2500/28,2600/22, and 2700/28. In addition, gradient-echo (GRE) images with the following pulse sequence parameters (TR/TE/flip angle) were also acquired: 120/10/70”, 200/11/10”, and 200/l l/50”. The SE and GRE images were obtained using a body coil and had an acquisition matrix of 256 x 256. Individual sections had a thickness of either 5 or 8 mm, and adjacent slices were acquired with a 50% gap. Following acquisition, the data for image analysis and processing studies were then transferred via magnetic tape to a DEC MicroVAX II computer (Digital Equipment Corporation, Maynard, MA). These studies were performed using computer software routines developed with Interactive Digital Language (IDL) (Research Systems, Inc., Boulder, CO). Color image output was directed to an Electrohome ECM 1301-X VGA monitor (Electrohome, Ltd., Ontario, Canada). For certain tissues within analyzed images, the average signal intensities of pixels within operatorselected regions of interest (ROIs) were calculated by a component of the image analysis program in order to provide quantitative data supporting qualitative assessments of tissue contrast behaviors. Tissue ROIs were selected from various anatomical and confirmed pathological sites using procedures that allowed the calculation of mean pixel signal intensities and their standard deviations for corresponding pixels within identical ROIs in a spatially aligned, simultaneously analyzed image set. With the program, actual and normalized (O-255) signal intensity values were reported. From the quantitative normalized data obtained, image color assignments (color coefficiency values in Tables 1, 2, and 4) were specifically selected so that the desired colors of specific tissues and fluids could be visually optimized in the final composites, based on standard red (R), green (G), and blue (B) computergenerated color combinations and on characteristic tissue intensity patterns in the component images. For composite generation, previously described image-processing routines were utilized. l6 These routines were designed to allow the combination of two or more spatially aligned MR images into one multicolored composite image in a process analogous to NASA’s LANDSAT multispectral false-color infrared image generation system. I9 In the two-color method for composite generation, based on normalized intensity characteristics of specific tissue ROIs, one image was assigned to the red output and the other to the green output. For the multichannel composite method, positive or negative integers were assigned for each of the three primary colors (R, G, and B). An assigned coefficiency value of zero indicated no contribution from that color for that particular image. Negative
Color display in MRI of the female pelvis 0 H.K. BROWN ET
coefficiency values were designed to subtract the selected color. In this latter method of composite generation, individual R, G, and B images were created from the input images, based on the signal intensities of spatially aligned pixels and on the coefficiency values assigned for each image. In both methods, the assigned R and G or the computer-generated R, G, and B images were then superimposed. Based on the various intensity levels of spatially aligned pixels within an image set, a single composite image with additive color assignments was then produced. Classified images generated in this study were created utilizing the maximum likelihood (ML) processing algorithm. ” With the use of a mouse-driven system, various tissue ROIs in a spatially aligned image set were selected from known anatomical and confirmed pathological locations for supervised classification by the computer. Each ROI was then assigned a numerical value that corresponded to a particular color in a Gamma-11 color table (D.M. Stern, Research Systems, Inc., Boulder, CO). Once an individual ROI was selected, the pattern recognition component of the image analysis routines then applied the ML algorithm to each pixel within the image set. This process was repeated for each tissue to be classified. Based on their characteristic signal intensity patterns within the multispectral set of MR images being analyzed, spatially aligned pixels were mapped in feature space, l9 with pixels possessing similar intensity patterns tending to cluster together. Based on their location in feature space, the computer statistically separated the mapped pixels into classes of tissues with intensity patterns similar to those of the original ROIs selected. Using an additional image-processing routine, the created ML classification scheme was applied to the original gray-tone image set in order to generate a classified color image analogous to the thematic maps created in multispectral remote sensing applications. ‘9,21*22This application was performed using sets of two, three, and four images. In order to highlight specific tissues, a threshold function was applied to permit display of pixels with greater probability of specific tissue classification. This threshold function allowed more accurate classification of individual pixels by decreasing the area in feature space into which pixels with similar intensity patterns could be classified as the tissue to be highlighted. To select a specific tissue, the numerical color assignment for the desired ROI in a two-image set was selected. The specific highlighted tissue was then overlaid onto a spatially aligned gray-tone image, and the color table was changed to a Blue-Red color table (D.M. Stern, Research Systems, Inc., Boulder, CO) to provide greater contrast between the selected tissue and the original gray-tone image.
AL.
145
For presentation in this report, color images were photographed on ASA 200 color slide film with the aid of a macro zoom lens using exposure times of l/8 and l/15 set, and an F setting of 3.5. RESULTS
Two-Color (RG) Composites When a composite using additive color combinations is formed, if one image is assigned red and the other green, tissues whose contrast behaviors are similar will appear various intensities of yellow, depending on whether their respective signal intensities are high, intermediate, or low. Tissues that are hyperintense on one image and hypointense on the other will appear either red or green depending on which color is assigned to the image possessing the hyperintense tissue. I6 Based on the normalized tissue intensity characteristics presented in Fig. 1, a color composite of the T2- and proton-density-weighted SE images shown in Fig. 2(A) and (B), respectively, was created by assigning the proton-density-weighted image to the R output and the T2-weighted image to the G output [Fig. 3(A)]. Since follicular fluid was bright on both the T2-weighted and proton-density-weighted images, the resultant additive color was yellow. Muscle, which had a relatively low intensity on the T2-weighted image and was somewhat brighter on the proton-densityweighted image, was brick-colored. Moreover, because fat was relatively bright on the proton-density-weighted image and somewhat darker on the T2-weighted image, a slight orange tone was imparted to this tissue. The intensities of the uterine fibroid on both the T2weighted and proton-density-weighted (slightly greater intensity) images gave regions of this structure a brownish hue in the composite image. Overall, this composite provided graded, seminatural color tones exhibiting the combined intensity characteristics of both original gray-tone images. A second composite [Fig. 3(B)] of the same scene was generated by combining the FISP GRE image in Fig. 2(C) (assigned to the R output) with the T2weighted SE image in Fig. 2(A) (assigned to the G output). The tissue contrast behaviors in the GRE image were such that differentiation between various tissues was observed to be enhanced in this composite compared with the proton-density-T, composite in Fig. 3(A), while a seminatural color scheme for subcutaneous fat, muscle, and uterine fibroid was preserved. In addition, the intense signal of flowing blood in the GRE image yielded a bright red angiographic effect within the iliac veins. This figure also illustrates the problem of slight spatial misalignment of images that can result from patient movement during an imaging session.
146
Magnetic Resonance Imaging 0 Volume 10, Number 1, 1992 250
200
150 Normalized Intensity
Mean Values
100
50
0 Fat
Mus.
B.M.
Vein
Tissue
Fol .
Fibr .
Air
ROIs
Fig. 1. Plot of average pixel signal intensity values for selected tissue ROIs in the T2- and proton-density (PD) weighted SE images, and the FISP GRE image in Fig. 2, used to generate the composites in Fig. 3. (Mus. = muscle; B.M. = bone marrow; Vein = iliac vein; Fol. = ovarian follicle; Fibr. = uterine fibroid.)
(B)
Fig. 2. Axial (A) T2- (2700/70) and (B) proton-density (2700/20) weighted SE images and (C) an axial FISP (200/ 1l/SO”) GRE image used to generate the composite images in Fig. 3. The images were obtained using a 0.35-T Siemens Magnatom imager.
Color display in MRI of the female pelvis 0 H.K. BROWNET
147
AL.
(4
Fig. 4. Two-color (RG) composite of sag&al proton-density (2500/28) and Tz-weighted (25&O/80)SE images demonstrating uterine zonal anatomy and differential color assignments between CSF and urine (arrows) and the more mutinous cervical and intrauterine contents (curved arrow). The component images were obtained using a 1.0-T Siemens Magnatom imager.
09 Fig. 3. Two-color (RG) composite of (A) axial protondensity and T2-weighted SE images and (B) an axial T2weighted SE image and an axial FISP GRE image. The use of a GRE image allowed better visual differentiation of tissues without the loss of the seminatural color scheme for muscle, fat, and uterine fibroid (F) and also yielded an angiograpbic effect within the iliac veins (curved arrow). Because the subject moved between pulse sequences, the images used to generate the composite in (B) were slightly misaligned (arrows).
An additional proton-density-T2 composite was generated by assigning a proton-density-weighted SE image to the R output and a T’-weighted SE image to the G output (Fig. 4). This figure demonstrates the zonal anatomy of the uterus within the context of a seminatural appearing image. In addition, in this composite, watery fluids [cerebrospinal fluid (CSF) and urine] appeared green, while the more mutinous cervical and intrauterine contents appeared yellow. Multichannel (RGB) Composites The image in Fig. 6 is a four-channel RGB composite created by combining axial T,-, T,-, and protondensity-weighted SE images [Fig. 5(A)-(C)] with an axial angiographic FISP GRE image [Fig. 5(D)] using
the coefficiency values in Table 1. Based on its intensity characteristics in the component images (bright on T2-weighted image, medium to dark on other images), a developing follicle was colored green in this composite. Seminatural colors for fat and muscle, as well as differential flow rates within the iliac vessels, were also noted. The four-channel RGB composite in Fig. 7 was created from T,-, T2-, and proton-density-weighted SE images, and a low-flip-angle GRASS GRE image. The
Table 1. Coefficiency values used to generate the composite images in Fig. 6* Colors Red
Green
Blue
Bits per color: 3 Coefficiency values Constant 0 Tl (600/20) 90 T2 (2600/W) 0 Proton density (2600/22) 50 FISP (120/10/70”) 120
3
2
0 90 200 50 0
0 85 0 0 0
*Coeffkiency value numbers represent RGB contributions for each of the images used to generate the composite in Fig. 6.
148
Magnetic Resonance Imaging 0 Volume 10, Number 1, 1992
(A)
(Cl
(W
Fig. 5. Axial (A) Tr- (600/20), (B) Tz-(2600/90), and (C) proton-density-weighted (2600/22) SE images and (D) an axial FISP (120/10/70”) GRE image used to generate the color composite in Fig. 6 and the classified images in Figs. 8 and 9. The images were obtained using a 1S-T Siemens Magnatom imager.
color coefficiency values used to generate this image appear in Table 2. This color table was designed to present uterine zonal anatomy within the context of a natural-appearing color scheme. Classified Image Generation The classified image in Fig. 8 was generated using the four gray-tone images in Fig. 5, and the tissue ROIs and color values listed in Table 3. This figure demonstrates some of the problems encountered in classified images created with statistical pattern recognition methods. In this figure, what appear to be improper tissue assignments (misclassified pixels) were apparent throughout the entire image and contributed to an overall degradation of image quality compared with
Table 2. Coefficiency values used to generate the composite images in Fig. 7* Colors Red Bits per color: Coefficiency values Constant T, (600/20) Tz (2500/90) Proton density (2500/20) FISP (200/11/10”)
Green
Blue
3
3
2
0 40
0 40
20
80
60 30
0 0
0 40 10 40 20
*Coefficiency value numbers represent RGB contributions for each of the images used to generate the composite in Fig. 7.
Color display in MRI of the female pelvis 0 H.K. BROWNETAL.
149
Fig. 6. Multichannel (RGB) composite created by combinand proton-density-weighted SE iming the axial T,-, T,-, ages in Fig. S(A)-(C) with the axial angiographic FISP image in Fig. S(D). Based on the intensity characteristics in the component images, near-natural color assignments were generated for both fat and muscle. In addition, a developing ovarian follicle was colored green (long arrow), and differential flow rates within the iliac arteries and veins generated different color assignments for each type of vessel (short arrows).
Fig. 8. Classified image generated using the four images in Fig. 5.
the original gray-tone images [Fig. 5(A)-(D)] and to the color composite created using the identical image set (Fig. 6). The image in Fig. 9 was produced by highlighting the pixels that were statistically similar to those included in the ROI selected for follicular fluid from a classified image generated using a two-image (T, /T,)
Case Study Application The images in Fig. 10 represent an application of the various color image display methods to MR images obtained from a female patient undergoing diagnostic workup for a suspected pelvic mass. The original gray-tone spin-echo images used to generate these color images are shown in Fig. 11. Initial evaluation of signal intensity patterns of selected tissues and confirmed pathologic lesions within the component images (Fig. 12) provided the basis upon which various
Fig. 7. Multichannel (RGB) composite created by combining sagittal T,-(600/20), T2-(2500/90), and proton-densityweighted (2500120) SE images with a sagittal GRASS (200/l I/10“) GRE image. This figure demonstrates uterine zonal anatomy within the context of a seminatural-appearing image. The component images were obtained using a 1.5-T General Electric Signa imager .
set. The highlighted pixels were overlaid onto the Tiweighted image to provide an anatomical reference. This image possesses the advantages of pattern recognition for tissue classification, without the image degradation problems inherent in creating a completely classified image using multiple tissue ROIs.
Fig. 9. Tl-weighted image displaying highlighted pixels that possess intensity patterns similar to that of follicular fluid in set [Fig. 5(A) and (B)]. a two-image (T,/T,)
150
Magnetic Resonance Imaging 0 Volume 10, Number 1, 1992
color assignments were made for composite image generation. The image in Fig. 10(A) was generated by combining spatially aligned Tl- and Tz-weighted SE images
using the two-color (RG) composite method. For this composite, the T1-weighted image was assigned to the R output and the T,-weighted image was assigned to the G output. In the resulting color display, hemorrhagic cysts within both ovaries appeared yellow due to their bright signal on both component images. The uterine fibroid possessed a greenish hue due to its moderately bright appearance on the T2-weighted image and relatively dark appearance on the Tl -weighted image. Urine, in anteriorly and posteriorly displaced segments of the bladder, was observed to be a more intense green. Overall, this composite provided adequate visual discrimination of important tissues and pathologic lesions, although the intensity of muscle was considerably hypointense relative to the other tissues in the image. In an effort to improve image appearance, the three-channel RGB composite in Fig. 10(B) was gen-
(4
(B)
(C)
CD)
Table 3. Gamma-l 1 color table values assigned for tissue ROIs selected for generation of the classified image in Fig. 8 Tissue ROI
Color value
Air Lumen of rectum
15
Muscle Junctional zone Myometrium Follicular fluid Fat
88 92 145 173 235
5
Fig. 10. Application of the various color display methods to images of a patient with a uterine fibroid (F) and bilateral hemorrhagic cysts @I) within the ovaries. Urine (U) is also visible in anteriorly and posteriorly displaced segments of the urinary bladder. (A) Two-color (RG) composite of the T1- and T2-weighted SE images in Fig. 1l(A) and (B). (B) Multichannel (RGB) and proton-density-weighted SE images in Fig. 11. (C) Classified image generated using the Tl-, composite of the TJ-, T,-, T2-, and proton-density-weighted SE images in Fig. 11. (D) T,-weighted image displaying highlighted pixels that possess inset [Fig. 11(A) and (B)]. tensity patterns similar to that of the hemorrhagic cysts in a two-image (T,/T,)
Color display in MRI of the female pelvis 0 H.K. BROWN ETAL.
151
Fig. 11. Axial (A) Tr- (600/U), (B) T2-(2000/60), and (C) proton-density-weighted (2000/30) SE images used to generate the composite and classified images in Fig. 10. The images were obtained using a 1.5-T Siemens Magnatom imager .
250
Normalized Mean Intensity Values
150
100
Fat
Mus .
Urine Tissue
Hem.
Fibr.
Air
ROIs
and proton-density Fig. 12. Plot of average pixe1 signal intensity values for selected tissue ROIs in the T,-, T,-, SE images in Fig. 11. (MU. = muscle; Hem. = hemorrhagic cyst; Fibr. = uterine fibroid.)
(PD) weighted
152
Magnetic Resonance Imaging 0 Volume 10, Number 1, 1992
Table 4. Coefficiency values used to generate the composite images in Fig. 10(B)* Colors Red
Green
3
3
Bits per color: Coefficiency values Constant r, @O/15) T2(2000&O) Proton density (2000/30)
0 95 -45 140
Blue 2
0 5.5 140 -4s
0 45 40 0
*Coefficiency value numbers represent RGB contributions for each of the images used to generate the composite in Fig. 10(B).
erated from T,-, T,-, and proton-density-weighted SE images using the coefficiency values shown in Table 4. While some visual contrast was lost between the right hemorrhagic cyst and urine in the posteriorly displaced segment of the bladder, the overall color assignments of other tissues were improved. Muscle, which was very dark in the T,/T2 composite, possessed a more natural color, and, in addition to obtaining a more natural appearance, the uterine fibroid became more distinguishable from surrounding tissues. The classified image in Fig. 10(C) was generated using the tissue ROIs and color values listed in Table 5. This classification scheme utilized duplicate tissue ROIs on both the right and the left sides of the image set in order to compensate for pixel intensity differences due to magnetic field inhomogeneities. This figure further demonstrates the pixel misassignment problems encountered with pattern recognition meth-
Table 5. Gamma-l 1 color table values assigned for tissue ROIs selected for generation of the classified image in Fig. 10(C) Tissue ROI Lumen of rectum Air Muscle (ROIL)* Muscle (ROIR)* Hemorrhagic cyst (L)* Hemorrhagic cyst (R)* Cervix Uterine fibroid Urine Fat Bone marrow (ROIL)* Bone marrow (ROIrJ* *L = patient’s left; R = patient’s right.
Color value 5 15 86 87 100 102 128 145 175 225 234 236
ods. In terms of delineating actual normal tissue and tumor margins, this method of color display appeared substantially less accurate in comparison to the original gray-tone images (Fig. 11) as well as to the color composites [Fig. 10(A) and (B)]. The image in Fig. IO(D) was generated by highlighting the pixels that were statistically similar to those included in the ROI selected for hemorrhagic cyst from a classified image generated using a two-image (T, /Tz) set. The highlighted pixels were overlaid onto the T2weighted image to provide an anatomical reference. The image in this figure possesses the advantages of pattern recognition for classification of specific tissues of interest, without many of the problems inherent in creating a completely classified image using multiple, potentially unnecessary tissue ROIs. DISCUSSION While it is likely that ultrasound will remain the screening technique of choice for most uterine and ovarian pathologic conditions, the application of MRI for the enhanced diagnosis and evaluation of various pelvic pathologic conditions for which further imaging studies are warranted retains considerable diagnostic promise, particularly with respect to the imaging of gynecological malignancies. In this report, several color image display methods that may further improve the diagnostic capabilities of MRI studies of the female pelvis have been presented. The results reported represent initial applications of two image processing techniques to MR images of the female pelvis. The purpose of this investigation was to demonstrate the application of these color display techniques -it was not intended to define their true clinical utility. In the future, receiver-operating characteristic (ROC) studies will need to be performed on a significant number of cases of various gynecological pathologic conditions in order to more accurately assess the possible benefits, if any, that these methods may provide. Currently, the diagnostic interpretation of MR images requires the back-and-forth comparison of images of the same anatomical scene in order to assess tissue-specific patterns of contrast behavior. It is possible that the combination of the diverse tissue contrast information present in several types of images of the same scene into one multicolored image might facilitate a more rapid, accurate diagnostic interpretation of MR images. Moreover, because of the increased conspicuity of specific tissues that is possible with color displays, particularly within the context of natural-appearing images using color composite methods, there is the potential for an enhanced ability to detect subtleties that might otherwise have been missed
Color display in MRI of the female pelvis 0 H.K. BROWN ET
using conventional gray-tone images. The increased tissue conspicuity that is potentially available with these color display methods may lead to an increase in the accuracy of MR image interpretation. This accuracy, however, is dependent on the reliability of the mechanism used to generate the color image. In addition, ultimate clinical acceptance of any color display technique is, in part, dependent on the ability of the diagnostician to understand the process of how the color image was generated. A distinct advantage of the composite generation methods used in this report, in comparison with the images generated with most pattern recognition algorithms, is the simplicity with which the composites are produced. In these composites, the final additive color assignments can be easily defined in terms of the intensity levels of various tissues within the component gray-tone images as well as by the colors (coefficiency values) these images have been assigned.16 In color composites, the gray-tone T1 and T2 tissue contrast patterns are translated into various color assignments that can be intuitively interpreted based on a basic knowledge of RGB color combinations. Because of this characteristic, the color composite method is readily applicable to MR images obtained using conventional pulse sequences. In contrast, many of the color display methods described in the literature3-l4 involve complex algorithms, statistical procedures, or image filters that have the potential to result in loss of image quality as well as in decreased accuracy of pixel tissue assignments. Such was the case with the classified images shown in Figs. 8 and IO(C). While it was possible to generate classified images of the female pelvis using the ML algorithm, overall image quality was observed to be poor relative to that of the original gray-tone images (Figs. 5 and 12) and the color composites [Figs. 6 and 10(A) and (B)]. Examples of this diminished image quality included the obscuring of the tumor margins in the image in Fig. 10(C), as well as the apparent misclassification of pixel assignments throughout both classified images. The former characteristic is extremely important in relation to the proper assessment of whether a particular neoplasm is isolated from or infiltrated into surrounding tissues. Despite these problems, pattern recognition methods still retain diagnostic promise for the visual characterization of specific tissues and warrant continued development. A potentially more appropriate application of pattern recognition techniques for the identification of specific tissues is illustrated in Figs. 9 and 10(D). For the generation of the images in these figures, the computer was used to statistically identify certain specific tissues of interest while other tissues were not classi-
AL.
153
fied in the final image that was presented. Thus, with this method, there was the advantage afforded by statistical identification of pixel sets with similar intensity patterns, yet overall image quality was preserved. A possible application of this technique is the statistical identification of probable sites of metastases in body regions other than the site of the primary tumor. Potential problems with the methods utilized in this report include the misregistration of component images [Fig. 3(B)] as well as improper tissue assignments in classified images if individuals move during or between different pulse sequences in an imaging session, Also, pulse sequences required for a particular composite or classified image generation protocol must be in the same sectional plane orientation and must include corresponding slice positions. The former requirement presents an additional problem in terms of increasing scan times past acceptable limits if multiple planes are required. In addition to these potential problems, the quality of the final composite images we have generated was limited by the presence of magnetic field inhomogeneities as well as by the graphics capabilities of the computer hardware. The images in Fig. 10(A) and (B) demonstrate an effect of magnetic field inhomogeneities on composite image color assignments. In both images in this figure, the left side of the imaged pelvis was observed to be somewhat darker than the right side, although overall differentiation of tissues was maintained. Magnetic field inhomogeneities also pose a significant problem with regard to confidence in proper tissue assignments in classified images. In terms of computer hardware capabilities, for composite images possessing more bits per color, there were more gradations of color available for display (number of possible colors = 2”, where n is the number of bits). Since images assigned 4 bits per color (16 possible colors) (Figs. 3 and 4) had more gradations of color available for display, they were of slightly better quality in comparison to images assigned only 2 bits (4 possible colors) or 3 bits (8 possible colors) per color [Figs. 6, 7, and IO(A) and (B)]. The potential for variability in appearance between color composites generated from image sets obtained using different models of MR imagers is not addressed in this report. However, the use of color composite techniques with image sets obtained from three different MR imagers appears to have little effect on the general color scheme achieved. l6 In summary, the application of color composite generation techniques to multiple MR images of the female pelvis represents a viable technique for displaying their diverse, diagnostically relevant tissue contrast information in one color image. In addition, while still in the developmental stage, artificial intelligence pat-
1.54
Magnetic Resonance Imaging 0 Volume 10, Number 1, 1992
tern recognition techniques for classified image generation continue to hold potential diagnostic promise for the enhancement of specific tissues, particularly for the identification of probable sites of tumor metastases. This promise will be better realized through the development of methods capable of more accurately assigning specific pixel sets into the correct tissue categories. With the continued development of composite generation and pattern recognition methods, appropriate clinical trials that include ROC studies, and specific case study applications, it will be possible to more fully assess the usefulness of these color image display techniques for enhancing the diagnostic interpretation of MR images of specific gynecological pathologic conditions. Acknowledgments- We gratefully acknowledge the technical and support assistance of the following individuals: M. Bryant, D. Diebel, T. Dula, A. Hayes, J. Lefler, C. Philips, J. Schellenberg, L. Sears, R. Shaw, D. Stern, R. Velthuizen, and C. Wood.
8.
9.
10.
11.
12.
13.
REFERENCES 14. 1. McCarthy, S.; Vaquero, E. Gynecologic anatomy with magnetic resonance imaging. Am. J. Obstet. Gynecol. 155:255-259; 1986. 2. Fishman-Javitt, M.C.; Lovecchio, J.L.; Stein, H.L. Imaging strategies for MRI of the pelvis. Radiol. Clin. North Am. 26:633-651; 1988. 3. Ortendahl, D.A.; Hylton, N.M.; Kaufman, L.; Crooks, L.E. Automated tissue characterization with NMR imaging. In: F. Deconick (Ed). Information Processing in Medical Imaging. Boston: Martinus Nijhoff Publishers; 1984: pp. 477-493. 4. Vannier, M.W.; Butterfield, R.L.; Jordan, D.; Murphy, W.A.; Levitt, R.G.; Gado, M. Multispectral analysis of magnetic resonance images. Radiology 154:221-224; 1985. 5. Koenig, H.A.; Bachus, R.; Reinhardt, E.R. Pattern recognition for tissue characterization in MR imaging. Health Care Instrumentation 1:184-187; 1986. 6. Gohagan, J.K.; Spitznagel, E.L.; Murphy, W.A.; Vannier, M.W.; Dixon, W.T.; Gersell, D.J.; Rossnick, S.L.; Totty, W.G.; Destouet, J.M.; Rickman, D.L.; Spraggins, T.A.; Butterfield, R.L. Multispectral analysis of MR images of the breast. Radiology 163:703-707; 1987. 7. Vannier, M.W.; Butterfield, R.L.; Rickman, D.L.; Jor-
15.
16.
17. 18. 19.
20.
21. 22.
dan, D.M.; Murphy, W.A.; Biondetti, P.R. Multispectral magnetic resonance image analysis. Crit. Rev. Biomed. Eng. 15:117-144; 1987. Jungke, M.; von Seelen, W.; Bielke, G.; Meindl, S.; Krone, G.; Grigat, M.; Higer, P.; Pfannenstiel, P. Information processing in nuclear magnetic resonance imaging. Magn. Reson. Imaging 6:683-693; 1988. van Dijk, P. Direct cardiac NMR imaging of heart wall and blood flow velocity. .Z.Comput, Assist. Tomogr. 8: 429-436; 1984. Klipstein, R.H.; Firmin, D.N.; Underwood, S.R.; Nayler, G.L.; Rees, R.S.O.; Longmore, D.B. Colour display of quantitative blood flow and cardiac anatomy in a single magnetic resonance tine loop. Br. J. Radiol. 60:105-111; 1987. Weiss, K.L.; Stiving, S.O.; Herderick, E.E.; Cornhill, J.F.; Chakeres, D.W. Hybrid color MR imaging display. AJR 149:825-829; 1987. Vannier, M.W.; Speidel, C.M.; Rickman, D.L. Magnetic resonance imaging multispectral tissue classification. News Physiol. Sci. 3:148-154; 1988. Vannier, M.W.; Rickman, D.L. Multispectral and color-aided displays, Invest. Radiol. 24:88-91; 1989. Kamman, R.L.; Stomp, G.P.; Berendsen, H. J.C. Unified multiple-feature color display for MR images. Magn. Reson. Med. 9:240-253; 1989. Brown, H.K.; Shellenberg, J.; Clarke, L.; Silbiger, M. Diagnostic utility of color composites generated from magnetic resonance images. Anat. Rec. 226:16A(abstract); 1990. Brown, H.K.; Hazelton, T.R.; Silbiger, M.L. Generation of color composites for enhanced tissue differentiation in magnetic resonance imaging of the brain. Am. .Z.Anat. 192(1):23-24; 1991. Thorell, L.G.; Smith, W.J. Using Computer Color Effectively. Englewood Cliffs: Prentice-Hall; 1990. Farrell, E.J. Visual interpretation of complex data. IBM Systems J. 26:174-200; 1987. Schowengerdt, R.A. Techniques for Image Processing and Classification in Remote Sensing. Orlando: Academic Press; 1983. Mantas, J. Methodologies in pattern recognition and image analysis -A brief survey. Pattern Recogn. 20: l6; 1987. Bernstein, R. (Ed). Digital Image Processing for Remote Sensing. New York: IEEE Press; 1978. Hord, R.M. Digital Image Processing of Remotely Sensed Data. New York: Academic Press; 1982.
Magnetic Resonance Imaging, Vol. 10. pp. 143-154, 1992 Printed in the USA. All rights reserved.
0 Original Contribution COMPOSITE AND CLASSIFIED COLOR DISPLAY IN MR IMAGING OF THE FEMALE PELVIS H. KEITH BROWN,*? TODD R. HA~ELTON,~ JAMES V. FIORICA,§ ANNA K. PARSONS,~ LAURENCE P. CLARKE,? AND MARTIN L. SILBIGER~ Departments of *Anatomy, tRadiology, and #Obstetrics and Gynecology, University of South Florida College of Medicine, Tampa, Florida 33612-4799,USA Because of its superior soft-tissue-imaging capabilities, MRI has proved to be an excellent modality for visualizing the contents of the female pelvis. In an effort to potentially improve gynecological MRI studies, we have applied color composite techniques to sets of spin-echo and gradient-echo gray-tone MR images obtained from various individuals. For composite generation, based on tissue region of interest calculated mean pixel intensity values, various colors were applied to spatially aligned images using a DEC MicroVAX II computer with interactive digital language (IDL) so that tissue contrast patterns could be optimized in the final image. The IDL procedures, which are similar to those used in NASA’s LANDSAT image processing system, allowed the generation of single composite images displaying the combined information present in a series of spatially aligned images acquired using different pulse sequences. With our composite generation techniques, it was possible to generate seminatural-appearing color images of the female pelvis that possessed enhanced conspicuity of specific tissues and fluids. For comparison with color composites, classified images were also generated based on computer recognition and statistical separation of distinct tissue intensity patterns in an image set using the maximum likelihood processing algorithm. Keywords:
Magnetic resonance imaging; Pelvis; MRI studies; Pelvis, female.
substantial amount of data, studies for assessing the diagnostic utility of MRI for the evaluation of various pathologic conditions within the female pelvis have yet to characterize the full clinical potential of this noninvasive imaging modality. Because the intensity characteristics of individual tissues in different types of MR images are dependent on the pulse sequence parameters selected for image acquisition, various investigators have endeavored to develop both gray-tone and color display methods for tissue characterization, based on pattern recognition techniques using multispectral MR image sets.3-8 In addition to the generation of classified images using pattern recognition techniques, various methods for
INTRODUCTION
Because of its unique ability to provide high contrast resolution and excellent soft-tissue contrast, magnetic resonance imaging (MRI) is well suited for evaluating the contents of the female pelvis. Moreover, MRI is capable of demonstrating gynecological anatomy not visible with other imaging techniques and has the additional benefit of being able to scan in multiple imaging planes.’ The advantages that these distinctive properties provide for the enhanced imaging of various gynecological pathologic conditions are evidenced by the sizable number of MRI studies of the abnormal female pelvis that have been reported.2 Even with this 2/8/9 1; ACCEPTED 6/28/91. $Medical Student, University of South Florida College of Medicine. Funding and resource support for this project were provided by H. Lee Moffitt Cancer Center and Research Institute, University Diagnostic Institute, University of South Florida President’s Council, and NASA-American Cancer Society (#89-25).Portions of this work were done during an
American Cancer Society R.G. Thompson Summer Research Fellowship (T.R.H.) and during the tenure of a Med-
RECEIVED
ical Student Research Fellowship of the American Heart Association (T.R.H.). Address correspondence and reprint requests to H. Keith Brown, Ph.D., Department of Anatomy, University of South Florida College of Medicine, Box 6, 12901 N. 30th Street, Tampa, FL 33612-4799, USA.
143
144
Magnetic Resonance Imaging 0 Volume 10, Number 1, 1992
producing hybrid, or composite, color images have been described.7v9-‘3 A method for simultaneously presenting the Ti and T2relaxation times of selected tissues in a single color image has also been reported. I4 Although the color display methods previously described in the literature have yielded images with enhanced differentiation of various tissues and fluids, most of the images that have been generated possess unnatural color combinations that often require special legends in order to facilitate their interpretation by the viewer. In contrast, methods for generating composite images that exhibit seminatural color assignments, in addition to enhanced tissue conspicuity, have been described. i5,i6 Advantages of color image display, in contrast to achromatic or monochromatic presentation, include the potential for a more realistic appearance, enhanced information processing, and increased ability of the viewer to discriminate and interpret related and unrelated data. I7 Moreover, while each display point in a gray-tone image possesses intensity as its only variable, each display point in a color image has three variable attributes (hue, saturation, and intensity), which allows a greater information capacity within the display.18 With the relatively high incidence rate and often difficult noninvasive assessment of various significant gynecological pathologic conditions, the advantages afforded by color display should not be overlooked with regard to MRI of the female pelvis. Presented in this report are applications of two methods of color image display that may potentially improve the diagnostic accuracy and efficiency of MRI studies of the abnormal female pelvis. METHODS The purpose of this investigation was to assess the feasibility of applying color composite and pattern recognition methods to various sets of MR images of the female pelvis, acquired using both routine and specialized pulse sequences. Images utilized in this study were obtained from patients and volunteers in accordance with the rules and regulations established for human studies by the Institutional Review Board of the University of South Florida Health Sciences Center. Imaging studies were performed on a 1S-tesla (T) General Electric Signa imager (General Electric Corporation, Medical Systems Group, Milwaukee, WI) and on 0.35, 1.O-, and 1.5-T Siemens Magnatom imagers (Siemens Medical Systems, Inc., Iselin, NJ). Spin-echo (SE) images were obtained using the following pulse sequences: T,-weighted images had repetition time and echo time (TRITE) values of 600/l 5 and 600/20; T,-weighted images had TR/TE values of 2000/60, 2500/80, 2500190, 2600190, and 2700/70;
and proton-density-weighted images had TR/TE values of 2000/30,2500/20,2500/28,2600/22, and 2700/28. In addition, gradient-echo (GRE) images with the following pulse sequence parameters (TR/TE/flip angle) were also acquired: 120/10/70”, 200/11/10”, and 200/l l/50”. The SE and GRE images were obtained using a body coil and had an acquisition matrix of 256 x 256. Individual sections had a thickness of either 5 or 8 mm, and adjacent slices were acquired with a 50% gap. Following acquisition, the data for image analysis and processing studies were then transferred via magnetic tape to a DEC MicroVAX II computer (Digital Equipment Corporation, Maynard, MA). These studies were performed using computer software routines developed with Interactive Digital Language (IDL) (Research Systems, Inc., Boulder, CO). Color image output was directed to an Electrohome ECM 1301-X VGA monitor (Electrohome, Ltd., Ontario, Canada). For certain tissues within analyzed images, the average signal intensities of pixels within operatorselected regions of interest (ROIs) were calculated by a component of the image analysis program in order to provide quantitative data supporting qualitative assessments of tissue contrast behaviors. Tissue ROIs were selected from various anatomical and confirmed pathological sites using procedures that allowed the calculation of mean pixel signal intensities and their standard deviations for corresponding pixels within identical ROIs in a spatially aligned, simultaneously analyzed image set. With the program, actual and normalized (O-255) signal intensity values were reported. From the quantitative normalized data obtained, image color assignments (color coefficiency values in Tables 1, 2, and 4) were specifically selected so that the desired colors of specific tissues and fluids could be visually optimized in the final composites, based on standard red (R), green (G), and blue (B) computergenerated color combinations and on characteristic tissue intensity patterns in the component images. For composite generation, previously described image-processing routines were utilized. l6 These routines were designed to allow the combination of two or more spatially aligned MR images into one multicolored composite image in a process analogous to NASA’s LANDSAT multispectral false-color infrared image generation system. I9 In the two-color method for composite generation, based on normalized intensity characteristics of specific tissue ROIs, one image was assigned to the red output and the other to the green output. For the multichannel composite method, positive or negative integers were assigned for each of the three primary colors (R, G, and B). An assigned coefficiency value of zero indicated no contribution from that color for that particular image. Negative
Color display in MRI of the female pelvis 0 H.K. BROWN ET
coefficiency values were designed to subtract the selected color. In this latter method of composite generation, individual R, G, and B images were created from the input images, based on the signal intensities of spatially aligned pixels and on the coefficiency values assigned for each image. In both methods, the assigned R and G or the computer-generated R, G, and B images were then superimposed. Based on the various intensity levels of spatially aligned pixels within an image set, a single composite image with additive color assignments was then produced. Classified images generated in this study were created utilizing the maximum likelihood (ML) processing algorithm. ” With the use of a mouse-driven system, various tissue ROIs in a spatially aligned image set were selected from known anatomical and confirmed pathological locations for supervised classification by the computer. Each ROI was then assigned a numerical value that corresponded to a particular color in a Gamma-11 color table (D.M. Stern, Research Systems, Inc., Boulder, CO). Once an individual ROI was selected, the pattern recognition component of the image analysis routines then applied the ML algorithm to each pixel within the image set. This process was repeated for each tissue to be classified. Based on their characteristic signal intensity patterns within the multispectral set of MR images being analyzed, spatially aligned pixels were mapped in feature space, l9 with pixels possessing similar intensity patterns tending to cluster together. Based on their location in feature space, the computer statistically separated the mapped pixels into classes of tissues with intensity patterns similar to those of the original ROIs selected. Using an additional image-processing routine, the created ML classification scheme was applied to the original gray-tone image set in order to generate a classified color image analogous to the thematic maps created in multispectral remote sensing applications. ‘9,21*22This application was performed using sets of two, three, and four images. In order to highlight specific tissues, a threshold function was applied to permit display of pixels with greater probability of specific tissue classification. This threshold function allowed more accurate classification of individual pixels by decreasing the area in feature space into which pixels with similar intensity patterns could be classified as the tissue to be highlighted. To select a specific tissue, the numerical color assignment for the desired ROI in a two-image set was selected. The specific highlighted tissue was then overlaid onto a spatially aligned gray-tone image, and the color table was changed to a Blue-Red color table (D.M. Stern, Research Systems, Inc., Boulder, CO) to provide greater contrast between the selected tissue and the original gray-tone image.
AL.
145
For presentation in this report, color images were photographed on ASA 200 color slide film with the aid of a macro zoom lens using exposure times of l/8 and l/15 set, and an F setting of 3.5. RESULTS
Two-Color (RG) Composites When a composite using additive color combinations is formed, if one image is assigned red and the other green, tissues whose contrast behaviors are similar will appear various intensities of yellow, depending on whether their respective signal intensities are high, intermediate, or low. Tissues that are hyperintense on one image and hypointense on the other will appear either red or green depending on which color is assigned to the image possessing the hyperintense tissue. I6 Based on the normalized tissue intensity characteristics presented in Fig. 1, a color composite of the T2- and proton-density-weighted SE images shown in Fig. 2(A) and (B), respectively, was created by assigning the proton-density-weighted image to the R output and the T2-weighted image to the G output [Fig. 3(A)]. Since follicular fluid was bright on both the T2-weighted and proton-density-weighted images, the resultant additive color was yellow. Muscle, which had a relatively low intensity on the T2-weighted image and was somewhat brighter on the proton-densityweighted image, was brick-colored. Moreover, because fat was relatively bright on the proton-density-weighted image and somewhat darker on the T2-weighted image, a slight orange tone was imparted to this tissue. The intensities of the uterine fibroid on both the T2weighted and proton-density-weighted (slightly greater intensity) images gave regions of this structure a brownish hue in the composite image. Overall, this composite provided graded, seminatural color tones exhibiting the combined intensity characteristics of both original gray-tone images. A second composite [Fig. 3(B)] of the same scene was generated by combining the FISP GRE image in Fig. 2(C) (assigned to the R output) with the T2weighted SE image in Fig. 2(A) (assigned to the G output). The tissue contrast behaviors in the GRE image were such that differentiation between various tissues was observed to be enhanced in this composite compared with the proton-density-T, composite in Fig. 3(A), while a seminatural color scheme for subcutaneous fat, muscle, and uterine fibroid was preserved. In addition, the intense signal of flowing blood in the GRE image yielded a bright red angiographic effect within the iliac veins. This figure also illustrates the problem of slight spatial misalignment of images that can result from patient movement during an imaging session.
146
Magnetic Resonance Imaging 0 Volume 10, Number 1, 1992 250
200
150 Normalized Intensity
Mean Values
100
50
0 Fat
Mus.
B.M.
Vein
Tissue
Fol .
Fibr .
Air
ROIs
Fig. 1. Plot of average pixel signal intensity values for selected tissue ROIs in the T2- and proton-density (PD) weighted SE images, and the FISP GRE image in Fig. 2, used to generate the composites in Fig. 3. (Mus. = muscle; B.M. = bone marrow; Vein = iliac vein; Fol. = ovarian follicle; Fibr. = uterine fibroid.)
(B)
Fig. 2. Axial (A) T2- (2700/70) and (B) proton-density (2700/20) weighted SE images and (C) an axial FISP (200/ 1l/SO”) GRE image used to generate the composite images in Fig. 3. The images were obtained using a 0.35-T Siemens Magnatom imager.
Color display in MRI of the female pelvis 0 H.K. BROWNET
147
AL.
(4
Fig. 4. Two-color (RG) composite of sag&al proton-density (2500/28) and Tz-weighted (25&O/80)SE images demonstrating uterine zonal anatomy and differential color assignments between CSF and urine (arrows) and the more mutinous cervical and intrauterine contents (curved arrow). The component images were obtained using a 1.0-T Siemens Magnatom imager.
09 Fig. 3. Two-color (RG) composite of (A) axial protondensity and T2-weighted SE images and (B) an axial T2weighted SE image and an axial FISP GRE image. The use of a GRE image allowed better visual differentiation of tissues without the loss of the seminatural color scheme for muscle, fat, and uterine fibroid (F) and also yielded an angiograpbic effect within the iliac veins (curved arrow). Because the subject moved between pulse sequences, the images used to generate the composite in (B) were slightly misaligned (arrows).
An additional proton-density-T2 composite was generated by assigning a proton-density-weighted SE image to the R output and a T’-weighted SE image to the G output (Fig. 4). This figure demonstrates the zonal anatomy of the uterus within the context of a seminatural appearing image. In addition, in this composite, watery fluids [cerebrospinal fluid (CSF) and urine] appeared green, while the more mutinous cervical and intrauterine contents appeared yellow. Multichannel (RGB) Composites The image in Fig. 6 is a four-channel RGB composite created by combining axial T,-, T,-, and protondensity-weighted SE images [Fig. 5(A)-(C)] with an axial angiographic FISP GRE image [Fig. 5(D)] using
the coefficiency values in Table 1. Based on its intensity characteristics in the component images (bright on T2-weighted image, medium to dark on other images), a developing follicle was colored green in this composite. Seminatural colors for fat and muscle, as well as differential flow rates within the iliac vessels, were also noted. The four-channel RGB composite in Fig. 7 was created from T,-, T2-, and proton-density-weighted SE images, and a low-flip-angle GRASS GRE image. The
Table 1. Coefficiency values used to generate the composite images in Fig. 6* Colors Red
Green
Blue
Bits per color: 3 Coefficiency values Constant 0 Tl (600/20) 90 T2 (2600/W) 0 Proton density (2600/22) 50 FISP (120/10/70”) 120
3
2
0 90 200 50 0
0 85 0 0 0
*Coeffkiency value numbers represent RGB contributions for each of the images used to generate the composite in Fig. 6.
148
Magnetic Resonance Imaging 0 Volume 10, Number 1, 1992
(A)
(Cl
(W
Fig. 5. Axial (A) Tr- (600/20), (B) Tz-(2600/90), and (C) proton-density-weighted (2600/22) SE images and (D) an axial FISP (120/10/70”) GRE image used to generate the color composite in Fig. 6 and the classified images in Figs. 8 and 9. The images were obtained using a 1S-T Siemens Magnatom imager.
color coefficiency values used to generate this image appear in Table 2. This color table was designed to present uterine zonal anatomy within the context of a natural-appearing color scheme. Classified Image Generation The classified image in Fig. 8 was generated using the four gray-tone images in Fig. 5, and the tissue ROIs and color values listed in Table 3. This figure demonstrates some of the problems encountered in classified images created with statistical pattern recognition methods. In this figure, what appear to be improper tissue assignments (misclassified pixels) were apparent throughout the entire image and contributed to an overall degradation of image quality compared with
Table 2. Coefficiency values used to generate the composite images in Fig. 7* Colors Red Bits per color: Coefficiency values Constant T, (600/20) Tz (2500/90) Proton density (2500/20) FISP (200/11/10”)
Green
Blue
3
3
2
0 40
0 40
20
80
60 30
0 0
0 40 10 40 20
*Coefficiency value numbers represent RGB contributions for each of the images used to generate the composite in Fig. 7.
Color display in MRI of the female pelvis 0 H.K. BROWNETAL.
149
Fig. 6. Multichannel (RGB) composite created by combinand proton-density-weighted SE iming the axial T,-, T,-, ages in Fig. S(A)-(C) with the axial angiographic FISP image in Fig. S(D). Based on the intensity characteristics in the component images, near-natural color assignments were generated for both fat and muscle. In addition, a developing ovarian follicle was colored green (long arrow), and differential flow rates within the iliac arteries and veins generated different color assignments for each type of vessel (short arrows).
Fig. 8. Classified image generated using the four images in Fig. 5.
the original gray-tone images [Fig. 5(A)-(D)] and to the color composite created using the identical image set (Fig. 6). The image in Fig. 9 was produced by highlighting the pixels that were statistically similar to those included in the ROI selected for follicular fluid from a classified image generated using a two-image (T, /T,)
Case Study Application The images in Fig. 10 represent an application of the various color image display methods to MR images obtained from a female patient undergoing diagnostic workup for a suspected pelvic mass. The original gray-tone spin-echo images used to generate these color images are shown in Fig. 11. Initial evaluation of signal intensity patterns of selected tissues and confirmed pathologic lesions within the component images (Fig. 12) provided the basis upon which various
Fig. 7. Multichannel (RGB) composite created by combining sagittal T,-(600/20), T2-(2500/90), and proton-densityweighted (2500120) SE images with a sagittal GRASS (200/l I/10“) GRE image. This figure demonstrates uterine zonal anatomy within the context of a seminatural-appearing image. The component images were obtained using a 1.5-T General Electric Signa imager .
set. The highlighted pixels were overlaid onto the Tiweighted image to provide an anatomical reference. This image possesses the advantages of pattern recognition for tissue classification, without the image degradation problems inherent in creating a completely classified image using multiple tissue ROIs.
Fig. 9. Tl-weighted image displaying highlighted pixels that possess intensity patterns similar to that of follicular fluid in set [Fig. 5(A) and (B)]. a two-image (T,/T,)
150
Magnetic Resonance Imaging 0 Volume 10, Number 1, 1992
color assignments were made for composite image generation. The image in Fig. 10(A) was generated by combining spatially aligned Tl- and Tz-weighted SE images
using the two-color (RG) composite method. For this composite, the T1-weighted image was assigned to the R output and the T,-weighted image was assigned to the G output. In the resulting color display, hemorrhagic cysts within both ovaries appeared yellow due to their bright signal on both component images. The uterine fibroid possessed a greenish hue due to its moderately bright appearance on the T2-weighted image and relatively dark appearance on the Tl -weighted image. Urine, in anteriorly and posteriorly displaced segments of the bladder, was observed to be a more intense green. Overall, this composite provided adequate visual discrimination of important tissues and pathologic lesions, although the intensity of muscle was considerably hypointense relative to the other tissues in the image. In an effort to improve image appearance, the three-channel RGB composite in Fig. 10(B) was gen-
(4
(B)
(C)
CD)
Table 3. Gamma-l 1 color table values assigned for tissue ROIs selected for generation of the classified image in Fig. 8 Tissue ROI
Color value
Air Lumen of rectum
15
Muscle Junctional zone Myometrium Follicular fluid Fat
88 92 145 173 235
5
Fig. 10. Application of the various color display methods to images of a patient with a uterine fibroid (F) and bilateral hemorrhagic cysts @I) within the ovaries. Urine (U) is also visible in anteriorly and posteriorly displaced segments of the urinary bladder. (A) Two-color (RG) composite of the T1- and T2-weighted SE images in Fig. 1l(A) and (B). (B) Multichannel (RGB) and proton-density-weighted SE images in Fig. 11. (C) Classified image generated using the Tl-, composite of the TJ-, T,-, T2-, and proton-density-weighted SE images in Fig. 11. (D) T,-weighted image displaying highlighted pixels that possess inset [Fig. 11(A) and (B)]. tensity patterns similar to that of the hemorrhagic cysts in a two-image (T,/T,)
Color display in MRI of the female pelvis 0 H.K. BROWN ETAL.
151
Fig. 11. Axial (A) Tr- (600/U), (B) T2-(2000/60), and (C) proton-density-weighted (2000/30) SE images used to generate the composite and classified images in Fig. 10. The images were obtained using a 1.5-T Siemens Magnatom imager .
250
Normalized Mean Intensity Values
150
100
Fat
Mus .
Urine Tissue
Hem.
Fibr.
Air
ROIs
and proton-density Fig. 12. Plot of average pixe1 signal intensity values for selected tissue ROIs in the T,-, T,-, SE images in Fig. 11. (MU. = muscle; Hem. = hemorrhagic cyst; Fibr. = uterine fibroid.)
(PD) weighted
152
Magnetic Resonance Imaging 0 Volume 10, Number 1, 1992
Table 4. Coefficiency values used to generate the composite images in Fig. 10(B)* Colors Red
Green
3
3
Bits per color: Coefficiency values Constant r, @O/15) T2(2000&O) Proton density (2000/30)
0 95 -45 140
Blue 2
0 5.5 140 -4s
0 45 40 0
*Coefficiency value numbers represent RGB contributions for each of the images used to generate the composite in Fig. 10(B).
erated from T,-, T,-, and proton-density-weighted SE images using the coefficiency values shown in Table 4. While some visual contrast was lost between the right hemorrhagic cyst and urine in the posteriorly displaced segment of the bladder, the overall color assignments of other tissues were improved. Muscle, which was very dark in the T,/T2 composite, possessed a more natural color, and, in addition to obtaining a more natural appearance, the uterine fibroid became more distinguishable from surrounding tissues. The classified image in Fig. 10(C) was generated using the tissue ROIs and color values listed in Table 5. This classification scheme utilized duplicate tissue ROIs on both the right and the left sides of the image set in order to compensate for pixel intensity differences due to magnetic field inhomogeneities. This figure further demonstrates the pixel misassignment problems encountered with pattern recognition meth-
Table 5. Gamma-l 1 color table values assigned for tissue ROIs selected for generation of the classified image in Fig. 10(C) Tissue ROI Lumen of rectum Air Muscle (ROIL)* Muscle (ROIR)* Hemorrhagic cyst (L)* Hemorrhagic cyst (R)* Cervix Uterine fibroid Urine Fat Bone marrow (ROIL)* Bone marrow (ROIrJ* *L = patient’s left; R = patient’s right.
Color value 5 15 86 87 100 102 128 145 175 225 234 236
ods. In terms of delineating actual normal tissue and tumor margins, this method of color display appeared substantially less accurate in comparison to the original gray-tone images (Fig. 11) as well as to the color composites [Fig. 10(A) and (B)]. The image in Fig. IO(D) was generated by highlighting the pixels that were statistically similar to those included in the ROI selected for hemorrhagic cyst from a classified image generated using a two-image (T, /Tz) set. The highlighted pixels were overlaid onto the T2weighted image to provide an anatomical reference. The image in this figure possesses the advantages of pattern recognition for classification of specific tissues of interest, without many of the problems inherent in creating a completely classified image using multiple, potentially unnecessary tissue ROIs. DISCUSSION While it is likely that ultrasound will remain the screening technique of choice for most uterine and ovarian pathologic conditions, the application of MRI for the enhanced diagnosis and evaluation of various pelvic pathologic conditions for which further imaging studies are warranted retains considerable diagnostic promise, particularly with respect to the imaging of gynecological malignancies. In this report, several color image display methods that may further improve the diagnostic capabilities of MRI studies of the female pelvis have been presented. The results reported represent initial applications of two image processing techniques to MR images of the female pelvis. The purpose of this investigation was to demonstrate the application of these color display techniques -it was not intended to define their true clinical utility. In the future, receiver-operating characteristic (ROC) studies will need to be performed on a significant number of cases of various gynecological pathologic conditions in order to more accurately assess the possible benefits, if any, that these methods may provide. Currently, the diagnostic interpretation of MR images requires the back-and-forth comparison of images of the same anatomical scene in order to assess tissue-specific patterns of contrast behavior. It is possible that the combination of the diverse tissue contrast information present in several types of images of the same scene into one multicolored image might facilitate a more rapid, accurate diagnostic interpretation of MR images. Moreover, because of the increased conspicuity of specific tissues that is possible with color displays, particularly within the context of natural-appearing images using color composite methods, there is the potential for an enhanced ability to detect subtleties that might otherwise have been missed
Color display in MRI of the female pelvis 0 H.K. BROWN ET
using conventional gray-tone images. The increased tissue conspicuity that is potentially available with these color display methods may lead to an increase in the accuracy of MR image interpretation. This accuracy, however, is dependent on the reliability of the mechanism used to generate the color image. In addition, ultimate clinical acceptance of any color display technique is, in part, dependent on the ability of the diagnostician to understand the process of how the color image was generated. A distinct advantage of the composite generation methods used in this report, in comparison with the images generated with most pattern recognition algorithms, is the simplicity with which the composites are produced. In these composites, the final additive color assignments can be easily defined in terms of the intensity levels of various tissues within the component gray-tone images as well as by the colors (coefficiency values) these images have been assigned.16 In color composites, the gray-tone T1 and T2 tissue contrast patterns are translated into various color assignments that can be intuitively interpreted based on a basic knowledge of RGB color combinations. Because of this characteristic, the color composite method is readily applicable to MR images obtained using conventional pulse sequences. In contrast, many of the color display methods described in the literature3-l4 involve complex algorithms, statistical procedures, or image filters that have the potential to result in loss of image quality as well as in decreased accuracy of pixel tissue assignments. Such was the case with the classified images shown in Figs. 8 and IO(C). While it was possible to generate classified images of the female pelvis using the ML algorithm, overall image quality was observed to be poor relative to that of the original gray-tone images (Figs. 5 and 12) and the color composites [Figs. 6 and 10(A) and (B)]. Examples of this diminished image quality included the obscuring of the tumor margins in the image in Fig. 10(C), as well as the apparent misclassification of pixel assignments throughout both classified images. The former characteristic is extremely important in relation to the proper assessment of whether a particular neoplasm is isolated from or infiltrated into surrounding tissues. Despite these problems, pattern recognition methods still retain diagnostic promise for the visual characterization of specific tissues and warrant continued development. A potentially more appropriate application of pattern recognition techniques for the identification of specific tissues is illustrated in Figs. 9 and 10(D). For the generation of the images in these figures, the computer was used to statistically identify certain specific tissues of interest while other tissues were not classi-
AL.
153
fied in the final image that was presented. Thus, with this method, there was the advantage afforded by statistical identification of pixel sets with similar intensity patterns, yet overall image quality was preserved. A possible application of this technique is the statistical identification of probable sites of metastases in body regions other than the site of the primary tumor. Potential problems with the methods utilized in this report include the misregistration of component images [Fig. 3(B)] as well as improper tissue assignments in classified images if individuals move during or between different pulse sequences in an imaging session, Also, pulse sequences required for a particular composite or classified image generation protocol must be in the same sectional plane orientation and must include corresponding slice positions. The former requirement presents an additional problem in terms of increasing scan times past acceptable limits if multiple planes are required. In addition to these potential problems, the quality of the final composite images we have generated was limited by the presence of magnetic field inhomogeneities as well as by the graphics capabilities of the computer hardware. The images in Fig. 10(A) and (B) demonstrate an effect of magnetic field inhomogeneities on composite image color assignments. In both images in this figure, the left side of the imaged pelvis was observed to be somewhat darker than the right side, although overall differentiation of tissues was maintained. Magnetic field inhomogeneities also pose a significant problem with regard to confidence in proper tissue assignments in classified images. In terms of computer hardware capabilities, for composite images possessing more bits per color, there were more gradations of color available for display (number of possible colors = 2”, where n is the number of bits). Since images assigned 4 bits per color (16 possible colors) (Figs. 3 and 4) had more gradations of color available for display, they were of slightly better quality in comparison to images assigned only 2 bits (4 possible colors) or 3 bits (8 possible colors) per color [Figs. 6, 7, and IO(A) and (B)]. The potential for variability in appearance between color composites generated from image sets obtained using different models of MR imagers is not addressed in this report. However, the use of color composite techniques with image sets obtained from three different MR imagers appears to have little effect on the general color scheme achieved. l6 In summary, the application of color composite generation techniques to multiple MR images of the female pelvis represents a viable technique for displaying their diverse, diagnostically relevant tissue contrast information in one color image. In addition, while still in the developmental stage, artificial intelligence pat-
1.54
Magnetic Resonance Imaging 0 Volume 10, Number 1, 1992
tern recognition techniques for classified image generation continue to hold potential diagnostic promise for the enhancement of specific tissues, particularly for the identification of probable sites of tumor metastases. This promise will be better realized through the development of methods capable of more accurately assigning specific pixel sets into the correct tissue categories. With the continued development of composite generation and pattern recognition methods, appropriate clinical trials that include ROC studies, and specific case study applications, it will be possible to more fully assess the usefulness of these color image display techniques for enhancing the diagnostic interpretation of MR images of specific gynecological pathologic conditions. Acknowledgments- We gratefully acknowledge the technical and support assistance of the following individuals: M. Bryant, D. Diebel, T. Dula, A. Hayes, J. Lefler, C. Philips, J. Schellenberg, L. Sears, R. Shaw, D. Stern, R. Velthuizen, and C. Wood.
8.
9.
10.
11.
12.
13.
REFERENCES 14. 1. McCarthy, S.; Vaquero, E. Gynecologic anatomy with magnetic resonance imaging. Am. J. Obstet. Gynecol. 155:255-259; 1986. 2. Fishman-Javitt, M.C.; Lovecchio, J.L.; Stein, H.L. Imaging strategies for MRI of the pelvis. Radiol. Clin. North Am. 26:633-651; 1988. 3. Ortendahl, D.A.; Hylton, N.M.; Kaufman, L.; Crooks, L.E. Automated tissue characterization with NMR imaging. In: F. Deconick (Ed). Information Processing in Medical Imaging. Boston: Martinus Nijhoff Publishers; 1984: pp. 477-493. 4. Vannier, M.W.; Butterfield, R.L.; Jordan, D.; Murphy, W.A.; Levitt, R.G.; Gado, M. Multispectral analysis of magnetic resonance images. Radiology 154:221-224; 1985. 5. Koenig, H.A.; Bachus, R.; Reinhardt, E.R. Pattern recognition for tissue characterization in MR imaging. Health Care Instrumentation 1:184-187; 1986. 6. Gohagan, J.K.; Spitznagel, E.L.; Murphy, W.A.; Vannier, M.W.; Dixon, W.T.; Gersell, D.J.; Rossnick, S.L.; Totty, W.G.; Destouet, J.M.; Rickman, D.L.; Spraggins, T.A.; Butterfield, R.L. Multispectral analysis of MR images of the breast. Radiology 163:703-707; 1987. 7. Vannier, M.W.; Butterfield, R.L.; Rickman, D.L.; Jor-
15.
16.
17. 18. 19.
20.
21. 22.
dan, D.M.; Murphy, W.A.; Biondetti, P.R. Multispectral magnetic resonance image analysis. Crit. Rev. Biomed. Eng. 15:117-144; 1987. Jungke, M.; von Seelen, W.; Bielke, G.; Meindl, S.; Krone, G.; Grigat, M.; Higer, P.; Pfannenstiel, P. Information processing in nuclear magnetic resonance imaging. Magn. Reson. Imaging 6:683-693; 1988. van Dijk, P. Direct cardiac NMR imaging of heart wall and blood flow velocity. .Z.Comput, Assist. Tomogr. 8: 429-436; 1984. Klipstein, R.H.; Firmin, D.N.; Underwood, S.R.; Nayler, G.L.; Rees, R.S.O.; Longmore, D.B. Colour display of quantitative blood flow and cardiac anatomy in a single magnetic resonance tine loop. Br. J. Radiol. 60:105-111; 1987. Weiss, K.L.; Stiving, S.O.; Herderick, E.E.; Cornhill, J.F.; Chakeres, D.W. Hybrid color MR imaging display. AJR 149:825-829; 1987. Vannier, M.W.; Speidel, C.M.; Rickman, D.L. Magnetic resonance imaging multispectral tissue classification. News Physiol. Sci. 3:148-154; 1988. Vannier, M.W.; Rickman, D.L. Multispectral and color-aided displays, Invest. Radiol. 24:88-91; 1989. Kamman, R.L.; Stomp, G.P.; Berendsen, H. J.C. Unified multiple-feature color display for MR images. Magn. Reson. Med. 9:240-253; 1989. Brown, H.K.; Shellenberg, J.; Clarke, L.; Silbiger, M. Diagnostic utility of color composites generated from magnetic resonance images. Anat. Rec. 226:16A(abstract); 1990. Brown, H.K.; Hazelton, T.R.; Silbiger, M.L. Generation of color composites for enhanced tissue differentiation in magnetic resonance imaging of the brain. Am. .Z.Anat. 192(1):23-24; 1991. Thorell, L.G.; Smith, W.J. Using Computer Color Effectively. Englewood Cliffs: Prentice-Hall; 1990. Farrell, E.J. Visual interpretation of complex data. IBM Systems J. 26:174-200; 1987. Schowengerdt, R.A. Techniques for Image Processing and Classification in Remote Sensing. Orlando: Academic Press; 1983. Mantas, J. Methodologies in pattern recognition and image analysis -A brief survey. Pattern Recogn. 20: l6; 1987. Bernstein, R. (Ed). Digital Image Processing for Remote Sensing. New York: IEEE Press; 1978. Hord, R.M. Digital Image Processing of Remotely Sensed Data. New York: Academic Press; 1982.
