THE AMERICAN JOURNAL OF ANATOMY 19223-34 (1991)
Generation of Color Composites for Enhanced Tissue Differentiation in Magnetic Resonance Imaging of the Brain H. KEITH BROWN, TODD R. HAZELTON, AND MARTIN L. SILBIGER Departments of Anatomy (H.K.B., T.R.H.) and Radiology (H.K.B., M.L.S.), University of South Florida College of Medicine, Tampa, Florida 336124799
ABSTRACT Currently, the diagnostic interpretation of magnetic resonance (MR) images requires that radiologists integrate specific tissue contrast information from several different images obtained at the same anatomic slice position. Each of these images has its own unique tissue contrast patterns which are based on the image acquisition parameters (pulse sequence) selected. The complex contrast patterns observable in these images reflect the inherent biophysical characteristics of the tissues and fluids present in the imaged section. In an effort to increase the diagnostic accuracy and efficiency of MR image interpretation, we have generated color composite images from quantitatively analyzed achromatic MR images of the brain, obtained while utilizing different pulse sequences. By using a DEC MicroVAX I1 computer with Interactive Digital Language (IDL), this color display method has been applied to images obtained from General Electric Signa and Siemens Magnatom imagers. For this study, our image sets included T1weighted, T2-weighted, and proton density spin echo sequences as well as both high and low flip angle gradient echo sequences. Advantages ofour color composite methods, in contrast to many other image processing techniques that have been described, are that minimal information is lost, computer misclassification of tissues is avoided, and the conspicuity of specific tissues is enhanced. Furthermore, with this method it is possible to produce composite images whose color renditions approach a natural anatomic tissue appearance. Availability of these color composites to radiologists may improve the efficiency and accuracy of the diagnostic interpretation of MR images.
ination of specific tissues, which are ultimately displayed as pixels comprising a two-dimensional MR image. The intensity of each individual pixel is determined by several biophysical characteristics of the tissue within the voxel. These characteristics include longitudinal relaxation rate (Tl), transverse relaxation rate (T2), proton density, and flow velocity and direction. To emphasize contrast patterns of specific tissues, different image acquisition parameters (pulse sequences) can be utilized to produce various types of “weighted” images. The selection of different pulse sequences allows for the generation of spin echo images that are T1-weighted, T2-weighted, or proton density weighted. Furthermore, gradient echo pulse sequences, or “fast scans,” can be utilized to obtain images that possess additional unique tissue-contrast patterns. In each type of image, whether spin echo or gradient echo, individual tissues appear differently, based on their own inherent biophysical characteristics (Sprawls, 1987). Because an individual tissue may have a characteristic appearance in a given type of MR image due to the particular pulse sequence parameters selected, effort has been directed toward developing color display methods for tissue characterization based on the multispectral analysis of MR image sets. Using NASA’s LANDSAT series satellite-image processing system and the nearest-neighbor classification algorithm, Vannier and co-workers (1985,1987) have successfully demonstrated the feasibility of computer-generated color displays by producing classified images with enhanced discrimination of different body tissues and fluids. Additional investigations in the area of pattern recognition for tissue classification have also been described by Koenig et al. (1986)and Jungke et al. (1988). Moreover, pattern recognition techniques have also been used successfully to demonstrate and enhance the appearance of certain pathologic conditions (Vannier et al., 1985; Koenig et al., 1986; Gohagan et al., 1987; Jungke et al., 1988). In addition to color image generation using pattern recognition techniques, hybrid or composite color imINTRODUCTION Magnetic resonance imaging (MRI) is a very rapidly ages have also been produced. The combination of both advancing technology which provides exciting new op- anatomical and parametric blood-flow velocity images portunities for the enhanced visualization of anatomic, into a single composite with range-defined color assignphysiologic, and pathologic features while using a sin- ments has been demonstrated by van Dijk (1984) and gle diagnostic imaging modality. This technology is Klipstein et al. (1987). In an image hybridization based on the principle that protons within body tissues method presented by Weiss et al. (19871, pixels from and fluids can absorb and then subsequently emit ra- one image were assigned varying spectral hues, while dio frequency (RF) signals when placed in a strong the luminance of these hues was derived from the inmagnetic field. It is the detection of the emitted RF signals from three-dimensional tissue voxels that provides data for the spatial location and contrast discrimReceived December 26, 1990. Accepted April 15, 1991. 0 1991 WILEY-LISS, INC.
24
H.K. BROWN ET AL.
Image 1
Image 2
-
Red Output
-
Green Output
C1R
Image 1
-+
4
CIG
- - - -:
I -I i
CIB
-7
C2R
-{--A
Red Output
I
I
I
I
'
! ++ I
Image 2 -+ C2G
I
~* C2B
-4
Green Output
L
Composite Image
,
1
1
,
I
I
Fig. 1. Schematic diagram illustrating (a)how a two-color composite is formed when one image is assigned red and the other green, and (b) how a multichannel (RGB) composite is formed from red, green, and
blue output images created by using an n number of input images (C = coeficiency value defining the contribution of images 1 , 2 , . . . ,n to the R, G, or B output channels).
tensity level of corresponding pixels in a second, spatially aligned image. Vannier et al. (1987, 1988) have created composites from MR images by assigning different images to the red, green, and blue (R, G, and B) guns of a color CRT monitor. A composite devised by using NASA's Earth Resources Application Software (ELAS) has also been presented by Vannier et al. (1988). Like the pattern recognition techniques previously described, the combining of different types of MR images into a composite has also been shown to enhance the appearance of certain pathologic conditions (Weiss et al., 1987; Levin et al., 1987). Recently, Kamman et al. (1989)proposed a color display method based on the calculated T1 and T2 relaxation times, as well as the proton density, of particular tissues of interest. In their display method, color images that simultaneously represented both the T1 and T2 relaxation times of selected tissues were generated by mixing the primary colors R, G, and B. This method is advantageous because its mechanism of display allows the generation of color images which represent inherent tissue properties and are not dependent on instrument settings as are conventional gray-tone MR images or the color displays derived from them. Although the color display methods described above have yielded images with enhanced tissue conspicuity, most possess unnatural color combinations that often
require special legends in order to be interpretable. In contrast, images with semi-natural color renditions, in addition to enhanced tissue conspicuity, have been presented (Brown et al., 1990). Such composites, when created from judiciously selected spin echo and/or gradient echo images, could contain important anatomic, physiologic, as well as pathologic information, all within the context of a single semi-natural-appearing image that might be more easily interpreted by the viewer. As mentioned previously, it has been demonstrated that it is possible to generate color MR composites in a manner that is similar to the mechanism by which images are displayed on color CRT systems (Vannier et al., 1987, 1988). In these systems, a composite image possessing a broad range of different colors is generated by additive combinations of the three primary colors R, G, and B. In an analogous process, color composites, such as those created by using NASA's LANDSAT satellite system, are also formed through the use of R, G, and B channels for each of the three bands that combine to generate the final image (Schowengerdt, 1983). In the three-primary-color system that is used for these techniques, additive combinations of spatially aligned pixels produce the various colors of the spectrum. With this method of display, in addition to the composite hues produced, pixels adjusted to different light intensities generate the appearance of different
COLOR COMPOSITES I N MRI O F THE BRAIN
25
250
200
Normalized Mean Intensity 100 Values 50
z
0 CSF
WM
GM
Mus.
Fat
Paro.
Air
Tissue Regions of Interest Fig. 2. Plot of average pixel signal intensity values for selected tissue regions of interest (ROIs) from the proton density (PD) and T2-weighted spin echo images in Figure 3. CSF, Cerebrospinal fluids; WM, white matter; GM, gray matter; Mus., muscle; Paro., parotid gland.
Fig. 3.Coronal proton density (TFUTE = 2600/22)(a)and T2-weighted (TFUTE = 2600/90)(b)spin echo images used to generate the color composite in Figure 8a. The images were obtained by using a 1.5 T Siemens Magnatom imager.
saturation and intensity values, and thus preserve both color and shading qualities within the final composite image (Thorell and Smith, 1990).
Gray Matter White Matter Cerebrospinal Fluid Red Nucleus Substantia Nigra Temporalis Muscle
pi:
Parotid Gland
4 Fig. 4. Schematic diagram of a coronal slice of the head illustrating important cranial tissues and structures discussed in the text.
MATERIALS AND METHODS Images utilized in this study were obtained from patients and volunteers in accordance with the rules and regulations established for human studies by the Institutional Review Board of the University of South Florida Health Sciences Center. Imaging studies were performed on a 1.5 Tesla (T) General Electric Signa imager (GE Medical Systems, Milwaukee, WI) and on 0.35,1.0,and 1.5T Siemens Magnetom imagers (Siemens Medical Systems, Iselin, NJ). Sets of images were acquired over a period of approximately 4 years and utilized a variety of pulse sequence parameters. Of the images acquired, the spin echo images that were ob-
H.K. BROWN ET AL.
26 25 0 200
Normalized Mean 150 Intensity 100 Values 50 0
5
CSF
WM
GM Mus.
Fat
Pa- A r t . Po.
Air
B1.
Tissue Regions of Interest Fig. 5. Plot of average pixel signal intensity values for selected tissue regions of interest (ROIs) from the FISP 70" gradient echo image and the T1-weighted spin echo image in Fig. 6. CSF, Cerebrospinal fluid; WM, white matter; GM,gray matter; Mus., muscle; Paro., parotid gland; Art. Bl., arterial blood.
Fig. 6. a: Coronal FISP gradient echo (TFb'TEIFlip Angle = 120/10/70")and b T1-weighted spin echo (TWTE = 600/20)images used to generate the color composite in Figure 9a.The images were obtained by using a 1.5T Siemens Magnatom imager.
tained possessed the following parameter values: T1weighted images had repetition time and echo time (TRITE)values (in milliseconds) of 400/20,500/17, and 600120; TPweighted images had TWTE values of 24001 90, 2600190, and 2700190; and proton density images had TRITE values of 2400130,2600122, and 2700130. In addition to spin echo images, gradient echo sequences were also obtained. For these images, the following parameters were utilized (TWTEIFlip Angle): 33/13/30", 120110170", 200113110", and 200115150". Each spin echo or gradient echo image had an acquisition matrix of 256 x 256 and a section thickness of 5 mm; and adjacent slices were obtained with a 50% gap.
The image data for generating the color composites were then transferred via magnetic tape to a DEC MicroVAX I1 computer (Digital Equipment, Maynard, MA.). Image analysis and processing were performed by using program routines developed with Interactive Digital Language (IDL) (Research Systems, Boulder, CO). Color image output was directed to an Electrohome ECM 1301-X VGA monitor (Electrohome, Ontario, Canada) with 8-bit display capabilities (256 possible colors). For certain tissues, the average signal intensities of the pixels within selected regions of interest (ROIs) were calculated in order to provide quantitative data
COLOR COMPOSITES IN MRI OF THE BRAIN
Fig. 7. a: Coronal T1-weighted (TWTE = 400/20), and, b, T2-weighted (TWTE = 2400/90) spin echo images, combined with a, c, GRASS gradient echo image (TR/TE/Flip Angle = 200/13/10")to form the composite in Figure 13a. The images were obtained by using a 1.5T GE Signa imager.
27
Figs. 8,9.
COLOR COMPOSITES I N MRI OF THE BRAIN
TABLE 1. Coefficiency values used to generate the composite images in Figure 13a, b, and c
Bits per color Coefficiency values Constant T1-weighted image T2-weighted image lo" fliD anele GRASS imaee
Red 3
Colors Green 3
Blue 2
0 50 100 100
0 50 100 0
0 0 150 0
supporting the empiric selection of image color assignments and color coefficiency values. Tissue ROTS were selected from various anatomical sites within spatially aligned MR images by using a mouse-driven system. By simultaneously analyzing all the data within a set, this method allowed the calculation of the mean signal intensities for corresponding pixels within identical ROIs for each of the images analyzed. Statistical calculations were performed on the DEC MicroVAX I1 computer by using a component of the IDL image-analysis program. With this particular program routine, the actual pixel intensity values were normalized to a 0 to 255 intensity scale so that the minimum intensity value (Datami,,)became 0 and the maximum (Data,,,,) became 255. This normalization was accomplished by using the equation below. Output
=
256 x
Data Data,,,,
-
Datami,. Datami,
-
A normalized output value was calculated for each actual pixel intensity value (Data) in the image matrix. The means and standard deviations of the actual signal intensities and normalized output values of all the pixels within a selected ROI were then reported. Based on the normalized mean intensity values obtained, certain color assignments or various color coefficiency values were empirically selected for each of the input images so that the desired colors of specific tissues could be optimized in the final composite. For color image display, the computer program routines were designed to allow the combination of data from two or more spatially aligned MR images in order to generate a single composite image. In the two-color method, based on characteristic signal intensities of selected tissues in the images to be combined, one im-
Fig. 8 . Two-color (WG) composites of coronal proton density/T2 image sets acquired from three different individuals on three different imagers. a: Individual 1imaged on a 1.5T Siemens Magnetom instrument with TWTE parameters of 2600122 and 2600/90 for the proton density and T2-weighted images, respectively. b: Individual 2 imaged on a 1.OT Siemens Magnetom instrument with TR/TE parameters of 2400/30 and 2400/90 for the proton density and T2-weighted images, respectively. c: Individual 3 imaged on a 1.5T GE Signa instrument with TWTE parameters of 2400/30 and 2400190 for the proton density and T2-weighted images, respectively.
Fig. 9. Two-color (RIG) composites of coronal FISP gradient echo (TWTE/Flip Angle = 120/10/70")and T1-weighted spin echo (TRITE = 600/20) images demonstrating enhanced vasculature (bright red) and the effects of magnetic field inhomogeneities on color assignments. The composite in a was created from the images in Figure 6. The composite image in b was produced from images obtained 15 mm away from those used to generate the image in a.
29
age was assigned to the R output channel and the other to the G output channel. To generate the final composite, the computer first partitioned the programmed color table into 4 bits (16 intensity levels) each for R and G, and then combined the assigned images (Fig. la). For each of the images used in the multichannel composite method, positive or negative integers were assigned for the three primary colors R, G, and B. An assigned coefficiency value of zero indicated no contribution from that color for that particular image. Negative coefficiency values were designed to subtract the selected color. For this latter method, individual R, G, and B output channels were created from the input images based on the coefficiency values assigned for each image. These three channels were then combined to form a single composite (Fig. lb). In order to generate composite images with this method, 3 bits (8 intensity levels) each for R and G and 2 bits (4 intensity levels) for B were utilized (D.M. Stern, Research Systems, Boulder, Co, written communication, February 1990). In both methods of display, based on the various intensity levels of spatially aligned pixels, additive combinations of the two or three primary colors resulted in a single image possessing a composite of all the information present in each of the original input images. For presentation in this report, the composites displayed on the monitor screen were photographed on ASA 200 color slide film with the aid of a macro zoom lens by using exposure times of 118 and 1/15 second and an F setting of 3.5. RESULTS Tissue Contrast Parameters and Two-Color (RIG) Composite Method
When forming a composite using additive color combinations, if one image is assigned red and the other green, tissues whose contrast patterns are similar will appear yellow, or varying shades of brown, depending on whether their respective signal intensities are high, intermediate, or low. Tissues which are hyperintense on one image, but hypointense on the other, will appear either red or green depending on which color is assigned to the image possessing the hyperintense tissue. Based on the normalized tissue intensity characteristics presented in Figure 2, a color composite of the proton density and TZ-weighted spin echo images shown in Figure 3 was created by assigning the proton density image to the R output channel and the T2-weighted image to the G output channel (Fig. 8a). Because cerebrospinal fluid (CSF) was very bright on the T2weighted image and relatively bright on the proton density image, the resultant additive color was greenish-yellow (See Fig. 4 for anatomical reference). Muscle, which had an intermediate intensity on the proton density image and a relatively low intensity on the T2-weighted image, was brick red. White matter, being hypointense to gray matter on both images, possessed a dark brown color, while gray matter appeared a lighter brown. Overall, this combination provided graded, semi-natural color tones exhibiting the desirable characteristics of both original gray-tone images. For the final composite, the proton density image possessed intensity characteristics that functioned to heighten the contrast between gray matter and white matter, while the intensities of the red nucleus, substantia nigra, and
30
H.K. BROWN ET AL.
Figs. 10-12.
COLOR COMPOSITES IN MRI OF THE BRAIN
CSF in the T2-weighted image were such that visual differentiation of these entities was enhanced. In addition to the heightened tissue conspicuity that has been demonstrated in this proton densityIT2 composite, this image combination also was found to provide a relatively consistent color scheme for differentiating tissues in several individuals imaged on the Siemens and GE instruments at different magnetic field strengths and using various pulse sequence parameters (Fig. 8). Normalized signal intensity values (Fig. 5) from the high flip angle FISP (Fast Imaging with Steady-State Precession; Siemens Magnetom) gradient echo and the T1-weighted spin echo images in Figure 6 revealed that the tissue contrast patterns for CSF, brain, and muscle were very similar for these two different pulse sequences. A color composite was generated from these two images by assigning the FISP image to R and the T1-weighted image to G (Fig. 9a). As would be predicted, muscle (which possessed a low intensity on both images, but was slightly more intense on the FISP image) was brick red; gray matter (which possessed an intermediate intensity on both images) was brown; and white matter (which possessed an intensity slightly greater than gray matter in both images) was strawcolored. Moreover, fat and parotid gland (which were brighter in the T1-weighted image and somewhat darker in the FISP image) had a greenish hue, and flowing blood (which was dark in the T1-weighted image and bright in the FISP image) yielded a bright red angiographic effect in the composite. The composite shown in Figure 9b was created from T1-weighted and FISP images obtained 15 mm away from the images used to generate the composite in Figure 9a. Both composites in this figure demonstrate some of the color assignment problems that may arise from magnetic field inhomogeneities both within a given image and between images of different sections. The changes in color assignment were particularly apparent in images utilizing gradient echo scans. Although pixel color assignments for gray and white matter varied both within an image and between two different images, visual differentiation of these tissues was preserved. Another example of a spin echolgradient echo compos-
31
ite is the image in Figure 10. This composite was generated by assigning a low flip angle FISP gradient echo image to the R output channel and a T1-weighted spin echo image to the G output channel. The resulting image possessed semi-natural tissue color assignments for brain, fat, and muscle. The composite in Figure 11was generated from coronal T1-weighted and T2-weighted spin echo images. To create this composite, the T1-weighted image was assigned to R and the T2-weighted image was assigned to G. Although the resulting contrast between gray and white matter was observed to be not as distinct as that in the proton densityIT2 composites (Fig. 8), this combination of images was found to provide adequate visual separation between these two brain tissues. This characteristic is important because this image set represents the two pulse sequences most frequently used in diagnostic MRI. An application of the two-color (WG) composite technique to a clinical case is illustrated in the proton densityIT2 composite in Figure 12a. For this composite, the proton density image was assigned to R and the T2-weighted image to G. This figure demonstrates various components of an arteriovenous malformation (AVM) in the brain of a patient. Another example of an enhanced pathology is the proton densityIT2 composite in Figure 12b of a patient with a large mass arising from the pituitary fossa. According to the evaluating radiologist's report, the anterior yellow region of the tumor most likely represents old hemorrhage, while the posterior red region probably depicts an area of more acute hemorrhage. On both the proton density and T2-weighted images, the anterior region was bright (normalized mean intensity values of 178 and 172, respectively) and, therefore, appeared yellow in the composite. In contrast, the posterior region was moderately bright on the proton density image (normalized mean intensity value of 101) and was dark on the T2-weighted image (normalized mean intensity value of 20). Since the proton density image was assigned to the R output channel and the T2-weighted image to the G output channel, these latter intensity characteristics correspond with the dark red color produced for this area of hemorrhage in the final composite. Multichannel (RGB) Composite Method
Fig. 10. Two-color (WG) composite demonstrating semi-natural color assignments for brain (flesh-colored) and muscle (brick red). This image was generated from a sagittal FISP gradient echo image (TWTE/Flip Angle = 200/11/50")and a T1-weighted spin echo image (TWTE = 500/17) obtained by using a 0.35T Siemens imager. Fig. 1 1. Color composite generated from coronal T2-weighted (TW TE = 600/20) and T2-weighted (TWTE = 2600/90) spin echo images obtained by using a 1.5T Siemens Magnetom imager. In this image, CSF is green, muscle is a very dark brick red, fat is orange-red, gray matter is light brown, and white matter is somewhat darker brown. Fig. 12. a: Two-color (WG)composite of an axial proton density/T2 image set (TWTE = 2400/30 and 2400/90, respectively) demonstrating the potential application of this technique to differentiate components within AVMs (arrow) in the brain. The images were obtained by using a 1.5T GE Signa imager. b Two-color (WG) composite of a sagittal proton density/T2 image set (TWTE = 2700/30 and 2700/90, respectively) illustrating the ability of this composite method t o characterize differentially old and new hemorrhage within a single image. The regions of hemorrhage are located within a pituitary adenoma (arrow). The images were obtained by using a 1.5T GE Signa imager.
The composites in Figure 13 were produced by combining coronal T1-weighted and T2-weighted spin echo images with a low flip angle GRASS (Gradient Refocused Acquisition in the Steady State; General Electric Signa) gradient echo image. The composite in Figure 13a was constructed from R, G, and B images formed by using the coefficiency values listed in Table 1 and the gray-tone images shown in Figure 7. The images in Figure 13 demonstrate the reproducibility of this technique between different individuals. Note that the color assignments for fat, muscle, CSF, gray matter, and white matter are fairly consistent between the three images. Due to the limited number of bits per color (3 for R, 3 for G, and 2 for B), the quality of these composite images was somewhat diminished compared to that of the two-color(R/G)composites in which 4 bits each were available for R and G (Figs. 8-12). The image in Figure 14 was generated by combining T2-weighted and proton density spin echo images with
32
H.K. BROWN ET AL.
Figs. 13, 14.
COLOR COMPOSITES IN MRI OF THE BRAIN
TABLE 2. Coefficiency values used to generate the composite image in Figure 14 Colors
Bits per color Coeficiency values Constant T2-weighted image Protonvdensity &age 30” flip angle GRASS image
Red 3
Green
3
Blue 2
0 0 -150 250
0 35 100 0
0 120 0 0
a low flip angle GRASS gradient echo image by using the coefficiency values listed in Table 2. Upon generation of the composite, the colors of the tissues were changed from their additive color combinations to colors with greater hue distinction by using a “16 Level” color table (D.M. Stern, Research Systems, Boulder, CO). This combination of images, with the applied color table, provided excellent contrast between gray matter, white matter, and CSF, although the color assignments in both the original composite (not shown) and the image in Figure 14 were observed to lack a naturalappearing color scheme. DISCUSSION
Benefits of color-image display, in contrast to achromatic or monochromatic presentation, include a more realistic appearance, enhanced information processing, and increased ability to interpret related and unrelated data (Thorell and Smith, 1990). Because of these benefits, the increased tissue conspicuity that is yielded with color MR images has the potential to allow the detection of subtleties that might otherwise have been missed by the interpreting radiologist. Currently, in order to assess tissue-specific contrast patterns, radiologists must analyze the information in different images of the same scene by back-and-forth comparison. It is possible, however, that the same information might be more easily interpreted in a single, appropriately formulated color composite. Intuitively, the efficiency and accuracy of achromatic MR image interpretation would be improved by preview or reference to a color composite, generated by using a well-established protocol in which color assignments were consistently reproducible regardless of the make of the instrument, magnetic field strength, or coil inhomogeneities.
Fig. 13. Multichannel (RGB) color composites generated by combining coronal T1-weighted (TWTE = 400120) and T2-weighted (TWTE = 2400190) spin echo images with a GRASS gradient echo image (TRITEIFlip Angle = 200/13/10”)by using the coefficiency values in Table 1.This figure demonstrates the reproducibility of the RGB composites in different individuals. The images from all three individuals (a+) were obtained by using a 1.5T GE Signa imager. The composite in a was generated by using the images shown in Figure 7. Fig. 14. Multichannel (RGB) color composite with a “16 Level” stepped color table that provides for greater contrast between gray matter, white matter, and CSF. This composite was created by combing axial proton density and T2-weighted spin echo images (TRITE = 2400/30 and 2400/90, respectively) with a GRASS gradient echo image (TWTE/Flip Angle = 33/13/30”).The images were obtained by using a 1.5T GE Signa imager.
33
As we have demonstrated, it is possible to generate such composites by using the methods presented in this report. Moreover, the design of these methods is such that almost any combination andlor number of images can be utilized, as long as the coeficiency tables are appropriately adjusted so that a meaningful composite can be generated. The composites in Figure 8 demonstrate the ability of the methods we have presented to meet the reproducibility criteria set forth above. Even with the variable color assignments demonstrated in Figure 9, this method can generate quality images without major color assignment problems as evidence by the composites in Figure 8. An additional benefit of the composite image generation method presented, in contrast to most pattern recognition techniques, is that no information is added or filtered out during the steps of image composition. Moreover, with the composite methods we have described, tissues are not statistically classified by a computer. Instead, pixel intensity information is merely being combined to form a unique color composite based on tissue contrast patterns in images obtained by using selected pulse sequences. With our composite method, because the identification of specific tissues is made by an anatomist, radiologist, or other trained observer, computer misclassification of tissues is avoided. Potential problems with the composite methods described in this report include possible misregistration of images if the patient moves between different pulse sequences, or motion artifacts if the patient moves during a particular pulse sequence. However, availability of original achromatic images would aid in defining image misregistration, chemical-shift, ghosting, wraparound, and other image artifacts. In addition to these potential problems, pulse sequences required for a particular composite generation protocol must be in the same sectional plane orientation and include corresponding slice positions. The former requirement presents an additional problem in terms of increasing scan times past acceptable limits if multiple planes are required. This problem could potentially be overcome through the use of rapidly acquired 3-D gradient echo pulse sequences because, owing to their short imaging time, there would be less likelihood of patient movement causing misregistration of spatially aligned images. In addition, because of the three-dimensional matrix acquisition of data, multiple orientations could be obtained from a single imaging session. From such sessions, a series of rapidly acquired images that exhibit T1, T2, T2*,1 and proton density characteristics could be generated (Siemens Medical Systems, Iselin, NJ). While these images, individually, might not be easily interpretable to the radiologist accustomed to conventional spin echo images, it is possible that they could be combined into a composite in which specific tissue features would be displayed in familiar colors. Such a method might prove to have the same diagnostic value as much longer scanning sessions with conventional spin echo pulse sequences. Even so, for the present, it is still possible to select appropriate spin echo pulse sequences and orientations that may be used to generate ’T2* gradient echo images are based on the effective transverse relaxation time, which is typically shorter than T2.
34
H.K. BROWN ET AL.
color composites that could potentially enhance the diagnostic evaluation of MR images. In addition to the potential problems described above, the quality of the final composite images we have generated is limited by the graphics capabilities of the computer hardware. This feature is well illustrated by comparing the images in Figure 13 in which R, G, and B were assigned 3 , 3 , and 2 bits each, respectively, with the images in Figures 8-12 in which Rand G were each assigned 4 bits. In images possessing more bits per color, there were more gradations of color available for display (number of possible colors = 2", where n is the number of bits). Therefore, with only 2 or 3 bits per color, more abrupt changes in color assignment resulted in the creation of artificial boundaries within various tissues (Fig. 13).In contrast, with 4 bits per color, these artificial boundaries appeared as a gradient that transitioned more subtly from one color to another (Figs. 812).Because the number ofpossible colors is determined by the number of assigned bits, the use of a high-resolution monitor with 24-bit display capabilities should dramatically improve the quality of the multichannel (RGB) composites, as well as further improve the appearance of the two-color (FUG) composites. With regard to the pathology cases presented in this report, although the pulse sequences used to generate these composites may not necessarily be the ideal set for diagnostic evaluation, they do demonstrate the potential this technique has for enhancing certain physiologic and pathologic components of MR images. This potential is well illustrated by the composite in Figure 12a. Through the prudent selection and combination of various f low-sensitive pulse sequences, a single MR image possessing various color-coded flow rates could be produced. Additionally, Figure 12b shows the possible application of this method to the assessment and dating of hemorrhage. This figure demonstrates that the composite generation methods described in this report have potential applications in terms of the diagnostic evaluation of traumatic intracranial hemorrhagic conditions, as well as for the evaluation of both intracranial and extracranial hemorrhagic neoplasms. Overall, we feel the methods of color display presented in this report are more advantageous than many of the other color display techniques that have been described in the literature. Because the color assignments are easily adjustable, this procedure allows the formulation of composite images possessing nearnatural colors that should function to enhance visual interpretability. Moreover, unlike pattern recognition techniques which vary only hue in order to separate statistically defined tissues, color composites generated by the methods we have presented maintain saturation and intensity values in addition to their composite hues, thus preserving all three of the important components of human visual perception. In terms of data presentation, this feature is important because the presence of all three of these components allows for a greater information capacity within a displayed image (Farrell, 1987). Additionally, one distinct advantage of our composite generation methods is the simplicity with which the composites are produced. Colors that appear in the final composites can be easily defined in terms of the intensity levels of various tissues within the component images as well as by the colors or coefficiency values these images have been assigned.
In summary, the composite generation methods proposed in this report represent a viable technique for displaying diagnostically relevant information of multiple MR images in one easily interpretable color image. Within appropriate clinical trials and case-study applications, it will be possible to assess more fully the usefulness of this method for enhancing the diagnostic interpretation of MR images of specific pathologies. ACKNOWLEDGMENTS
We gratefully acknowledge the technical and support assistance of the following individuals: M. Bryant, L. Clarke, T. Dula, J. Lefler, J. Madden, C. Philips, J. Schellenberg, L. Sears, R. Shaw, D. Stern, R. VelthuiZen, and C. Wood. Funding and resource support for this project were provided by the H. Lee Mofitt Cancer Center and Research Institute, the University Diagnostic Institute, the University of South Florida President's Council, and the NASAIAmerican Cancer Society (#89-25). Portions of this work were done during the tenure of a Medical Student Research Fellowship of the American Heart Association (T.R.H.) and during an American Cancer Society R.G. Thompson Summer Research Fellowship (T.R.H.). LITERATURE CITED Brown, H.K., J. Schellenberg, L. Clarke, and M. Silbiger 1990 Diagnostic utility of color composites generated from magnetic resonance images. Anat. Rec., 226:16A (Abstr.). Farrell, E.J. 1987 Visual interpretation of complex data. IBM Syst. J., 26: 174-200. Gohagan, J.K., E.L. Spitznagei, W.A. Murphy, M.W. Vannier, W.T. Dixon, D.J. Gersell, S.L. Rossnick, W.G. Totty, J.M. Destouet, D.L. Rickman, T.A. Spraggins, and R.L. Butterfield 1987 Multispectral analysis of MR images of the breast. Radiology, 163: 703-707. h n g k e , M., W. von Seelen, G. Bielke, S. Meindl, G. Krone, M. Grigat, P. Higer, and P. Pfannenstiel 1988 Information processing in nuclear magnetic resonance imaging. Magn. Reson. Imaging, 6: 683-693. Kamman, R.L., G.P. Stomp, and H.J.C. Berendsen 1989 Unified multiple-feature color display for MR images. Magn. Reson. Med., 9:240-253. Klipstein, R.H., D.N. Firmin, S.R. Underwood, G.L. Nayler, R.S.O. Rees, and D.B. Longmore 1987 Colour display of quantitative blood flow and cardiac anatomy in a single magnetic resonance cine loop. Br. J . Radiol., 60:105-111. Koenig, H.A., R. Bachus, and E.R. Reinhardt 1986 Pattern recogni' tion for tissue characterization in MR imaging. Health Care Instrum., 1~184-187. Levin, D.N., A. Herrmann, T. Spraggins, P.A. Collins, L.B. Dixon, M.A. Simon, and A.E. Stillman 1987 Musculoskelatal tumors: improved detection with linear combinations of MR images. Radiology, 163:545-549. Schowengerdt, R.A. 1983 Techniques for Image Processing and Classification in Remote Sensing. Academic Press, Orlando, pp. 117118. Sprawls, P. 1987 Nuclear magnetic resonance imaging. In: Physical Principles of Medical Imaging. Aspen, Rockville, pp. 381-425. 'rhorell, L.G., and W.J. Smith 1990 Using Computer Color Effectively. Prentice Hall, Englewood Cliffs, pp. 16-17, 118. van Dijk, P. 1984 Direct cardiac NMR imaging of heart wall and blood flow velocity. J . Comput. Assist. Tomogr., 8:429-436. Vannier, M.W., R.L. Butterfield, D. Jordan, W.A. Murphy, R.G. Levitt, and M. Gad0 1985 Multispectral analysis of magnetic resonance images. Radiology, 154:221-224. Vannier, M.W., R.L. Butterfield, D.L. Rickman, D.M. Jordan, W.A. Murphy, and P.R. Biondetti 1987 Multispectral magnetic resonance image analysis. Crit. Rev. Biomed. Eng., 15:117-144. Vannier, M.W., C.M. Speidel, and D.L. Rickman 1988 Magnetic resonance imaging multispectral tissue classification. News Physiol. Sci., 3:148-154. Weiss, K.L., S.O. Stiving, E.E. Herderick, J.F. Cornhill, D.W. Chakeres 1987 Hybrid color MR imaging display. Am. J . Radiol., 149: 825-829.
Generation of Color Composites for Enhanced Tissue Differentiation in Magnetic Resonance Imaging of the Brain H. KEITH BROWN, TODD R. HAZELTON, AND MARTIN L. SILBIGER Departments of Anatomy (H.K.B., T.R.H.) and Radiology (H.K.B., M.L.S.), University of South Florida College of Medicine, Tampa, Florida 336124799
ABSTRACT Currently, the diagnostic interpretation of magnetic resonance (MR) images requires that radiologists integrate specific tissue contrast information from several different images obtained at the same anatomic slice position. Each of these images has its own unique tissue contrast patterns which are based on the image acquisition parameters (pulse sequence) selected. The complex contrast patterns observable in these images reflect the inherent biophysical characteristics of the tissues and fluids present in the imaged section. In an effort to increase the diagnostic accuracy and efficiency of MR image interpretation, we have generated color composite images from quantitatively analyzed achromatic MR images of the brain, obtained while utilizing different pulse sequences. By using a DEC MicroVAX I1 computer with Interactive Digital Language (IDL), this color display method has been applied to images obtained from General Electric Signa and Siemens Magnatom imagers. For this study, our image sets included T1weighted, T2-weighted, and proton density spin echo sequences as well as both high and low flip angle gradient echo sequences. Advantages ofour color composite methods, in contrast to many other image processing techniques that have been described, are that minimal information is lost, computer misclassification of tissues is avoided, and the conspicuity of specific tissues is enhanced. Furthermore, with this method it is possible to produce composite images whose color renditions approach a natural anatomic tissue appearance. Availability of these color composites to radiologists may improve the efficiency and accuracy of the diagnostic interpretation of MR images.
ination of specific tissues, which are ultimately displayed as pixels comprising a two-dimensional MR image. The intensity of each individual pixel is determined by several biophysical characteristics of the tissue within the voxel. These characteristics include longitudinal relaxation rate (Tl), transverse relaxation rate (T2), proton density, and flow velocity and direction. To emphasize contrast patterns of specific tissues, different image acquisition parameters (pulse sequences) can be utilized to produce various types of “weighted” images. The selection of different pulse sequences allows for the generation of spin echo images that are T1-weighted, T2-weighted, or proton density weighted. Furthermore, gradient echo pulse sequences, or “fast scans,” can be utilized to obtain images that possess additional unique tissue-contrast patterns. In each type of image, whether spin echo or gradient echo, individual tissues appear differently, based on their own inherent biophysical characteristics (Sprawls, 1987). Because an individual tissue may have a characteristic appearance in a given type of MR image due to the particular pulse sequence parameters selected, effort has been directed toward developing color display methods for tissue characterization based on the multispectral analysis of MR image sets. Using NASA’s LANDSAT series satellite-image processing system and the nearest-neighbor classification algorithm, Vannier and co-workers (1985,1987) have successfully demonstrated the feasibility of computer-generated color displays by producing classified images with enhanced discrimination of different body tissues and fluids. Additional investigations in the area of pattern recognition for tissue classification have also been described by Koenig et al. (1986)and Jungke et al. (1988). Moreover, pattern recognition techniques have also been used successfully to demonstrate and enhance the appearance of certain pathologic conditions (Vannier et al., 1985; Koenig et al., 1986; Gohagan et al., 1987; Jungke et al., 1988). In addition to color image generation using pattern recognition techniques, hybrid or composite color imINTRODUCTION Magnetic resonance imaging (MRI) is a very rapidly ages have also been produced. The combination of both advancing technology which provides exciting new op- anatomical and parametric blood-flow velocity images portunities for the enhanced visualization of anatomic, into a single composite with range-defined color assignphysiologic, and pathologic features while using a sin- ments has been demonstrated by van Dijk (1984) and gle diagnostic imaging modality. This technology is Klipstein et al. (1987). In an image hybridization based on the principle that protons within body tissues method presented by Weiss et al. (19871, pixels from and fluids can absorb and then subsequently emit ra- one image were assigned varying spectral hues, while dio frequency (RF) signals when placed in a strong the luminance of these hues was derived from the inmagnetic field. It is the detection of the emitted RF signals from three-dimensional tissue voxels that provides data for the spatial location and contrast discrimReceived December 26, 1990. Accepted April 15, 1991. 0 1991 WILEY-LISS, INC.
24
H.K. BROWN ET AL.
Image 1
Image 2
-
Red Output
-
Green Output
C1R
Image 1
-+
4
CIG
- - - -:
I -I i
CIB
-7
C2R
-{--A
Red Output
I
I
I
I
'
! ++ I
Image 2 -+ C2G
I
~* C2B
-4
Green Output
L
Composite Image
,
1
1
,
I
I
Fig. 1. Schematic diagram illustrating (a)how a two-color composite is formed when one image is assigned red and the other green, and (b) how a multichannel (RGB) composite is formed from red, green, and
blue output images created by using an n number of input images (C = coeficiency value defining the contribution of images 1 , 2 , . . . ,n to the R, G, or B output channels).
tensity level of corresponding pixels in a second, spatially aligned image. Vannier et al. (1987, 1988) have created composites from MR images by assigning different images to the red, green, and blue (R, G, and B) guns of a color CRT monitor. A composite devised by using NASA's Earth Resources Application Software (ELAS) has also been presented by Vannier et al. (1988). Like the pattern recognition techniques previously described, the combining of different types of MR images into a composite has also been shown to enhance the appearance of certain pathologic conditions (Weiss et al., 1987; Levin et al., 1987). Recently, Kamman et al. (1989)proposed a color display method based on the calculated T1 and T2 relaxation times, as well as the proton density, of particular tissues of interest. In their display method, color images that simultaneously represented both the T1 and T2 relaxation times of selected tissues were generated by mixing the primary colors R, G, and B. This method is advantageous because its mechanism of display allows the generation of color images which represent inherent tissue properties and are not dependent on instrument settings as are conventional gray-tone MR images or the color displays derived from them. Although the color display methods described above have yielded images with enhanced tissue conspicuity, most possess unnatural color combinations that often
require special legends in order to be interpretable. In contrast, images with semi-natural color renditions, in addition to enhanced tissue conspicuity, have been presented (Brown et al., 1990). Such composites, when created from judiciously selected spin echo and/or gradient echo images, could contain important anatomic, physiologic, as well as pathologic information, all within the context of a single semi-natural-appearing image that might be more easily interpreted by the viewer. As mentioned previously, it has been demonstrated that it is possible to generate color MR composites in a manner that is similar to the mechanism by which images are displayed on color CRT systems (Vannier et al., 1987, 1988). In these systems, a composite image possessing a broad range of different colors is generated by additive combinations of the three primary colors R, G, and B. In an analogous process, color composites, such as those created by using NASA's LANDSAT satellite system, are also formed through the use of R, G, and B channels for each of the three bands that combine to generate the final image (Schowengerdt, 1983). In the three-primary-color system that is used for these techniques, additive combinations of spatially aligned pixels produce the various colors of the spectrum. With this method of display, in addition to the composite hues produced, pixels adjusted to different light intensities generate the appearance of different
COLOR COMPOSITES I N MRI O F THE BRAIN
25
250
200
Normalized Mean Intensity 100 Values 50
z
0 CSF
WM
GM
Mus.
Fat
Paro.
Air
Tissue Regions of Interest Fig. 2. Plot of average pixel signal intensity values for selected tissue regions of interest (ROIs) from the proton density (PD) and T2-weighted spin echo images in Figure 3. CSF, Cerebrospinal fluids; WM, white matter; GM, gray matter; Mus., muscle; Paro., parotid gland.
Fig. 3.Coronal proton density (TFUTE = 2600/22)(a)and T2-weighted (TFUTE = 2600/90)(b)spin echo images used to generate the color composite in Figure 8a. The images were obtained by using a 1.5 T Siemens Magnatom imager.
saturation and intensity values, and thus preserve both color and shading qualities within the final composite image (Thorell and Smith, 1990).
Gray Matter White Matter Cerebrospinal Fluid Red Nucleus Substantia Nigra Temporalis Muscle
pi:
Parotid Gland
4 Fig. 4. Schematic diagram of a coronal slice of the head illustrating important cranial tissues and structures discussed in the text.
MATERIALS AND METHODS Images utilized in this study were obtained from patients and volunteers in accordance with the rules and regulations established for human studies by the Institutional Review Board of the University of South Florida Health Sciences Center. Imaging studies were performed on a 1.5 Tesla (T) General Electric Signa imager (GE Medical Systems, Milwaukee, WI) and on 0.35,1.0,and 1.5T Siemens Magnetom imagers (Siemens Medical Systems, Iselin, NJ). Sets of images were acquired over a period of approximately 4 years and utilized a variety of pulse sequence parameters. Of the images acquired, the spin echo images that were ob-
H.K. BROWN ET AL.
26 25 0 200
Normalized Mean 150 Intensity 100 Values 50 0
5
CSF
WM
GM Mus.
Fat
Pa- A r t . Po.
Air
B1.
Tissue Regions of Interest Fig. 5. Plot of average pixel signal intensity values for selected tissue regions of interest (ROIs) from the FISP 70" gradient echo image and the T1-weighted spin echo image in Fig. 6. CSF, Cerebrospinal fluid; WM, white matter; GM,gray matter; Mus., muscle; Paro., parotid gland; Art. Bl., arterial blood.
Fig. 6. a: Coronal FISP gradient echo (TFb'TEIFlip Angle = 120/10/70")and b T1-weighted spin echo (TWTE = 600/20)images used to generate the color composite in Figure 9a.The images were obtained by using a 1.5T Siemens Magnatom imager.
tained possessed the following parameter values: T1weighted images had repetition time and echo time (TRITE)values (in milliseconds) of 400/20,500/17, and 600120; TPweighted images had TWTE values of 24001 90, 2600190, and 2700190; and proton density images had TRITE values of 2400130,2600122, and 2700130. In addition to spin echo images, gradient echo sequences were also obtained. For these images, the following parameters were utilized (TWTEIFlip Angle): 33/13/30", 120110170", 200113110", and 200115150". Each spin echo or gradient echo image had an acquisition matrix of 256 x 256 and a section thickness of 5 mm; and adjacent slices were obtained with a 50% gap.
The image data for generating the color composites were then transferred via magnetic tape to a DEC MicroVAX I1 computer (Digital Equipment, Maynard, MA.). Image analysis and processing were performed by using program routines developed with Interactive Digital Language (IDL) (Research Systems, Boulder, CO). Color image output was directed to an Electrohome ECM 1301-X VGA monitor (Electrohome, Ontario, Canada) with 8-bit display capabilities (256 possible colors). For certain tissues, the average signal intensities of the pixels within selected regions of interest (ROIs) were calculated in order to provide quantitative data
COLOR COMPOSITES IN MRI OF THE BRAIN
Fig. 7. a: Coronal T1-weighted (TWTE = 400/20), and, b, T2-weighted (TWTE = 2400/90) spin echo images, combined with a, c, GRASS gradient echo image (TR/TE/Flip Angle = 200/13/10")to form the composite in Figure 13a. The images were obtained by using a 1.5T GE Signa imager.
27
Figs. 8,9.
COLOR COMPOSITES I N MRI OF THE BRAIN
TABLE 1. Coefficiency values used to generate the composite images in Figure 13a, b, and c
Bits per color Coefficiency values Constant T1-weighted image T2-weighted image lo" fliD anele GRASS imaee
Red 3
Colors Green 3
Blue 2
0 50 100 100
0 50 100 0
0 0 150 0
supporting the empiric selection of image color assignments and color coefficiency values. Tissue ROTS were selected from various anatomical sites within spatially aligned MR images by using a mouse-driven system. By simultaneously analyzing all the data within a set, this method allowed the calculation of the mean signal intensities for corresponding pixels within identical ROIs for each of the images analyzed. Statistical calculations were performed on the DEC MicroVAX I1 computer by using a component of the IDL image-analysis program. With this particular program routine, the actual pixel intensity values were normalized to a 0 to 255 intensity scale so that the minimum intensity value (Datami,,)became 0 and the maximum (Data,,,,) became 255. This normalization was accomplished by using the equation below. Output
=
256 x
Data Data,,,,
-
Datami,. Datami,
-
A normalized output value was calculated for each actual pixel intensity value (Data) in the image matrix. The means and standard deviations of the actual signal intensities and normalized output values of all the pixels within a selected ROI were then reported. Based on the normalized mean intensity values obtained, certain color assignments or various color coefficiency values were empirically selected for each of the input images so that the desired colors of specific tissues could be optimized in the final composite. For color image display, the computer program routines were designed to allow the combination of data from two or more spatially aligned MR images in order to generate a single composite image. In the two-color method, based on characteristic signal intensities of selected tissues in the images to be combined, one im-
Fig. 8 . Two-color (WG) composites of coronal proton density/T2 image sets acquired from three different individuals on three different imagers. a: Individual 1imaged on a 1.5T Siemens Magnetom instrument with TWTE parameters of 2600122 and 2600/90 for the proton density and T2-weighted images, respectively. b: Individual 2 imaged on a 1.OT Siemens Magnetom instrument with TR/TE parameters of 2400/30 and 2400/90 for the proton density and T2-weighted images, respectively. c: Individual 3 imaged on a 1.5T GE Signa instrument with TWTE parameters of 2400/30 and 2400190 for the proton density and T2-weighted images, respectively.
Fig. 9. Two-color (RIG) composites of coronal FISP gradient echo (TWTE/Flip Angle = 120/10/70")and T1-weighted spin echo (TRITE = 600/20) images demonstrating enhanced vasculature (bright red) and the effects of magnetic field inhomogeneities on color assignments. The composite in a was created from the images in Figure 6. The composite image in b was produced from images obtained 15 mm away from those used to generate the image in a.
29
age was assigned to the R output channel and the other to the G output channel. To generate the final composite, the computer first partitioned the programmed color table into 4 bits (16 intensity levels) each for R and G, and then combined the assigned images (Fig. la). For each of the images used in the multichannel composite method, positive or negative integers were assigned for the three primary colors R, G, and B. An assigned coefficiency value of zero indicated no contribution from that color for that particular image. Negative coefficiency values were designed to subtract the selected color. For this latter method, individual R, G, and B output channels were created from the input images based on the coefficiency values assigned for each image. These three channels were then combined to form a single composite (Fig. lb). In order to generate composite images with this method, 3 bits (8 intensity levels) each for R and G and 2 bits (4 intensity levels) for B were utilized (D.M. Stern, Research Systems, Boulder, Co, written communication, February 1990). In both methods of display, based on the various intensity levels of spatially aligned pixels, additive combinations of the two or three primary colors resulted in a single image possessing a composite of all the information present in each of the original input images. For presentation in this report, the composites displayed on the monitor screen were photographed on ASA 200 color slide film with the aid of a macro zoom lens by using exposure times of 118 and 1/15 second and an F setting of 3.5. RESULTS Tissue Contrast Parameters and Two-Color (RIG) Composite Method
When forming a composite using additive color combinations, if one image is assigned red and the other green, tissues whose contrast patterns are similar will appear yellow, or varying shades of brown, depending on whether their respective signal intensities are high, intermediate, or low. Tissues which are hyperintense on one image, but hypointense on the other, will appear either red or green depending on which color is assigned to the image possessing the hyperintense tissue. Based on the normalized tissue intensity characteristics presented in Figure 2, a color composite of the proton density and TZ-weighted spin echo images shown in Figure 3 was created by assigning the proton density image to the R output channel and the T2-weighted image to the G output channel (Fig. 8a). Because cerebrospinal fluid (CSF) was very bright on the T2weighted image and relatively bright on the proton density image, the resultant additive color was greenish-yellow (See Fig. 4 for anatomical reference). Muscle, which had an intermediate intensity on the proton density image and a relatively low intensity on the T2-weighted image, was brick red. White matter, being hypointense to gray matter on both images, possessed a dark brown color, while gray matter appeared a lighter brown. Overall, this combination provided graded, semi-natural color tones exhibiting the desirable characteristics of both original gray-tone images. For the final composite, the proton density image possessed intensity characteristics that functioned to heighten the contrast between gray matter and white matter, while the intensities of the red nucleus, substantia nigra, and
30
H.K. BROWN ET AL.
Figs. 10-12.
COLOR COMPOSITES IN MRI OF THE BRAIN
CSF in the T2-weighted image were such that visual differentiation of these entities was enhanced. In addition to the heightened tissue conspicuity that has been demonstrated in this proton densityIT2 composite, this image combination also was found to provide a relatively consistent color scheme for differentiating tissues in several individuals imaged on the Siemens and GE instruments at different magnetic field strengths and using various pulse sequence parameters (Fig. 8). Normalized signal intensity values (Fig. 5) from the high flip angle FISP (Fast Imaging with Steady-State Precession; Siemens Magnetom) gradient echo and the T1-weighted spin echo images in Figure 6 revealed that the tissue contrast patterns for CSF, brain, and muscle were very similar for these two different pulse sequences. A color composite was generated from these two images by assigning the FISP image to R and the T1-weighted image to G (Fig. 9a). As would be predicted, muscle (which possessed a low intensity on both images, but was slightly more intense on the FISP image) was brick red; gray matter (which possessed an intermediate intensity on both images) was brown; and white matter (which possessed an intensity slightly greater than gray matter in both images) was strawcolored. Moreover, fat and parotid gland (which were brighter in the T1-weighted image and somewhat darker in the FISP image) had a greenish hue, and flowing blood (which was dark in the T1-weighted image and bright in the FISP image) yielded a bright red angiographic effect in the composite. The composite shown in Figure 9b was created from T1-weighted and FISP images obtained 15 mm away from the images used to generate the composite in Figure 9a. Both composites in this figure demonstrate some of the color assignment problems that may arise from magnetic field inhomogeneities both within a given image and between images of different sections. The changes in color assignment were particularly apparent in images utilizing gradient echo scans. Although pixel color assignments for gray and white matter varied both within an image and between two different images, visual differentiation of these tissues was preserved. Another example of a spin echolgradient echo compos-
31
ite is the image in Figure 10. This composite was generated by assigning a low flip angle FISP gradient echo image to the R output channel and a T1-weighted spin echo image to the G output channel. The resulting image possessed semi-natural tissue color assignments for brain, fat, and muscle. The composite in Figure 11was generated from coronal T1-weighted and T2-weighted spin echo images. To create this composite, the T1-weighted image was assigned to R and the T2-weighted image was assigned to G. Although the resulting contrast between gray and white matter was observed to be not as distinct as that in the proton densityIT2 composites (Fig. 8), this combination of images was found to provide adequate visual separation between these two brain tissues. This characteristic is important because this image set represents the two pulse sequences most frequently used in diagnostic MRI. An application of the two-color (WG) composite technique to a clinical case is illustrated in the proton densityIT2 composite in Figure 12a. For this composite, the proton density image was assigned to R and the T2-weighted image to G. This figure demonstrates various components of an arteriovenous malformation (AVM) in the brain of a patient. Another example of an enhanced pathology is the proton densityIT2 composite in Figure 12b of a patient with a large mass arising from the pituitary fossa. According to the evaluating radiologist's report, the anterior yellow region of the tumor most likely represents old hemorrhage, while the posterior red region probably depicts an area of more acute hemorrhage. On both the proton density and T2-weighted images, the anterior region was bright (normalized mean intensity values of 178 and 172, respectively) and, therefore, appeared yellow in the composite. In contrast, the posterior region was moderately bright on the proton density image (normalized mean intensity value of 101) and was dark on the T2-weighted image (normalized mean intensity value of 20). Since the proton density image was assigned to the R output channel and the T2-weighted image to the G output channel, these latter intensity characteristics correspond with the dark red color produced for this area of hemorrhage in the final composite. Multichannel (RGB) Composite Method
Fig. 10. Two-color (WG) composite demonstrating semi-natural color assignments for brain (flesh-colored) and muscle (brick red). This image was generated from a sagittal FISP gradient echo image (TWTE/Flip Angle = 200/11/50")and a T1-weighted spin echo image (TWTE = 500/17) obtained by using a 0.35T Siemens imager. Fig. 1 1. Color composite generated from coronal T2-weighted (TW TE = 600/20) and T2-weighted (TWTE = 2600/90) spin echo images obtained by using a 1.5T Siemens Magnetom imager. In this image, CSF is green, muscle is a very dark brick red, fat is orange-red, gray matter is light brown, and white matter is somewhat darker brown. Fig. 12. a: Two-color (WG)composite of an axial proton density/T2 image set (TWTE = 2400/30 and 2400/90, respectively) demonstrating the potential application of this technique to differentiate components within AVMs (arrow) in the brain. The images were obtained by using a 1.5T GE Signa imager. b Two-color (WG) composite of a sagittal proton density/T2 image set (TWTE = 2700/30 and 2700/90, respectively) illustrating the ability of this composite method t o characterize differentially old and new hemorrhage within a single image. The regions of hemorrhage are located within a pituitary adenoma (arrow). The images were obtained by using a 1.5T GE Signa imager.
The composites in Figure 13 were produced by combining coronal T1-weighted and T2-weighted spin echo images with a low flip angle GRASS (Gradient Refocused Acquisition in the Steady State; General Electric Signa) gradient echo image. The composite in Figure 13a was constructed from R, G, and B images formed by using the coefficiency values listed in Table 1 and the gray-tone images shown in Figure 7. The images in Figure 13 demonstrate the reproducibility of this technique between different individuals. Note that the color assignments for fat, muscle, CSF, gray matter, and white matter are fairly consistent between the three images. Due to the limited number of bits per color (3 for R, 3 for G, and 2 for B), the quality of these composite images was somewhat diminished compared to that of the two-color(R/G)composites in which 4 bits each were available for R and G (Figs. 8-12). The image in Figure 14 was generated by combining T2-weighted and proton density spin echo images with
32
H.K. BROWN ET AL.
Figs. 13, 14.
COLOR COMPOSITES IN MRI OF THE BRAIN
TABLE 2. Coefficiency values used to generate the composite image in Figure 14 Colors
Bits per color Coeficiency values Constant T2-weighted image Protonvdensity &age 30” flip angle GRASS image
Red 3
Green
3
Blue 2
0 0 -150 250
0 35 100 0
0 120 0 0
a low flip angle GRASS gradient echo image by using the coefficiency values listed in Table 2. Upon generation of the composite, the colors of the tissues were changed from their additive color combinations to colors with greater hue distinction by using a “16 Level” color table (D.M. Stern, Research Systems, Boulder, CO). This combination of images, with the applied color table, provided excellent contrast between gray matter, white matter, and CSF, although the color assignments in both the original composite (not shown) and the image in Figure 14 were observed to lack a naturalappearing color scheme. DISCUSSION
Benefits of color-image display, in contrast to achromatic or monochromatic presentation, include a more realistic appearance, enhanced information processing, and increased ability to interpret related and unrelated data (Thorell and Smith, 1990). Because of these benefits, the increased tissue conspicuity that is yielded with color MR images has the potential to allow the detection of subtleties that might otherwise have been missed by the interpreting radiologist. Currently, in order to assess tissue-specific contrast patterns, radiologists must analyze the information in different images of the same scene by back-and-forth comparison. It is possible, however, that the same information might be more easily interpreted in a single, appropriately formulated color composite. Intuitively, the efficiency and accuracy of achromatic MR image interpretation would be improved by preview or reference to a color composite, generated by using a well-established protocol in which color assignments were consistently reproducible regardless of the make of the instrument, magnetic field strength, or coil inhomogeneities.
Fig. 13. Multichannel (RGB) color composites generated by combining coronal T1-weighted (TWTE = 400120) and T2-weighted (TWTE = 2400190) spin echo images with a GRASS gradient echo image (TRITEIFlip Angle = 200/13/10”)by using the coefficiency values in Table 1.This figure demonstrates the reproducibility of the RGB composites in different individuals. The images from all three individuals (a+) were obtained by using a 1.5T GE Signa imager. The composite in a was generated by using the images shown in Figure 7. Fig. 14. Multichannel (RGB) color composite with a “16 Level” stepped color table that provides for greater contrast between gray matter, white matter, and CSF. This composite was created by combing axial proton density and T2-weighted spin echo images (TRITE = 2400/30 and 2400/90, respectively) with a GRASS gradient echo image (TWTE/Flip Angle = 33/13/30”).The images were obtained by using a 1.5T GE Signa imager.
33
As we have demonstrated, it is possible to generate such composites by using the methods presented in this report. Moreover, the design of these methods is such that almost any combination andlor number of images can be utilized, as long as the coeficiency tables are appropriately adjusted so that a meaningful composite can be generated. The composites in Figure 8 demonstrate the ability of the methods we have presented to meet the reproducibility criteria set forth above. Even with the variable color assignments demonstrated in Figure 9, this method can generate quality images without major color assignment problems as evidence by the composites in Figure 8. An additional benefit of the composite image generation method presented, in contrast to most pattern recognition techniques, is that no information is added or filtered out during the steps of image composition. Moreover, with the composite methods we have described, tissues are not statistically classified by a computer. Instead, pixel intensity information is merely being combined to form a unique color composite based on tissue contrast patterns in images obtained by using selected pulse sequences. With our composite method, because the identification of specific tissues is made by an anatomist, radiologist, or other trained observer, computer misclassification of tissues is avoided. Potential problems with the composite methods described in this report include possible misregistration of images if the patient moves between different pulse sequences, or motion artifacts if the patient moves during a particular pulse sequence. However, availability of original achromatic images would aid in defining image misregistration, chemical-shift, ghosting, wraparound, and other image artifacts. In addition to these potential problems, pulse sequences required for a particular composite generation protocol must be in the same sectional plane orientation and include corresponding slice positions. The former requirement presents an additional problem in terms of increasing scan times past acceptable limits if multiple planes are required. This problem could potentially be overcome through the use of rapidly acquired 3-D gradient echo pulse sequences because, owing to their short imaging time, there would be less likelihood of patient movement causing misregistration of spatially aligned images. In addition, because of the three-dimensional matrix acquisition of data, multiple orientations could be obtained from a single imaging session. From such sessions, a series of rapidly acquired images that exhibit T1, T2, T2*,1 and proton density characteristics could be generated (Siemens Medical Systems, Iselin, NJ). While these images, individually, might not be easily interpretable to the radiologist accustomed to conventional spin echo images, it is possible that they could be combined into a composite in which specific tissue features would be displayed in familiar colors. Such a method might prove to have the same diagnostic value as much longer scanning sessions with conventional spin echo pulse sequences. Even so, for the present, it is still possible to select appropriate spin echo pulse sequences and orientations that may be used to generate ’T2* gradient echo images are based on the effective transverse relaxation time, which is typically shorter than T2.
34
H.K. BROWN ET AL.
color composites that could potentially enhance the diagnostic evaluation of MR images. In addition to the potential problems described above, the quality of the final composite images we have generated is limited by the graphics capabilities of the computer hardware. This feature is well illustrated by comparing the images in Figure 13 in which R, G, and B were assigned 3 , 3 , and 2 bits each, respectively, with the images in Figures 8-12 in which Rand G were each assigned 4 bits. In images possessing more bits per color, there were more gradations of color available for display (number of possible colors = 2", where n is the number of bits). Therefore, with only 2 or 3 bits per color, more abrupt changes in color assignment resulted in the creation of artificial boundaries within various tissues (Fig. 13).In contrast, with 4 bits per color, these artificial boundaries appeared as a gradient that transitioned more subtly from one color to another (Figs. 812).Because the number ofpossible colors is determined by the number of assigned bits, the use of a high-resolution monitor with 24-bit display capabilities should dramatically improve the quality of the multichannel (RGB) composites, as well as further improve the appearance of the two-color (FUG) composites. With regard to the pathology cases presented in this report, although the pulse sequences used to generate these composites may not necessarily be the ideal set for diagnostic evaluation, they do demonstrate the potential this technique has for enhancing certain physiologic and pathologic components of MR images. This potential is well illustrated by the composite in Figure 12a. Through the prudent selection and combination of various f low-sensitive pulse sequences, a single MR image possessing various color-coded flow rates could be produced. Additionally, Figure 12b shows the possible application of this method to the assessment and dating of hemorrhage. This figure demonstrates that the composite generation methods described in this report have potential applications in terms of the diagnostic evaluation of traumatic intracranial hemorrhagic conditions, as well as for the evaluation of both intracranial and extracranial hemorrhagic neoplasms. Overall, we feel the methods of color display presented in this report are more advantageous than many of the other color display techniques that have been described in the literature. Because the color assignments are easily adjustable, this procedure allows the formulation of composite images possessing nearnatural colors that should function to enhance visual interpretability. Moreover, unlike pattern recognition techniques which vary only hue in order to separate statistically defined tissues, color composites generated by the methods we have presented maintain saturation and intensity values in addition to their composite hues, thus preserving all three of the important components of human visual perception. In terms of data presentation, this feature is important because the presence of all three of these components allows for a greater information capacity within a displayed image (Farrell, 1987). Additionally, one distinct advantage of our composite generation methods is the simplicity with which the composites are produced. Colors that appear in the final composites can be easily defined in terms of the intensity levels of various tissues within the component images as well as by the colors or coefficiency values these images have been assigned.
In summary, the composite generation methods proposed in this report represent a viable technique for displaying diagnostically relevant information of multiple MR images in one easily interpretable color image. Within appropriate clinical trials and case-study applications, it will be possible to assess more fully the usefulness of this method for enhancing the diagnostic interpretation of MR images of specific pathologies. ACKNOWLEDGMENTS
We gratefully acknowledge the technical and support assistance of the following individuals: M. Bryant, L. Clarke, T. Dula, J. Lefler, J. Madden, C. Philips, J. Schellenberg, L. Sears, R. Shaw, D. Stern, R. VelthuiZen, and C. Wood. Funding and resource support for this project were provided by the H. Lee Mofitt Cancer Center and Research Institute, the University Diagnostic Institute, the University of South Florida President's Council, and the NASAIAmerican Cancer Society (#89-25). Portions of this work were done during the tenure of a Medical Student Research Fellowship of the American Heart Association (T.R.H.) and during an American Cancer Society R.G. Thompson Summer Research Fellowship (T.R.H.). LITERATURE CITED Brown, H.K., J. Schellenberg, L. Clarke, and M. Silbiger 1990 Diagnostic utility of color composites generated from magnetic resonance images. Anat. Rec., 226:16A (Abstr.). Farrell, E.J. 1987 Visual interpretation of complex data. IBM Syst. J., 26: 174-200. Gohagan, J.K., E.L. Spitznagei, W.A. Murphy, M.W. Vannier, W.T. Dixon, D.J. Gersell, S.L. Rossnick, W.G. Totty, J.M. Destouet, D.L. Rickman, T.A. Spraggins, and R.L. Butterfield 1987 Multispectral analysis of MR images of the breast. Radiology, 163: 703-707. h n g k e , M., W. von Seelen, G. Bielke, S. Meindl, G. Krone, M. Grigat, P. Higer, and P. Pfannenstiel 1988 Information processing in nuclear magnetic resonance imaging. Magn. Reson. Imaging, 6: 683-693. Kamman, R.L., G.P. Stomp, and H.J.C. Berendsen 1989 Unified multiple-feature color display for MR images. Magn. Reson. Med., 9:240-253. Klipstein, R.H., D.N. Firmin, S.R. Underwood, G.L. Nayler, R.S.O. Rees, and D.B. Longmore 1987 Colour display of quantitative blood flow and cardiac anatomy in a single magnetic resonance cine loop. Br. J . Radiol., 60:105-111. Koenig, H.A., R. Bachus, and E.R. Reinhardt 1986 Pattern recogni' tion for tissue characterization in MR imaging. Health Care Instrum., 1~184-187. Levin, D.N., A. Herrmann, T. Spraggins, P.A. Collins, L.B. Dixon, M.A. Simon, and A.E. Stillman 1987 Musculoskelatal tumors: improved detection with linear combinations of MR images. Radiology, 163:545-549. Schowengerdt, R.A. 1983 Techniques for Image Processing and Classification in Remote Sensing. Academic Press, Orlando, pp. 117118. Sprawls, P. 1987 Nuclear magnetic resonance imaging. In: Physical Principles of Medical Imaging. Aspen, Rockville, pp. 381-425. 'rhorell, L.G., and W.J. Smith 1990 Using Computer Color Effectively. Prentice Hall, Englewood Cliffs, pp. 16-17, 118. van Dijk, P. 1984 Direct cardiac NMR imaging of heart wall and blood flow velocity. J . Comput. Assist. Tomogr., 8:429-436. Vannier, M.W., R.L. Butterfield, D. Jordan, W.A. Murphy, R.G. Levitt, and M. Gad0 1985 Multispectral analysis of magnetic resonance images. Radiology, 154:221-224. Vannier, M.W., R.L. Butterfield, D.L. Rickman, D.M. Jordan, W.A. Murphy, and P.R. Biondetti 1987 Multispectral magnetic resonance image analysis. Crit. Rev. Biomed. Eng., 15:117-144. Vannier, M.W., C.M. Speidel, and D.L. Rickman 1988 Magnetic resonance imaging multispectral tissue classification. News Physiol. Sci., 3:148-154. Weiss, K.L., S.O. Stiving, E.E. Herderick, J.F. Cornhill, D.W. Chakeres 1987 Hybrid color MR imaging display. Am. J . Radiol., 149: 825-829.
