796
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL 37. NO. 8. AUGUST 1990
A Workstation-Based System for 2-D Echocardiography Visualization and Image Processing
Abstract-Parameters of cardiac function can be drawn from the analysis of echocardiographic image sequences, especially the motion of the ventricular wall, heart wall thickness, and shape parameters. Automatic image analysis and visualization allows reduced manual operations and, above all, ensures objectivity and repetition of analysis, which is essential when one wishes to calculate parameters based on variations, i.e., on image sequence analysis. In this paper, a system and the related software package for interactive echocardiographic image analysis and visualization are illustrated and discussed. Furthermore, the full model for smoothing, edge enhancement, and contour detection is discussed and a new technique based on the heat anisotropic diffusion model is presented. The results of automatic detection of the left ventricle contours are presented and discussed.
I. INTRODUCTION AGING and visualization techniques in medical diagnostics are used more and more frequently and their application field is expanding rapidly. Particularly within the cardiological field, echocardiography has demonstrated a high level of effectiveness at reasonably low cost, even though the image quality obtained by using such technique is not as good as those obtained by using others. Moreover, important deductions as to heart size and function can still be drawn from echo images. Since the local blood rate has been added to the 2-D echocardiographic images (echo color coded Doppler), many investigations, once requiring X-ray angiography, can now be solved by echography . Ventriculography analysis techniques (i.e., contour detection for volume computation and wall motion estimation) can also be used, suitably corrected, in echography analysis, as more than one author has suggested [2]-[4], [7], [12], [16]. These analyses are based on the detection of left ventricular internal wall edge and on the analysis of wall displacement in an image sequence. Usually the ventricular contours are manually traced by experts. This operation is particularly onerous when several or all frames which correspond to an entire cardiac
I"
Manuscript received July 28, 1989; revised January IO, 1990. This work was supported in part by a grant from the Consiglio Nazionale delle Ricerche. C. Lamberti is with the Dipartimento di Elettronica, Informatica e Sistemistica, Universita' di Bologna, 40136 Bologna, Italy. F. Sgallari is with the Dipartimento di Matematica, Universita' di Bologna, 40123 Bologna, Italy. IEEE Log Number 9036240.
cycle must be analyzed. This occurs, for example, when one wishes to study the diastolic or systolic phase. Furthermore, the automatic detection of the contours eliminates the subjectivity introduced by the operator and ensures the repetition which is an essential requirement in such analyses. Automatic detection of the echocardiography image contours presents serious difficulties because of the poor image quality, the possibility of drop-outs, high noise level, valve and papillary muscle interference in the images, etc. Therefore the clinician takes great advantage of the availability of an interactive system for visualization and correction of any step of the analysis process. In this paper we present a system for echocardiography image digital processing and visualization oriented to the assessment of features such as contours, volumes, and wall motion. This system allows the operator an interactive approach in the acquisition of preselected image sequences and in the processing of the digitized and stored images. It will be shown how a good final result of the process is obtained only by a close interaction between the operator and the system; in fact, the operator can choose from a proposed menu of various techniques, suggest the value of the parameters and, after the visualization of the intermediate results, modify and continue the analysis until the best final result is obtained. In Section I1 the hardware and software of the workstation which we have implemented are presented. In Section I11 the full model for smoothing, edge enhancement, and contour detection is illustrated. Particular attention has been devoted to the implementation of anisotropic diffusion technique for 2-D echocardiography analysis. Several examples are presented and discussed in Section IV. PROCESSING, AND 11. THE SYSTEMFOR ACQUISITION, VISUALIZATION A scheme of the system for the acquisition, processing, and visualization of echocardiographic image sequences is presented in Fig. 1. The system is based on a personal computer with two auxiliary boards: a frame-grabber board and a VTR interface. The frame-grabber board is one of the commercially available (Matrox PIP 1024, with a 512 X 512 pixels resolution, 256 gray levels, four-frame
00 18-9294/90/0800-0796$0 1.OO O 1990 IEEE
LAMBERT1 A N D SGALLARI 2-D ECHOCARDIOGRAPHY VISUALIZATION A N D IMAGE PROCESSING
797
VIDEO TAPE RECORDER
j I
j
PERSONAL COMPUTER
.
,
I
j. !
.j V I # I n I E I F n C f .......................
i
f l l l l f S I I I I L I 1111)
I
L.....
OPTICAL DISK
.
.
.
.........................
Fig. 1. General scheme of the system for acquisition, processing and visualization of 2-D echographic frames.
memory). The VTR interface board has been built by the authors to allow the operator some functions such as control of VTR by PC; automatic assignment of numerical codes to the video frames by utilizing one of the two audio channels; visualization of the recorded sequences with the codes superimposed; acquisition of one or more frames according to the input of the corresponding code by the operator; ECG signal analysis and QRS gated acquisition. A particular frame-grabber management software package written in C language has been developed and implemented. The main features of such a package are as follows: a) b) c) d)
user-friendly frame-grabber management; high capability of expansion; very easy configuration; modular function organization.
Feature a) was obtained by designing a mouse-based system, together with an overlapping-window presentation. Features b)-d) were obtained by total parametrization of the software code. 111. THE EDGEDETECTION MODEL Unfortunately for the clinician, the low-quality of echocardiograms makes detection of small or faint structures difficult. To promote easier interpretation of ventricular contour-detected data, we consider an interactive approach.
The complete model for eGge detection consists of a low-pass filter, a Laplace filter, and a zero-crossing detector performed in sequence. We have extended this model with an edge strength process and a threshold operator to obtain a binary image. A block diagram of the sequence of processing procedures is shown in Fig. 2. The edge detection model consists of five major modules associated with their parameters value. With all the possible choices of parameters, the operator can construct many edge detectors with different properties, characteristics, and performances, taking previous experiences or comparisons into account. In this section the possible choices for each module will be summarized with a brief description of the interaction and visualization process. For the detailed description of these modules together with their implementation we refer to [61, [81, [ill, [ W , 1151. In echocardiography we are confronted with low-quality images. Improving the image may thus be seen as a first step towards the edge detection phase. Moreover, the ill-posed nature of the problem suggests the use of lowpass filters in order to transform it into a well-posed problem [5], [ 151, [ 171. We allowed a flexible approach, where the operator is free to choose among three kinds of filters (moving average filter, finite prolate spheroidal window, and median filter), to compare the results obtained, and to decide on repetition of the smoothing step or continuation of the elaboration. Prolate functions were chosen because they constitute the optimal band-limited filter which maximizes the output energy in a given region [ 11, [6], [ 141, [ 181. The other smoothing filters considered (moving average and median
798
l F E t TRANSACTIONS ON BIOMEDICAL ENGINEERING. VOL. 37. NO 8. AUGUST 1990
Edgr. linage -.
I
Fig. 2 . The edge detection model.
filter) are able to suppress noise by applying some form is much greater than K , we have c ( x , y , t ) = 0 and diffusion is hampered. On the contrary, where V I is much of averaging to the input image [6]. We must be careful in using filters of this kind, how- smaller than K , then c(x, y, t ) = C and diffusion is high, ever, because averaging blurs the image and thus de- thus producing image smoothing. Parameter K therefore creases resolution [5], [ 171. Thus, visualization of the re- is like a threshold referred to VI: it separates noise from sults is fundamental in order to choose or change the edges. In our model there is a suggested value for K , but parameters. Default values for the parameters of these fil- a change is possible and advisable where the operator is ters (spatial and frequency domain sizes) are suggested by interested in the enhancement of a particular region or details. the system but can be varied by the operator. If the low-pass filtering is judged unsatisfactory, it may The new image Z(x, y ) is derived from the original imbe followed by the edge strength process. In it, the heat age Zo(x, y ) by applying the anisotropic diffusion techanisotropic diffusion technique, proposed by Perona and nique: Malik [13], is used to reduce the effect of noise superI ( x , y ) = Io(x, y ) AZ imposed on the image and to emphasize the edges of the objects [8]. This technique estimates the luminous inten- where A I results from discretization of the fundamental sity gradient locally and, referring to such a value, pro- equation: duces a smoothing only inside the semantically homogeAZ neous regions, while the contrast is stressed close to the - = div ( c ( x , y, t ) V I ) contours. Let us consider a parallel between temperature At changes in a bidimensional structure due to the heat aniFor appropriate discretization schemes and values for At sotropic diffusion during time and intensity changes we refer to [8], [13]. through an image due to low-pass filtering with different The anisotropic diffusion technique produces different scale-space parameter. The fundamental equation behomogeneous regions so that segmentation is now much comes easier. In fact, if the ventricular contour has to be deI: intensity tected, as in our case, a binary image is obtained by comar = div ( c ( x , y, t ) V I ) t : scale-space parameter paring the pixel value with a suitable threshold. Due to the poor quality of echocardiograms and the at c: conductivity possible low signal-to-noise ratio, the threshold level is where t is the scale-space parameter, and a l l a t the interactively varied by the operator by visual inspection. The next step consists of Laplace filtering, which is brightness intensity change due to the application of the thoroughly explained in the literature [5], 161, 1111, 1171. anisotropic diffusion process. The conduction coefficient In our model we consider the Marr-Hildreth operator aladopted is as follows: lowing the choice of the standard deviation of the Gaussian. The size of the kernel is determined by imposing that all the significant components are included. Many papers have pointed out the relationship between the size of the Gaussian and the spatial accuracy of the detected edge position obtained by the Marr-Hildreth opwhere C and K are constants. In the pixel in which the gradient of the brightness ( V I ) erator [ 111, [ 151, [ 171. The system suggests a large op-
+
LAMBERT1 A N D SGALLARI: 2-D ECHOCARDIOGRAPHY VISUALIZATION A N D IMAGE PROCESSING
799
erator (or Gaussian filter) to find the approximate position of strong edges, while a small operator is suggested to obtain high positional accuracy. Such differentiation module produces an image in which the zero crossing must be detected. In fact, it is the task of the zero-crossing detector to assign the changes in sign of a Laplace filtered image as the location of an edge. It is, however, unlikely that most of the zero crossing will coincide with the specified spatial coordinates called pixels, thus the nearest neighbor in the direction of the gradient should be assigned as the edge pixel to achieve accuracy. IV. VENTRICULAR CONTOURDETECTION In this section we present the results obtained by the operator interacting with the workstation on which the edge detection model presented above has been implemented. The examples presented here refer to detection of the contour of the left ventricular cavity, and in particular by analyzing a parasternal view and an apical view. Such views are, in fact, among those most commonly used by the cardiologist to evaluate the functionality of the left ventricle: on the basis of the analysis of the ventricular detected contours in a sequence of frames, quantitative evaluations of the characteristics of contraction and relaxation of the various regions of the ventricular wall can be made. The images presented constitute a portion of 256 x 256 pixels taken from the original echo images.
Low-Pass Filtering In Fig. 3(a) and (b) are shown the results obtained by filtering the original image (upper left part) with a moving-average filter 9 X 9 (upper right part), with a median filter 9 X 9 (lower right part), and with a spheroidal prolate filter 9 X 9 with a Fourier transform equal to zero out of the region of radius R = 0.6 (lower left part) [ l ] , [6]. The results obtained using such methods are satisfactory, showing little difference from one another. The median filter, however, seems to be the most suitable for eliminating the background noise typical of echocardiographic images and being the least time consuming. Laplacian Operator In Fig. 4(a) and (b) are reproduced the images prefiltered with a median filter 9 x 9 (upper left part), and the images obtained by means of a convolution of such image with the Laplacian of Gaussian operator corresponding to different value for standard deviation U ( U = 0.5 upper right, U = 0.8 lower right, U = 1.0 lower left). The values assigned to U are within the optimal range for edge detection in echo images; nevertheless the obtained images (Laplacian images) show that small differences occur. Edge Strength Detector and Thresholding In Fig. 5(a) and (b) the results obtained by using the option of “edge strength” are reported. Such option consists in processing the original image or the prefiltered one
(b) Fig. 3 . Low-pass filtering of a parasternal view (a) and of an apical view (b). Original image (upper left); moving average (upper right); median filter (lower right); spheroidal prolate (lower left).
(upper left part) by means of the anisotropic diffusion technique. In the resulting image (upper right part) a smoothing had been produced inside the homogeneous regions, while the contrast close to the contours had been stressed. Upon such an image an operation of thresholding can be made to produce a binary image (lower right part) in which the ventricular cavity appears very clearly. Such a binary image is then convolved by means of a Laplacian of Gaussian operator to obtain the Laplacian image (lower left part).
Zero-Crossing and Contours The points at which the Laplacian image (second derivative) cross the zero are considered to be points of con-
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING. VOL 37. NO
800
8 . AUGUST 1990
(b) Fig. 4. Laplacian of Gaussian operator applied to a parasternal view (a) and to an apical view (b). Prefiltered image by median filter (upper left): Laplacian of Gaussian with U = 0.5 (upper right), U = 0.8 (lower right), U = 1.0 (lower left).
tour. Fig. 6(a) and (b) show the points of zero-crossing superimposed on the original images. It is evident how the treatment of the image with the method of edge strength and the successive binarization make it possible to individuate with greater accuracy the points of the contour. V . CONCLUSION The system presented and described in this paper has shown itself to be particularly useful in the analysis of echocardiographic tiacings by ;he medical operator: Since
(b) Fig. 5 . Edge strength by anisotropic diffusion technique on a parasternal view (a) and on an apical view (b). Original image (upper left); result of anisotropic diffusion technique with K = 8, C = 5, Ar = 1 (upper right); binary image achieved by thresholding (lower right); Laplacian of Gaussian of the binary image with U = I .O.
it shows the intermediate results, and the interaction with the system is simple, it is possible to optimize the process of analysis and achieve an optimal result. The results obtained during the first phase of the system experimentation have made clear how a preliminary treatment of low-pass filtering is essential, and how interaction and visualization allow for a selection of the optimal values to assign to the parameters. Moreover, the procedure of edge strength (anisotropic diffusion technique and thresholding) has shown itself particularly suitable for the echo images, given the low quality of the original images.
-
LAMBERT1 A N D SGALLARI: 2-D ECHOCARDIOGRAPHY VISUALIZATION A N D IMAGE PROCESSING
80 1
whose geometry, dynamics, and noise distribution are known a priori. ACKNOWLEDGMENT The authors wish to thank G. Curti for his invaluable technical assistance, and Prof. Dr. G. Binetti of the Istituto di Malattie dell’ Apparato Cardiovascolare, Universita’ di Bologna, for his cooperation and valuable discussions.
REFERENCES
(b) Fig. 6. Edge detection on a parasternal view (a) and on an apical view (b). Zero-crossings of the Laplacian of Gaussian with U = 0 . 5 (upper right). U = 0.8 (lower right), U = 1 .O (lower left), and U = 1.O after the edge strength procedure (upper left).
It is furthermore maintained that the analysis of static images is penalizing in that it does not allow the temporal continuity of the scene and the action of the operator eye when observing the recording to be taken into consideration. Thus, the next step will be to include time as a third dimension of analysis. Moreover, in order to be able to validate all the procedures of analysis, we need a physical model (phantom) fully representative of the structures and of the dynamics of the human heart. The difficulty of creating such a phantom has suggested to investigate the feasibility of a numerical model which would permit the synthetization of images similar to those obtained experimentally, but
S . Alliney and F . Sgallari, “Two-dimensional almost bandlimited discrete sequences,” Pubblicazioni IAC, Ser. 111, no. 165, 1978. C. E. Angermann, R. J. Hart, C. H. Spes, W. Zwehl, M. Marquart, D. P. Schmid, and K. Theisen, “Computerized quantitative evaluation of the endocardium in serial two-dimensional echocardiograms of the left ventricular short axis,” in Proc. Comp. in Cardiol., Leuven, Belgium, 1987, New York: IEEE Comput. Soc., 1987, pp. 437440. J. G . Bosch, J. H . C. Reiber, G.van Burken, J. J. Gerbrands, W. J. Gussenhoven, N. Bom, and J. R. T . C. Roelandt, “Automated endocardial contour detection in short-axis 2-D echocardiograms; methodology and assessment of variability,” in Proc. Comp. in Cardiol., Washington, 1988, New York: IEEE Comput. Soc., 1988. C . H. Chu, E. J . Delp, and A. J . Buda, “Detecting left ventricular endocardial and epicardial boundaries by digital two-dimensional echocardiography,” IEEE Trans. Med. Imaging, vol. MI-7, pp. 8190, 1988. E. De Micheli, B. Caprile, P. Ottonello, and V. Torre, “Localization and noise in edge detection,” IEEE Trans. Pattern Anal. Mach. htell., vol. PAMI-11, pp. 1106-1117, 1989. C . Lamberti, A. Guidazzoli, and F . Sgallari, “Image regularization for echocardiography digital processing,” in Proc. Time-Varying Image Processing and Moving Object Recognition, Florence, Italy, V. Cappellini, Ed. Amsterdam: North-Holland, 1989, pp. 173-180. C . Lamberti, A. Martelli, and A. Guidazzoli, “Postprocessing techniques for 2-D echocardiographic imaging,” in Proc. IEEE EMBS 10th Ann. Intern. Con$, New Orleans, LA, G. H a m s and C. Walker, Eds. New York: IEEE Press, 1988, pp. 450-451. C. Lamberti, S. Lusvardi, and C. Truzzi, “Workstation for 2-D echocardiography image processing,” Computers in Cardiology, Jerusalem, Israel, New York: IEEE Comput. Soc., 1989. W . H. H. J . Lunscher and M. P. Beddoes, “Optimal edge detector design I: Parameter selection and noise effects,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-8, pp. 164-177, 1986. -, “Optimal edge detector design 11: Coefficient quantization,” IEEE Trans. Pattern Anal. Mach. Inrell., vol. PAMI-8, pp. 178-187, 1986. D. Marr and E. Hildreth, “Theory of edge detection,” Proc. R. Soc. Land. E., vol. 207, pp. 187-217, 1980. E. G. Melendo and E. J . Delp, “A technique for the visualization and analysis of cardiac wall motion by two-dimensional echocardiography,” IEEE Trans. Med. Imaging, vol. MI-8, pp. 104-106, 1989. P. Perona and J. Malik, “A network for multiscale image segmentation,” in Proc. 1988 IEEE Intern. S p p . on Circuits and Systems, Epsoo, Finland, New York: IEEE Press, 1988, pp. 2565-2568. D. Slepian, “Prolate spheroidal wavefunctions, Fourier analysis and uncertainty V: The discrete case,” Bell Syst. Tech. J . , vol. 57, pp. 1317-1430, 1978. G. Torre and T . A. Poggio, “On edge detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-8, pp. 147-163, 1986. L. Torres, E. Sangra’, A. Gasull, and S . Sallent, “A new algorithm for automatic border detection of two-dimensional echocardiographic images,” in Proc. Time-Varying Image Processing and Moving Object Recognition, Florence, Italy, V . Cappellini, Ed. Amsterdam: North-Holland, 1989, pp. 181-188. [I71 L. J . van Vliet, I . T. Young, and G. L. Beckers, “A nonlinear Laplace operator as edge detector in noisy images,” Comput. Graph. Image Process., vol. 45, pp. 167-195, 1989. [IS] R. Wilson, “Finite spheroidal sequences and their applications I: Generation and properties,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-9, pp, 787-795, 1987.
802
IEEE TRANSACTIONS ON BlOMEDlCAL ENGlNEERING. VOL. 37. NO. 8. AUGUST 1990
Claudio Lamberti (M’87) was born in Bologna, Italy, in 1948. He received the degree in mechanical engineering from the University of Bologna. Bologna, Italy, in 1974, and the postgraduate degree in biomedical technology from the University of Bologna School o f Medicine in 1978. He was a Research Fellow in bioengineering at Istituto di Automatica from 1976 to 1980. He joined the Division of Cardiology at the University of Bologna Hospital from 1980 to 1981. Since 1981 he has been working as a Research Associate at the Department of Electronics, Computer Science. and Systems of the University of Bologna. His principal activity is research in the field of cardiovascular signal processing, cardiovascular image processing and technology assessment.
Fiorella Sgallari (M’88) was born in Bologna, Italy, in 1953. She received the degree in mathematics from University of Bologna, Bologna, Italy, in 1976. She is currently an Associate Professor of Numerical Analysis at the University of BolognabHer main research is at present concerned with parallel computation and digital processing for medical imaging.
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL 37. NO. 8. AUGUST 1990
A Workstation-Based System for 2-D Echocardiography Visualization and Image Processing
Abstract-Parameters of cardiac function can be drawn from the analysis of echocardiographic image sequences, especially the motion of the ventricular wall, heart wall thickness, and shape parameters. Automatic image analysis and visualization allows reduced manual operations and, above all, ensures objectivity and repetition of analysis, which is essential when one wishes to calculate parameters based on variations, i.e., on image sequence analysis. In this paper, a system and the related software package for interactive echocardiographic image analysis and visualization are illustrated and discussed. Furthermore, the full model for smoothing, edge enhancement, and contour detection is discussed and a new technique based on the heat anisotropic diffusion model is presented. The results of automatic detection of the left ventricle contours are presented and discussed.
I. INTRODUCTION AGING and visualization techniques in medical diagnostics are used more and more frequently and their application field is expanding rapidly. Particularly within the cardiological field, echocardiography has demonstrated a high level of effectiveness at reasonably low cost, even though the image quality obtained by using such technique is not as good as those obtained by using others. Moreover, important deductions as to heart size and function can still be drawn from echo images. Since the local blood rate has been added to the 2-D echocardiographic images (echo color coded Doppler), many investigations, once requiring X-ray angiography, can now be solved by echography . Ventriculography analysis techniques (i.e., contour detection for volume computation and wall motion estimation) can also be used, suitably corrected, in echography analysis, as more than one author has suggested [2]-[4], [7], [12], [16]. These analyses are based on the detection of left ventricular internal wall edge and on the analysis of wall displacement in an image sequence. Usually the ventricular contours are manually traced by experts. This operation is particularly onerous when several or all frames which correspond to an entire cardiac
I"
Manuscript received July 28, 1989; revised January IO, 1990. This work was supported in part by a grant from the Consiglio Nazionale delle Ricerche. C. Lamberti is with the Dipartimento di Elettronica, Informatica e Sistemistica, Universita' di Bologna, 40136 Bologna, Italy. F. Sgallari is with the Dipartimento di Matematica, Universita' di Bologna, 40123 Bologna, Italy. IEEE Log Number 9036240.
cycle must be analyzed. This occurs, for example, when one wishes to study the diastolic or systolic phase. Furthermore, the automatic detection of the contours eliminates the subjectivity introduced by the operator and ensures the repetition which is an essential requirement in such analyses. Automatic detection of the echocardiography image contours presents serious difficulties because of the poor image quality, the possibility of drop-outs, high noise level, valve and papillary muscle interference in the images, etc. Therefore the clinician takes great advantage of the availability of an interactive system for visualization and correction of any step of the analysis process. In this paper we present a system for echocardiography image digital processing and visualization oriented to the assessment of features such as contours, volumes, and wall motion. This system allows the operator an interactive approach in the acquisition of preselected image sequences and in the processing of the digitized and stored images. It will be shown how a good final result of the process is obtained only by a close interaction between the operator and the system; in fact, the operator can choose from a proposed menu of various techniques, suggest the value of the parameters and, after the visualization of the intermediate results, modify and continue the analysis until the best final result is obtained. In Section I1 the hardware and software of the workstation which we have implemented are presented. In Section I11 the full model for smoothing, edge enhancement, and contour detection is illustrated. Particular attention has been devoted to the implementation of anisotropic diffusion technique for 2-D echocardiography analysis. Several examples are presented and discussed in Section IV. PROCESSING, AND 11. THE SYSTEMFOR ACQUISITION, VISUALIZATION A scheme of the system for the acquisition, processing, and visualization of echocardiographic image sequences is presented in Fig. 1. The system is based on a personal computer with two auxiliary boards: a frame-grabber board and a VTR interface. The frame-grabber board is one of the commercially available (Matrox PIP 1024, with a 512 X 512 pixels resolution, 256 gray levels, four-frame
00 18-9294/90/0800-0796$0 1.OO O 1990 IEEE
LAMBERT1 A N D SGALLARI 2-D ECHOCARDIOGRAPHY VISUALIZATION A N D IMAGE PROCESSING
797
VIDEO TAPE RECORDER
j I
j
PERSONAL COMPUTER
.
,
I
j. !
.j V I # I n I E I F n C f .......................
i
f l l l l f S I I I I L I 1111)
I
L.....
OPTICAL DISK
.
.
.
.........................
Fig. 1. General scheme of the system for acquisition, processing and visualization of 2-D echographic frames.
memory). The VTR interface board has been built by the authors to allow the operator some functions such as control of VTR by PC; automatic assignment of numerical codes to the video frames by utilizing one of the two audio channels; visualization of the recorded sequences with the codes superimposed; acquisition of one or more frames according to the input of the corresponding code by the operator; ECG signal analysis and QRS gated acquisition. A particular frame-grabber management software package written in C language has been developed and implemented. The main features of such a package are as follows: a) b) c) d)
user-friendly frame-grabber management; high capability of expansion; very easy configuration; modular function organization.
Feature a) was obtained by designing a mouse-based system, together with an overlapping-window presentation. Features b)-d) were obtained by total parametrization of the software code. 111. THE EDGEDETECTION MODEL Unfortunately for the clinician, the low-quality of echocardiograms makes detection of small or faint structures difficult. To promote easier interpretation of ventricular contour-detected data, we consider an interactive approach.
The complete model for eGge detection consists of a low-pass filter, a Laplace filter, and a zero-crossing detector performed in sequence. We have extended this model with an edge strength process and a threshold operator to obtain a binary image. A block diagram of the sequence of processing procedures is shown in Fig. 2. The edge detection model consists of five major modules associated with their parameters value. With all the possible choices of parameters, the operator can construct many edge detectors with different properties, characteristics, and performances, taking previous experiences or comparisons into account. In this section the possible choices for each module will be summarized with a brief description of the interaction and visualization process. For the detailed description of these modules together with their implementation we refer to [61, [81, [ill, [ W , 1151. In echocardiography we are confronted with low-quality images. Improving the image may thus be seen as a first step towards the edge detection phase. Moreover, the ill-posed nature of the problem suggests the use of lowpass filters in order to transform it into a well-posed problem [5], [ 151, [ 171. We allowed a flexible approach, where the operator is free to choose among three kinds of filters (moving average filter, finite prolate spheroidal window, and median filter), to compare the results obtained, and to decide on repetition of the smoothing step or continuation of the elaboration. Prolate functions were chosen because they constitute the optimal band-limited filter which maximizes the output energy in a given region [ 11, [6], [ 141, [ 181. The other smoothing filters considered (moving average and median
798
l F E t TRANSACTIONS ON BIOMEDICAL ENGINEERING. VOL. 37. NO 8. AUGUST 1990
Edgr. linage -.
I
Fig. 2 . The edge detection model.
filter) are able to suppress noise by applying some form is much greater than K , we have c ( x , y , t ) = 0 and diffusion is hampered. On the contrary, where V I is much of averaging to the input image [6]. We must be careful in using filters of this kind, how- smaller than K , then c(x, y, t ) = C and diffusion is high, ever, because averaging blurs the image and thus de- thus producing image smoothing. Parameter K therefore creases resolution [5], [ 171. Thus, visualization of the re- is like a threshold referred to VI: it separates noise from sults is fundamental in order to choose or change the edges. In our model there is a suggested value for K , but parameters. Default values for the parameters of these fil- a change is possible and advisable where the operator is ters (spatial and frequency domain sizes) are suggested by interested in the enhancement of a particular region or details. the system but can be varied by the operator. If the low-pass filtering is judged unsatisfactory, it may The new image Z(x, y ) is derived from the original imbe followed by the edge strength process. In it, the heat age Zo(x, y ) by applying the anisotropic diffusion techanisotropic diffusion technique, proposed by Perona and nique: Malik [13], is used to reduce the effect of noise superI ( x , y ) = Io(x, y ) AZ imposed on the image and to emphasize the edges of the objects [8]. This technique estimates the luminous inten- where A I results from discretization of the fundamental sity gradient locally and, referring to such a value, pro- equation: duces a smoothing only inside the semantically homogeAZ neous regions, while the contrast is stressed close to the - = div ( c ( x , y, t ) V I ) contours. Let us consider a parallel between temperature At changes in a bidimensional structure due to the heat aniFor appropriate discretization schemes and values for At sotropic diffusion during time and intensity changes we refer to [8], [13]. through an image due to low-pass filtering with different The anisotropic diffusion technique produces different scale-space parameter. The fundamental equation behomogeneous regions so that segmentation is now much comes easier. In fact, if the ventricular contour has to be deI: intensity tected, as in our case, a binary image is obtained by comar = div ( c ( x , y, t ) V I ) t : scale-space parameter paring the pixel value with a suitable threshold. Due to the poor quality of echocardiograms and the at c: conductivity possible low signal-to-noise ratio, the threshold level is where t is the scale-space parameter, and a l l a t the interactively varied by the operator by visual inspection. The next step consists of Laplace filtering, which is brightness intensity change due to the application of the thoroughly explained in the literature [5], 161, 1111, 1171. anisotropic diffusion process. The conduction coefficient In our model we consider the Marr-Hildreth operator aladopted is as follows: lowing the choice of the standard deviation of the Gaussian. The size of the kernel is determined by imposing that all the significant components are included. Many papers have pointed out the relationship between the size of the Gaussian and the spatial accuracy of the detected edge position obtained by the Marr-Hildreth opwhere C and K are constants. In the pixel in which the gradient of the brightness ( V I ) erator [ 111, [ 151, [ 171. The system suggests a large op-
+
LAMBERT1 A N D SGALLARI: 2-D ECHOCARDIOGRAPHY VISUALIZATION A N D IMAGE PROCESSING
799
erator (or Gaussian filter) to find the approximate position of strong edges, while a small operator is suggested to obtain high positional accuracy. Such differentiation module produces an image in which the zero crossing must be detected. In fact, it is the task of the zero-crossing detector to assign the changes in sign of a Laplace filtered image as the location of an edge. It is, however, unlikely that most of the zero crossing will coincide with the specified spatial coordinates called pixels, thus the nearest neighbor in the direction of the gradient should be assigned as the edge pixel to achieve accuracy. IV. VENTRICULAR CONTOURDETECTION In this section we present the results obtained by the operator interacting with the workstation on which the edge detection model presented above has been implemented. The examples presented here refer to detection of the contour of the left ventricular cavity, and in particular by analyzing a parasternal view and an apical view. Such views are, in fact, among those most commonly used by the cardiologist to evaluate the functionality of the left ventricle: on the basis of the analysis of the ventricular detected contours in a sequence of frames, quantitative evaluations of the characteristics of contraction and relaxation of the various regions of the ventricular wall can be made. The images presented constitute a portion of 256 x 256 pixels taken from the original echo images.
Low-Pass Filtering In Fig. 3(a) and (b) are shown the results obtained by filtering the original image (upper left part) with a moving-average filter 9 X 9 (upper right part), with a median filter 9 X 9 (lower right part), and with a spheroidal prolate filter 9 X 9 with a Fourier transform equal to zero out of the region of radius R = 0.6 (lower left part) [ l ] , [6]. The results obtained using such methods are satisfactory, showing little difference from one another. The median filter, however, seems to be the most suitable for eliminating the background noise typical of echocardiographic images and being the least time consuming. Laplacian Operator In Fig. 4(a) and (b) are reproduced the images prefiltered with a median filter 9 x 9 (upper left part), and the images obtained by means of a convolution of such image with the Laplacian of Gaussian operator corresponding to different value for standard deviation U ( U = 0.5 upper right, U = 0.8 lower right, U = 1.0 lower left). The values assigned to U are within the optimal range for edge detection in echo images; nevertheless the obtained images (Laplacian images) show that small differences occur. Edge Strength Detector and Thresholding In Fig. 5(a) and (b) the results obtained by using the option of “edge strength” are reported. Such option consists in processing the original image or the prefiltered one
(b) Fig. 3 . Low-pass filtering of a parasternal view (a) and of an apical view (b). Original image (upper left); moving average (upper right); median filter (lower right); spheroidal prolate (lower left).
(upper left part) by means of the anisotropic diffusion technique. In the resulting image (upper right part) a smoothing had been produced inside the homogeneous regions, while the contrast close to the contours had been stressed. Upon such an image an operation of thresholding can be made to produce a binary image (lower right part) in which the ventricular cavity appears very clearly. Such a binary image is then convolved by means of a Laplacian of Gaussian operator to obtain the Laplacian image (lower left part).
Zero-Crossing and Contours The points at which the Laplacian image (second derivative) cross the zero are considered to be points of con-
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING. VOL 37. NO
800
8 . AUGUST 1990
(b) Fig. 4. Laplacian of Gaussian operator applied to a parasternal view (a) and to an apical view (b). Prefiltered image by median filter (upper left): Laplacian of Gaussian with U = 0.5 (upper right), U = 0.8 (lower right), U = 1.0 (lower left).
tour. Fig. 6(a) and (b) show the points of zero-crossing superimposed on the original images. It is evident how the treatment of the image with the method of edge strength and the successive binarization make it possible to individuate with greater accuracy the points of the contour. V . CONCLUSION The system presented and described in this paper has shown itself to be particularly useful in the analysis of echocardiographic tiacings by ;he medical operator: Since
(b) Fig. 5 . Edge strength by anisotropic diffusion technique on a parasternal view (a) and on an apical view (b). Original image (upper left); result of anisotropic diffusion technique with K = 8, C = 5, Ar = 1 (upper right); binary image achieved by thresholding (lower right); Laplacian of Gaussian of the binary image with U = I .O.
it shows the intermediate results, and the interaction with the system is simple, it is possible to optimize the process of analysis and achieve an optimal result. The results obtained during the first phase of the system experimentation have made clear how a preliminary treatment of low-pass filtering is essential, and how interaction and visualization allow for a selection of the optimal values to assign to the parameters. Moreover, the procedure of edge strength (anisotropic diffusion technique and thresholding) has shown itself particularly suitable for the echo images, given the low quality of the original images.
-
LAMBERT1 A N D SGALLARI: 2-D ECHOCARDIOGRAPHY VISUALIZATION A N D IMAGE PROCESSING
80 1
whose geometry, dynamics, and noise distribution are known a priori. ACKNOWLEDGMENT The authors wish to thank G. Curti for his invaluable technical assistance, and Prof. Dr. G. Binetti of the Istituto di Malattie dell’ Apparato Cardiovascolare, Universita’ di Bologna, for his cooperation and valuable discussions.
REFERENCES
(b) Fig. 6. Edge detection on a parasternal view (a) and on an apical view (b). Zero-crossings of the Laplacian of Gaussian with U = 0 . 5 (upper right). U = 0.8 (lower right), U = 1 .O (lower left), and U = 1.O after the edge strength procedure (upper left).
It is furthermore maintained that the analysis of static images is penalizing in that it does not allow the temporal continuity of the scene and the action of the operator eye when observing the recording to be taken into consideration. Thus, the next step will be to include time as a third dimension of analysis. Moreover, in order to be able to validate all the procedures of analysis, we need a physical model (phantom) fully representative of the structures and of the dynamics of the human heart. The difficulty of creating such a phantom has suggested to investigate the feasibility of a numerical model which would permit the synthetization of images similar to those obtained experimentally, but
S . Alliney and F . Sgallari, “Two-dimensional almost bandlimited discrete sequences,” Pubblicazioni IAC, Ser. 111, no. 165, 1978. C. E. Angermann, R. J. Hart, C. H. Spes, W. Zwehl, M. Marquart, D. P. Schmid, and K. Theisen, “Computerized quantitative evaluation of the endocardium in serial two-dimensional echocardiograms of the left ventricular short axis,” in Proc. Comp. in Cardiol., Leuven, Belgium, 1987, New York: IEEE Comput. Soc., 1987, pp. 437440. J. G . Bosch, J. H . C. Reiber, G.van Burken, J. J. Gerbrands, W. J. Gussenhoven, N. Bom, and J. R. T . C. Roelandt, “Automated endocardial contour detection in short-axis 2-D echocardiograms; methodology and assessment of variability,” in Proc. Comp. in Cardiol., Washington, 1988, New York: IEEE Comput. Soc., 1988. C . H. Chu, E. J . Delp, and A. J . Buda, “Detecting left ventricular endocardial and epicardial boundaries by digital two-dimensional echocardiography,” IEEE Trans. Med. Imaging, vol. MI-7, pp. 8190, 1988. E. De Micheli, B. Caprile, P. Ottonello, and V. Torre, “Localization and noise in edge detection,” IEEE Trans. Pattern Anal. Mach. htell., vol. PAMI-11, pp. 1106-1117, 1989. C . Lamberti, A. Guidazzoli, and F . Sgallari, “Image regularization for echocardiography digital processing,” in Proc. Time-Varying Image Processing and Moving Object Recognition, Florence, Italy, V. Cappellini, Ed. Amsterdam: North-Holland, 1989, pp. 173-180. C . Lamberti, A. Martelli, and A. Guidazzoli, “Postprocessing techniques for 2-D echocardiographic imaging,” in Proc. IEEE EMBS 10th Ann. Intern. Con$, New Orleans, LA, G. H a m s and C. Walker, Eds. New York: IEEE Press, 1988, pp. 450-451. C. Lamberti, S. Lusvardi, and C. Truzzi, “Workstation for 2-D echocardiography image processing,” Computers in Cardiology, Jerusalem, Israel, New York: IEEE Comput. Soc., 1989. W . H. H. J . Lunscher and M. P. Beddoes, “Optimal edge detector design I: Parameter selection and noise effects,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-8, pp. 164-177, 1986. -, “Optimal edge detector design 11: Coefficient quantization,” IEEE Trans. Pattern Anal. Mach. Inrell., vol. PAMI-8, pp. 178-187, 1986. D. Marr and E. Hildreth, “Theory of edge detection,” Proc. R. Soc. Land. E., vol. 207, pp. 187-217, 1980. E. G. Melendo and E. J . Delp, “A technique for the visualization and analysis of cardiac wall motion by two-dimensional echocardiography,” IEEE Trans. Med. Imaging, vol. MI-8, pp. 104-106, 1989. P. Perona and J. Malik, “A network for multiscale image segmentation,” in Proc. 1988 IEEE Intern. S p p . on Circuits and Systems, Epsoo, Finland, New York: IEEE Press, 1988, pp. 2565-2568. D. Slepian, “Prolate spheroidal wavefunctions, Fourier analysis and uncertainty V: The discrete case,” Bell Syst. Tech. J . , vol. 57, pp. 1317-1430, 1978. G. Torre and T . A. Poggio, “On edge detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-8, pp. 147-163, 1986. L. Torres, E. Sangra’, A. Gasull, and S . Sallent, “A new algorithm for automatic border detection of two-dimensional echocardiographic images,” in Proc. Time-Varying Image Processing and Moving Object Recognition, Florence, Italy, V . Cappellini, Ed. Amsterdam: North-Holland, 1989, pp. 181-188. [I71 L. J . van Vliet, I . T. Young, and G. L. Beckers, “A nonlinear Laplace operator as edge detector in noisy images,” Comput. Graph. Image Process., vol. 45, pp. 167-195, 1989. [IS] R. Wilson, “Finite spheroidal sequences and their applications I: Generation and properties,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-9, pp, 787-795, 1987.
802
IEEE TRANSACTIONS ON BlOMEDlCAL ENGlNEERING. VOL. 37. NO. 8. AUGUST 1990
Claudio Lamberti (M’87) was born in Bologna, Italy, in 1948. He received the degree in mechanical engineering from the University of Bologna. Bologna, Italy, in 1974, and the postgraduate degree in biomedical technology from the University of Bologna School o f Medicine in 1978. He was a Research Fellow in bioengineering at Istituto di Automatica from 1976 to 1980. He joined the Division of Cardiology at the University of Bologna Hospital from 1980 to 1981. Since 1981 he has been working as a Research Associate at the Department of Electronics, Computer Science. and Systems of the University of Bologna. His principal activity is research in the field of cardiovascular signal processing, cardiovascular image processing and technology assessment.
Fiorella Sgallari (M’88) was born in Bologna, Italy, in 1953. She received the degree in mathematics from University of Bologna, Bologna, Italy, in 1976. She is currently an Associate Professor of Numerical Analysis at the University of BolognabHer main research is at present concerned with parallel computation and digital processing for medical imaging.
