SCANNING VOL. 37, 381–388 (2015) © Wiley Periodicals, Inc.
Local Dynamic Range Compensation for Scanning Electron Microscope Imaging System K.S. SIM AND Y.H. HUANG Faculty of Engineering and Technology, Multimedia University, Melaka, Malaysia
Summary: This is the extended project by introducing the modified dynamic range histogram modification (MDRHM) and is presented in this paper. This technique is used to enhance the scanning electron microscope (SEM) imaging system. By comparing with the conventional histogram modification compensators, this technique utilizes histogram profiling by extending the dynamic range of each tile of an image to the limit of 0–255 range while retains its histogram shape. The proposed technique yields better image compensation compared to conventional methods. SCANNING 37:381–388, 2015. © 2015 Wiley Periodicals, Inc. Key words: SEM images, contrast enhancement, local dynamic range enhancement
2005, 2007) but the conventional ABC methods still restrict by its constraints such that the region of interest (ROI) does not occupy a significant portion of field-ofvision (FOV). In order to have a greater achievement of the image enhancement level, modified dynamic range histogram modification (MDRHM) is proposed to enhance images in certain conditions. It is a continuation from the dynamic range histogram modification technique. It is known that the SEM image is a two-dimension intensity map either in analog or digital domain. Each image pixel is corresponding to a point on the sample. Due to the limitations, manipulation of the image pixel is necessary through several enhanced techniques to extract more useful information from SEM images. Specifically, three aims are addressed as describe below:
Introduction Contrast is a critical issue of the image; hence, image enhancement techniques are introduced to overcome the poor contrast image. For instance, images formed by scanning electron microscope (SEM) may suffer from contrast issue and this will prevent user from analyzing and extracting the desire information from the images. Hence, researchers have been investigated in a number of ways by introducing several image enhancement techniques to resolve the problems from time to time. By tracing back to few decades, several approaches regarding to Auto Brightness Contrast (ABC) techniques had been proposed to enhance the image contrast. Though they achieved some degree of success (Tolat et al., 1991; De Medeiros Martins et al., 2002; Sim et al., Conflict of interest: none. Address for reprints: Ir. Prof. Dr. Sim Kok Swee, Faculty of Engineering and Technology, Multimedia University, 75450, Melaka, Malaysia. E-mail: [email protected]; [email protected] Received 24 February 2015; revised 20 April 2015; Accepted with revision 29 April 2015 DOI: 10.1002/sca.21226 Published online 13 May 2015 in Wiley Online Library (wileyonlinelibrary.com).
a. First, a few set of SEM images are tested to verify the feasibility of MDRHM. b. Second, the findings are compared between MDRHM and Histogram Modification (HM) with three types of distribution such as uniform, Rayleigh and exponential. c. Third, justification of the proposed technique in various types of image. MDRHM is tested with a series of SEM images that comprised of different types of image including images with less objects, more objects, low magnification, high magnification, high, and low dynamic range. With the aid of quality assessment tools, it can be concluded that MDRHM outperforms the selected three histogram modification techniques and HE under the condition of more objects and narrow dynamic range.
Research and Methodology Image enhancement is a process of modifying the pixel intensity of an image and there are various techniques proposed to solve the brightness deterioration (Arjun et al., 2013). In other words, if the contrast
382
SCANNING VOL. 37, 6 (2015)
of an image is highly concentrated on a specific range, the information will be lost in some areas. For instance, it is difficult to extract the useful information for a sample image X with a dynamic range from 59 to 158 because it does not occupy the full range of gray level (0–255) so some objects may not appear in the image. Histogram equalization is one of the well-known techniques used to enhance the poor contrast images. It distributes pixel values uniformly and stretches the contrast of the high histogram region, meanwhile; compresses the contrast with low histogram region (Cheng and Ramli, 2003). In spite of its good performance in improving contrast of images, however, it changes the brightness of the images and results in undesirable artifacts to the images. In addition, theoretically after histogram equalization is applied to an image, its mean value is changed regardless of the input mean (Manpreet et al., 2011). This is undesirable because brightness preservation is required in some applications. Thereafter, Bi-histogram equalization (BBHE) technique is proposed by Kim (1997) to overcome the brightness issue. BBHE divides an input image into two sub-images regarding the mean of input image (Raju et al., 2013). By taking xmin as the sub-image with values less than or equal to the mean and xmax sub-image with values of greater or equal than mean; it is then applied HE independently to the sub-images by mapping into the range from minimum to input mean and to maximum gray level. Therefore, the mean brightness is preserved. Unfortunately, the assumption of its symmetric histogram is seldom applied and required high computation time (Yoon et al., 2009). Furthermore, Wongsritong et al. introduced multipeak HE in year 1998. Multi-peak HE equalized the input histogram based on detected local maxima of the histogram. It can eliminate the effect of brightness saturation so it is more acceptable compare to BBHE in brightness preservation.
proposed by Pizer et al. such that histograms are generated only at a rectangular grid of points. Interpolating mappings of the four nearest grid points are used to map each pixel of an image. The window size is adjustable that can be either smaller or bigger based on the application which it is to be used for that specific occasion. The process is repeated at all grid points and bilinear interpolation is used to combine the overlapped mapping in order to minimize the boundary effect. Figure 1 shows the interpolating mappings with its window size. By using the defined window, the contrast of each tile is enhanced by matching with the histogram shape specified by uniform, Rayleigh and exponential distribution to form a desired output image. These distributions are selected depend on the nature of images. Uniform distribution tends to flatten the image histogram whereas exponential tends to distribute gray level of higher frequency in the higher gray level (Azeddine and Alain, 1988). Lastly, Rayleigh distribution is based on the bell-shaped which focuses most of the pixels at the middle of the intensity level (Abdul Ghani and Mat Isa, 2014). Therefore, the intensity level of both end sides has the minimum number of pixels. With this property, it is suitable for underwater images. Figure 2 shows the histogram shape regarding to uniform, Rayleigh and exponential distributions.
Drawbacks of HM Implementation
Histogram Modification
As mentioned before, during the processes of combining the gray level mappings; boundary effect is minimized by bilinear interpolation. However, in some applications the windows size must be adjusted accordingly to enhance the image but there will be an increase of noise in the image. In addition, the adaptive or local property has a tendency to increase the contrast excessively and amplify noise within the interest region (tiles). Figure 3 illustrates the effect of different window size.It indicates that the window size of 3232 pixels introduces higher level of noise as compare to window size of 44 pixels.
Since the aforementioned techniques are not efficient to process all types of image, therefore, histogram modification (HM) is proposed to improve the enhancement efficiency. HM is a technique that rescales the histogram of the original image to form the output image that follows some desired form and it can be grouped into non-adaptive and adaptive histogram modification. Non-adaptive histogram modification involve in mapping function to each pixel in an image such that mapping function is formed regarding to the histogram of the image. This process can be made with regard to a defined space or moving window onto each of the image pixels. Adaptive histogram modification is then
Fig. 1.
Interpolating mappings.
Sim and Huang: MDRHM Compensator for SEM Imaging System
Fig. 2.
Histogram shape of (a) Uniform, (b) Rayleigh, (c) Exponential.
Modified Dynamic Range Histogram Modification (MDRHM)
Histogram hyperbolization is one of the histogram modification proposed by Frei in year 1977. It is used to emphasize the image details based on transformations made by human peripheral visual system. Brightness and contrast are the significant variables to be manipulated in SEM imaging system because the resulting images produced by SEM are not stable. Therefore, stable output with the aid of enhancement technique is necessary to handle the fluctuation problems in SEM images. Dynamic Range Histogram Modification (DRHM) is first proposed to stretch the input dynamic range without changing its histogram to improve the brightness and contrast (Sim et al., 2005).
Fig. 3.
383
Modified dynamic range histogram modification is proposed to further increase the performance of the technique; to enhance the contrast of images locally. In calculus, monotonic transformation defined as a set of numbers to transform into another set that preserves the order. HM uses this concept where Dj ¼ T {Ci} (transformation function) by transforming the input dynamic range, C1 Ci CM into output dynamic range, D1 Di DN. The input and output probability distribution must comply with the property of unity as shown in the equations (1) and (2):
(a) Window size: 44 pixels (b) Window size: 3232 pixels.
M X PR {C i ¼ f i } ¼ 1 i¼1
ð1Þ
SCANNING VOL. 37, 6 (2015)
384 N X PR {Di ¼ gi } ¼ 1
ð2Þ
i¼1
where fi and gj are reconstruction values of ith and jth levels. Mathematically, the algorithm is expressed as
Dði; jÞ ¼ ½ðdÞPC ðCÞ ðCði; jÞ C 0 Þ þ Dof f ;
ð3Þ
1 where PC ðCÞ ¼ C1 C , d is the new dynamic range 0 with preset range from 0 to 255, C(i,j) is the input image, C1 is the maximum intensity value of input image, C0 is the minimum intensity value of input image, Doff is the offset values of reconstructed dynamic range and D(i,j) is the output image with reconstructed dynamic range. Figure 4 shows the MDRHM transformation function by mapping input dynamic range into output dynamic range with predefined range of 0–255 where input range is configurable either manually or automatically. Equation (4) shows the contrast enhancement ratio where E represents the slope of the transformation function between input and output:
E¼
Fig. 4.
D1 D0 C1 C0
ð4Þ
Therefore, the enhancement ratio can be adjusted regarding to the input pixel’s intensity to prevent image over-saturation. The mean of output image can be retained since the input histogram is preserved and its output range is fixed at a dynamic range from D0 to D1. Assume a sample image is divided into four tiles as shown in Figure 4; each tile is enhanced by mapping the local input range into local output range. The input range of each tile may vary from each other. There are two main steps in the proposed method: (a) to equalize the tiles independently; and (b) to reduce the intensity discontinuity along the tile’s boundary with the use of smoothing filter. To define a local function of each pixel (let (x0, y0) to be the initial pixel) it is based on the neighboring pixels within mn matrix centered at this pixel; I0(x0, y0) ¼ f(I(x, y)) (Cheng and Shi, 2004). Equation (3) is then mapped into this local function to enhance the contrast level of each tile since each tile may have different contrast level. Therefore, the enhancement performance may be varied from each other. By enhancing the sample images locally will extract the details at each tile. Even though it is effective in local enhancement, however, it may distort input images because the transformation is only monotonic in local enhancement (independent tile) but not monotonic as a global image when combining each tiles to form a
Modified Dynamic Range Histogram Modification (MDRHM).
Sim and Huang: MDRHM Compensator for SEM Imaging System
385
filter is defined as in equation (5) and (6):
Pc1;r1 r¼
c¼0;r¼0
Zðw; xÞ ¼
c 2
Z n ðw; xÞ þ Z m ðw; xÞ 2
ð5Þ
1 þ w 2r 1 þ x ; R R
ð6Þ
where w and x is the variables of pixel’s intensity, R is the average of two levels of pixel’s intensity, m is the previous intensity value, and n is the current intensity value. Fig. 5.
Boundary effects.
complete output image where the order of gray levels may change noticeably at the image boundary. Hence, MDRHM will introduce boundary artifacts in some sample images which has a noticeable drop in intensity level from each tile along the boundary area. Boundary artifacts are introduced in horizontal and vertical directions (Kawaldeep and Parveen, 2012) which are illustrated in Figure 5. By making assumptions, the pixel intensity across the boundary is very large as comparing to the pixel intensity on the left or right which is away from boundary area. Smoothing filter is applied to the boundary area to normalize the pixels lied on the boundary area (Zhou and Wu, 2010). The smoothing
Fig. 6.
Work flow of the algorithms.
General Work Flow
Figure 6 shows the work flow of the algorithms (George, 2013).
Results and Discussions The first test trial is by comparing conventional HE with DRHM and MDRHM. This test is carried out to show that the conventional HE is suffered from brightness issue. For SEM sample image 1, it is very obvious that the proposed MDRHM transformation is able to stabilize the brightness fluctuation of the sequential input images and enhance input image locally as shown in Figure 7.
386
SCANNING VOL. 37, 6 (2015)
Fig. 7. (a) SEM sample image 1 with image resolution of 256256 pixels. (c) Output image after HE transformation (e) output image after DRHM (g) output image of MDRHMwith its corresponding histogram at (b), (d), (f), and (h), respectively.
Image Quality Assessment
Trials are performed on different types of images including low contrast, high contrast, more objects, less objects, narrow, and wide dynamic range images regarding to the three histogram modifications and proposed method. Hence, a set of image quality assessment tools are used to test the performance of each method. By taking a set of sample images, the observations of these images show that MDRHM method outperforms the three histogram modification.
Fig. 8. (a) SEM sample image 2 with image resolution of 256256 pixels (c) Output image after Uniform transformation (e) output image after Rayleigh transformation (g) output image after Exponential transformation (i) output image after MDRHM with its corresponding histograms at (b), (d), (f), (i), and (j).
Sim and Huang: MDRHM Compensator for SEM Imaging System TABLE I Comparison of PSNR, SSIM, Entropy and EMEE with different histogram modification transfer functions for SEM sample image 2
PSNR SSIM Entropy EMEE
Uniform
Rayleigh
Exponential
MDRHM
10.01 0.59 1.17 3.05
9.58 0.63 1.13 0.37
9.20 0.55 1.17 3.45
13.73 0.77 1.00 2.74
However, it is necessary to compare the results of quality assessment tools with subjective assessment such as human visual perception. Therefore, four quality assessment tools will be used to evaluate the performance of the three histogram modification techniques and namely MDRHM including Peak-to-noise ratio (PSNR), entropy, Measure of Enhancement by Entropy (EMEE), and Structural Similarity Index (SSIM). Mean Square Error (MSE) is defined as in equation (7):
MSE ¼
x1;y1 1 X ðI 1 ðx; yÞ I 2 ðx; yÞÞ; x; y x;y¼0
ð7Þ
where x and y are the size of an image, I1(x,y) and I (x,y) are two images to be compared PSNR is defined as in equation (8): Maximum intensity2 PSNR ¼ 10log10 ð8Þ MSE 2
It is a ratio between power signal and power noise in which the higher the value of PSNR the better the quality of an image and therefore, the MSE value is lower. Entropy is defined as in equation (9): X Entropy ¼ I½x; ylog2 I½x; y ð9Þ x;y
Under this section, the results of comparison between four methods are explained with the aid of the quality assessment tools. It is tested with different types of SEM images. Four histogram modification techniques are TABLE II
387
used in this enhancement comparison which are uniform, Rayleigh, exponential, and MDRHM. Figure 8 shows the sample image with output images after the implementation of the four selected techniques with their corresponding histograms. Table I shows the comparison of the four selected techniques with their PSNR, entropy, EMEE, and SSIM values. Table I shows that MDRHM obtains the highest PSNR value of 13.73 follows by Uniform, Rayleigh, and Exponential for SEM sample image 2 as shown in Figure 8. Besides, MDRHM also achieves the highest SSIM value of 0.77 follows by Rayleigh, Uniform, and Exponential. Assume that the entropy with value 1 is acted as the reference point; MDRHM has a value of 1.00 which is the nearest value to the reference point. Therefore, the randomness percentage is the least for MDRHM. It is then followed by Rayleigh, Uniform, and Exponential with values of 1.13, 1.17, and 1.17, respectively. As for EMEE, Exponential obtains the highest value of EMEE with a value of 3.45 follow by Uniform, MDRHM, and Rayleigh. When PSNR value is higher, it indicates that the output image has significant improvement in quality but sometime PSNR may vary significantly even though the input and output images are almost indistinguishable. Quality assessment tools are used to measure the quality and distortion level of images but it may be different when it is assessed by human visual response. Therefore, the quality assessment measures may not be perfected yet for human perception but it acts as a guideline for judgment. From the selected sample images, it can be concluded that MDRHM performs better in images with more objects and narrow dynamic range. In order to confirm that MDRHM performs better in images with more objects and narrow dynamic range, 100 trials had been taken. The average data in two conditions are shown in Table II. The results shown in Table II indicate that there is advantage for MDRHM to perform well in these two conditions. MDRHM obtains the highest values of PSNR and SSIM in more objects type of SEM images whereas the MDRHM only obtains the highest value of SSIM in narrow dynamic range but the PSNR values
Statistical study of 100 sample images with two conditions applied Techniques Uniform
Conditions More objects Narrow dynamic range Standard deviation More objects Narrow dynamic range
Rayleigh
Exponential
MDRHM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
14 12.5
0.76 0.64
15.41 13.21
0.84 0.73
13.72 11.73
0.75 0.62
23 13.18
0.84 0.75
4.7121 2.0209
0.2808 0.0455
7.9310 3.6748
0.2783 0.0748
5.9774 2.2191
0.3214 0.0668
21.4070 2.2292
0.1049 0.0990
388
SCANNING VOL. 37, 6 (2015)
between Rayleigh and MDRHM is only 0.03 which shows that MDRHM is still able to maintain its quality and capability in image enhancement.
Conclusions In this paper, some of the common image enhancement techniques are reviewed and explained. These techniques are useful indeed but the results may not be desirable from one application to another. Conventional histogram equalization is widely used for contrast enhancement because of its effectiveness and simplicity of algorithm. However, it would introduce artifacts in the region where it has more or less frequency of gray levels. So, the proposed MDRHM is able to overcome these issues by stretching the dynamic range in each region into its greatest gray levels. Since the pixel intensities in each region may be varied from each other so the contrast enhancement performance may be different. The proposed MDRHM is tested with a series of SEM images with variety types of images as mentioned in discussion section. In the quantitative measure of contrast enhancement, MDRHM outperforms the selected three histogram modification techniques and HE in two categories of image type.
References Ahmad SAG, Nor Ashidi MI. 2014. Underwater image quality enhancement through composition of dual-intensity images and Rayleigh-stretching. Springer Open Journal, 3: 1–14. Arjun N, Pradnyawant K, Amit L, Vrushali P, Bhagyashri S. 2013. A novel approach to medical & gray scale image enhancement. International Journal of Engineering Research and Applications 3:653–657. Azeddine B, Alain LN. 1988. Contrast enhancement technique based on local detection of edges. Computer Vision, Graphics, and Image Processing 46:162–174.
Cheng HD, Shi XJ. 2004. A simple and effective histogram equalization approach to image enhancement. Digital Signal Proccessing 14:158–170. Cheng SD, Ramli R. 2003. Contrast enhancement using recursive mean-separate histogram equalization for scalable brightness preservation. Institute of Electrical and Electronics Engineers Transactions on Consumer Electronics 49:1301–1309. De Medeiros Martins A, Torres de Almeida Filho W, Medeiros Brito Junior A, Duarte Doria Neto A. 2002. A new method for multi-texture segmentation using neural networks. Proc of the 2002 International Joint Conference 3, 12–17:2064–2069. George S. (2013) VIsual media processing using Matlab. Birmingham - Mumbai: PACKT. Kawaldeep SR, Parveen K. 2012. A novel approach for blocking artifact reduction in JPEG compressed images. International Journal of Emerging Technology and Advanced Engineering 2:150–157. Manpreet K, Jasdeep K, Jappreet K. 2011. Survey of contrast enhancement techniques based on histogram equalization. International Journal of Advanced Computer Science and Applications 2:138–141. Raju A, Dwarakish GS, Venkat R. 2013. A comparative analysis of histogram equalization based techniques for contrast enhancement and brightness preserving. International Journal of Signal Processing, Image Processing and Pattern Recognition 4:353–366. Sim KS, Kamel NS, Chuah HT. 2005. A real-time image dynamic range compensation for scanning electron microscope imaging system. Scanning 27:199–207. Sim KS, Tso CP, Tan YY, Law KK, Teoh ABJ. 2007. Realtime image dynamic range compensation for an optical imaging system. Journal of Electronic Imaging 16: 033006–033006-17. Tolat AR, McNeill SR, Sutton MA. 1991. Effects of contrast and brightness on subpixel image correlation. System Theory Proceedings, 23rd Southeastern Symposium:604–608. Wongsritong K, Kittayaruasiriwat K, Cheevasuvit F, Dejhan K, Somboonkaew A. 1998. A contrast enhancement using multipeak histogram equalization with brightness preserving, circuit and systems. IEEE APCCAS Asia-Pacific Conference:455–458. Yoon HS, Han YJ, Hahn HS. 2009. Image contrast enhancement based sub-histogram equalization technique without overequalization noise World Academy of Science. Engineering and Technology 3:145–151. Zhou HY, Wu JH. (2010) Digital image processing: Part 1. © 2010 Huiyu Zhou, Jiahua Wu, Jianguo Zhang & bookboon. com, London.
Local Dynamic Range Compensation for Scanning Electron Microscope Imaging System K.S. SIM AND Y.H. HUANG Faculty of Engineering and Technology, Multimedia University, Melaka, Malaysia
Summary: This is the extended project by introducing the modified dynamic range histogram modification (MDRHM) and is presented in this paper. This technique is used to enhance the scanning electron microscope (SEM) imaging system. By comparing with the conventional histogram modification compensators, this technique utilizes histogram profiling by extending the dynamic range of each tile of an image to the limit of 0–255 range while retains its histogram shape. The proposed technique yields better image compensation compared to conventional methods. SCANNING 37:381–388, 2015. © 2015 Wiley Periodicals, Inc. Key words: SEM images, contrast enhancement, local dynamic range enhancement
2005, 2007) but the conventional ABC methods still restrict by its constraints such that the region of interest (ROI) does not occupy a significant portion of field-ofvision (FOV). In order to have a greater achievement of the image enhancement level, modified dynamic range histogram modification (MDRHM) is proposed to enhance images in certain conditions. It is a continuation from the dynamic range histogram modification technique. It is known that the SEM image is a two-dimension intensity map either in analog or digital domain. Each image pixel is corresponding to a point on the sample. Due to the limitations, manipulation of the image pixel is necessary through several enhanced techniques to extract more useful information from SEM images. Specifically, three aims are addressed as describe below:
Introduction Contrast is a critical issue of the image; hence, image enhancement techniques are introduced to overcome the poor contrast image. For instance, images formed by scanning electron microscope (SEM) may suffer from contrast issue and this will prevent user from analyzing and extracting the desire information from the images. Hence, researchers have been investigated in a number of ways by introducing several image enhancement techniques to resolve the problems from time to time. By tracing back to few decades, several approaches regarding to Auto Brightness Contrast (ABC) techniques had been proposed to enhance the image contrast. Though they achieved some degree of success (Tolat et al., 1991; De Medeiros Martins et al., 2002; Sim et al., Conflict of interest: none. Address for reprints: Ir. Prof. Dr. Sim Kok Swee, Faculty of Engineering and Technology, Multimedia University, 75450, Melaka, Malaysia. E-mail: [email protected]; [email protected] Received 24 February 2015; revised 20 April 2015; Accepted with revision 29 April 2015 DOI: 10.1002/sca.21226 Published online 13 May 2015 in Wiley Online Library (wileyonlinelibrary.com).
a. First, a few set of SEM images are tested to verify the feasibility of MDRHM. b. Second, the findings are compared between MDRHM and Histogram Modification (HM) with three types of distribution such as uniform, Rayleigh and exponential. c. Third, justification of the proposed technique in various types of image. MDRHM is tested with a series of SEM images that comprised of different types of image including images with less objects, more objects, low magnification, high magnification, high, and low dynamic range. With the aid of quality assessment tools, it can be concluded that MDRHM outperforms the selected three histogram modification techniques and HE under the condition of more objects and narrow dynamic range.
Research and Methodology Image enhancement is a process of modifying the pixel intensity of an image and there are various techniques proposed to solve the brightness deterioration (Arjun et al., 2013). In other words, if the contrast
382
SCANNING VOL. 37, 6 (2015)
of an image is highly concentrated on a specific range, the information will be lost in some areas. For instance, it is difficult to extract the useful information for a sample image X with a dynamic range from 59 to 158 because it does not occupy the full range of gray level (0–255) so some objects may not appear in the image. Histogram equalization is one of the well-known techniques used to enhance the poor contrast images. It distributes pixel values uniformly and stretches the contrast of the high histogram region, meanwhile; compresses the contrast with low histogram region (Cheng and Ramli, 2003). In spite of its good performance in improving contrast of images, however, it changes the brightness of the images and results in undesirable artifacts to the images. In addition, theoretically after histogram equalization is applied to an image, its mean value is changed regardless of the input mean (Manpreet et al., 2011). This is undesirable because brightness preservation is required in some applications. Thereafter, Bi-histogram equalization (BBHE) technique is proposed by Kim (1997) to overcome the brightness issue. BBHE divides an input image into two sub-images regarding the mean of input image (Raju et al., 2013). By taking xmin as the sub-image with values less than or equal to the mean and xmax sub-image with values of greater or equal than mean; it is then applied HE independently to the sub-images by mapping into the range from minimum to input mean and to maximum gray level. Therefore, the mean brightness is preserved. Unfortunately, the assumption of its symmetric histogram is seldom applied and required high computation time (Yoon et al., 2009). Furthermore, Wongsritong et al. introduced multipeak HE in year 1998. Multi-peak HE equalized the input histogram based on detected local maxima of the histogram. It can eliminate the effect of brightness saturation so it is more acceptable compare to BBHE in brightness preservation.
proposed by Pizer et al. such that histograms are generated only at a rectangular grid of points. Interpolating mappings of the four nearest grid points are used to map each pixel of an image. The window size is adjustable that can be either smaller or bigger based on the application which it is to be used for that specific occasion. The process is repeated at all grid points and bilinear interpolation is used to combine the overlapped mapping in order to minimize the boundary effect. Figure 1 shows the interpolating mappings with its window size. By using the defined window, the contrast of each tile is enhanced by matching with the histogram shape specified by uniform, Rayleigh and exponential distribution to form a desired output image. These distributions are selected depend on the nature of images. Uniform distribution tends to flatten the image histogram whereas exponential tends to distribute gray level of higher frequency in the higher gray level (Azeddine and Alain, 1988). Lastly, Rayleigh distribution is based on the bell-shaped which focuses most of the pixels at the middle of the intensity level (Abdul Ghani and Mat Isa, 2014). Therefore, the intensity level of both end sides has the minimum number of pixels. With this property, it is suitable for underwater images. Figure 2 shows the histogram shape regarding to uniform, Rayleigh and exponential distributions.
Drawbacks of HM Implementation
Histogram Modification
As mentioned before, during the processes of combining the gray level mappings; boundary effect is minimized by bilinear interpolation. However, in some applications the windows size must be adjusted accordingly to enhance the image but there will be an increase of noise in the image. In addition, the adaptive or local property has a tendency to increase the contrast excessively and amplify noise within the interest region (tiles). Figure 3 illustrates the effect of different window size.It indicates that the window size of 3232 pixels introduces higher level of noise as compare to window size of 44 pixels.
Since the aforementioned techniques are not efficient to process all types of image, therefore, histogram modification (HM) is proposed to improve the enhancement efficiency. HM is a technique that rescales the histogram of the original image to form the output image that follows some desired form and it can be grouped into non-adaptive and adaptive histogram modification. Non-adaptive histogram modification involve in mapping function to each pixel in an image such that mapping function is formed regarding to the histogram of the image. This process can be made with regard to a defined space or moving window onto each of the image pixels. Adaptive histogram modification is then
Fig. 1.
Interpolating mappings.
Sim and Huang: MDRHM Compensator for SEM Imaging System
Fig. 2.
Histogram shape of (a) Uniform, (b) Rayleigh, (c) Exponential.
Modified Dynamic Range Histogram Modification (MDRHM)
Histogram hyperbolization is one of the histogram modification proposed by Frei in year 1977. It is used to emphasize the image details based on transformations made by human peripheral visual system. Brightness and contrast are the significant variables to be manipulated in SEM imaging system because the resulting images produced by SEM are not stable. Therefore, stable output with the aid of enhancement technique is necessary to handle the fluctuation problems in SEM images. Dynamic Range Histogram Modification (DRHM) is first proposed to stretch the input dynamic range without changing its histogram to improve the brightness and contrast (Sim et al., 2005).
Fig. 3.
383
Modified dynamic range histogram modification is proposed to further increase the performance of the technique; to enhance the contrast of images locally. In calculus, monotonic transformation defined as a set of numbers to transform into another set that preserves the order. HM uses this concept where Dj ¼ T {Ci} (transformation function) by transforming the input dynamic range, C1 Ci CM into output dynamic range, D1 Di DN. The input and output probability distribution must comply with the property of unity as shown in the equations (1) and (2):
(a) Window size: 44 pixels (b) Window size: 3232 pixels.
M X PR {C i ¼ f i } ¼ 1 i¼1
ð1Þ
SCANNING VOL. 37, 6 (2015)
384 N X PR {Di ¼ gi } ¼ 1
ð2Þ
i¼1
where fi and gj are reconstruction values of ith and jth levels. Mathematically, the algorithm is expressed as
Dði; jÞ ¼ ½ðdÞPC ðCÞ ðCði; jÞ C 0 Þ þ Dof f ;
ð3Þ
1 where PC ðCÞ ¼ C1 C , d is the new dynamic range 0 with preset range from 0 to 255, C(i,j) is the input image, C1 is the maximum intensity value of input image, C0 is the minimum intensity value of input image, Doff is the offset values of reconstructed dynamic range and D(i,j) is the output image with reconstructed dynamic range. Figure 4 shows the MDRHM transformation function by mapping input dynamic range into output dynamic range with predefined range of 0–255 where input range is configurable either manually or automatically. Equation (4) shows the contrast enhancement ratio where E represents the slope of the transformation function between input and output:
E¼
Fig. 4.
D1 D0 C1 C0
ð4Þ
Therefore, the enhancement ratio can be adjusted regarding to the input pixel’s intensity to prevent image over-saturation. The mean of output image can be retained since the input histogram is preserved and its output range is fixed at a dynamic range from D0 to D1. Assume a sample image is divided into four tiles as shown in Figure 4; each tile is enhanced by mapping the local input range into local output range. The input range of each tile may vary from each other. There are two main steps in the proposed method: (a) to equalize the tiles independently; and (b) to reduce the intensity discontinuity along the tile’s boundary with the use of smoothing filter. To define a local function of each pixel (let (x0, y0) to be the initial pixel) it is based on the neighboring pixels within mn matrix centered at this pixel; I0(x0, y0) ¼ f(I(x, y)) (Cheng and Shi, 2004). Equation (3) is then mapped into this local function to enhance the contrast level of each tile since each tile may have different contrast level. Therefore, the enhancement performance may be varied from each other. By enhancing the sample images locally will extract the details at each tile. Even though it is effective in local enhancement, however, it may distort input images because the transformation is only monotonic in local enhancement (independent tile) but not monotonic as a global image when combining each tiles to form a
Modified Dynamic Range Histogram Modification (MDRHM).
Sim and Huang: MDRHM Compensator for SEM Imaging System
385
filter is defined as in equation (5) and (6):
Pc1;r1 r¼
c¼0;r¼0
Zðw; xÞ ¼
c 2
Z n ðw; xÞ þ Z m ðw; xÞ 2
ð5Þ
1 þ w 2r 1 þ x ; R R
ð6Þ
where w and x is the variables of pixel’s intensity, R is the average of two levels of pixel’s intensity, m is the previous intensity value, and n is the current intensity value. Fig. 5.
Boundary effects.
complete output image where the order of gray levels may change noticeably at the image boundary. Hence, MDRHM will introduce boundary artifacts in some sample images which has a noticeable drop in intensity level from each tile along the boundary area. Boundary artifacts are introduced in horizontal and vertical directions (Kawaldeep and Parveen, 2012) which are illustrated in Figure 5. By making assumptions, the pixel intensity across the boundary is very large as comparing to the pixel intensity on the left or right which is away from boundary area. Smoothing filter is applied to the boundary area to normalize the pixels lied on the boundary area (Zhou and Wu, 2010). The smoothing
Fig. 6.
Work flow of the algorithms.
General Work Flow
Figure 6 shows the work flow of the algorithms (George, 2013).
Results and Discussions The first test trial is by comparing conventional HE with DRHM and MDRHM. This test is carried out to show that the conventional HE is suffered from brightness issue. For SEM sample image 1, it is very obvious that the proposed MDRHM transformation is able to stabilize the brightness fluctuation of the sequential input images and enhance input image locally as shown in Figure 7.
386
SCANNING VOL. 37, 6 (2015)
Fig. 7. (a) SEM sample image 1 with image resolution of 256256 pixels. (c) Output image after HE transformation (e) output image after DRHM (g) output image of MDRHMwith its corresponding histogram at (b), (d), (f), and (h), respectively.
Image Quality Assessment
Trials are performed on different types of images including low contrast, high contrast, more objects, less objects, narrow, and wide dynamic range images regarding to the three histogram modifications and proposed method. Hence, a set of image quality assessment tools are used to test the performance of each method. By taking a set of sample images, the observations of these images show that MDRHM method outperforms the three histogram modification.
Fig. 8. (a) SEM sample image 2 with image resolution of 256256 pixels (c) Output image after Uniform transformation (e) output image after Rayleigh transformation (g) output image after Exponential transformation (i) output image after MDRHM with its corresponding histograms at (b), (d), (f), (i), and (j).
Sim and Huang: MDRHM Compensator for SEM Imaging System TABLE I Comparison of PSNR, SSIM, Entropy and EMEE with different histogram modification transfer functions for SEM sample image 2
PSNR SSIM Entropy EMEE
Uniform
Rayleigh
Exponential
MDRHM
10.01 0.59 1.17 3.05
9.58 0.63 1.13 0.37
9.20 0.55 1.17 3.45
13.73 0.77 1.00 2.74
However, it is necessary to compare the results of quality assessment tools with subjective assessment such as human visual perception. Therefore, four quality assessment tools will be used to evaluate the performance of the three histogram modification techniques and namely MDRHM including Peak-to-noise ratio (PSNR), entropy, Measure of Enhancement by Entropy (EMEE), and Structural Similarity Index (SSIM). Mean Square Error (MSE) is defined as in equation (7):
MSE ¼
x1;y1 1 X ðI 1 ðx; yÞ I 2 ðx; yÞÞ; x; y x;y¼0
ð7Þ
where x and y are the size of an image, I1(x,y) and I (x,y) are two images to be compared PSNR is defined as in equation (8): Maximum intensity2 PSNR ¼ 10log10 ð8Þ MSE 2
It is a ratio between power signal and power noise in which the higher the value of PSNR the better the quality of an image and therefore, the MSE value is lower. Entropy is defined as in equation (9): X Entropy ¼ I½x; ylog2 I½x; y ð9Þ x;y
Under this section, the results of comparison between four methods are explained with the aid of the quality assessment tools. It is tested with different types of SEM images. Four histogram modification techniques are TABLE II
387
used in this enhancement comparison which are uniform, Rayleigh, exponential, and MDRHM. Figure 8 shows the sample image with output images after the implementation of the four selected techniques with their corresponding histograms. Table I shows the comparison of the four selected techniques with their PSNR, entropy, EMEE, and SSIM values. Table I shows that MDRHM obtains the highest PSNR value of 13.73 follows by Uniform, Rayleigh, and Exponential for SEM sample image 2 as shown in Figure 8. Besides, MDRHM also achieves the highest SSIM value of 0.77 follows by Rayleigh, Uniform, and Exponential. Assume that the entropy with value 1 is acted as the reference point; MDRHM has a value of 1.00 which is the nearest value to the reference point. Therefore, the randomness percentage is the least for MDRHM. It is then followed by Rayleigh, Uniform, and Exponential with values of 1.13, 1.17, and 1.17, respectively. As for EMEE, Exponential obtains the highest value of EMEE with a value of 3.45 follow by Uniform, MDRHM, and Rayleigh. When PSNR value is higher, it indicates that the output image has significant improvement in quality but sometime PSNR may vary significantly even though the input and output images are almost indistinguishable. Quality assessment tools are used to measure the quality and distortion level of images but it may be different when it is assessed by human visual response. Therefore, the quality assessment measures may not be perfected yet for human perception but it acts as a guideline for judgment. From the selected sample images, it can be concluded that MDRHM performs better in images with more objects and narrow dynamic range. In order to confirm that MDRHM performs better in images with more objects and narrow dynamic range, 100 trials had been taken. The average data in two conditions are shown in Table II. The results shown in Table II indicate that there is advantage for MDRHM to perform well in these two conditions. MDRHM obtains the highest values of PSNR and SSIM in more objects type of SEM images whereas the MDRHM only obtains the highest value of SSIM in narrow dynamic range but the PSNR values
Statistical study of 100 sample images with two conditions applied Techniques Uniform
Conditions More objects Narrow dynamic range Standard deviation More objects Narrow dynamic range
Rayleigh
Exponential
MDRHM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
14 12.5
0.76 0.64
15.41 13.21
0.84 0.73
13.72 11.73
0.75 0.62
23 13.18
0.84 0.75
4.7121 2.0209
0.2808 0.0455
7.9310 3.6748
0.2783 0.0748
5.9774 2.2191
0.3214 0.0668
21.4070 2.2292
0.1049 0.0990
388
SCANNING VOL. 37, 6 (2015)
between Rayleigh and MDRHM is only 0.03 which shows that MDRHM is still able to maintain its quality and capability in image enhancement.
Conclusions In this paper, some of the common image enhancement techniques are reviewed and explained. These techniques are useful indeed but the results may not be desirable from one application to another. Conventional histogram equalization is widely used for contrast enhancement because of its effectiveness and simplicity of algorithm. However, it would introduce artifacts in the region where it has more or less frequency of gray levels. So, the proposed MDRHM is able to overcome these issues by stretching the dynamic range in each region into its greatest gray levels. Since the pixel intensities in each region may be varied from each other so the contrast enhancement performance may be different. The proposed MDRHM is tested with a series of SEM images with variety types of images as mentioned in discussion section. In the quantitative measure of contrast enhancement, MDRHM outperforms the selected three histogram modification techniques and HE in two categories of image type.
References Ahmad SAG, Nor Ashidi MI. 2014. Underwater image quality enhancement through composition of dual-intensity images and Rayleigh-stretching. Springer Open Journal, 3: 1–14. Arjun N, Pradnyawant K, Amit L, Vrushali P, Bhagyashri S. 2013. A novel approach to medical & gray scale image enhancement. International Journal of Engineering Research and Applications 3:653–657. Azeddine B, Alain LN. 1988. Contrast enhancement technique based on local detection of edges. Computer Vision, Graphics, and Image Processing 46:162–174.
Cheng HD, Shi XJ. 2004. A simple and effective histogram equalization approach to image enhancement. Digital Signal Proccessing 14:158–170. Cheng SD, Ramli R. 2003. Contrast enhancement using recursive mean-separate histogram equalization for scalable brightness preservation. Institute of Electrical and Electronics Engineers Transactions on Consumer Electronics 49:1301–1309. De Medeiros Martins A, Torres de Almeida Filho W, Medeiros Brito Junior A, Duarte Doria Neto A. 2002. A new method for multi-texture segmentation using neural networks. Proc of the 2002 International Joint Conference 3, 12–17:2064–2069. George S. (2013) VIsual media processing using Matlab. Birmingham - Mumbai: PACKT. Kawaldeep SR, Parveen K. 2012. A novel approach for blocking artifact reduction in JPEG compressed images. International Journal of Emerging Technology and Advanced Engineering 2:150–157. Manpreet K, Jasdeep K, Jappreet K. 2011. Survey of contrast enhancement techniques based on histogram equalization. International Journal of Advanced Computer Science and Applications 2:138–141. Raju A, Dwarakish GS, Venkat R. 2013. A comparative analysis of histogram equalization based techniques for contrast enhancement and brightness preserving. International Journal of Signal Processing, Image Processing and Pattern Recognition 4:353–366. Sim KS, Kamel NS, Chuah HT. 2005. A real-time image dynamic range compensation for scanning electron microscope imaging system. Scanning 27:199–207. Sim KS, Tso CP, Tan YY, Law KK, Teoh ABJ. 2007. Realtime image dynamic range compensation for an optical imaging system. Journal of Electronic Imaging 16: 033006–033006-17. Tolat AR, McNeill SR, Sutton MA. 1991. Effects of contrast and brightness on subpixel image correlation. System Theory Proceedings, 23rd Southeastern Symposium:604–608. Wongsritong K, Kittayaruasiriwat K, Cheevasuvit F, Dejhan K, Somboonkaew A. 1998. A contrast enhancement using multipeak histogram equalization with brightness preserving, circuit and systems. IEEE APCCAS Asia-Pacific Conference:455–458. Yoon HS, Han YJ, Hahn HS. 2009. Image contrast enhancement based sub-histogram equalization technique without overequalization noise World Academy of Science. Engineering and Technology 3:145–151. Zhou HY, Wu JH. (2010) Digital image processing: Part 1. © 2010 Huiyu Zhou, Jiahua Wu, Jianguo Zhang & bookboon. com, London.
