Int. J. Computational Biology and Drug Design, Vol. 7, Nos. 2/3, 2014
157
Localisation of Drosophila embryos using active contours in channel spaces Qi Li* and Soujanya Siddavaram Ananta Department of Computer Science, Western Kentucky University, Bowling Green, KY, 42101, USA E-mail: [email protected] E-mail: [email protected] *Corresponding author Abstract: In this paper, we introduce an active contour based scheme to localise Drosophila embryos in RGB images. An active contour (initiated as a closed one) maybe converge to an open contour, e.g., in the case that a targeting embryo is touched by a neighbouring one. We propose an algorithmic strategy to detect and restore open active contours. The experiment results show the promise of the proposed scheme. Keywords: Drosophila embryo; localisation; active contour; RGB channels. Reference to this paper should be made as follows: Li, Q. and Ananta, S.S. (2014) ‘Localisation of Drosophila embryos using active contours in channel spaces’, Int. J. Computational Biology and Drug Design, Vol. 7, Nos. 2/3, pp.157–167. Biographical notes: Qi Li received his PhD in Computer Science from University of Delaware in 2006. He is an Associate Professor of the Department of Computer Science at Western Kentucky University. His current research interest include pattern recognition, computer vision, machine learning, and bioinformatics. He is an Associate Editor of Neurocomputing; International Journal of Data Mining, Modelling, and Management, and International Journal of Data Analysis Techniques and Strategies. Soujanya Siddavaram Ananta received her Master degree in Computer Science from Western Kentucky University in 2012. Her thesis research is focused on Contour Extraction of Drosophila Embryos Using Active Contours in Scale Space. This paper is a revised and expanded version of a paper entitled ‘Localisation of Drosophila embryos using active contours in channel spaces’ presented at the International Conference on Intelligent Biology and Medicine (ICIBM 2013), Nashville, TN, USA, 11–13 August, 2013.
1
Introduction
Drosophila embryonic images provide detailed spatial and temporal information of gene expression, which becomes an important tool for micro-biologists to study gene-gene interaction (Tomancak et al., 2002). Copyright © 2014 Inderscience Enterprises Ltd.
158
Q. Li and S.S. Ananta
Localisation of a targeting embryo in an embryonic image is the first step of an automatic computational system for the exploration of gene-gene interaction on Drosophila. Embryonic images usually contain significant amount of variations: •
imaging conditions, such as the contrast, scale, orientation, and neighbouring embryos
•
gene expression patterns
•
developmental stages.
The straight application of existing techniques on edge detection and contour extraction failed to obtain desirable results (Peng and Myers, 2004; Pan et al., 2006; Li and Kambhamettu, 2011). Several approaches have been proposed to localise embryos in images (Gargesha et al., 2005; Peng and Myers, 2004; Pan et al., 2006). Peng and Myers (2004) proposed an approach that computes the standard deviation of the local windows of pixels to characterise pixels as foreground and background pixels, and applies 8-neighbourconnectivity region-growing method to localise the contour of the embryo. Pan et al. (2006) used a variant of Marquardt-Levenberg algorithm to compute an optimal affine transformation to register localised embryos into an ellipsoidal region. Frise et al. (2010) proposed a heuristic algorithm to separate the embryo of interest from multiple touching embryos, with the assumption that the centre of the embryo of interest is the image centre. Mace et al. (2010) proposed an eigen-embryo method to extract the contour of embryos, where particle swarm optimiser was used to reduce the computational cost of searching optimal eigen parameters. Puniyani et al. (2010) proposed an edge detection based method that involves a set of heuristic constraints, including object size, convexity, shape features (e.g., ratio of the major over minor axis of an object), and the percentage of overlapping region. In this paper, we propose a localisation scheme based on the exploration of active contours of an embryonic image in a channel space: red, green, blue, and greyscale. Specifically, the proposed localisation scheme contains three main stages: •
extraction of active contours in a channel space
•
detection and restoration of an open active contour (see Figure 1)
•
maximisation of the areas of closed active contours in channel spaces.
We use the Chan-Vese active contour model (Chan and Vese, 2001) in this paper. One advantage of using the Chan-Vese model is that extracted active contours are not sensitive to their initialisation, and the method contains relatively few parameters to tune in. Moreover, our preliminary study (Ananta, 2012) shows that contours extracted by the Chan-Vese model are generally closer to the boundary of a targeting embryo than other contour extraction models, e.g., distance regularised level set evolution (DRLSE) (Li et al., 2010). We demonstrate the effectiveness of the proposed localisation scheme in experiments.
2
Related work
Contour extraction can be achieved by configuring global constraints (Kass et al., 1988; Chan and Vese, 2001; Fussenegger et al., 2009; Li et al., 2010; Arbelaez et al., 2011;
Localisation of Drosophila embryos using active contours
159
Boykov and Funka-Lea, 2006; Grady and Alvino, 2009; Salah et al., 2011; Shi and Malik, 2000; Tao et al., 2007). In terms of internal and external energy, an active contour approach can incorporate generic shape constraints, such as smoothness and elastic, and image contrast in order to drive an initialised contour to the boundary of an object (Kass et al., 1988). Chan and Vese (2001) proposed an active contour method without edge, which is essentially a level set method. Fussenegger et al. (2009) integrated shape priors of objects in a level set in an incremental manner to improve the performance of image segmentation and tracking. A level set function can accumulate irregularity during its evolution. As a solution to the irregularity problem, re-initialisation can degrade the numerical accuracy. Li et al. (2010) proposed a energy function with a distance regularisation term so that the derived level set evolution has a unique forward-andbackward diffusion effect, which is in turn able to maintain a desired shape of the level set function. Figure 1
An active contour may be open if the targeting embryo in an image is touched by a neighbouring embryo. Red solid dots are open points detected by the proposed detection algorithm. The closed contour is obtained by the proposed restoration algorithm. (a) An open contour and (b) a closed contour (see online version for colours)
(a)
(b)
Graph cut is an alternative to active contour and level set approaches for contour extraction. Graph cut can integrate region, boundary and shape, in addition to topology constraints (Boykov and Funka-Lea, 2006). Shi and Malik (2000) proposed a normalisation scheme in graph cut to take between-cluster difference in consideration by introducing an association between a subgraph and a graph. Boykov and Funka-Lea (2006) revealed interesting relationships between graph cut and active contours, levelsets. Grady and Alvino (2009) extended the Mumford-Shah functional from active contours and level sets to graph cut to reduce its computational cost. Salah et al. (2011) proposed a multi-region graph cut approach in terms of a kernel mapping, aiming to provide an effective alternative to complex modelling of original image data while taking advantage of the computational benefits of graph cuts. In general, global constraints used in active contours, level sets, or graph cut are not easily adaptive to objects with significant appearance variations. Besides this, a graph cut approach has some other limitations, e.g., shrinking bias (tending to produce a small contour) (Sinop and Grady, 2007), the expensive memory usage, etc. Under the graph formulation, Zhu et al. (2007) proposed a grouping criterion, called untangling cycles, to exploit the inherent topological 1D structure of salient contours. Kennedy et al. (2011) proposed contour cut to generalise the cost function used in untangling cycles criterion and give a computational solution based on a Hermitian eigenvalue formulation. Arbelaez et al. (2011) proposed a spectral clustering based global method that integrates multiscale local brightness, colour, and texture cues.
160
3
Q. Li and S.S. Ananta
Active contours in channel spaces
We first give an overview of the Chan-Vese active contour model (Chan and Vese, 2001). Denote I an image (in a channel), ȍ an image plane, Ȧ a region, C = Ȧ the boundary of the region, inside(C) an alternative notation of Ȧ, outside(C) = ȍ\Ȧ. Denote c1 the average of I(inside(C)) and c2 the average of I(outside(C)). The Chan-Vese model aims to minimise the following functional: F ( c1 , c2 , C ) = µLength ( C ) + vArea ( inside ( C ) ) + λ1 ³
inside( C )
+ λ2 ³
| I ( x, y ) − c1 |2 dx dy
outside( C )
| I ( x, y ) − c2 |2 dx dy,
where µ 0, v 0, Ȝ1 > 0, and Ȝ2 > 0. The last three parameters are generally set as follows: v = 0, Ȝ1 = 1, and Ȝ2 = 1 (Chan and Vese, 2001). Note that the Chan-Vese model with v = 0 is consistent with the Mumford-Shah model (Mumford and Shah, 1989). For embryonic images, we set µ to be 0.8, based on our preliminary study (Ananta, 2012). We initialise a Chan-Vese active contour as a rectangle with a centre same as the image centre and with the width and height equal to the half width and half height of the image, respectively. Figure 2 illustrates the initialisation and the convergence behaviour of a Chan-Vese active contour on an embryonic image. Figure 2
Initialisation and behaviours of a Chan-Vese active contour (Chan and Vese, 2001) for an input embryonic image: (a) initialisation; (b) 250 iterations; (c) 500 iterations and (d) 1000 iterations (see online version for colours)
(a)
(b)
(c)
(d)
Figure 3 shows three examples of active contours in red, green, blue, and greyscale channels. It is interesting to observe that the first two active contours positioned on embryonic images containing blue regions tend to expand in the blue channel and to shrink in the other three channels. However, the third active contour positioned on the embryo without blue regions converges to the boundary of the targeting embryo in all channels. Note that blue regions in an embryonic image in the BDGP
Localisation of Drosophila embryos using active contours
161
(Berkley Drosophila Genome Project) (Tomancak et al., 2002) indicate gene expressions under the dye of blue colour. As shown in Figures 1 and 2, a final active contour may be open due to the existence of some partial embryos touching the targeting embryo. Note that touching embryos form a homogeneous region that can enforce a Chan-Vese contour to converge to the union of boundaries of touching embryos. In next section, we will propose algorithms to remove external points (i.e., points not belonging to the boundary of the targeting embryo) in an active contour, and restore it as a closed one. Based on closed contours extracted in channel space of an embryonic image, we propose a maximisation criterion to localise the optimal contour as follows: r* = argmaxrArea(Cr), where Cr is a closed contour in a channel r. Figure 3
4
Three examples of active contours in channel spaces. From the second row to last rows are final active contours in red, green, blue, and greyscale channels, respectively (see online version for colours)
Detection and restoration of an open contour
In this section, we first propose an algorithm to detect open contour points in an active contour, then a method to remove external contour points, i.e., points not belonging to the boundary of a targeting embryo, and finally a two-step method to restore an open contour. For convenience of illustration, we denote P = {pi = (xi, yi)} an active contour, i.e., a sequence of points.
4.1 Detection of open points We propose an approach to detect open points if an active contour is open, as illustrated in Figure 4.
162 Figure 4
Q. Li and S.S. Ananta Detection of an open contour (see online version for colours)
The basic idea of the proposed approach is the following. First, we normalise each point to a unit vector based on the centroid. We use a bounding box to determine a centroid of an active contour, i.e., x = ( xmin + xmax ) / 2, y = ( ymin + ymax ) / 2. Then, we accumulate normalised points to an angular bin according to their orientations. Based on the characteristic that the angular bin (ș1) of one side of an end point should be empty and the angular bin (ș2) of the other side should not be empty, we can locate two angular bins associated with two end points. Finally, we compute pair-wise distances of points in bins ș1 and ș2 to find out the shortest distance, and the two points that contribute the shortest distance are output as end points. Algorithm 1 describes the details of the procedure of detection of open points in an active contour.
Localisation of Drosophila embryos using active contours
163
4.2 Removal of external contour points Based on open points, we propose an algorithm (Algorithm 2) to remove external contour points, i.e., points not belonging to the boundary of a targeting embryo.
4.3 Restoration of an open contour Without loss of generality, we now assume that the orientation of a contour (of internal points) is horizontal. If the orientation of the contour is not horizontal, we can apply SVD to the centralised points to compute the orientation, and then rotate points to a horizontal orientation. We propose a two-step method to recover missing points in an active contour. Step 1: Find a base model This step aims to locate a potential segment (a subsequence of points) P by a horizontal or vertical reflection of end points p0 and pn. For convenience, we denote p0′ a reflection point of p0 and pn′ a reflection point of pn. A reflection operation is expected to locate a subsequence from the input sequence of points as a base model to morph and then fill the missing part of a contour. A reflection is called safe if two line segments p0pn and pn p0′ has an intersection point. Otherwise, the reflection is called unsafe. Figure 5 shows two samples of open contours. In the first example, the horizontal reflection of two end points is safe, while the vertical reflection is unsafe. In the second sample, both horizontal and vertical reflections are safe to apply to end points to locate a base model. Which one is better to use? One measurement to select an optimal reflection is based on the overall displacement of reflections of end points from the existing points. More specifically, the overall displacement of reflections is defined as ∆p0′ + ∆pn′ ,
where ∆p0′ = min p∈P || p − p0′ ||, and ∆pn′ = min p∈P || p − pn′ || . The reflection that leads to smaller displacement ∆p0′ + ∆pn′ is considered as the optimal reflection. Denote pi = arg min p || p − p0′ ||, and p j = arg min p || p − pn′ || . Then the subsequence of points is the base model used for next two steps.
164
Q. Li and S.S. Ananta
Step 2: Morph the base model Morphing a base model B = aims to warp the base model smoothly with the preservation of the shape and fit p0′ and pn′ . Denote pi p0′ = ( p0′ − pi ) and p j pn′ = ( pn′ − p j ). Our basic strategy of morphing the base model is to applying a weighting scheme to the two directional vectors: pi p0′ and p j pn′ . Specifically, given a p ∈ B we use the following formula to warp: p ′ = p + [1 − t ( p ) pi p0′ + t ( p ) p j pn′ ],
where t(p) = ||p – pi||/(||p – pi|| + ||p – pj||). Note that t(pi) = 0, t(pj) = 1, and 0 t(p) 1, for any p ∈ B. Figure 5
5
A reflection is safe if there is an intersection point between p0pn′ and pnp0′, where p0′ and pn′ are reflection points of p0 and pn, respectively (see online version for colours)
Experiments
We will test the proposed framework on BDGP (Berkley Drosophila Genome Project) (Tomancak et al., 2002). BDGP images are available at a public webpage (http://www. fruitfly.org/insituimages/insitu_images/). Figure 6 shows examples of optimal closed contours. The optimal contours are visually consistent with the boundary of a targeting embryo. It is worth noting that the image channel space under the criterion of attention maximisation is typically effective to ‘circumvent’ the high contrast between geneexpression (dark/blue) regions and non-expression regions inside an embryo (see the last image in Figure 6). Figure 7 shows examples of optimal contours that are open and successfully restored contours. Figure 8 shows three failure examples of the proposed approach. In case (a), the active contour technique does not localise the boundary of an embryo correctly. In case (b), the open-point detection algorithm fails to detect open points since the contour fills every angular bin (from 0° to 359°). In case (c), the detection algorithm fails to detect correct open points due to the large intersection between the targeting embryo and a neighbouring embryo. To have a quantified evaluation of the proposed method, we define a localisation result to be successful if the average displacement between a final contour and the ground truth (manually extracted contour) is less than 5%. Tested on the dataset of 1000 BDGP
Localisation of Drosophila embryos using active contours
165
images, the localisation accuracy achieved by the proposed method is 89%. As a comparison, the straight application of Chan-Vese active contour method only achieved an accuracy of 62%. We also applied a distance regularised level set evolution (DRLSE) method (Li et al., 2010) to the dataset, and got an accuracy of 56%, which showed the limitation of level set methods with shape constraints in the context of contour extraction of Drosophila embryonic images. As a conclusion, experimental results show the effectiveness of processing channel spaces, and algorithms to process open contours. In the future, we plan to tune the proposed method to extract gene expression regions in embryonic images. Figure 6
Optimal closed active contours (see online version for colours)
Figure 7
Successful examples of the proposed restoration algorithm. (a1/a2) Open contours. (b1/b2) Restored contours (see online version for colours)
166 Figure 8
Q. Li and S.S. Ananta Failure cases of the proposed approach (see online version for colours)
Acknowledgements This work was supported by National Science Foundation Grant IIS-1016668.
References Ananta, S. (2012) Contour Extraction of Drosophila Embryos using Active Contours in Scale Space, Master Thesis, Western Kentucky University. Arbelaez, P., Maire, M., Fowlkes, C. and Malik, J. (2011) ‘Contour detection and hierarchical image segmentation’, IEEE Trans. Pattern Anal. Mach. Intell., Vol. 33, No. 5, pp.898–916. Boykov, Y. and Funka-Lea, G. (2006) ‘Graph cuts and efficient N-D image segmentation’, International Journal of Computer Vision, Vol. 70, No. 2, pp.109–131. Chan, T.F. and Vese, L.A. (2001) ‘Active contours without edges’, IEEE Transactions on Image Processing, Vol. 10, pp.266–277. Frise, E., Hammonds, A.S. and Celniker, S.E. (2010) ‘Systematic image-driven analysis of the spatial drosophila embryonic expression landscape’, Molecular Systems Biology, Vol. 6, No. 345, doi:10.1038/msb.2009.102. Fussenegger, M., Roth, P.M., Bischof, H., Deriche, R. and Pinz, A. (2009) ‘A level set framework using a new incremental, robust Active Shape Model for object segmentation and tracking’, Image Vision Comput., Vol. 27, No. 8, pp.1157–1168. Gargesha, M., Yang, J., Emden, B.V., Panchanathan, S. and Kumar, S. (2005) ‘Automatic annotation techniques for gene expression images of the fruit fly embryo’, Visual Communications and Image Processing, in Li, S., Pereira, F., Shum, H-Y. and Tescher, A.G. (Eds.): Proceedings of the SPIE, Vol. 5960, pp.576–583. Grady, L. and Alvino, C.V. (2009) ‘The piecewise smooth mumford-shah functional on an arbitrary graph’, IEEE Transactions on Image Processing, Vol. 18, No. 11, pp.2547–2561. Kass, M., Witkin, A.P. and Terzopoulos, D. (1988) ‘Snakes: active contour models’, International Journal of Computer Vision, Vol. 1, No. 4, pp.321–331. Kennedy, R., Gallier, J.H. and Shi, J. (2011) ‘Contour cut: Identifying salient contours in images by solving a Hermitian eigenvalue problem’, Proceedings of Computer Vision and Pattern Recognition, pp.2065–2072. Li, C., Xu, C., Gui, C. and Fox, M.D. (2010) ‘Distance regularized level set evolution and its application to image segmentation’, IEEE Transactions on Image Processing, Vol. 19, No. 12, pp.3243–3254.
Localisation of Drosophila embryos using active contours
167
Li, Q. and Kambhamettu, C. (2011) ‘Contour extraction of drosophila embryos’, IEEE/ACM Trans. Comput. Biology Bioinform., Vol. 8, No. 6, pp.1509–1521. Mace, D.L., Varnado, N., Zhang, W., Frise, E. and Ohler, U. (2010) ‘Extraction and comparison of gene expression patterns from 2d RNA in situ hybridization images’, Bioinformatics, Vol. 26, No. 6, pp.761–769. Mumford, D. and Shah, J. (1989) ‘Optimal approximation by piecewise smooth functions and associated variational problems’, Commun. Pure Appl. Math., Vol. 42, pp.577–685. Pan, J.Y., Balan, A.G.R., Xing, E.P., Traina, A.J.M. and Faloutsos, C. (2006) ‘Automatic mining of fruit fly embryo images’, ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.693–698. Peng, H. and Myers, E.W. (2004) ‘Comparing n situ mRNA expression patterns of drosophila embryos’, RECOMB, pp.157–166. Puniyani, K., Faloutsos, C. and Xing, E.P. (2010) ‘Spex2: Automated concise extraction of spatial gene expression patterns from fly embryo ISH images’, Bioinformatics, Vol. 26, No. 12, pp.i47–i56. Salah, M.B., Mitiche, A. and Ayed, I.B. (2011) ‘Multiregion image segmentation by parametric kernel graph cuts’, IEEE Transactions on Image Processing, Vol. 20, No. 2, pp.545–557. Shi, J. and Malik, J. (2000) ‘Normalized cuts and image segmentation’, IEEE Trans. Pattern Anal. Mach. Intell., Vol. 22, No. 8, pp.888–905. Sinop, A.K. and Grady, L. (2007) ‘A seeded image segmentation framework unifying graph cuts and random walker which yields a new algorithm’, International Conference on Computer Vision, pp.1–8. Tao, W., Jin, H. and Zhang, Y. (2007) ‘Color image segmentation based on mean shift and normalized cuts’, IEEE Transactions on Systems, Man, and Cybernetics, Part B, Vol. 37, No. 5, pp.1382–1389. Tomancak, P., Beaton, A., Weiszmann, R., Kwan, E., Shu, S., Lewis, S.E., Richards, S., Ashburner, M., Hartenstein, V., Celniker, S.E. and Rubin, G.M. (2002) ‘Systematic determination of patterns of gene expression during drosophila embryogenesis’, Genome Biology, Vol. 3, No. 12, pp.1–14. Zhu, Q., Song, G. and Shi, J. (2007) ‘Untangling cycles for contour grouping’, International Conference on Computer Vision, pp.1–8.
157
Localisation of Drosophila embryos using active contours in channel spaces Qi Li* and Soujanya Siddavaram Ananta Department of Computer Science, Western Kentucky University, Bowling Green, KY, 42101, USA E-mail: [email protected] E-mail: [email protected] *Corresponding author Abstract: In this paper, we introduce an active contour based scheme to localise Drosophila embryos in RGB images. An active contour (initiated as a closed one) maybe converge to an open contour, e.g., in the case that a targeting embryo is touched by a neighbouring one. We propose an algorithmic strategy to detect and restore open active contours. The experiment results show the promise of the proposed scheme. Keywords: Drosophila embryo; localisation; active contour; RGB channels. Reference to this paper should be made as follows: Li, Q. and Ananta, S.S. (2014) ‘Localisation of Drosophila embryos using active contours in channel spaces’, Int. J. Computational Biology and Drug Design, Vol. 7, Nos. 2/3, pp.157–167. Biographical notes: Qi Li received his PhD in Computer Science from University of Delaware in 2006. He is an Associate Professor of the Department of Computer Science at Western Kentucky University. His current research interest include pattern recognition, computer vision, machine learning, and bioinformatics. He is an Associate Editor of Neurocomputing; International Journal of Data Mining, Modelling, and Management, and International Journal of Data Analysis Techniques and Strategies. Soujanya Siddavaram Ananta received her Master degree in Computer Science from Western Kentucky University in 2012. Her thesis research is focused on Contour Extraction of Drosophila Embryos Using Active Contours in Scale Space. This paper is a revised and expanded version of a paper entitled ‘Localisation of Drosophila embryos using active contours in channel spaces’ presented at the International Conference on Intelligent Biology and Medicine (ICIBM 2013), Nashville, TN, USA, 11–13 August, 2013.
1
Introduction
Drosophila embryonic images provide detailed spatial and temporal information of gene expression, which becomes an important tool for micro-biologists to study gene-gene interaction (Tomancak et al., 2002). Copyright © 2014 Inderscience Enterprises Ltd.
158
Q. Li and S.S. Ananta
Localisation of a targeting embryo in an embryonic image is the first step of an automatic computational system for the exploration of gene-gene interaction on Drosophila. Embryonic images usually contain significant amount of variations: •
imaging conditions, such as the contrast, scale, orientation, and neighbouring embryos
•
gene expression patterns
•
developmental stages.
The straight application of existing techniques on edge detection and contour extraction failed to obtain desirable results (Peng and Myers, 2004; Pan et al., 2006; Li and Kambhamettu, 2011). Several approaches have been proposed to localise embryos in images (Gargesha et al., 2005; Peng and Myers, 2004; Pan et al., 2006). Peng and Myers (2004) proposed an approach that computes the standard deviation of the local windows of pixels to characterise pixels as foreground and background pixels, and applies 8-neighbourconnectivity region-growing method to localise the contour of the embryo. Pan et al. (2006) used a variant of Marquardt-Levenberg algorithm to compute an optimal affine transformation to register localised embryos into an ellipsoidal region. Frise et al. (2010) proposed a heuristic algorithm to separate the embryo of interest from multiple touching embryos, with the assumption that the centre of the embryo of interest is the image centre. Mace et al. (2010) proposed an eigen-embryo method to extract the contour of embryos, where particle swarm optimiser was used to reduce the computational cost of searching optimal eigen parameters. Puniyani et al. (2010) proposed an edge detection based method that involves a set of heuristic constraints, including object size, convexity, shape features (e.g., ratio of the major over minor axis of an object), and the percentage of overlapping region. In this paper, we propose a localisation scheme based on the exploration of active contours of an embryonic image in a channel space: red, green, blue, and greyscale. Specifically, the proposed localisation scheme contains three main stages: •
extraction of active contours in a channel space
•
detection and restoration of an open active contour (see Figure 1)
•
maximisation of the areas of closed active contours in channel spaces.
We use the Chan-Vese active contour model (Chan and Vese, 2001) in this paper. One advantage of using the Chan-Vese model is that extracted active contours are not sensitive to their initialisation, and the method contains relatively few parameters to tune in. Moreover, our preliminary study (Ananta, 2012) shows that contours extracted by the Chan-Vese model are generally closer to the boundary of a targeting embryo than other contour extraction models, e.g., distance regularised level set evolution (DRLSE) (Li et al., 2010). We demonstrate the effectiveness of the proposed localisation scheme in experiments.
2
Related work
Contour extraction can be achieved by configuring global constraints (Kass et al., 1988; Chan and Vese, 2001; Fussenegger et al., 2009; Li et al., 2010; Arbelaez et al., 2011;
Localisation of Drosophila embryos using active contours
159
Boykov and Funka-Lea, 2006; Grady and Alvino, 2009; Salah et al., 2011; Shi and Malik, 2000; Tao et al., 2007). In terms of internal and external energy, an active contour approach can incorporate generic shape constraints, such as smoothness and elastic, and image contrast in order to drive an initialised contour to the boundary of an object (Kass et al., 1988). Chan and Vese (2001) proposed an active contour method without edge, which is essentially a level set method. Fussenegger et al. (2009) integrated shape priors of objects in a level set in an incremental manner to improve the performance of image segmentation and tracking. A level set function can accumulate irregularity during its evolution. As a solution to the irregularity problem, re-initialisation can degrade the numerical accuracy. Li et al. (2010) proposed a energy function with a distance regularisation term so that the derived level set evolution has a unique forward-andbackward diffusion effect, which is in turn able to maintain a desired shape of the level set function. Figure 1
An active contour may be open if the targeting embryo in an image is touched by a neighbouring embryo. Red solid dots are open points detected by the proposed detection algorithm. The closed contour is obtained by the proposed restoration algorithm. (a) An open contour and (b) a closed contour (see online version for colours)
(a)
(b)
Graph cut is an alternative to active contour and level set approaches for contour extraction. Graph cut can integrate region, boundary and shape, in addition to topology constraints (Boykov and Funka-Lea, 2006). Shi and Malik (2000) proposed a normalisation scheme in graph cut to take between-cluster difference in consideration by introducing an association between a subgraph and a graph. Boykov and Funka-Lea (2006) revealed interesting relationships between graph cut and active contours, levelsets. Grady and Alvino (2009) extended the Mumford-Shah functional from active contours and level sets to graph cut to reduce its computational cost. Salah et al. (2011) proposed a multi-region graph cut approach in terms of a kernel mapping, aiming to provide an effective alternative to complex modelling of original image data while taking advantage of the computational benefits of graph cuts. In general, global constraints used in active contours, level sets, or graph cut are not easily adaptive to objects with significant appearance variations. Besides this, a graph cut approach has some other limitations, e.g., shrinking bias (tending to produce a small contour) (Sinop and Grady, 2007), the expensive memory usage, etc. Under the graph formulation, Zhu et al. (2007) proposed a grouping criterion, called untangling cycles, to exploit the inherent topological 1D structure of salient contours. Kennedy et al. (2011) proposed contour cut to generalise the cost function used in untangling cycles criterion and give a computational solution based on a Hermitian eigenvalue formulation. Arbelaez et al. (2011) proposed a spectral clustering based global method that integrates multiscale local brightness, colour, and texture cues.
160
3
Q. Li and S.S. Ananta
Active contours in channel spaces
We first give an overview of the Chan-Vese active contour model (Chan and Vese, 2001). Denote I an image (in a channel), ȍ an image plane, Ȧ a region, C = Ȧ the boundary of the region, inside(C) an alternative notation of Ȧ, outside(C) = ȍ\Ȧ. Denote c1 the average of I(inside(C)) and c2 the average of I(outside(C)). The Chan-Vese model aims to minimise the following functional: F ( c1 , c2 , C ) = µLength ( C ) + vArea ( inside ( C ) ) + λ1 ³
inside( C )
+ λ2 ³
| I ( x, y ) − c1 |2 dx dy
outside( C )
| I ( x, y ) − c2 |2 dx dy,
where µ 0, v 0, Ȝ1 > 0, and Ȝ2 > 0. The last three parameters are generally set as follows: v = 0, Ȝ1 = 1, and Ȝ2 = 1 (Chan and Vese, 2001). Note that the Chan-Vese model with v = 0 is consistent with the Mumford-Shah model (Mumford and Shah, 1989). For embryonic images, we set µ to be 0.8, based on our preliminary study (Ananta, 2012). We initialise a Chan-Vese active contour as a rectangle with a centre same as the image centre and with the width and height equal to the half width and half height of the image, respectively. Figure 2 illustrates the initialisation and the convergence behaviour of a Chan-Vese active contour on an embryonic image. Figure 2
Initialisation and behaviours of a Chan-Vese active contour (Chan and Vese, 2001) for an input embryonic image: (a) initialisation; (b) 250 iterations; (c) 500 iterations and (d) 1000 iterations (see online version for colours)
(a)
(b)
(c)
(d)
Figure 3 shows three examples of active contours in red, green, blue, and greyscale channels. It is interesting to observe that the first two active contours positioned on embryonic images containing blue regions tend to expand in the blue channel and to shrink in the other three channels. However, the third active contour positioned on the embryo without blue regions converges to the boundary of the targeting embryo in all channels. Note that blue regions in an embryonic image in the BDGP
Localisation of Drosophila embryos using active contours
161
(Berkley Drosophila Genome Project) (Tomancak et al., 2002) indicate gene expressions under the dye of blue colour. As shown in Figures 1 and 2, a final active contour may be open due to the existence of some partial embryos touching the targeting embryo. Note that touching embryos form a homogeneous region that can enforce a Chan-Vese contour to converge to the union of boundaries of touching embryos. In next section, we will propose algorithms to remove external points (i.e., points not belonging to the boundary of the targeting embryo) in an active contour, and restore it as a closed one. Based on closed contours extracted in channel space of an embryonic image, we propose a maximisation criterion to localise the optimal contour as follows: r* = argmaxrArea(Cr), where Cr is a closed contour in a channel r. Figure 3
4
Three examples of active contours in channel spaces. From the second row to last rows are final active contours in red, green, blue, and greyscale channels, respectively (see online version for colours)
Detection and restoration of an open contour
In this section, we first propose an algorithm to detect open contour points in an active contour, then a method to remove external contour points, i.e., points not belonging to the boundary of a targeting embryo, and finally a two-step method to restore an open contour. For convenience of illustration, we denote P = {pi = (xi, yi)} an active contour, i.e., a sequence of points.
4.1 Detection of open points We propose an approach to detect open points if an active contour is open, as illustrated in Figure 4.
162 Figure 4
Q. Li and S.S. Ananta Detection of an open contour (see online version for colours)
The basic idea of the proposed approach is the following. First, we normalise each point to a unit vector based on the centroid. We use a bounding box to determine a centroid of an active contour, i.e., x = ( xmin + xmax ) / 2, y = ( ymin + ymax ) / 2. Then, we accumulate normalised points to an angular bin according to their orientations. Based on the characteristic that the angular bin (ș1) of one side of an end point should be empty and the angular bin (ș2) of the other side should not be empty, we can locate two angular bins associated with two end points. Finally, we compute pair-wise distances of points in bins ș1 and ș2 to find out the shortest distance, and the two points that contribute the shortest distance are output as end points. Algorithm 1 describes the details of the procedure of detection of open points in an active contour.
Localisation of Drosophila embryos using active contours
163
4.2 Removal of external contour points Based on open points, we propose an algorithm (Algorithm 2) to remove external contour points, i.e., points not belonging to the boundary of a targeting embryo.
4.3 Restoration of an open contour Without loss of generality, we now assume that the orientation of a contour (of internal points) is horizontal. If the orientation of the contour is not horizontal, we can apply SVD to the centralised points to compute the orientation, and then rotate points to a horizontal orientation. We propose a two-step method to recover missing points in an active contour. Step 1: Find a base model This step aims to locate a potential segment (a subsequence of points) P by a horizontal or vertical reflection of end points p0 and pn. For convenience, we denote p0′ a reflection point of p0 and pn′ a reflection point of pn. A reflection operation is expected to locate a subsequence from the input sequence of points as a base model to morph and then fill the missing part of a contour. A reflection is called safe if two line segments p0pn and pn p0′ has an intersection point. Otherwise, the reflection is called unsafe. Figure 5 shows two samples of open contours. In the first example, the horizontal reflection of two end points is safe, while the vertical reflection is unsafe. In the second sample, both horizontal and vertical reflections are safe to apply to end points to locate a base model. Which one is better to use? One measurement to select an optimal reflection is based on the overall displacement of reflections of end points from the existing points. More specifically, the overall displacement of reflections is defined as ∆p0′ + ∆pn′ ,
where ∆p0′ = min p∈P || p − p0′ ||, and ∆pn′ = min p∈P || p − pn′ || . The reflection that leads to smaller displacement ∆p0′ + ∆pn′ is considered as the optimal reflection. Denote pi = arg min p || p − p0′ ||, and p j = arg min p || p − pn′ || . Then the subsequence of points is the base model used for next two steps.
164
Q. Li and S.S. Ananta
Step 2: Morph the base model Morphing a base model B = aims to warp the base model smoothly with the preservation of the shape and fit p0′ and pn′ . Denote pi p0′ = ( p0′ − pi ) and p j pn′ = ( pn′ − p j ). Our basic strategy of morphing the base model is to applying a weighting scheme to the two directional vectors: pi p0′ and p j pn′ . Specifically, given a p ∈ B we use the following formula to warp: p ′ = p + [1 − t ( p ) pi p0′ + t ( p ) p j pn′ ],
where t(p) = ||p – pi||/(||p – pi|| + ||p – pj||). Note that t(pi) = 0, t(pj) = 1, and 0 t(p) 1, for any p ∈ B. Figure 5
5
A reflection is safe if there is an intersection point between p0pn′ and pnp0′, where p0′ and pn′ are reflection points of p0 and pn, respectively (see online version for colours)
Experiments
We will test the proposed framework on BDGP (Berkley Drosophila Genome Project) (Tomancak et al., 2002). BDGP images are available at a public webpage (http://www. fruitfly.org/insituimages/insitu_images/). Figure 6 shows examples of optimal closed contours. The optimal contours are visually consistent with the boundary of a targeting embryo. It is worth noting that the image channel space under the criterion of attention maximisation is typically effective to ‘circumvent’ the high contrast between geneexpression (dark/blue) regions and non-expression regions inside an embryo (see the last image in Figure 6). Figure 7 shows examples of optimal contours that are open and successfully restored contours. Figure 8 shows three failure examples of the proposed approach. In case (a), the active contour technique does not localise the boundary of an embryo correctly. In case (b), the open-point detection algorithm fails to detect open points since the contour fills every angular bin (from 0° to 359°). In case (c), the detection algorithm fails to detect correct open points due to the large intersection between the targeting embryo and a neighbouring embryo. To have a quantified evaluation of the proposed method, we define a localisation result to be successful if the average displacement between a final contour and the ground truth (manually extracted contour) is less than 5%. Tested on the dataset of 1000 BDGP
Localisation of Drosophila embryos using active contours
165
images, the localisation accuracy achieved by the proposed method is 89%. As a comparison, the straight application of Chan-Vese active contour method only achieved an accuracy of 62%. We also applied a distance regularised level set evolution (DRLSE) method (Li et al., 2010) to the dataset, and got an accuracy of 56%, which showed the limitation of level set methods with shape constraints in the context of contour extraction of Drosophila embryonic images. As a conclusion, experimental results show the effectiveness of processing channel spaces, and algorithms to process open contours. In the future, we plan to tune the proposed method to extract gene expression regions in embryonic images. Figure 6
Optimal closed active contours (see online version for colours)
Figure 7
Successful examples of the proposed restoration algorithm. (a1/a2) Open contours. (b1/b2) Restored contours (see online version for colours)
166 Figure 8
Q. Li and S.S. Ananta Failure cases of the proposed approach (see online version for colours)
Acknowledgements This work was supported by National Science Foundation Grant IIS-1016668.
References Ananta, S. (2012) Contour Extraction of Drosophila Embryos using Active Contours in Scale Space, Master Thesis, Western Kentucky University. Arbelaez, P., Maire, M., Fowlkes, C. and Malik, J. (2011) ‘Contour detection and hierarchical image segmentation’, IEEE Trans. Pattern Anal. Mach. Intell., Vol. 33, No. 5, pp.898–916. Boykov, Y. and Funka-Lea, G. (2006) ‘Graph cuts and efficient N-D image segmentation’, International Journal of Computer Vision, Vol. 70, No. 2, pp.109–131. Chan, T.F. and Vese, L.A. (2001) ‘Active contours without edges’, IEEE Transactions on Image Processing, Vol. 10, pp.266–277. Frise, E., Hammonds, A.S. and Celniker, S.E. (2010) ‘Systematic image-driven analysis of the spatial drosophila embryonic expression landscape’, Molecular Systems Biology, Vol. 6, No. 345, doi:10.1038/msb.2009.102. Fussenegger, M., Roth, P.M., Bischof, H., Deriche, R. and Pinz, A. (2009) ‘A level set framework using a new incremental, robust Active Shape Model for object segmentation and tracking’, Image Vision Comput., Vol. 27, No. 8, pp.1157–1168. Gargesha, M., Yang, J., Emden, B.V., Panchanathan, S. and Kumar, S. (2005) ‘Automatic annotation techniques for gene expression images of the fruit fly embryo’, Visual Communications and Image Processing, in Li, S., Pereira, F., Shum, H-Y. and Tescher, A.G. (Eds.): Proceedings of the SPIE, Vol. 5960, pp.576–583. Grady, L. and Alvino, C.V. (2009) ‘The piecewise smooth mumford-shah functional on an arbitrary graph’, IEEE Transactions on Image Processing, Vol. 18, No. 11, pp.2547–2561. Kass, M., Witkin, A.P. and Terzopoulos, D. (1988) ‘Snakes: active contour models’, International Journal of Computer Vision, Vol. 1, No. 4, pp.321–331. Kennedy, R., Gallier, J.H. and Shi, J. (2011) ‘Contour cut: Identifying salient contours in images by solving a Hermitian eigenvalue problem’, Proceedings of Computer Vision and Pattern Recognition, pp.2065–2072. Li, C., Xu, C., Gui, C. and Fox, M.D. (2010) ‘Distance regularized level set evolution and its application to image segmentation’, IEEE Transactions on Image Processing, Vol. 19, No. 12, pp.3243–3254.
Localisation of Drosophila embryos using active contours
167
Li, Q. and Kambhamettu, C. (2011) ‘Contour extraction of drosophila embryos’, IEEE/ACM Trans. Comput. Biology Bioinform., Vol. 8, No. 6, pp.1509–1521. Mace, D.L., Varnado, N., Zhang, W., Frise, E. and Ohler, U. (2010) ‘Extraction and comparison of gene expression patterns from 2d RNA in situ hybridization images’, Bioinformatics, Vol. 26, No. 6, pp.761–769. Mumford, D. and Shah, J. (1989) ‘Optimal approximation by piecewise smooth functions and associated variational problems’, Commun. Pure Appl. Math., Vol. 42, pp.577–685. Pan, J.Y., Balan, A.G.R., Xing, E.P., Traina, A.J.M. and Faloutsos, C. (2006) ‘Automatic mining of fruit fly embryo images’, ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.693–698. Peng, H. and Myers, E.W. (2004) ‘Comparing n situ mRNA expression patterns of drosophila embryos’, RECOMB, pp.157–166. Puniyani, K., Faloutsos, C. and Xing, E.P. (2010) ‘Spex2: Automated concise extraction of spatial gene expression patterns from fly embryo ISH images’, Bioinformatics, Vol. 26, No. 12, pp.i47–i56. Salah, M.B., Mitiche, A. and Ayed, I.B. (2011) ‘Multiregion image segmentation by parametric kernel graph cuts’, IEEE Transactions on Image Processing, Vol. 20, No. 2, pp.545–557. Shi, J. and Malik, J. (2000) ‘Normalized cuts and image segmentation’, IEEE Trans. Pattern Anal. Mach. Intell., Vol. 22, No. 8, pp.888–905. Sinop, A.K. and Grady, L. (2007) ‘A seeded image segmentation framework unifying graph cuts and random walker which yields a new algorithm’, International Conference on Computer Vision, pp.1–8. Tao, W., Jin, H. and Zhang, Y. (2007) ‘Color image segmentation based on mean shift and normalized cuts’, IEEE Transactions on Systems, Man, and Cybernetics, Part B, Vol. 37, No. 5, pp.1382–1389. Tomancak, P., Beaton, A., Weiszmann, R., Kwan, E., Shu, S., Lewis, S.E., Richards, S., Ashburner, M., Hartenstein, V., Celniker, S.E. and Rubin, G.M. (2002) ‘Systematic determination of patterns of gene expression during drosophila embryogenesis’, Genome Biology, Vol. 3, No. 12, pp.1–14. Zhu, Q., Song, G. and Shi, J. (2007) ‘Untangling cycles for contour grouping’, International Conference on Computer Vision, pp.1–8.
