AUTOMATED IMAGE ALIGNMENT AND SEGMENTATION TO FOLLOW PROGRESSION OF GEOGRAPHIC ATROPHY IN AGE-RELATED MACULAR DEGENERATION DAVID J. RAMSEY, MD, PHD, MPH,* JANET S. SUNNESS, MD,† POORVA MALVIYA, MA,‡ CAROL APPLEGATE, BS,† GREGORY D. HAGER, PHD,‡ JAMES T. HANDA, MD* Purpose: To develop a computer-based image segmentation method for standardizing the quantification of geographic atrophy (GA). Methods: The authors present an automated image segmentation method based on the fuzzy c-means clustering algorithm for the detection of GA lesions. The method is evaluated by comparing computerized segmentation against outlines of GA drawn by an expert grader for a longitudinal series of fundus autofluorescence images with paired 30° color fundus photographs for 10 patients. Results: The automated segmentation method showed excellent agreement with an expert grader for fundus autofluorescence images, achieving a performance level of 94 ± 5% sensitivity and 98 ± 2% specificity on a per-pixel basis for the detection of GA area, but performed less well on color fundus photographs with a sensitivity of 47 ± 26% and specificity of 98 ± 2%. The segmentation algorithm identified 75 ± 16% of the GA border correctly in fundus autofluorescence images compared with just 42 ± 25% for color fundus photographs. Conclusion: The results of this study demonstrate a promising computerized segmentation method that may enhance the reproducibility of GA measurement and provide an objective strategy to assist an expert in the grading of images. RETINA 34:1296–1307, 2014
A
Western industrialized countries.1,2 Recent advances in anti–vascular endothelial growth factor therapy have revolutionized the treatment of neovascular disease.3 However, nearly because many patients have severe central vision loss from geographic atrophy (GA), the advanced form of atrophic (dry) AMD is with almost one million individuals exhibiting GA in at least one eye. High-resolution color fundus photographs (CFPs) and fundus autofluorescence (FAF) imaging are clinically established diagnostic and documentary tools for monitoring GA associated with dry AMD.4–10 Most epidemiologic and natural history studies have relied on the tracing of retinal atrophy in CFPs with careful reference to retinal landmarks, including retinal blood vessels and the optic nerve, and using computerized planimetry to record the size of GA.5,8–10 Increasingly, FAF imaging, a noninvasive technique for mapping
ge-related macular degeneration (AMD) remains the leading cause of irreversible central visual loss among individuals of 60 years of age and older in
From the *Wilmer Eye Institute, Johns Hopkins School of Medicine, Johns Hopkins University, Baltimore, Maryland; †Hoover Rehabilitation Low Vision Services, Greater Baltimore Medical Center, Baltimore, Maryland; and ‡Department of Computer Science, Johns Hopkins University, Baltimore, Maryland. Supported in part by the National Institutes of Health (EY08552 JSS), Research to Prevent Blindness (Wilmer), and Johns Hopkins Internal funds. None of the authors have any financial/conflicting interests to disclose. Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www.retinajournal.com). J. T. Handa is the Robert Bond Welch Professor. Reprint requests: James T. Handa, MD, Room 3015, Smith Building, 400 North Broadway, Baltimore, MD 21287; e-mail: [email protected]
1296
AUTOMATED SEGMENTATION OF GEOGRAPHIC ATROPHY RAMSEY ET AL
naturally or pathologically occurring fluorescence in the ocular fundus, has been used to follow the progression of GA.6,7 A loss of autofluorescence is a consequence of retinal pigment epithelium atrophy and loss of lipofuscin. Fundus autofluorescence recently has been used in FDA-approved clinical trials of GA,11–15 including large-scale treatment trials of GA, such as the Geographic Atrophy Treatment Evaluation (GATE) study.13 Because there is presently no effective treatment for GA,1,2,4,5,16 accurate quantification of the shape and extent of GA lesions over time is vital to a successful clinical trial aimed at detecting differences in the progression of atrophy and will require large numbers of patients with longitudinal data to detect meaningful changes.16 Currently, quantification of atrophy in serial photographs by expert graders is time consuming and prone to inter- and intra-observer variability, differing between graders and imaging modalities.4,8–10 Even expert graders at accredited reading centers have difficulty reproducing the measurement of atrophic areas because of irregular lesion borders, multifocal foci of GA, variability in fundus pigmentation, changes in the optical media over time, and the presence of drusen.4,15,17 Variation in the imaging parameters and equipment between institutions may also impede serial diagnostic interpretation of images by introducing differences in exposure, alignment, focus, magnification, and other optical aberrations.9 The systematic monitoring of GA progression with the aid of serial image alignment and computer-assisted quantification methods could improve the reproducibility of human graders, thereby saving time, reducing cost, and improving overall precision. Computer-assisted segmentation, that is, the partitioning of an image into objects or boundaries to aid in interpretation, could overcome some of the challenges in GA quantification but remains an unsolved strategy in computer vision. Computational techniques have been applied to quantify drusen progression,18–21 choroidal neovascularization,22 and GA in CFPs23 and FAF images.7,15,24,25 Although many different techniques have been used to delineate retinal lesions, including statistical classification,26 dynamic clustering of contrast and threshold information,18 segmenting by morphology,27 and a combination of recursive region growing and adaptive thresholding,15,28 each has its limitations. We became interested in an interactive fuzzy c-means (FCM) clustering-based method for the segmentation of GA lesions paired with CFPs and FAF images because it is objective, unsupervised, and always converges,29 which can remove inter-reader variability and will identify GA using objective criteria. Fuzzy c-means is a class-based soft segmentation
1297
method frequently used in pattern recognition that allows one piece of data to belong to two or more clusters.29,30 Oversegmentation is a recognized shortcoming of FCM-based segmentation. To overcome this potential problem, we allowed the expert to guide the segmentation process by first selecting a region of interest (ROI) uniquely drawn for each patient, followed by an interactive candidate region selection process for each image.31 Selection of an ROI has been successfully used in lesion segmentation in contrast-enhanced magnetic resonance images.32 Color fundus photograph and FAF are commonly used to monitor GA in clinical practice. The purpose of this investigation is to develop a computer-based image alignment and segmentation method as a tool for standardizing the quantification of GA in CFPs and FAFs images. We evaluate our method by comparing computerized segmentation against expert outlines on a clinical database of GA lesions. We also compare the proposed algorithm between color and FAF images. The results of this study demonstrate a promising computerized segmentation method that may enhance the precision, reproducibility, and speed of GA measurement, and could assist an expert in the manual grading of images. These methods could be helpful in planning future treatment trials for GA by helping to standardize the quantification of GA or with the clinician who is trying to determine whether a patient has progression of GA. Methods Image Library Longitudinal series of 30° CFPs matched with FAF images were selected from a database of patients from the Fenretinide study.33 The subset of patients used for our study was among the subgroup of patients with bilateral GA and no choroidal neovascularization, confirmed by fluorescein angiography performed at baseline. All patients met the criteria for advanced AMD, having GA of at least 500 mm in diameter with visible choroidal vessels and no neovascular disease.34 Ten patients, each with an average of three image pairs taken at baseline, Year 1, and Year 2 were selected for the analysis. All 10 patients had GA with varying degrees of foveal involvement; 4 had a solid area of atrophy, 2 multifocal, and 4 a horseshoe configuration with varying degrees of central sparing. One patient with an infrared (IR) image paired with a corresponding color photograph is also presented. In this data set, one patient had only 2 years of imaging data available, two patients were missing FAF images in Years 2 and 3 of the study, respectively, and one FAF image was
1298 RETINA, THE JOURNAL OF RETINAL AND VITREOUS DISEASES
of such poor quality that it could not be manually graded. The Johns Hopkins University and Greater Baltimore Medical Center Institutional Review Boards reviewed and approved the study. Development of Ground Truth An expert grader (J.S.S.) outlined the GA in the best macular image for each modality with careful reference to retinal landmarks, including retinal blood vessels and the optic nerve,4 to identify the area of GA in each photograph using IMAGEnet R4 (Topcon Medical Systems, Inc, Tokyo, Japan). The manual computerized planimetry drawings were then converted to binary TIFFs in Adobe Photoshop CS4 (Adobe Systems, Inc, San Jose, CA) and used as ground truth maps for software development. Image Registration Serial image registration compensates for differences in photograph alignment, focus, magnification, and other optical aberrations, both for images taken at different time points or with different imaging modalities. This not only provides a clearer visual representation of evolving fundus features for the clinician to evaluate but also to allow a more precise measure of GA progression over time. Registering images to the same coordinate plane also permits “flicker” analysis of the serial images, whereby a series of aligned images are alternately displayed in rapid sequence to the observer to accentuate the areas of the image that change over time or between modalities.35,36 We aligned our longitudinal data sets of paired images for each patient with i2k Align Retina (DualAlign LLC, Clifton Park, NY; see Figure, Supplemental Digital Content 1, http://links.lww.com/IAE/A213), a commercially available alignment program, and an identical alignment transformation was performed on each corresponding GA map drawn by the expert grader (Figure 1). Using a proprietary image registration and recognition algorithm, this program is capable of registering almost any set of two-dimensional images, finding common structures in the images, and ignoring regions that vary. In the case of fundus images, the retinal arcades provide a built-in, relatively static series of landmarks that allow for alignment of serial images taken from the same patient; however, there are many more subtle features that may be used by the software to refine the accuracy of image registration. Regions with differences, for example, in areas with evolving GA or remodeling of drusen deposits, are ignored as the software matches and scales the paired images. The software also uses proprietary camera and motion correction algorithms. The image alignment software generates inter-image transformations. It then
2014 VOLUME 34 NUMBER 7
refines the most promising initial estimates, and decides if resulting images align the same structures. The research version of this software also calculates a distortion factor from the transformation matrix, which provides an estimate of the impact of image alignment on the transformed image. This feature was used to generate an estimate for the impact of registration on quantification of GA in the central macula in serial fundus photographs.37 Registration of images across imaging modalities was performed with i2k Align Retina. To register FAF images to their corresponding CFPs, each pair of images was sequentially tested until the software found a match; there is enough variation in the spatial information present in CFPs and FAF images that prevents individual pair alignment in some cases. Once a pair was found that could be coregistered, all of the color images were aligned to the color image of the pair as the target. Most often, this was the first image in the patient data set. Fundus autofluorescence images for that patient were next registered to the corresponding FAF image of the pair, and then the image transformation between the FAF and corresponding CFP was performed sequentially using the CFP as the target. An identical series of alignment transformations were then performed on each corresponding GA map drawn by the expert grader, a feature found in the research version of the i2k Align Retina software. Blood Vessel Elimination Vessel borders are prone to interfere with the detection of the edge of GA.7 Blood vessel pixels have low-intensity values when compared with non-vessel pixels. We used this property to represent blood vessels as wavelets in a time–frequency representation called a continuous wavelet transform after the work by Arneodo et al.38 We used a Morlet wavelet as analyzing wavelet. A Morlet wavelet is obtained from localizing a sine wave with a Gaussian curve. It has the ability to be tuned to specific spatial frequencies, filtering out noise in the fundus images while retaining image structure. For blood vessel segmentation, the Morlet transform was computed over different orientations, and the maximum modulus value of Morlet transform was selected. This analysis was implemented in MatLab (MathWorks, Natick, MA). With this modification, the vessels attain more prominence than the background noise, which helps capture even the smallest retinal capillaries and choroidal vessels present in the fundus image.39 Figure 2 illustrates blood vessel elimination using this method, where retinal and prominent choroidal blood vessels are digitally subtracted from the green
AUTOMATED SEGMENTATION OF GEOGRAPHIC ATROPHY RAMSEY ET AL
1299
Fig. 1. i2K Align Retina aligns images across modalities. A and B. Aligned CFP and FAF image pair from a representative patient is displayed on the same coordinate plane. The degree of foveal involvement between the two imaging modalities differs substantially with additional central sparing apparent in the FAF image compared with the corresponding CFP. Each image was aligned to the first color photograph to produce a continuous series per patient. The i2k Align Retina is the only image registration program capable of correcting for image distortion between modalities and over time (not shown).
channel of a CFP (Figure 2B) and from its paired FAF image (Figure 2C). Of note, the vessel elimination works better on the color image because the vessel map is derived from this source. In the FAF image, the retinal vessels are muted but not completely eliminated from the resulting image. Optic Disk Detection and Fovea Localization Because the optic disk appears as a bright, generally circular ellipse where the retinal blood vessel network converges, it can be misidentified as GA by the segmentation algorithm because of its high-intensity level. It is therefore necessary to identify and mask the optic disk when analyzing GA. We have used the fuzzy segment model described by Hover and Goldbaum48 to find the point of convergence of the blood vessel network. To detect the optic disk location, we used the vessel map generated from the vessel elimination step and represented vessels as line segments. Every pixel in the segment model casts a vote to produce a convergence image. The pixel that has the most number of votes in the convergence image is marked as the optic disk center. Before GA segmentation in the image, the user
has the opportunity to manually adjust the location of the optic disk center and radius of the mask. We then used a vessel model approach to localize the fovea.40 This method first fits a parabolic shape to the central retinal blood vessel network using an Active Shape Model based on the calculated optic disk location. Next, it localizes the fovea from the fitted parabolic shape. We trained two separate models for right and left fundus images. After identifying the structure of central retinal blood vessel network (Figure 3A), the fovea is located 2 MPS disk areas from the optic disk center on the axis of the fitted parabola (Figure 3C). The same optic disk map and putative location of the fovea can be used for all images for a given patient after registration of all of the image pairs as described above (Figure 3D). Implementation of Fuzzy c-Means We have implemented FCM, a soft segmentation method, where an image pixel can be classified into multiple classes with different degrees of membership using MatLab (MathWorks). The FCM method was developed by Dunn30 and improved by Bezdek et al.29
Fig. 2. Example of blood vessel elimination. A. Vessel borders are prone to interfere with the detection of the edge of GA.7 Retinal and (large) choroidal blood vessels were eliminated through digital subtraction based on a vessel map generated by the maximum modulus value of the Morlet transform computed over different orientations. Thresholding the algorithm’s output generates a mask suitable for digital subtraction of the major vessels (A). B and C. Example CFP from the same patient shown in Figure 1 with the blood vessels removed from the green channel of the original image (shown in gray scale). The same vessel map can be used to subtract the blood vessels from the paired FAF image after the registration step; however, the vessels are not completely removed.
1300 RETINA, THE JOURNAL OF RETINAL AND VITREOUS DISEASES
2014 VOLUME 34 NUMBER 7
Fig. 3. Detection of the optic disk and foveal location. A. Example of a thinned blood vessel map obtained, where each blood vessel in the vessel map is condensed to a single point. B. Convergence image obtained after the application of fuzzy convergence algorithm. The brighter the region is, the more blood vessels converge in that area and the higher the probability of that pixel being the optic disk center. C. Points along the major retinal arcade vessels detected by Active Shape Model are shown in blue. The position of the fovea (marked by a green cross) is two disk areas away from the optic disk center along the axis of a parabola fit to these points. D. The original fundus photograph with the foveal center (black cross) and optic disk center (white cross), as well as a one disk area region of interest centered over the fovea (green circle) and mask superimposed over the optic disk (black disk).
In our FCM-based method, all brightness and contrast data are used as a feature vector in differentiating lesions from the background retina. Clustering, in general, allows one piece of data to belong to two or more classes or clusters. This is an important feature for medical diagnostic systems because it increases the sensitivity of segmentation. In the FCM algorithm, the degree of membership for each data point is related to the inverse of the distance from the centroid of the cluster, computed as the mean of all points, weighted by their degree of belonging to the cluster. In our implementation, we have elected to use only two classes, one for atrophy and the other for nonatrophy based on the brightness and contrast feature space of image. The FCM segmentation used K-means clustering method to obtain the initial centroid values, and the centroid values were updated with each iteration to minimize the objective function. Our method takes the cluster with maximum association value as the cluster the data associates. After segmentation, we present the segmentation results for each iteration of the algorithm graphically and asked the user to choose the best segmentation result. Interactive Geographic Atrophy Segmentation Method Automated segmentation of the GA present in CFPs and FAF images, and more importantly delineating the
border between normal and atrophic regions of retina, would save time, reduce cost, and improve overall precision of quantification of GA. Figure 4 illustrates the steps for the expert-guided automated image segmentation process. First, the retinal and (large) choroidal blood vessels are eliminated from each retinal image through digital subtraction because vessel borders are prone to interfere with detecting the edge of GA. Using the vessel map, the putative locations of the optic disk and fovea are calculated for each patient, and a mask is created to exclude the optic disk, which is a bright object prone to misclassification as atrophy. Next, the expert user draws a ROI around the portion of the central macula where the GA is to be detected. The same ROI can be used for all images in a given patient, or between patients, if the scale of images is roughly identical. Fuzzy c-means segmentation is then performed on the CFP or FAF image, generating a series of clusters of pixels. An example of the FCM segmentation result for a representative IR image is also presented (see Figure, Supplemental Digital Content 2, http://links.lww.com/IAE/A207). Next, the user is simultaneously provided with the results of several iterations of the algorithm and may select a segmentation result before the convergence of the algorithm, which generally occurred between four and nine iterations. Each cluster (area) is displayed as a separate topography overlaid on a copy of the original image in descending order of size. The user
AUTOMATED SEGMENTATION OF GEOGRAPHIC ATROPHY RAMSEY ET AL
1301
correctly identified in the FCM segmentation result, the edge of the ground truth map was identified (±3 pixels) and an ROI created. A pixel in the border ROI was scored as correctly identified if it was internally bordered by a pixel scored as atrophic and externally bordered by a pixel identified as nonatrophy. Statistical Analyses Comparisons of the mean values between aligned and unaligned areas in color images and comparisons between the GA areas in aligned paired CFP and FAF images were performed using two-sample two-sided t-tests. No imputation of missing values was performed. Statistical significance was defined as P , 0.05. All analyses were conducted using statistical analysis package, Microsoft Excel (Version 12.3.3; Redmond, WA). Results Impact of Image Registration on Geographic Atrophy Identified by an Expert Grader in Color Fundus Photographs and Fundus Autofluorescence Images
Fig. 4. Flow diagram of interactive GA segmentation method. The GA segmentation strategy consists of six consecutive stages: 1) image registration, 2) blood vessel segmentation and digital subtraction, 3) ROI selection by a human operator with reference to the imputed location of the fovea and optic disk (mask), 4) GA segmentation by the application of FCM algorithm, 5) selection of the relevant topographies by the human operator, and 6) GA quantification and border detection based on the final segmentation result. Automated steps are shown enclosed with ellipses; those requiring user input are boxed.
then selects one or more topographies that correspond to the visible GA in the image. As drusen and other fundus lesions are often small relative to regions of atrophy, this step serves as a size exclusion function because only the nine largest clusters were displayed. Finally, the resulting user-defined FCM segmentation is compared with the registered ground truth map, noting pixels correctly identified (true positives), those incorrectly identified (false positives), and those missed by the software method (false negatives). To estimate the portion of the GA border that was
The registration of the 10 patient series included in our automated segmentation study by the i2k Align Retina software provides a unique opportunity to assess the impact of automated image alignment on both the calculated GA area and mean enlargement rate of the total atrophic area. To assess the impact of technical differences in the serial imaging of GA, we first manually calculated the GA area using CFP and FAF image pairs. We then aligned the image pairs and performed an identical alignment transformation on the corresponding GA maps drawn by our expert. Image alignment did not significantly alter the area of GA measured by the expert grader, affecting the manually segmented areas by only 0.19 ± 0.42 mm2 (1.6 ± 3.0% as percent of lesion area, not significant) for CFPs and 0.69 ± 1.14 mm2 (5.4 ± 8.6% as percent of lesion area, not significant) for FAF images. This compares with an estimated impact of image alignment on the central macula of digitized CFPs reported from the Wilmer Geographic Atrophy Study of 1.0 ± 5.4%.37 Although the differences are small on the basis of lesion area, focal differences in atrophy enlargement in the macula can often account for disproportionately large changes in vision; image alignment may assist the clinician in visually detecting these subtle structural changes. No difference in the rate of atrophy enlargement was detected between the pre- and post-processed images. The mean enlargement of the total atrophic area in the
1302 RETINA, THE JOURNAL OF RETINAL AND VITREOUS DISEASES
study eye by 2 years was 2.06 ± 1.23 mm2 before versus 2.00 ± 1.19 mm2 after the image alignment for CFPs (not significant), and it was 2.40 ± 1.43 mm2 before versus 2.06 ± 1.16 mm2 after the image alignment for FAF images (not significant). Of note, the enlargement rate between the two modalities more closely agree for the aligned versus the unaligned impairs, although neither difference achieved statistical significance. Impact of Cross-Modality Registration on Geographic Atrophy Identified by an Expert Grader in Color Fundus Photographs Versus Fundus Autofluorescence Images Experts grading CFP and FAF image pairs do not always agree. Some of these differences may be overcome by advanced image registration techniques, such as those used by the i2k Align Retina platform, which may compensate for differences in image alignment, magnification, or other types of imaging artifacts that lead to distortion of the image. Although FAF imaging is extremely sensitive for the presence of GA, it is not specific for GA. For example, areas of pigmentary alteration, presumed to have retinal pigment epithelium attenuation but not GA, may have reduced FAF that may persist for years without the development of GA.41 Although there were differences in the GA identified by manual grading of CFP and FAF image pairs, the difference in the measured atrophic area between the CFP and FAF images was quite small in our patient series (1.9 ± 11.4% as percent of lesion area, not significant). After image registration by the i2k Align Retina software, the difference in lesion measurements between the two modalities was slightly, but not significantly, reduced (1.3 ± 7.0% as percent of lesion area, not significant). These results suggest that the difference observed in the GA area between paired CFPs and FAF images, quite pronounced for some patients, is unlikely to be simply the result of technical differences introduced by the image acquisition system. Automated Segmentation of Geographic Atrophy Area Using the Fuzzy c-Means Algorithm We next compared our automated system with manual grading of GA on FAF and CFPs by characterizing GA area and the GA border. The automated FCM-based segmentation detection of atrophic lesion area in retinal images showed good agreement with an expert grader for FAF images but performed less well on CFPs. Automated segmentation of FAF images achieved a performance level of 94 ± 5% sensitivity and 98 ± 2% specificity on a per-pixel basis. In contrast, the segmentation of CFPs for GA area varied greatly with
2014 VOLUME 34 NUMBER 7
a sensitivity of only 47 ± 26% and specificity of 98 ± 2%. The segmentation algorithm identified 75 ± 16% of the GA border correctly in FAF images compared with just 42 ± 25% for CFPs. On FAF images, the FCM algorithm’s high sensitivity segmented suspicious regions as atrophy rather than background. Figure 5 presents a representative case of a patient with parafoveal GA, where the foveal center has pigmentary changes and reduced fundus FAF. These pigmentary areas are incorrectly segmented as atrophy by the FCM algorithm. Although these changes are likely indicative of retinal disease, they are not atrophy. It is often difficult to delineate parafoveal atrophy in fundus FAF images because of the absorption of blue light by xanthophyll pigment in the macula. This phenomenon may contribute to our segmentation errors, particularly in these cases. Figure 6 shows a patient with a large area of fovealinvolving GA. The false-positive rate is significantly less in this case compared with the patient shown in Figure 5 who has foveal sparing. The area incorrectly identified as atrophy (falsepositive pixels) relative to the total area of GA present in the corresponding ground truth images was 9 ± 8% for the CFPs analyzed in the study. For the FAF images, the area of incorrectly identified atrophy relative to the total area of GA present in the corresponding ground truth images was 17 ± 13%. Atrophy was more likely to be missed in the segmentation of CFPs, and this difference likely accounts for why the false-positive pixels also were fewer. The significantly higher sensitivity for FAF images accounts for the relatively greater number of false-positive pixels. In either case, these errors can be manually corrected or eliminated through alternative segmentation or postprocessing methods.
Discussion Segmentation of lesions in retinal images is an area of intense investigation in many ophthalmic conditions, including AMD. In particular, future treatment trials for GA will require the accurate analysis of large numbers of patients with longitudinal data to detect meaningful changes in the atrophy progression rate.16 Automated segmentation that accurately identifies GA would greatly improve the efficiency of image processing during such a clinical trial. Furthermore, analysis software that could easily identify and measure GA progression would be a welcome addition to the clinician who is following patients with GA. The expert-guided automated image segmentation approach presented in this article is reproducible, easy to apply to serial fundus photographs across multiple
AUTOMATED SEGMENTATION OF GEOGRAPHIC ATROPHY RAMSEY ET AL
1303
Fig. 5. Fuzzy c-means segmentation results for a patient with parafoveal GA. Representative CFP (A) and FAF image (F) from a patient with parafoveal GA. Ground truth maps for GA in the same CFP (B) and FAF image (G). Corresponding segmentation result produced by the FCM detection method (C and H). Color-coded maps of the segmentation result for GA area (D and I), and GA border (E and J) that was correctly identified by the FCM algorithm (green), incorrectly identified (false positive, red), and missed by the software method (false negative, blue). Note that most of the false positives identified by the FCM method are in the central macula of the FAF image, an area rich in xanthophyll pigment, and in regions that show changes in pigmentation that are likely indicative of retinal disease.
imaging modalities, and relatively fast. The FCM algorithm always converges to the same output for a given image, making the segmentation approach automated and reproducible. The speed, sensitivity, and specificity of this method are enhanced by a simple user-drawn ROI before lesion segmentation that effectively limits processing of irrelevant portions of the image. The ROI can be applied serially to images for a given patient provided the bounding box is drawn on the most recent image, where the GA lesion would be expected to be of maximal extent. In the final segmentation step, the user selects candidate regions from the automated segmentation results for final summation to limit the problem of oversegmentation. Obvious misclassifications of background nonatrophic retina, or lesions too small to be atrophy such as drusen, can be efficiently eliminated by this added step (see Figure, Supplemental Digital Content 3, http://links.lww.com/IAE/A214). The image segmentation approach presented in this article can be applied to serial fundus photographs across multiple imaging modalities. Automated image alignment not only permits the elimination of invariant features such as retinal blood vessels but also identifies other features in a given image that might impinge upon the analysis, such as areas of variable fundus pigmentation or drusen. The cross-modality image registration presented here is an integral part of the method used for preprocessing FAF (and IR) images to allow the digital subtraction of blood vessels based on the corresponding CFPs. Importantly, image registration permits these operations to be applied simultaneously across all of the images in a patient data set. Limiting the amount of user input makes the method relatively fast and reproducible without the need for special training. Although manual segmentation
methods can take several minutes to evaluate a single fundus photograph, the software-based methods presented in this study can reduce the time needed to process an image by about 10-fold. The ability of our FCM-based segmentation method to correctly identify the GA lesion border is a unique feature of our analysis. At first blush, our results may not seem to stand out given that only three-quarters of the GA border was correctly identified for FAF images and less than half for CFPs. However, no comparable numbers are available in the literature, because previous studies have not reported this more stringent index of performance. This is especially relevant for clinical trials, where a high level of precision is required and because the time and effort required by an expert grader to manually validate and correct a softwarebased segmentation result is directly proportional to the extent that lesion borders are correctly identified. Furthermore, GA is most likely to advance from a lesion edge than from de novo; we therefore believe that special attention should be placed on carefully delineating the lesion border in serial image analysis. Further development of this segmentation method, and its validation in a clinical setting, promises to significantly improve the precision of GA measurement and reduce the effort involved while grading images. Few automated image analysis techniques have been developed for the segmentation of GA; most approaches for segmenting GA rely on interactive segmentation schemes for GA. Köse et al23 used texture segmentation to automate the identification of GA in CFPs. However, in this study, the threshold value for texture segmentation, determined by supervised experiments, was manually adjusted to improved the results in some cases. In particular, this method can fail to give good results for GA segmentation in
1304 RETINA, THE JOURNAL OF RETINAL AND VITREOUS DISEASES
2014 VOLUME 34 NUMBER 7
Fig. 6. Fuzzy c-means segmentation results for a patient with foveal-involving GA. Representative serial CFPs (A–C) and FAF images (D–F) from a patient with central GA. Ground truth maps for GA in CFP (G–I) and FAF images (J–M). Color-coded maps of the segmentation result for GA area (M–R) and GA border (S–X), that was correctly identified by the FCM algorithm (green), incorrectly identified (false positive, red), and missed by the software method (false negative, blue). Note that bright signals from prominent calcific drusen visible in the CFP are preferentially segmented by the FCM method. These features are silent in the FAF image, where the FCM algorithm does a better job of identifying the atrophy. However, in the FAF image, a small satellite focus of GA was not correctly segmented from the background nonatrophic retina. Finally, the outline of the FCM segmentation result is displayed on the original FAF images (Y–Z, ZZ) to illustrate the close agreement between the GA border and the appearance of GA on the FAF image.
images that have low contrast between healthy and atrophic regions. Cluster-based analysis, such as the FCM segmentation method, partially overcomes such difficulties by virtue of weighting regions in relation to their neighbors. Another semiautomated segmentation strategy developed by Hwang et al7 used a user-defined threshold applied to a Gaussian-filtered original FAF image that was supplemented by manual segmentation. Additional image preprocessing such as color normalization and contrast enhancement31,42 could also be applied to our FCM segmentation method; but in our experience, these adjustments tend to accentuate background variations as much if not more than the GA lesions, resulting in oversegmentation of the image by the FCM algorithm (data not shown). More recently, Schmitz-Valckenberg et al15 presented a so-called region-growing algorithm for the
quantification of GA in FAF images. In this method, trained graders define one or more GA centers (seedpoints) and then manually adjust a user-defined threshold value for each until the resulting area “grows” to encompass the atrophy. A similar approach is available as a proprietary software tool from Heidelberg Engineering known as RegionFinder.43 This method has good intra- and inter-observer agreement and has been validated in a clinical trial of the concordance rate of GA44; but unlike the FCM segmentation method, it is not automated and can only analyze regions sequentially, meaning that it takes considerably more effort to analyze a patient with multiple foci of GA compared with one with a solitary lesion. In addition, areas of atrophy may be overlooked, something unlikely to happen in a method that relies on data clustering, like the FCM-based method.
AUTOMATED SEGMENTATION OF GEOGRAPHIC ATROPHY RAMSEY ET AL
Finally, Devisetti et al24 recently presented the first successful application of a neural network classifier in performing segmentation of GA in IR and FAF images of eight eyes belonging to six patients with GA. They used a supervised learning approach to train a classifier provided with seven image features derived from both the FAF and IR images that included both intensity and regional intensity data. Their approach performed reasonably well, achieving a sensitivity of 82.5% and specificity of 92.9% as measured on a per-pixel basis and averaged across all the input image pairs. Our FCM-based classifier performs reasonably well compared with this method and is computationally less intensive. In the future, candidate regions could be intelligently classified in an automated fashion using a simple machine learning approach such as a neural network.42 If such a method were implemented, the full power of the FCM approach could be used with more than two classes to better differentiate GA from the surrounding normal retina. This would require a much larger data set for training of a classifier. Although clinically both CFPs and FAF images are used to characterize GA, the relative ease of defining GA by FAF and the wide availability of FAF along with spectral domain optical coherence tomography using the Heidelberg Spectralis, most clinical trials currently use FAF to quantify GA enlargement.6,7,11,12,14,15 Improved analytical techniques using spectral domain optical coherence tomography45 or new technologies like polarization sensitive optical coherence tomography46,47 will likely make these other important ways of quantifying progression of GA.45 The ability to characterize the extent of foveal involvement, one of the limitations of FAF imaging, is enhanced by the use of IR reflectance images,15 in addition to spectral domain optical coherence tomography findings. Most current trials are not using CFP for this purpose. Our results also highlight the difficulty with quantifying GA changes using CFP. This study suggests that FAF imaging may slightly underestimate the GA area relative to CFP, but the enlargement rates of the two modalities are not significantly different, and the impact of image alignment is small. Also, of note is that the enlargement rates using both modalities in this study are close to the rates found by Sunness,16 independent of the modality used. Thus, we suggest that the use of FAFs does not explain the somewhat lower enlargement rates found in the FAM study and others studies.25 In our sample population, we also measured the differences in the GA area between CFP and FAF image pairs. The proprietary image alignment algorithms implemented in the i2k Align Retina software program perform point-by-point image registration correcting for nonlinear differences between images,
1305
including across imaging modalities. We found only a modest effect of image alignment on the measurement GA area and mean enlargement rate of the total atrophic area, and correction for image alignment by itself, did not fully account for the difference in the GA area between the two imaging modalities. Therefore, the differences in the measurements between the CFP and FAF images pairs appear to be due to the differences in what each image modality highlights rather than from flaws in our technology. We recognize that our study was not powered to answer whether CFPs or FAFs better quantify GA area. However, the results do suggest that the application of cross-modality image registration coupled with computer-assisted segmentation could help better define the differences between color and FAF images. Further work examining the differences between GA quantification between imaging modalities is needed to determine the best method for following GA, balancing the sensitivity of the method with its specificity for lesion identification. We have developed an interactive computer-based image segmentation method for the serial measurements of GA lesions in CFPs and FAF images. Although image alignment did not have a significant impact on the measured atrophy area, image registration is a clinically useful tool in monitoring the progression of atrophy in a given patient. The segmentation method achieves a high level of sensitivity and specificity. However, the accuracy, particularly in identification of the lesion border, is not sufficient to replace the expert grader. The results of this study are proof of principle that automated segmentation may enhance the speed, precision, and reproducibility of GA measurement in an unbiased fashion that could assist an expert in the manual grading of images. These methods are potentially useful to the practicing clinician, as well as to natural history studies, and future treatment trials designed, to evaluate new pharmacologic agents that limit the progression of GA. Key words: age-related macular degeneration, automated image segmentation, fundus autofluorescence, geographic atrophy. Acknowledgment The authors thank Dr. Charles Stewart (Rensselaer Polytechnic Institute and DualAlign LLC [Founder and Chief Scientist]) and Dr. Gary Yang (DualAlign LLC) for custom image alignment software and support. References 1. Friedman DS, O’Colmain BJ, Muñoz B, et al. Prevalence of agerelated macular degeneration in the United States. Arch Ophthalmol 2004;122:564–572. doi:10.1001/archopht.122.4.564.
1306 RETINA, THE JOURNAL OF RETINAL AND VITREOUS DISEASES 2. Rein DB, Wittenborn JS, Zhang X, et al. Forecasting agerelated macular degeneration through the year 2050: the potential impact of new treatments. Arch Ophthalmol 2009; 127:533–540. doi:10.1001/archophthalmol.2009.58. 3. Bressler NM, Doan QV, Varma R, et al. Estimated cases of legal blindness and visual impairment avoided using ranibizumab for choroidal neovascularization: non-Hispanic white population in the United States with age-related macular degeneration. Arch Ophthalmol 2011;129:709–717. doi: 10.1001/archophthalmol.2011.140. 4. Sunness JS, Bressler NM, Tian Y, et al. Measuring geographic atrophy in advanced age-related macular degeneration. Invest Ophthalmol Vis Sci 1999;40:1761–1769. 5. Sunness JS, Gonzalez-Baron J, Applegate CA, et al. Enlargement of atrophy and visual acuity loss in the geographic atrophy form of age-related macular degeneration. Ophthalmology 1999;106:1768–1779. doi:10.1016/S0161-6420(99)90340-8. 6. Holz FG, Bellman C, Staudt S, et al. Fundus autofluorescence and development of geographic atrophy in age-related macular degeneration. Invest Ophthalmol Vis Sci 2001;42:1051–1056. 7. Hwang JC, Chan JW, Chang S, Smith RT. Predictive value of fundus autofluorescence for development of geographic atrophy in age-related macular degeneration. Invest Ophthalmol Vis Sci 2006;47:2655–2661. doi:10.1167/iovs.05-1027. 8. Klein ML, Ferris FL 3rd, Francis PJ, et al. Progression of geographic atrophy and genotype in age-related macular degeneration. Ophthalmology 2010;117:1554–1559, 1559 e1551. doi: 10.1016/j.ophtha.2009.12.012. 9. Lindblad AS, Lloyd PC, Clemons TE, et al. Change in area of geographic atrophy in the Age-Related Eye Disease Study: AREDS report number 26. Arch Ophthalmol 2009;127: 1168–1174. doi:10.1001/archophthalmol.2009.198. 10. Maguire P, Vine AK. Geographic atrophy of the retinal pigment epithelium. Am J Ophthalmol 1986;102:621–625. 11. Intravitreal LFG316 in Patients With Age-related Macular Degeneration (AMD). Available at: http://clinicaltrials.gov/ show/NCT01527500. Accessed June 25, 2012. 12. A Multicenter, Proof-of-Concept Study of Intravitreal AL-78898A In Patients With Geographic Atrophy (GA) Associated With Age-Related Macular Degeneration (AMD). Available at: http://clinicaltrials.gov/show/NCT01603043. Accessed June 25, 2012. 13. Geographic Atrophy Treatment Evaluation (GATE). Available at: http://clinicaltrials.gov/ct2/show/NCT00890097. Accessed June 25, 2012. 14. Efficacy, Safety and Tolerability Study of RN6G in Subjects With Geographic Atrophy Secondary to Age-related Macular Degeneration. Available at: http://clinicaltrials.gov/ct2/show/ NCT01577381. Accessed June 25, 2012. 15. Schmitz-Valckenberg S, Brinkmann CK, Alten F, et al. Semiautomated image processing method for identification and quantification of geographic atrophy in age-related macular degeneration. Invest Ophthalmol Vis Sci 2011;52:7640– 7646. doi:10.1167/iovs.11-7457. 16. Sunness JS, Applegate CA, Bressler NM, Hawkins BS. Designing clinical trials for age-related geographic atrophy of the macula: enrollment data from the geographic atrophy natural history study. Retina 2007;27:204–210. doi:10.1097/01. iae.0000248148.56560.b1. 17. Scholl HP, Dandekar SS, Peto T, et al. What is lost by digitizing stereoscopic fundus color slides for macular grading in age-related maculopathy and degeneration? Ophthalmology 2004;111:125–132. doi:10.1016/j.ophtha.2003.05.003.
2014 VOLUME 34 NUMBER 7
18. Hsu W, Pallawala PMDS, Lee ML, Au-Eong KG. The Role of domain knowledge in the detection of retinal hard exudates. Proc IEEE Computer Vis Pattern Recognition II-246-II-251 2001. 19. Ben Sbeh Z, Cohen LD, Mimoun G, Coscas G. A new approach of geodesic reconstruction for drusen segmentation in eye fundus images. IEEE Trans Med Imaging 2001;20: 1321–1333. doi:10.1109/42.974927. 20. Kankanahalli S, Burlina PM, Wolfson Y, et al. Automated classification of severity of age-related macular degeneration from fundus photographs. Invest Ophthalmol Vis Sci 2013;54: 1789–1796. doi:10.1167/iovs.12-10928. 21. Rapantzikos K, Zervakis M, Balas K. Detection and segmentation of drusen deposits on human retina: potential in the diagnosis of age-related macular degeneration. Med Image Anal 2003;7:95–108. 22. Chakravarthy U, Walsh AC, Muldrew A, et al. Quantitative fluorescein angiographic analysis of choroidal neovascular membranes: validation and correlation with visual function. Invest Ophthalmol Vis Sci 2007;48:349–354. doi: 10.1167/iovs.06-0493. 23. Köse C, Sevik U, Gencalioglu O, et al. A statistical segmentation method for measuring age-related macular degeneration in retinal fundus images. J Med Syst 2010;34:1–13. 24. Devisetti K, Karnowski TP, Giancardo L, et al. Geographic atrophy segmentation in infrared and autofluorescent retina images using supervised learning. Conf Proc IEEE Eng Med Biol Soc 2011;2011:3958–3961. doi:10.1109/IEMBS.2011.6090983. 25. Schmitz-Valckenberg S, Bindewald-Wittich A, DolarSzczasny J, et al. Correlation between the area of increased autofluorescence surrounding geographic atrophy and disease progression in patients with AMD. Invest Ophthalmol Vis Sci 2006;47:2648–2654. doi:10.1167/iovs.05-0892. 26. Wang H, Hsu W, Goh KG, Lee KG. Proceedings of IEEE Conf. On computer vision and pattern recognition. Proceedings IEEE Conf Computer Vision Pattern Recognition, 2000;181-186. 27. Walter T, Klein JC. Automatic Detection of Microaneurysms in Color Fundus Images of the Human Retina by Means of the Bounding Box Closing. Vol 2526. Berlin, Germany: Springer, 2002:210–220. 28. Usher D, Dumskyj M, Himaga M, et al. Automated detection of diabetic retinopathy in digital retinal images: a tool for diabetic retinopathy screening. Diabet Med 2004;21:84–90. 29. Bezdek JC, Trivedi M, Ehrlich R, Full W. Fuzzy clustering: a new approach for geostatistical analysis. Interntional J Syst Meas Decis 1981;1-2:13–24. 30. Dunn J. Well separated clusters and optimal fuzzy partitions. J Cybernetics 1974;95–104. 31. Osareh A, Shadgar B, Markham R. A computational-intelligencebased approach for detection of exudates in diabetic retinopathy images. IEEE Trans Inf Technol Biomed 2009;13:535–545. doi: 10.1109/TITB.2008.2007493. 32. Chen W, Giger ML, Bick U. A fuzzy c-means (FCM)-based approach for computerized segmentation of breast lesions in dynamic contrast-enhanced MR images. Acad Radiol 2006;13: 63–72. doi:10.1016/j.acra.2005.08.035. 33. Mata NL, Lichter JB, Vogel R, et al. Investigation of oral fenretinide for treatment of geographic atrophy in age-related macular degeneration. Retina 2013;33:498–507. doi:10.1097/ IAE.0b013e318265801d. 34. Age-Related Eye Disease Study Research Group. The AgeRelated Eye Disease Study (AREDS): design implications. AREDS report no. 1. Control Clin Trials 1999;20:573–600.
AUTOMATED SEGMENTATION OF GEOGRAPHIC ATROPHY RAMSEY ET AL 35. Heijl A, Bengtsson B. Diagnosis of early glaucoma with flicker comparisons of serial disc photographs. Invest Ophthalmol Vis Sci 1989;30:2376–2384. 36. You QS, Xu L, Jonas JB, et al. Change in choroidal nevi during a 5-year follow-up study: the Beijing Eye Study. Br J Ophthalmol 2010;94:575–578. doi:10.1136/bjo.2009.165720. 37. Ramsey DJ, Sunness JS, Handa JT. Automated Image Alignment to Follow Progression of Geographic Atrophy in AgeRelated Macular Degeneration. In: Association for Research in Vision and Ophthalmology. Ft. Lauderdale, FL; 2011. 38. Arneodo A, Decoster N, Roux SG. A wavelet-based method for multifractal image analysis. I. Methodology and test applications on isotropic and anisotropic random rough surfaces. Eur Phys J B 2000;15:567–600. 39. Leandro JJG, Cesar RM Jr, Jelinek HF. Blood vessels segmentation in retina: preliminary assessment of the mathematical morphology and of the wavelet transform techniques. SIBGRAPI-01, Florianópolis—SC, IEEE Computer Society 2001;84–90. 40. Chutatape O. Fundus foveal localization based on vessel model. Conf Proc IEEE Eng Med Biol Soc. 2006;1:4440–4444. 41. Sunness JS, Ziegler MD, Applegate CA. Issues in quantifying atrophic macular disease using retinal autofluorescence. Retina 2006;26:666–672. doi:10.1097/01.iae.0000236472.56195.e9. 42. Osareh A, Mirmehdi M, Thomas B, Markham R. Automated identification of diabetic retinal exudates in digital colour images. Br J Ophthalmol 2003;87:1220–1223.
1307
43. Mansouri K, Tung JD, Medeiros FA, et al. Semiautomated quantification of beta-zone parapapillary atrophy using blue light fundus autofluorescence. Acta Ophthalmol 2013;91: e379–e385. doi:10.1111/aos.12057. 44. Fleckenstein M, Adrion C, Schmitz-Valckenberg S, et al. Concordance of disease progression in bilateral geographic atrophy due to AMD. Invest Ophthalmol Vis Sci 2010;51:637–642. doi:10.1167/iovs.09-3547. 45. Stopa M, Bower BA, Davies E, et al. Correlation of pathologic features in spectral domain optical coherence tomography with conventional retinal studies. Retina 2008;28:298–308. doi: 10.1097/IAE.0b013e3181567798. 46. Ahlers C, Götzinger E, Pircher M, et al. Imaging of the retinal pigment epithelium in age-related macular degeneration using polarization-sensitive optical coherence tomography. Invest Ophthalmol Vis Sci 2010;51:2149–2157. doi: 10.1167/iovs.09-3817. 47. Baumann B, Gotzinger E, Pircher M, et al. Segmentation and quantification of retinal lesions in age-related macular degeneration using polarization-sensitive optical coherence tomography. J Biomed Opt 2010;15:061704. doi:10.1117/1.3499420. 48. Hoover A, Goldbaum M. Locating the optic nerve in a retinal image using the fuzzy convergence of the blood vessels. IEEE Trans Med Imaging 2003;22:951–958.
A
Western industrialized countries.1,2 Recent advances in anti–vascular endothelial growth factor therapy have revolutionized the treatment of neovascular disease.3 However, nearly because many patients have severe central vision loss from geographic atrophy (GA), the advanced form of atrophic (dry) AMD is with almost one million individuals exhibiting GA in at least one eye. High-resolution color fundus photographs (CFPs) and fundus autofluorescence (FAF) imaging are clinically established diagnostic and documentary tools for monitoring GA associated with dry AMD.4–10 Most epidemiologic and natural history studies have relied on the tracing of retinal atrophy in CFPs with careful reference to retinal landmarks, including retinal blood vessels and the optic nerve, and using computerized planimetry to record the size of GA.5,8–10 Increasingly, FAF imaging, a noninvasive technique for mapping
ge-related macular degeneration (AMD) remains the leading cause of irreversible central visual loss among individuals of 60 years of age and older in
From the *Wilmer Eye Institute, Johns Hopkins School of Medicine, Johns Hopkins University, Baltimore, Maryland; †Hoover Rehabilitation Low Vision Services, Greater Baltimore Medical Center, Baltimore, Maryland; and ‡Department of Computer Science, Johns Hopkins University, Baltimore, Maryland. Supported in part by the National Institutes of Health (EY08552 JSS), Research to Prevent Blindness (Wilmer), and Johns Hopkins Internal funds. None of the authors have any financial/conflicting interests to disclose. Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www.retinajournal.com). J. T. Handa is the Robert Bond Welch Professor. Reprint requests: James T. Handa, MD, Room 3015, Smith Building, 400 North Broadway, Baltimore, MD 21287; e-mail: [email protected]
1296
AUTOMATED SEGMENTATION OF GEOGRAPHIC ATROPHY RAMSEY ET AL
naturally or pathologically occurring fluorescence in the ocular fundus, has been used to follow the progression of GA.6,7 A loss of autofluorescence is a consequence of retinal pigment epithelium atrophy and loss of lipofuscin. Fundus autofluorescence recently has been used in FDA-approved clinical trials of GA,11–15 including large-scale treatment trials of GA, such as the Geographic Atrophy Treatment Evaluation (GATE) study.13 Because there is presently no effective treatment for GA,1,2,4,5,16 accurate quantification of the shape and extent of GA lesions over time is vital to a successful clinical trial aimed at detecting differences in the progression of atrophy and will require large numbers of patients with longitudinal data to detect meaningful changes.16 Currently, quantification of atrophy in serial photographs by expert graders is time consuming and prone to inter- and intra-observer variability, differing between graders and imaging modalities.4,8–10 Even expert graders at accredited reading centers have difficulty reproducing the measurement of atrophic areas because of irregular lesion borders, multifocal foci of GA, variability in fundus pigmentation, changes in the optical media over time, and the presence of drusen.4,15,17 Variation in the imaging parameters and equipment between institutions may also impede serial diagnostic interpretation of images by introducing differences in exposure, alignment, focus, magnification, and other optical aberrations.9 The systematic monitoring of GA progression with the aid of serial image alignment and computer-assisted quantification methods could improve the reproducibility of human graders, thereby saving time, reducing cost, and improving overall precision. Computer-assisted segmentation, that is, the partitioning of an image into objects or boundaries to aid in interpretation, could overcome some of the challenges in GA quantification but remains an unsolved strategy in computer vision. Computational techniques have been applied to quantify drusen progression,18–21 choroidal neovascularization,22 and GA in CFPs23 and FAF images.7,15,24,25 Although many different techniques have been used to delineate retinal lesions, including statistical classification,26 dynamic clustering of contrast and threshold information,18 segmenting by morphology,27 and a combination of recursive region growing and adaptive thresholding,15,28 each has its limitations. We became interested in an interactive fuzzy c-means (FCM) clustering-based method for the segmentation of GA lesions paired with CFPs and FAF images because it is objective, unsupervised, and always converges,29 which can remove inter-reader variability and will identify GA using objective criteria. Fuzzy c-means is a class-based soft segmentation
1297
method frequently used in pattern recognition that allows one piece of data to belong to two or more clusters.29,30 Oversegmentation is a recognized shortcoming of FCM-based segmentation. To overcome this potential problem, we allowed the expert to guide the segmentation process by first selecting a region of interest (ROI) uniquely drawn for each patient, followed by an interactive candidate region selection process for each image.31 Selection of an ROI has been successfully used in lesion segmentation in contrast-enhanced magnetic resonance images.32 Color fundus photograph and FAF are commonly used to monitor GA in clinical practice. The purpose of this investigation is to develop a computer-based image alignment and segmentation method as a tool for standardizing the quantification of GA in CFPs and FAFs images. We evaluate our method by comparing computerized segmentation against expert outlines on a clinical database of GA lesions. We also compare the proposed algorithm between color and FAF images. The results of this study demonstrate a promising computerized segmentation method that may enhance the precision, reproducibility, and speed of GA measurement, and could assist an expert in the manual grading of images. These methods could be helpful in planning future treatment trials for GA by helping to standardize the quantification of GA or with the clinician who is trying to determine whether a patient has progression of GA. Methods Image Library Longitudinal series of 30° CFPs matched with FAF images were selected from a database of patients from the Fenretinide study.33 The subset of patients used for our study was among the subgroup of patients with bilateral GA and no choroidal neovascularization, confirmed by fluorescein angiography performed at baseline. All patients met the criteria for advanced AMD, having GA of at least 500 mm in diameter with visible choroidal vessels and no neovascular disease.34 Ten patients, each with an average of three image pairs taken at baseline, Year 1, and Year 2 were selected for the analysis. All 10 patients had GA with varying degrees of foveal involvement; 4 had a solid area of atrophy, 2 multifocal, and 4 a horseshoe configuration with varying degrees of central sparing. One patient with an infrared (IR) image paired with a corresponding color photograph is also presented. In this data set, one patient had only 2 years of imaging data available, two patients were missing FAF images in Years 2 and 3 of the study, respectively, and one FAF image was
1298 RETINA, THE JOURNAL OF RETINAL AND VITREOUS DISEASES
of such poor quality that it could not be manually graded. The Johns Hopkins University and Greater Baltimore Medical Center Institutional Review Boards reviewed and approved the study. Development of Ground Truth An expert grader (J.S.S.) outlined the GA in the best macular image for each modality with careful reference to retinal landmarks, including retinal blood vessels and the optic nerve,4 to identify the area of GA in each photograph using IMAGEnet R4 (Topcon Medical Systems, Inc, Tokyo, Japan). The manual computerized planimetry drawings were then converted to binary TIFFs in Adobe Photoshop CS4 (Adobe Systems, Inc, San Jose, CA) and used as ground truth maps for software development. Image Registration Serial image registration compensates for differences in photograph alignment, focus, magnification, and other optical aberrations, both for images taken at different time points or with different imaging modalities. This not only provides a clearer visual representation of evolving fundus features for the clinician to evaluate but also to allow a more precise measure of GA progression over time. Registering images to the same coordinate plane also permits “flicker” analysis of the serial images, whereby a series of aligned images are alternately displayed in rapid sequence to the observer to accentuate the areas of the image that change over time or between modalities.35,36 We aligned our longitudinal data sets of paired images for each patient with i2k Align Retina (DualAlign LLC, Clifton Park, NY; see Figure, Supplemental Digital Content 1, http://links.lww.com/IAE/A213), a commercially available alignment program, and an identical alignment transformation was performed on each corresponding GA map drawn by the expert grader (Figure 1). Using a proprietary image registration and recognition algorithm, this program is capable of registering almost any set of two-dimensional images, finding common structures in the images, and ignoring regions that vary. In the case of fundus images, the retinal arcades provide a built-in, relatively static series of landmarks that allow for alignment of serial images taken from the same patient; however, there are many more subtle features that may be used by the software to refine the accuracy of image registration. Regions with differences, for example, in areas with evolving GA or remodeling of drusen deposits, are ignored as the software matches and scales the paired images. The software also uses proprietary camera and motion correction algorithms. The image alignment software generates inter-image transformations. It then
2014 VOLUME 34 NUMBER 7
refines the most promising initial estimates, and decides if resulting images align the same structures. The research version of this software also calculates a distortion factor from the transformation matrix, which provides an estimate of the impact of image alignment on the transformed image. This feature was used to generate an estimate for the impact of registration on quantification of GA in the central macula in serial fundus photographs.37 Registration of images across imaging modalities was performed with i2k Align Retina. To register FAF images to their corresponding CFPs, each pair of images was sequentially tested until the software found a match; there is enough variation in the spatial information present in CFPs and FAF images that prevents individual pair alignment in some cases. Once a pair was found that could be coregistered, all of the color images were aligned to the color image of the pair as the target. Most often, this was the first image in the patient data set. Fundus autofluorescence images for that patient were next registered to the corresponding FAF image of the pair, and then the image transformation between the FAF and corresponding CFP was performed sequentially using the CFP as the target. An identical series of alignment transformations were then performed on each corresponding GA map drawn by the expert grader, a feature found in the research version of the i2k Align Retina software. Blood Vessel Elimination Vessel borders are prone to interfere with the detection of the edge of GA.7 Blood vessel pixels have low-intensity values when compared with non-vessel pixels. We used this property to represent blood vessels as wavelets in a time–frequency representation called a continuous wavelet transform after the work by Arneodo et al.38 We used a Morlet wavelet as analyzing wavelet. A Morlet wavelet is obtained from localizing a sine wave with a Gaussian curve. It has the ability to be tuned to specific spatial frequencies, filtering out noise in the fundus images while retaining image structure. For blood vessel segmentation, the Morlet transform was computed over different orientations, and the maximum modulus value of Morlet transform was selected. This analysis was implemented in MatLab (MathWorks, Natick, MA). With this modification, the vessels attain more prominence than the background noise, which helps capture even the smallest retinal capillaries and choroidal vessels present in the fundus image.39 Figure 2 illustrates blood vessel elimination using this method, where retinal and prominent choroidal blood vessels are digitally subtracted from the green
AUTOMATED SEGMENTATION OF GEOGRAPHIC ATROPHY RAMSEY ET AL
1299
Fig. 1. i2K Align Retina aligns images across modalities. A and B. Aligned CFP and FAF image pair from a representative patient is displayed on the same coordinate plane. The degree of foveal involvement between the two imaging modalities differs substantially with additional central sparing apparent in the FAF image compared with the corresponding CFP. Each image was aligned to the first color photograph to produce a continuous series per patient. The i2k Align Retina is the only image registration program capable of correcting for image distortion between modalities and over time (not shown).
channel of a CFP (Figure 2B) and from its paired FAF image (Figure 2C). Of note, the vessel elimination works better on the color image because the vessel map is derived from this source. In the FAF image, the retinal vessels are muted but not completely eliminated from the resulting image. Optic Disk Detection and Fovea Localization Because the optic disk appears as a bright, generally circular ellipse where the retinal blood vessel network converges, it can be misidentified as GA by the segmentation algorithm because of its high-intensity level. It is therefore necessary to identify and mask the optic disk when analyzing GA. We have used the fuzzy segment model described by Hover and Goldbaum48 to find the point of convergence of the blood vessel network. To detect the optic disk location, we used the vessel map generated from the vessel elimination step and represented vessels as line segments. Every pixel in the segment model casts a vote to produce a convergence image. The pixel that has the most number of votes in the convergence image is marked as the optic disk center. Before GA segmentation in the image, the user
has the opportunity to manually adjust the location of the optic disk center and radius of the mask. We then used a vessel model approach to localize the fovea.40 This method first fits a parabolic shape to the central retinal blood vessel network using an Active Shape Model based on the calculated optic disk location. Next, it localizes the fovea from the fitted parabolic shape. We trained two separate models for right and left fundus images. After identifying the structure of central retinal blood vessel network (Figure 3A), the fovea is located 2 MPS disk areas from the optic disk center on the axis of the fitted parabola (Figure 3C). The same optic disk map and putative location of the fovea can be used for all images for a given patient after registration of all of the image pairs as described above (Figure 3D). Implementation of Fuzzy c-Means We have implemented FCM, a soft segmentation method, where an image pixel can be classified into multiple classes with different degrees of membership using MatLab (MathWorks). The FCM method was developed by Dunn30 and improved by Bezdek et al.29
Fig. 2. Example of blood vessel elimination. A. Vessel borders are prone to interfere with the detection of the edge of GA.7 Retinal and (large) choroidal blood vessels were eliminated through digital subtraction based on a vessel map generated by the maximum modulus value of the Morlet transform computed over different orientations. Thresholding the algorithm’s output generates a mask suitable for digital subtraction of the major vessels (A). B and C. Example CFP from the same patient shown in Figure 1 with the blood vessels removed from the green channel of the original image (shown in gray scale). The same vessel map can be used to subtract the blood vessels from the paired FAF image after the registration step; however, the vessels are not completely removed.
1300 RETINA, THE JOURNAL OF RETINAL AND VITREOUS DISEASES
2014 VOLUME 34 NUMBER 7
Fig. 3. Detection of the optic disk and foveal location. A. Example of a thinned blood vessel map obtained, where each blood vessel in the vessel map is condensed to a single point. B. Convergence image obtained after the application of fuzzy convergence algorithm. The brighter the region is, the more blood vessels converge in that area and the higher the probability of that pixel being the optic disk center. C. Points along the major retinal arcade vessels detected by Active Shape Model are shown in blue. The position of the fovea (marked by a green cross) is two disk areas away from the optic disk center along the axis of a parabola fit to these points. D. The original fundus photograph with the foveal center (black cross) and optic disk center (white cross), as well as a one disk area region of interest centered over the fovea (green circle) and mask superimposed over the optic disk (black disk).
In our FCM-based method, all brightness and contrast data are used as a feature vector in differentiating lesions from the background retina. Clustering, in general, allows one piece of data to belong to two or more classes or clusters. This is an important feature for medical diagnostic systems because it increases the sensitivity of segmentation. In the FCM algorithm, the degree of membership for each data point is related to the inverse of the distance from the centroid of the cluster, computed as the mean of all points, weighted by their degree of belonging to the cluster. In our implementation, we have elected to use only two classes, one for atrophy and the other for nonatrophy based on the brightness and contrast feature space of image. The FCM segmentation used K-means clustering method to obtain the initial centroid values, and the centroid values were updated with each iteration to minimize the objective function. Our method takes the cluster with maximum association value as the cluster the data associates. After segmentation, we present the segmentation results for each iteration of the algorithm graphically and asked the user to choose the best segmentation result. Interactive Geographic Atrophy Segmentation Method Automated segmentation of the GA present in CFPs and FAF images, and more importantly delineating the
border between normal and atrophic regions of retina, would save time, reduce cost, and improve overall precision of quantification of GA. Figure 4 illustrates the steps for the expert-guided automated image segmentation process. First, the retinal and (large) choroidal blood vessels are eliminated from each retinal image through digital subtraction because vessel borders are prone to interfere with detecting the edge of GA. Using the vessel map, the putative locations of the optic disk and fovea are calculated for each patient, and a mask is created to exclude the optic disk, which is a bright object prone to misclassification as atrophy. Next, the expert user draws a ROI around the portion of the central macula where the GA is to be detected. The same ROI can be used for all images in a given patient, or between patients, if the scale of images is roughly identical. Fuzzy c-means segmentation is then performed on the CFP or FAF image, generating a series of clusters of pixels. An example of the FCM segmentation result for a representative IR image is also presented (see Figure, Supplemental Digital Content 2, http://links.lww.com/IAE/A207). Next, the user is simultaneously provided with the results of several iterations of the algorithm and may select a segmentation result before the convergence of the algorithm, which generally occurred between four and nine iterations. Each cluster (area) is displayed as a separate topography overlaid on a copy of the original image in descending order of size. The user
AUTOMATED SEGMENTATION OF GEOGRAPHIC ATROPHY RAMSEY ET AL
1301
correctly identified in the FCM segmentation result, the edge of the ground truth map was identified (±3 pixels) and an ROI created. A pixel in the border ROI was scored as correctly identified if it was internally bordered by a pixel scored as atrophic and externally bordered by a pixel identified as nonatrophy. Statistical Analyses Comparisons of the mean values between aligned and unaligned areas in color images and comparisons between the GA areas in aligned paired CFP and FAF images were performed using two-sample two-sided t-tests. No imputation of missing values was performed. Statistical significance was defined as P , 0.05. All analyses were conducted using statistical analysis package, Microsoft Excel (Version 12.3.3; Redmond, WA). Results Impact of Image Registration on Geographic Atrophy Identified by an Expert Grader in Color Fundus Photographs and Fundus Autofluorescence Images
Fig. 4. Flow diagram of interactive GA segmentation method. The GA segmentation strategy consists of six consecutive stages: 1) image registration, 2) blood vessel segmentation and digital subtraction, 3) ROI selection by a human operator with reference to the imputed location of the fovea and optic disk (mask), 4) GA segmentation by the application of FCM algorithm, 5) selection of the relevant topographies by the human operator, and 6) GA quantification and border detection based on the final segmentation result. Automated steps are shown enclosed with ellipses; those requiring user input are boxed.
then selects one or more topographies that correspond to the visible GA in the image. As drusen and other fundus lesions are often small relative to regions of atrophy, this step serves as a size exclusion function because only the nine largest clusters were displayed. Finally, the resulting user-defined FCM segmentation is compared with the registered ground truth map, noting pixels correctly identified (true positives), those incorrectly identified (false positives), and those missed by the software method (false negatives). To estimate the portion of the GA border that was
The registration of the 10 patient series included in our automated segmentation study by the i2k Align Retina software provides a unique opportunity to assess the impact of automated image alignment on both the calculated GA area and mean enlargement rate of the total atrophic area. To assess the impact of technical differences in the serial imaging of GA, we first manually calculated the GA area using CFP and FAF image pairs. We then aligned the image pairs and performed an identical alignment transformation on the corresponding GA maps drawn by our expert. Image alignment did not significantly alter the area of GA measured by the expert grader, affecting the manually segmented areas by only 0.19 ± 0.42 mm2 (1.6 ± 3.0% as percent of lesion area, not significant) for CFPs and 0.69 ± 1.14 mm2 (5.4 ± 8.6% as percent of lesion area, not significant) for FAF images. This compares with an estimated impact of image alignment on the central macula of digitized CFPs reported from the Wilmer Geographic Atrophy Study of 1.0 ± 5.4%.37 Although the differences are small on the basis of lesion area, focal differences in atrophy enlargement in the macula can often account for disproportionately large changes in vision; image alignment may assist the clinician in visually detecting these subtle structural changes. No difference in the rate of atrophy enlargement was detected between the pre- and post-processed images. The mean enlargement of the total atrophic area in the
1302 RETINA, THE JOURNAL OF RETINAL AND VITREOUS DISEASES
study eye by 2 years was 2.06 ± 1.23 mm2 before versus 2.00 ± 1.19 mm2 after the image alignment for CFPs (not significant), and it was 2.40 ± 1.43 mm2 before versus 2.06 ± 1.16 mm2 after the image alignment for FAF images (not significant). Of note, the enlargement rate between the two modalities more closely agree for the aligned versus the unaligned impairs, although neither difference achieved statistical significance. Impact of Cross-Modality Registration on Geographic Atrophy Identified by an Expert Grader in Color Fundus Photographs Versus Fundus Autofluorescence Images Experts grading CFP and FAF image pairs do not always agree. Some of these differences may be overcome by advanced image registration techniques, such as those used by the i2k Align Retina platform, which may compensate for differences in image alignment, magnification, or other types of imaging artifacts that lead to distortion of the image. Although FAF imaging is extremely sensitive for the presence of GA, it is not specific for GA. For example, areas of pigmentary alteration, presumed to have retinal pigment epithelium attenuation but not GA, may have reduced FAF that may persist for years without the development of GA.41 Although there were differences in the GA identified by manual grading of CFP and FAF image pairs, the difference in the measured atrophic area between the CFP and FAF images was quite small in our patient series (1.9 ± 11.4% as percent of lesion area, not significant). After image registration by the i2k Align Retina software, the difference in lesion measurements between the two modalities was slightly, but not significantly, reduced (1.3 ± 7.0% as percent of lesion area, not significant). These results suggest that the difference observed in the GA area between paired CFPs and FAF images, quite pronounced for some patients, is unlikely to be simply the result of technical differences introduced by the image acquisition system. Automated Segmentation of Geographic Atrophy Area Using the Fuzzy c-Means Algorithm We next compared our automated system with manual grading of GA on FAF and CFPs by characterizing GA area and the GA border. The automated FCM-based segmentation detection of atrophic lesion area in retinal images showed good agreement with an expert grader for FAF images but performed less well on CFPs. Automated segmentation of FAF images achieved a performance level of 94 ± 5% sensitivity and 98 ± 2% specificity on a per-pixel basis. In contrast, the segmentation of CFPs for GA area varied greatly with
2014 VOLUME 34 NUMBER 7
a sensitivity of only 47 ± 26% and specificity of 98 ± 2%. The segmentation algorithm identified 75 ± 16% of the GA border correctly in FAF images compared with just 42 ± 25% for CFPs. On FAF images, the FCM algorithm’s high sensitivity segmented suspicious regions as atrophy rather than background. Figure 5 presents a representative case of a patient with parafoveal GA, where the foveal center has pigmentary changes and reduced fundus FAF. These pigmentary areas are incorrectly segmented as atrophy by the FCM algorithm. Although these changes are likely indicative of retinal disease, they are not atrophy. It is often difficult to delineate parafoveal atrophy in fundus FAF images because of the absorption of blue light by xanthophyll pigment in the macula. This phenomenon may contribute to our segmentation errors, particularly in these cases. Figure 6 shows a patient with a large area of fovealinvolving GA. The false-positive rate is significantly less in this case compared with the patient shown in Figure 5 who has foveal sparing. The area incorrectly identified as atrophy (falsepositive pixels) relative to the total area of GA present in the corresponding ground truth images was 9 ± 8% for the CFPs analyzed in the study. For the FAF images, the area of incorrectly identified atrophy relative to the total area of GA present in the corresponding ground truth images was 17 ± 13%. Atrophy was more likely to be missed in the segmentation of CFPs, and this difference likely accounts for why the false-positive pixels also were fewer. The significantly higher sensitivity for FAF images accounts for the relatively greater number of false-positive pixels. In either case, these errors can be manually corrected or eliminated through alternative segmentation or postprocessing methods.
Discussion Segmentation of lesions in retinal images is an area of intense investigation in many ophthalmic conditions, including AMD. In particular, future treatment trials for GA will require the accurate analysis of large numbers of patients with longitudinal data to detect meaningful changes in the atrophy progression rate.16 Automated segmentation that accurately identifies GA would greatly improve the efficiency of image processing during such a clinical trial. Furthermore, analysis software that could easily identify and measure GA progression would be a welcome addition to the clinician who is following patients with GA. The expert-guided automated image segmentation approach presented in this article is reproducible, easy to apply to serial fundus photographs across multiple
AUTOMATED SEGMENTATION OF GEOGRAPHIC ATROPHY RAMSEY ET AL
1303
Fig. 5. Fuzzy c-means segmentation results for a patient with parafoveal GA. Representative CFP (A) and FAF image (F) from a patient with parafoveal GA. Ground truth maps for GA in the same CFP (B) and FAF image (G). Corresponding segmentation result produced by the FCM detection method (C and H). Color-coded maps of the segmentation result for GA area (D and I), and GA border (E and J) that was correctly identified by the FCM algorithm (green), incorrectly identified (false positive, red), and missed by the software method (false negative, blue). Note that most of the false positives identified by the FCM method are in the central macula of the FAF image, an area rich in xanthophyll pigment, and in regions that show changes in pigmentation that are likely indicative of retinal disease.
imaging modalities, and relatively fast. The FCM algorithm always converges to the same output for a given image, making the segmentation approach automated and reproducible. The speed, sensitivity, and specificity of this method are enhanced by a simple user-drawn ROI before lesion segmentation that effectively limits processing of irrelevant portions of the image. The ROI can be applied serially to images for a given patient provided the bounding box is drawn on the most recent image, where the GA lesion would be expected to be of maximal extent. In the final segmentation step, the user selects candidate regions from the automated segmentation results for final summation to limit the problem of oversegmentation. Obvious misclassifications of background nonatrophic retina, or lesions too small to be atrophy such as drusen, can be efficiently eliminated by this added step (see Figure, Supplemental Digital Content 3, http://links.lww.com/IAE/A214). The image segmentation approach presented in this article can be applied to serial fundus photographs across multiple imaging modalities. Automated image alignment not only permits the elimination of invariant features such as retinal blood vessels but also identifies other features in a given image that might impinge upon the analysis, such as areas of variable fundus pigmentation or drusen. The cross-modality image registration presented here is an integral part of the method used for preprocessing FAF (and IR) images to allow the digital subtraction of blood vessels based on the corresponding CFPs. Importantly, image registration permits these operations to be applied simultaneously across all of the images in a patient data set. Limiting the amount of user input makes the method relatively fast and reproducible without the need for special training. Although manual segmentation
methods can take several minutes to evaluate a single fundus photograph, the software-based methods presented in this study can reduce the time needed to process an image by about 10-fold. The ability of our FCM-based segmentation method to correctly identify the GA lesion border is a unique feature of our analysis. At first blush, our results may not seem to stand out given that only three-quarters of the GA border was correctly identified for FAF images and less than half for CFPs. However, no comparable numbers are available in the literature, because previous studies have not reported this more stringent index of performance. This is especially relevant for clinical trials, where a high level of precision is required and because the time and effort required by an expert grader to manually validate and correct a softwarebased segmentation result is directly proportional to the extent that lesion borders are correctly identified. Furthermore, GA is most likely to advance from a lesion edge than from de novo; we therefore believe that special attention should be placed on carefully delineating the lesion border in serial image analysis. Further development of this segmentation method, and its validation in a clinical setting, promises to significantly improve the precision of GA measurement and reduce the effort involved while grading images. Few automated image analysis techniques have been developed for the segmentation of GA; most approaches for segmenting GA rely on interactive segmentation schemes for GA. Köse et al23 used texture segmentation to automate the identification of GA in CFPs. However, in this study, the threshold value for texture segmentation, determined by supervised experiments, was manually adjusted to improved the results in some cases. In particular, this method can fail to give good results for GA segmentation in
1304 RETINA, THE JOURNAL OF RETINAL AND VITREOUS DISEASES
2014 VOLUME 34 NUMBER 7
Fig. 6. Fuzzy c-means segmentation results for a patient with foveal-involving GA. Representative serial CFPs (A–C) and FAF images (D–F) from a patient with central GA. Ground truth maps for GA in CFP (G–I) and FAF images (J–M). Color-coded maps of the segmentation result for GA area (M–R) and GA border (S–X), that was correctly identified by the FCM algorithm (green), incorrectly identified (false positive, red), and missed by the software method (false negative, blue). Note that bright signals from prominent calcific drusen visible in the CFP are preferentially segmented by the FCM method. These features are silent in the FAF image, where the FCM algorithm does a better job of identifying the atrophy. However, in the FAF image, a small satellite focus of GA was not correctly segmented from the background nonatrophic retina. Finally, the outline of the FCM segmentation result is displayed on the original FAF images (Y–Z, ZZ) to illustrate the close agreement between the GA border and the appearance of GA on the FAF image.
images that have low contrast between healthy and atrophic regions. Cluster-based analysis, such as the FCM segmentation method, partially overcomes such difficulties by virtue of weighting regions in relation to their neighbors. Another semiautomated segmentation strategy developed by Hwang et al7 used a user-defined threshold applied to a Gaussian-filtered original FAF image that was supplemented by manual segmentation. Additional image preprocessing such as color normalization and contrast enhancement31,42 could also be applied to our FCM segmentation method; but in our experience, these adjustments tend to accentuate background variations as much if not more than the GA lesions, resulting in oversegmentation of the image by the FCM algorithm (data not shown). More recently, Schmitz-Valckenberg et al15 presented a so-called region-growing algorithm for the
quantification of GA in FAF images. In this method, trained graders define one or more GA centers (seedpoints) and then manually adjust a user-defined threshold value for each until the resulting area “grows” to encompass the atrophy. A similar approach is available as a proprietary software tool from Heidelberg Engineering known as RegionFinder.43 This method has good intra- and inter-observer agreement and has been validated in a clinical trial of the concordance rate of GA44; but unlike the FCM segmentation method, it is not automated and can only analyze regions sequentially, meaning that it takes considerably more effort to analyze a patient with multiple foci of GA compared with one with a solitary lesion. In addition, areas of atrophy may be overlooked, something unlikely to happen in a method that relies on data clustering, like the FCM-based method.
AUTOMATED SEGMENTATION OF GEOGRAPHIC ATROPHY RAMSEY ET AL
Finally, Devisetti et al24 recently presented the first successful application of a neural network classifier in performing segmentation of GA in IR and FAF images of eight eyes belonging to six patients with GA. They used a supervised learning approach to train a classifier provided with seven image features derived from both the FAF and IR images that included both intensity and regional intensity data. Their approach performed reasonably well, achieving a sensitivity of 82.5% and specificity of 92.9% as measured on a per-pixel basis and averaged across all the input image pairs. Our FCM-based classifier performs reasonably well compared with this method and is computationally less intensive. In the future, candidate regions could be intelligently classified in an automated fashion using a simple machine learning approach such as a neural network.42 If such a method were implemented, the full power of the FCM approach could be used with more than two classes to better differentiate GA from the surrounding normal retina. This would require a much larger data set for training of a classifier. Although clinically both CFPs and FAF images are used to characterize GA, the relative ease of defining GA by FAF and the wide availability of FAF along with spectral domain optical coherence tomography using the Heidelberg Spectralis, most clinical trials currently use FAF to quantify GA enlargement.6,7,11,12,14,15 Improved analytical techniques using spectral domain optical coherence tomography45 or new technologies like polarization sensitive optical coherence tomography46,47 will likely make these other important ways of quantifying progression of GA.45 The ability to characterize the extent of foveal involvement, one of the limitations of FAF imaging, is enhanced by the use of IR reflectance images,15 in addition to spectral domain optical coherence tomography findings. Most current trials are not using CFP for this purpose. Our results also highlight the difficulty with quantifying GA changes using CFP. This study suggests that FAF imaging may slightly underestimate the GA area relative to CFP, but the enlargement rates of the two modalities are not significantly different, and the impact of image alignment is small. Also, of note is that the enlargement rates using both modalities in this study are close to the rates found by Sunness,16 independent of the modality used. Thus, we suggest that the use of FAFs does not explain the somewhat lower enlargement rates found in the FAM study and others studies.25 In our sample population, we also measured the differences in the GA area between CFP and FAF image pairs. The proprietary image alignment algorithms implemented in the i2k Align Retina software program perform point-by-point image registration correcting for nonlinear differences between images,
1305
including across imaging modalities. We found only a modest effect of image alignment on the measurement GA area and mean enlargement rate of the total atrophic area, and correction for image alignment by itself, did not fully account for the difference in the GA area between the two imaging modalities. Therefore, the differences in the measurements between the CFP and FAF images pairs appear to be due to the differences in what each image modality highlights rather than from flaws in our technology. We recognize that our study was not powered to answer whether CFPs or FAFs better quantify GA area. However, the results do suggest that the application of cross-modality image registration coupled with computer-assisted segmentation could help better define the differences between color and FAF images. Further work examining the differences between GA quantification between imaging modalities is needed to determine the best method for following GA, balancing the sensitivity of the method with its specificity for lesion identification. We have developed an interactive computer-based image segmentation method for the serial measurements of GA lesions in CFPs and FAF images. Although image alignment did not have a significant impact on the measured atrophy area, image registration is a clinically useful tool in monitoring the progression of atrophy in a given patient. The segmentation method achieves a high level of sensitivity and specificity. However, the accuracy, particularly in identification of the lesion border, is not sufficient to replace the expert grader. The results of this study are proof of principle that automated segmentation may enhance the speed, precision, and reproducibility of GA measurement in an unbiased fashion that could assist an expert in the manual grading of images. These methods are potentially useful to the practicing clinician, as well as to natural history studies, and future treatment trials designed, to evaluate new pharmacologic agents that limit the progression of GA. Key words: age-related macular degeneration, automated image segmentation, fundus autofluorescence, geographic atrophy. Acknowledgment The authors thank Dr. Charles Stewart (Rensselaer Polytechnic Institute and DualAlign LLC [Founder and Chief Scientist]) and Dr. Gary Yang (DualAlign LLC) for custom image alignment software and support. References 1. Friedman DS, O’Colmain BJ, Muñoz B, et al. Prevalence of agerelated macular degeneration in the United States. Arch Ophthalmol 2004;122:564–572. doi:10.1001/archopht.122.4.564.
1306 RETINA, THE JOURNAL OF RETINAL AND VITREOUS DISEASES 2. Rein DB, Wittenborn JS, Zhang X, et al. Forecasting agerelated macular degeneration through the year 2050: the potential impact of new treatments. Arch Ophthalmol 2009; 127:533–540. doi:10.1001/archophthalmol.2009.58. 3. Bressler NM, Doan QV, Varma R, et al. Estimated cases of legal blindness and visual impairment avoided using ranibizumab for choroidal neovascularization: non-Hispanic white population in the United States with age-related macular degeneration. Arch Ophthalmol 2011;129:709–717. doi: 10.1001/archophthalmol.2011.140. 4. Sunness JS, Bressler NM, Tian Y, et al. Measuring geographic atrophy in advanced age-related macular degeneration. Invest Ophthalmol Vis Sci 1999;40:1761–1769. 5. Sunness JS, Gonzalez-Baron J, Applegate CA, et al. Enlargement of atrophy and visual acuity loss in the geographic atrophy form of age-related macular degeneration. Ophthalmology 1999;106:1768–1779. doi:10.1016/S0161-6420(99)90340-8. 6. Holz FG, Bellman C, Staudt S, et al. Fundus autofluorescence and development of geographic atrophy in age-related macular degeneration. Invest Ophthalmol Vis Sci 2001;42:1051–1056. 7. Hwang JC, Chan JW, Chang S, Smith RT. Predictive value of fundus autofluorescence for development of geographic atrophy in age-related macular degeneration. Invest Ophthalmol Vis Sci 2006;47:2655–2661. doi:10.1167/iovs.05-1027. 8. Klein ML, Ferris FL 3rd, Francis PJ, et al. Progression of geographic atrophy and genotype in age-related macular degeneration. Ophthalmology 2010;117:1554–1559, 1559 e1551. doi: 10.1016/j.ophtha.2009.12.012. 9. Lindblad AS, Lloyd PC, Clemons TE, et al. Change in area of geographic atrophy in the Age-Related Eye Disease Study: AREDS report number 26. Arch Ophthalmol 2009;127: 1168–1174. doi:10.1001/archophthalmol.2009.198. 10. Maguire P, Vine AK. Geographic atrophy of the retinal pigment epithelium. Am J Ophthalmol 1986;102:621–625. 11. Intravitreal LFG316 in Patients With Age-related Macular Degeneration (AMD). Available at: http://clinicaltrials.gov/ show/NCT01527500. Accessed June 25, 2012. 12. A Multicenter, Proof-of-Concept Study of Intravitreal AL-78898A In Patients With Geographic Atrophy (GA) Associated With Age-Related Macular Degeneration (AMD). Available at: http://clinicaltrials.gov/show/NCT01603043. Accessed June 25, 2012. 13. Geographic Atrophy Treatment Evaluation (GATE). Available at: http://clinicaltrials.gov/ct2/show/NCT00890097. Accessed June 25, 2012. 14. Efficacy, Safety and Tolerability Study of RN6G in Subjects With Geographic Atrophy Secondary to Age-related Macular Degeneration. Available at: http://clinicaltrials.gov/ct2/show/ NCT01577381. Accessed June 25, 2012. 15. Schmitz-Valckenberg S, Brinkmann CK, Alten F, et al. Semiautomated image processing method for identification and quantification of geographic atrophy in age-related macular degeneration. Invest Ophthalmol Vis Sci 2011;52:7640– 7646. doi:10.1167/iovs.11-7457. 16. Sunness JS, Applegate CA, Bressler NM, Hawkins BS. Designing clinical trials for age-related geographic atrophy of the macula: enrollment data from the geographic atrophy natural history study. Retina 2007;27:204–210. doi:10.1097/01. iae.0000248148.56560.b1. 17. Scholl HP, Dandekar SS, Peto T, et al. What is lost by digitizing stereoscopic fundus color slides for macular grading in age-related maculopathy and degeneration? Ophthalmology 2004;111:125–132. doi:10.1016/j.ophtha.2003.05.003.
2014 VOLUME 34 NUMBER 7
18. Hsu W, Pallawala PMDS, Lee ML, Au-Eong KG. The Role of domain knowledge in the detection of retinal hard exudates. Proc IEEE Computer Vis Pattern Recognition II-246-II-251 2001. 19. Ben Sbeh Z, Cohen LD, Mimoun G, Coscas G. A new approach of geodesic reconstruction for drusen segmentation in eye fundus images. IEEE Trans Med Imaging 2001;20: 1321–1333. doi:10.1109/42.974927. 20. Kankanahalli S, Burlina PM, Wolfson Y, et al. Automated classification of severity of age-related macular degeneration from fundus photographs. Invest Ophthalmol Vis Sci 2013;54: 1789–1796. doi:10.1167/iovs.12-10928. 21. Rapantzikos K, Zervakis M, Balas K. Detection and segmentation of drusen deposits on human retina: potential in the diagnosis of age-related macular degeneration. Med Image Anal 2003;7:95–108. 22. Chakravarthy U, Walsh AC, Muldrew A, et al. Quantitative fluorescein angiographic analysis of choroidal neovascular membranes: validation and correlation with visual function. Invest Ophthalmol Vis Sci 2007;48:349–354. doi: 10.1167/iovs.06-0493. 23. Köse C, Sevik U, Gencalioglu O, et al. A statistical segmentation method for measuring age-related macular degeneration in retinal fundus images. J Med Syst 2010;34:1–13. 24. Devisetti K, Karnowski TP, Giancardo L, et al. Geographic atrophy segmentation in infrared and autofluorescent retina images using supervised learning. Conf Proc IEEE Eng Med Biol Soc 2011;2011:3958–3961. doi:10.1109/IEMBS.2011.6090983. 25. Schmitz-Valckenberg S, Bindewald-Wittich A, DolarSzczasny J, et al. Correlation between the area of increased autofluorescence surrounding geographic atrophy and disease progression in patients with AMD. Invest Ophthalmol Vis Sci 2006;47:2648–2654. doi:10.1167/iovs.05-0892. 26. Wang H, Hsu W, Goh KG, Lee KG. Proceedings of IEEE Conf. On computer vision and pattern recognition. Proceedings IEEE Conf Computer Vision Pattern Recognition, 2000;181-186. 27. Walter T, Klein JC. Automatic Detection of Microaneurysms in Color Fundus Images of the Human Retina by Means of the Bounding Box Closing. Vol 2526. Berlin, Germany: Springer, 2002:210–220. 28. Usher D, Dumskyj M, Himaga M, et al. Automated detection of diabetic retinopathy in digital retinal images: a tool for diabetic retinopathy screening. Diabet Med 2004;21:84–90. 29. Bezdek JC, Trivedi M, Ehrlich R, Full W. Fuzzy clustering: a new approach for geostatistical analysis. Interntional J Syst Meas Decis 1981;1-2:13–24. 30. Dunn J. Well separated clusters and optimal fuzzy partitions. J Cybernetics 1974;95–104. 31. Osareh A, Shadgar B, Markham R. A computational-intelligencebased approach for detection of exudates in diabetic retinopathy images. IEEE Trans Inf Technol Biomed 2009;13:535–545. doi: 10.1109/TITB.2008.2007493. 32. Chen W, Giger ML, Bick U. A fuzzy c-means (FCM)-based approach for computerized segmentation of breast lesions in dynamic contrast-enhanced MR images. Acad Radiol 2006;13: 63–72. doi:10.1016/j.acra.2005.08.035. 33. Mata NL, Lichter JB, Vogel R, et al. Investigation of oral fenretinide for treatment of geographic atrophy in age-related macular degeneration. Retina 2013;33:498–507. doi:10.1097/ IAE.0b013e318265801d. 34. Age-Related Eye Disease Study Research Group. The AgeRelated Eye Disease Study (AREDS): design implications. AREDS report no. 1. Control Clin Trials 1999;20:573–600.
AUTOMATED SEGMENTATION OF GEOGRAPHIC ATROPHY RAMSEY ET AL 35. Heijl A, Bengtsson B. Diagnosis of early glaucoma with flicker comparisons of serial disc photographs. Invest Ophthalmol Vis Sci 1989;30:2376–2384. 36. You QS, Xu L, Jonas JB, et al. Change in choroidal nevi during a 5-year follow-up study: the Beijing Eye Study. Br J Ophthalmol 2010;94:575–578. doi:10.1136/bjo.2009.165720. 37. Ramsey DJ, Sunness JS, Handa JT. Automated Image Alignment to Follow Progression of Geographic Atrophy in AgeRelated Macular Degeneration. In: Association for Research in Vision and Ophthalmology. Ft. Lauderdale, FL; 2011. 38. Arneodo A, Decoster N, Roux SG. A wavelet-based method for multifractal image analysis. I. Methodology and test applications on isotropic and anisotropic random rough surfaces. Eur Phys J B 2000;15:567–600. 39. Leandro JJG, Cesar RM Jr, Jelinek HF. Blood vessels segmentation in retina: preliminary assessment of the mathematical morphology and of the wavelet transform techniques. SIBGRAPI-01, Florianópolis—SC, IEEE Computer Society 2001;84–90. 40. Chutatape O. Fundus foveal localization based on vessel model. Conf Proc IEEE Eng Med Biol Soc. 2006;1:4440–4444. 41. Sunness JS, Ziegler MD, Applegate CA. Issues in quantifying atrophic macular disease using retinal autofluorescence. Retina 2006;26:666–672. doi:10.1097/01.iae.0000236472.56195.e9. 42. Osareh A, Mirmehdi M, Thomas B, Markham R. Automated identification of diabetic retinal exudates in digital colour images. Br J Ophthalmol 2003;87:1220–1223.
1307
43. Mansouri K, Tung JD, Medeiros FA, et al. Semiautomated quantification of beta-zone parapapillary atrophy using blue light fundus autofluorescence. Acta Ophthalmol 2013;91: e379–e385. doi:10.1111/aos.12057. 44. Fleckenstein M, Adrion C, Schmitz-Valckenberg S, et al. Concordance of disease progression in bilateral geographic atrophy due to AMD. Invest Ophthalmol Vis Sci 2010;51:637–642. doi:10.1167/iovs.09-3547. 45. Stopa M, Bower BA, Davies E, et al. Correlation of pathologic features in spectral domain optical coherence tomography with conventional retinal studies. Retina 2008;28:298–308. doi: 10.1097/IAE.0b013e3181567798. 46. Ahlers C, Götzinger E, Pircher M, et al. Imaging of the retinal pigment epithelium in age-related macular degeneration using polarization-sensitive optical coherence tomography. Invest Ophthalmol Vis Sci 2010;51:2149–2157. doi: 10.1167/iovs.09-3817. 47. Baumann B, Gotzinger E, Pircher M, et al. Segmentation and quantification of retinal lesions in age-related macular degeneration using polarization-sensitive optical coherence tomography. J Biomed Opt 2010;15:061704. doi:10.1117/1.3499420. 48. Hoover A, Goldbaum M. Locating the optic nerve in a retinal image using the fuzzy convergence of the blood vessels. IEEE Trans Med Imaging 2003;22:951–958.
