Design of an Optimum Computer Vision-Based Automatic Abalone (Haliotis discus hannai) Grading Algorithm Abstract: An automatic abalone grading algorithm that estimates abalone weights on the basis of computer vision using 2D images is developed and tested. The algorithm overcomes the problems experienced by conventional abalone grading methods that utilize manual sorting and mechanical automatic grading. To design an optimal algorithm, a regression formula and R2 value were investigated by performing a regression analysis for each of total length, body width, thickness, view area, and actual volume against abalone weights. The R2 value between the actual volume and abalone weight was 0.999, showing a relatively high correlation. As a result, to easily estimate the actual volumes of abalones based on computer vision, the volumes were calculated under the assumption that abalone shapes are half-oblate ellipsoids, and a regression formula was derived to estimate the volumes of abalones through linear regression analysis between the calculated and actual volumes. The final automatic abalone grading algorithm is designed using the abalone volume estimation regression formula derived from test results, and the actual volumes and abalone weights regression formula. In the range of abalones weighting from 16.51 to 128.01 g, the results of evaluation of the performance of algorithm via cross-validation indicate root mean square and worst-case prediction errors of are 2.8 and ±8 g, respectively. Keywords: abalone, automatic grading, computer vision, volume, weight
Introduction
because the rebound speed of the device is slow, grading cannot be processed rapidly. Several challenges preventing high accuracy sorting/grading of fish based on computer vision have been reported (Strachan 1994; Zion and others 2000, 2012; G¨um¨u¸s and others 2011), and a number of computer vision-based automated methods for measuring the length or estimating the weight of fish in order to sort/grade them have been proposed. Recently, those methods are also being used to estimate vaccine injection site of flatfish (Lee and others 2013). With regard to length, most of these methods measure the link length between the fish width midpoints (Strachan 1993; Ibrahim and Wang, 2009; Jeong and others 2013). With regard to weight estimation, the methods use a regression formula between the projected two-dimensional (2D) view area and the weight (G¨um¨u¸s and Balaban, 2010; Balaban and others 2010a, 2010b). However, these methods are subject to measurement inaccuracies because they disregard thickness information. Lee and others (2001) propose an oyster volume measurement algorithm that grades oysters automatically via a three-dimensional (3D) measurement method that considers thickness information. The algorithm combines a laser triangulation technique with 2D measurements to reconstruct a 3D surface for volume measurement. These 3D measurement methods facilitate accurate volume measurement, but their measurement systems are complex and the measurements are difficult to correct. Furthermore, they require complex processing procedures to collect the tomographic image data and reconstruct the surface (Lee and others 2003). This paper propose an automatic abalone grading algorithm on the basis of computer vision using 2D images that uses regression analysis of total length (TL), body length (BL), thickness (T), view MS 20140618 Submitted 4/13/2014, Accepted 12/4/2014. Authors are area (A), and actual volume of abalone against measured weight with Fisheries System Engineering Div., Natl. Fisheries Research & Development to surmount the challenges identified and accurately sort/grade Inst., Korea. Direct inquiries to author Yang (E-mail: [email protected]). abalone.
The value and price of marine products are dependent on the size of the products; therefore, accurate grading is required, and the grading process must be performed as quickly as possible in order to preserve the freshness of the product. However, because in most cases conventional methods manually measure the length and weight of the products, they are labor-intensive and time-consuming. Consequently, automatic grading technologies that can sort/grade marine products quickly and accurately, and thereby preserve their freshness, are required. As present, mechanical sorting/grading machines, such as those with increasing gap belts and roller graders, are used to sort/grade fish. These enable the processing of several fish in a short period of time; however, the resulting friction exerted on the fish may decrease their commercial value. Further, accurate grading cannot be expected because the weight and length of the fish are not measured during the processing (Veliyulin and others 2011). In South Korea, mechanical automatic abalone graders, which perform the grading process by comparing abalone weight to weights established according to grades, have been developed and are used in fish farms to grade the abalones automatically. In this method, abalone are placed, one at a time, on grading cups that are rotated on an oval-shaped rail and, as a result of the seesaw principle, abalone that are heavier than the weight set up on each grading cup fall off into a container. However, this equipment has a limitation: each grading cup and weight combination is designed to return to its original position after the abalone falls; however,
R C 2015 Institute of Food Technologists
doi: 10.1111/1750-3841.12799 Further reproduction without permission is prohibited
Vol. 80, Nr. 4, 2015 r Journal of Food Science E729
E: Food Engineering & Materials Science
Donggil Lee, Kyounghoon Lee, Seonghun Kim, and Yongsu Yang
An optimum abalone grading algorithm . . . Table 1–CCD camera specifications and test conditions.
Abalone features measurement and image processing In our study, the total length (TL) and body length (BL) of an CCD cameras abalone are measured simply by detecting the edge points of the Setting Base Side projected image from the bottom side of the abalone; in addition, Lens 16 (mm) 12 (mm) least squares fitting of ellipses with the detected edge points (Rosin CCD size 1/3 in, mono 1/3 in, mono 1993) was conducted to measure the longest and shortest lengths Pixels 640 × 480 640 × 480 of the ellipse. The view area of the abalones is easily calculated by Field of view (X) 223.36 (mm) 168.32 (mm) Resolution 0.349 (mm/pixel) 0.263 (mm/pixel) substituting the measured TL and BL of the abalones into the area equation of the ellipse (Eq. 1). Working distance 740 (mm) 400 (mm) Max. error 1 (pixel) 1 (pixel) The abalone’s thickness (T) is measured as the maximum height of the abalone’s side image, and the view area is measured using the abalone’s TL and BL values and the area equation of the ellipse, Materials and Methods as indicated by Eq. (1):
E: Food Engineering & Materials Science
Experimental animals In January 2014, 500 abalones, weighting in the range 16.51 π View Area A(cm2 ) = · T L · B L (1) to 128.01 g, and cultivated in the abalone hatchery of the South4 western Sea Research Center of the National Fisheries Research and Development Institute, were transferred to an indoor recirFigure 2 is the image processing flowchart for measuring the culation tank in the institute’s System Engineering Department abalone’s TL, BL, T, and view area: the preprocessing procedure for breeding and adaptation. All abalones were placed in water at 12.8 °C and were not fed for 24 h before the experiment to is shown in Figure 2(a) and the measuring procedure in Figure 2(b). As shown in Figure 2(a), during the preprocessing process, first, the minimize the effects of excretion on weight. image of the abalone’s base and the side projections are captured The experimental system simultaneously, and the brightness information of the images is To continuously measure various abalone sizes, a laboratory- converted to 8-bit grayscale. When the height of the abalone is scale system for measuring abalone features, shown in Figure 1, being measured, the closing computation is performed using 9 × 9 was manufactured. Abalones are measured by falling free from matrix structure images to flatten the extruding parts of the shell guide rail of an abalone feature measurement system with a grade (Gonzalez and Woods 2007). Next, in the binarization process, the pixel brightness of the 8-bit images is respectively converted of 35° and it is feasible to measure up to 3 abalones per second. The system consists of 0.3 megapixel charge-coupled device to 0 and 225 levels on the basis of 50 levels, to vividly show the (CCD) cameras (Panasonic Electric Works SUNX Co., Ltd., AN- edge points and the lines of base and side images of the projected PVC1040, Japan) installed at its base and sides, and a 30 W infrared abalone. After preprocessing and during the measuring process, as shown LED backlight installed across from the CCD camera to prevent worker’s dazzle and to generate a silhouette of the abalone. A trans- in Figure 2(b), first, the edge points are detected from the base mission photosensor (Autonics Co., Ltd., BYS500-TDT, Korea) image projected to measure the abalone’s TL, BL, and view area. is installed at the side to provide a trigger signal for capturing With the detected edge points, the external looped curve is dean image of the abalone for the vision system. A vision system tected from the base image of the projected abalone; the center (Panasonic Co., Ltd., PV200, Japan) and data collection system point and the angle of the area connected to the looped curve are (National Instruments, cRIO-9024, USA) were also present for extracted; and, while referring to the extracted center point and image processing, measurement, data collection, and analysis of the the angle, the coordinate is converted to a reference coordinate. abalones. Data collected from the vision system were transmitted Following coordinate conversion and by scanning from the outer over an Ethernet network and stored in a data collection system. edge to the direction of the center point with each circular pattern Table 1 lists the specifications of the CCD cameras and test and straight line pattern, 100 edge points are detected. Through conditions installed at the base and sides of the abalone feature least squares fitting of ellipses using the detected edge points, the TL and BL of the abalone are measured with the longest and measurement system.
Figure 1–Abalone features measurement system.
E730 Journal of Food Science r Vol. 80, Nr. 4, 2015
An optimum abalone grading algorithm . . . towel. The average value of 20 consecutive measurements was determined to be the weight of each abalone. Subsequently, each regression equation capable of weighing by measuring abalone features is extracted using a power curve regression model or linear regression analysis for each of the TL, BL, T, view area, and volume with respect to the measured abalone weight.
Abalone volume measurement and estimation The average of values measured 20 times repetitively using the density meter (Shinko Denshi Co., Ltd., DME-220, Japan) is used as to be the actual volume of the abalone. The abalone actual volume measurements are also obtained using a density meter Abalone weight measurement and estimation following removal of external water from the specimens by blotting The weight measurements of the abalones were obtained, using them on a paper towel. Using Archimedes’ principle, the actual a digital balance (A&D Co., Ltd., ER-180A, Japan), after external volume of each abalone is measured by substituting the weight of water was removed from the specimens by blotting them on a paper the abalone in the air, the abalone weight measured in distilled
Figure 2–Image processing flowchart for measuring TL, BL, T, and view area of the abalone: (a) preprocessing process, (b) measuring process.
Vol. 80, Nr. 4, 2015 r Journal of Food Science E731
E: Food Engineering & Materials Science
shortest sides of the ellipse, respectively, and the view area of the abalone is calculating Eq. (1). The abalone’s T is measured with the maximum thickness of the side image of the projected abalone via a method of that scants from brightness to darkness using the edge detection technique (Davis 1975). Here, because measurement variation may occur depending on the position of the CCD cameras and the abalone given the cameras’ perspective, using the algorithm proposed by Lee and others (2012), the center point coordinate of the measurement subject’s area is found and corrected by the amount of thickness variations, which occur according to measurement distance.
An optimum abalone grading algorithm . . . Table 2–Equation fit results for abalone weight compared to TL, sensor, which has a suitable property for rectilinear propagation, is BL, T, A, Cv , and Av . used instead of CCD cameras to measure abalone T, it might not Equation I y=
a·xb
TL
BL
T
A
a 0.071 0.319 8.029 0.202 b 3.285 3.141 2.440 1.635 R2 0.969 0.973 0.923 0.979
Equation II y = a·x
Cv
Av
a 1.222 1.224 R2 0.998 0.999
∗
Equation I: The y denotes weight (g), x represents the TL, BL, T (cm), and A (cm2 ), which represent view area. ∗ Equation II: The y denotes weight (g), x represents the Cv and Av (cm3 ), respectively. ∗ The a and b represents regression coefficients.
E: Food Engineering & Materials Science
water, and the density of the distilled water into Eq. (2). In our study, we considered the density of water to be 1 because the density of pure water at room temperature (25 °C) is 0.997 g/cm3 (Damar and others 2007) Actual volume Av (cm3 ) =
weight in air (g) − weight in water(g) density of water(g/cm3 ) (2)
To calculate the abalone volume using the simplified method, the abalones are assumed to have the shape of half-oblate ellipsoids (Wu and others 2009); consequently, using the measured TL, BL, and T of the abalone and Eq. (3) for irregular half-oblate ellipsoids (Du and Sun 2006), the volume of the abalone is calculated Calculated volume Cv (cm3 ) =
π · TL · BL · T 6
(3)
To compare and analyze the calculated volume and the actual volume of the abalone, and to predict the actual volume using Eq. (3), which comprises of the measurement parameters of TL, BL, and T, a linear regression analysis without the intercept was performed between the calculated volume (Cv ) and the actual volume (Av ), as shown in Eq. (4), in which a is regression coefficient: Av (c m 3 ) = a · Cv
Experimental Results
(4)
be necessary to correct T according to the location of the abalone, and a more accurate measurement might be obtained. The R2 value between the view area and the abalone weight is 0.979, which is similar to the R2 value between the view area and the fish weight from previous studies (Balaban and others 2010b). The experimental results indicate that the regression formulas between the weight and each of TL, BL, T, and view area are nonlinear distribution distributed, and that a more precise measurement technology is required to subdivide the grades by abalone weight. However, if there is no need to subdivide the grades by abalone weight and if there is no need to develop an automatic grading system at low cost, then the automatic grading method, which estimates weights through the conventional view area measurement, is more efficient (Balaban and others 1994; Eifert and others 2006). As listed in Table 2, compared to the R2 values between the weight and each abalone TL, BL, T, and view area, the R2 value between the actual volume and abalone weight, 0.999, is the highest. The next highest R2 value is 0.998, which is the value between the calculated volumes and the weights. From these results, it can be deduced that (1) the volume and the abalone weight have a linear relationship, (2) it is possible to subdivide the grades by abalone weight, and (3) accurate weight measurement is possible because, relative to the R2 values between the weight and each abalone TL, BL, T, and view area, the R2 value between the actual volume and the abalone weight is higher. Further, we ascertained that the slope is 1.224 g/cm3 which means the abalone’s density from regression analysis between the weight and actual volume of the abalone. Figure 3 shows the relationship between the calculated volumes and the actual volumes as indicated in Eq. (4), regression analysis results relationship between the show that the regression coefficient a and R2 value are 0.999 and 0.998, respectively. In the experimental results, the slope of the regression formula between the calculated volume of the abalone and the actual volume is close to 1. By means of this basic experiment, we ascertained that our method for estimating weight via volume measurement of the abalone is effective. Furthermore, we discovered that the actual volume of the abalone can be estimated using the volume
The relationship between the measured weight and each of TL, BL, T, and view area shows that the nonlinear distribution and the regression analysis were performed using the power curve regression mode, respectively. The relationship between measured weight and each of calculated volume and actual volume show that the linear distributed, and regression analysis was performed using the linear regression analysis without the intercept. The results of the regression analysis between the measured weight and each abalone measurement factor are summarized in Table 2. Table 2 indicates that, the R2 value between the measured weight and each TL, BL, and the view area is more than 0.969, but the R2 value between T and weight is 0.923, which shows a relatively lower correlation. It is assumed that even a small change in the T of the abalone has a major impact on the weight because the degree of meat thickness varies by individual abalone pertaining to the T. In addition, the organisms attached to the surface impact the results. Consequently, we believe that a technology that can filter the images of meat and attached organisms is required in order to estimate the weight along with the thickness of the abalone. Furthermore, it appears that if a through-beam laser photoelectric Figure 3–Linear curve regression for abalone Cv compared to Av . E732 Journal of Food Science r Vol. 80, Nr. 4, 2015
An optimum abalone grading algorithm . . . weight of abalones, can be effectively applied for growth analysis of abalones.
Acknowledgment This work was supported by a grant from the National Fisheries Research and Development Institute (NFRDI), Republic of Korea. (RP-2015-FE-002).
Author Contributions D. Lee designed and performed the study. K. Lee and S. Kim collected test data and analyzed data. Y. Yang interpreted the results and drafted the manuscript.
Figure 4–Weight estimation error distribution evaluated via leave-one-out cross-validation of fresh abalone.
calculated under the assumption that abalones have the shape of half-oblate ellipsoids. From the results of our study, we derived the automatic abalone grading algorithm for abalone’s weight estimation indicated in Eq. (5) using the regression formula shown in Eq. (4) between the actual volumes and the abalone weights listed in Table 2: this algorithm facilitates automatic abalone grading based on computer vision π (5) Estimated weight (g) = · T L · B L · T 5 We evaluated the performance of this algorithm via leave-oneout cross-validation (Lachenbruch and Mickey 1968; Theodoridis and Koutroumbas 1999) and thereby ascertained that the root mean square error and the weight prediction error spread are 2.8 and ±8 g, for a sample (range of abalones weight: 16.51 to 128.01 g), respectively, as shown in Figure 4. These results show improvements over the results obtained from the methods used to estimate fish weight via the 3D volume measurements in previous studies (Storbeck and Daan 1991; Mathiassen and others 2011).
Conclusion In this paper, we showed that abalone volume can be calculated by measuring the abalone’s TL, BT, and T via a simple method that assumes that the shape of the abalones is a half-oblate ellipsoid, without using conventional expensive equipment or complex formulas. Further, we showed that estimating the abalone’s weight by measuring its volume is more effective than estimating the weight by measuring the TL, BL, T, and view area. From the experimental results, in the range of abalones weighting from 16.51 to 128.01 g, the results of evaluation of the performance of algorithm via cross-validation indicate root mean square and worst-case prediction errors of are 2.8 and ±8 g, respectively. We also found that estimating the abalone’s weight using its view area is also useful, and is similar to conventional methods that estimate fish weight using a regression formula between view areas and fish weight, and therefore can be used to minimize development cost. We expect that these finding will enable abalone farmers to provide fresh abalones to consumers and generate profits. Furthermore, we believe that in the field of marine research, this algorithm, developed to estimate the volume and
Balaban MO, Bergmann Y, Yeralan S, Otwell WS. 1994. Determination of count and uniformity ratio of shrimp by machine vision. J Aquat Food Prod Technol 3(3):43–58. doi: 10.1300/J030v03n03_04 ¨ Balaban MO, Unal-S ¸ eng¨or GF, Soriano MG, Guill´en Ruiz E. 2010a. Using image analysis to predict the weight of Alaskan salmon of different species. J Food Sci 75(3):E157–62. doi: 10.1111/j.1750-841.2010.01522.x Balaban MO, Chombeau M, Cırban D, G¨um¨u¸s B. 2010b. Prediction of the weight of Alaskan Pollock using image analysis. J Food Sci 75(8):E552–6. doi: 10.1111/j.17503841.2010.01813.x Damar S, Yagiz Y, Balaban MO, Ural S, Oliveira ACM, Crapo CA. 2007. Prediction of oyster volume and weight using machine vision. J Aquat Food Prod Technol 15(4):3–15. doi: 10.1300/J030v15n04_02 Davis LS. 1975. A survey of edge detection techniques. Comput Vision Graph 4(3):248–70. doi: http://dx.doi.org/ 10.1016/0146-664X(75)90012-X Du CJ, Sun DW. 2006. Estimating the surface area and volume of ellipsoidal ham using computer vision. J Food Eng 73(3):260–8. doi:http://dx.doi.org/ 10.1016/j.jfoodeng.2005.01.029 Eifert JD, Sanglay GC, Lee D-J, Summer SS, Pierson MD. 2006. Prediction of raw produce surface area from weight measurement. J Food Eng 74(4):552–6. doi:http://dx.doi.org/ 10.1016/j.jfoodeng.2005.02.030 Gonzalez R, Woods R. 2007. Digital image processing. Upper Saddle River, N.J.: Prentice Hall. G¨um¨u¸s B, Balaban MO. 2010. Prediction of the weight of aquacultured rainbow trout (Oncorhynchusmykiss) by image analysis. J Aquat Food Prod Technol 19(3–4):227–37. doi: 10.1080/10498850.2010.508869 ¨ Unl¨ ¨ usayın M. 2011. Machine vision applications to aquatic foods: a G¨um¨u¸s B, Balaban MO, review. Turkish J Fish Aqua. Sci 11:71–81. doi: 10.4194/trjfas.2011.0124 Ibrahim MY, Wang J. 2009. Mechatronics applications to fish sorting Part 1: fish size identification. IEEE International Symposium on Industrial Electronics. ISIE 2009:1978–83. doi: 10.1109/ISIE.2009.5215660 Jeong SJ, Yang YS, Lee K, Kang JG, Lee DG. 2013. Vision-based automatic system for non-contact measurement of morphometric characteristics of flatfish. J Electr Eng Technol 8(5):1194–201. Lachenbruch PA, Mickey RM. 1968. Estimation of error rates in discriminant analysis. Technometrics 10:1–11. doi: 10.1080/00401706.1968.10490530 Lee DG, Yang YS, Kim S, Choi JH, Kang JG, Kim HJ. 2012. A study on system for measuring morphometric characteristic of fish using morphological image processing. J Kor Soc Fish Technol 48:469–78. Lee DG, Yang YS, Kang JG. 2013. Conveyor belt automatic vaccine injection system (AVIS) for flatfish based on computer vision analysis of fish shape. Aquacult Eng. 57:54–62. doi: 10.1016/j.aquaeng.2013.08.004 Lee DJ, Lane RM, Chang GH. 2001. Three-dimensional reconstruction for highspeed volume measurement. In: Intelligent systems and smart manufacturing. International Society for Optics and Photonics. p 258–67. http://proceedings.spiedigitallibrary. org/proceeding.aspx?articleid=926941. doi: 10.1117/12.417201 Lee DJ, Eifert J, Zhan P, Westover B. 2003. Fast surface approximation for volume and surface area measurements using distance transform. Opt Eng 42(10):2947–55. doi:10.1117/1.1605737 Mathiassen JR, Misimi E, Toldnes B, Bondø M, Østvik SO. 2011. High-speed weight estimation of whole herring (Clupeaharengus) using 3D machine vision. J Food Sci 76(6):E458–64. doi: 10.1111/j.1750-3841.2011.02226.x Rosin PL. 1993. A note on the least squares fitting of ellipses. Pattern Recogn Lett 14(10):799– 808. Storbeck F, Daan B. 1991. Weight estimation of flatfish by means of structured light and image analysis. Fish Res 11:99–108. doi:http://dx.doi.org/ 10.106/0165-7836(91)90101-K Strachan NJC. 1993. Length measurement of fish by computer vision. Comput Electron Agric 8(2):93–104. doi: http://dx.doi.org/ 10.1016/0168-1699(93)90009-P Strachan NJC. 1994. Sea trials of a computer vision based fish species sorting and size grading machine. Mechatronics 4(8):773–83. doi: http://dx.doi.org/ 10.1016/0957-4158(94)900523 Theodoridis S, Koutroumbas K. 1999. Pattern recognition. San Diego, Calif.: Academic Press. Veliyulin E, Misimi E, Bondø M, Vebenstad PA, Østvik SO. 2011. A simple method for weight estimation of whole herring (Clupeaharengus) using planar X-ray imaging. J Food Sci 76(3):E328–31. doi: 10.1111/j.1750-3841.2011.02093.x Wu F, Liu X, Zhang G, Wang C. 2009. Effects of the initial size, stocking density and sorting on the growth of juvenile Pacific abalone, Haliotis discus hannaiIno. Aquac Res 40(10):1103–10. doi: 10.1111/j.1365-2109.2009.02205.x Zion B. 2012. The use of computer vision technologies in aquaculture—a review. Comput Electron Agric 88:125–32. doi: http://dx.doi.org/ 10.1016/j.compag.2012.07.010 Zion B, Shklyar A, Karplus I. 2000. In-vivo fish sorting by computer vision. Aquacult Eng 22:165–79. doi: http://dx.doi.org/ 10.1016/S0144-8609(99)00037-0
Vol. 80, Nr. 4, 2015 r Journal of Food Science E733
E: Food Engineering & Materials Science
References
Introduction
because the rebound speed of the device is slow, grading cannot be processed rapidly. Several challenges preventing high accuracy sorting/grading of fish based on computer vision have been reported (Strachan 1994; Zion and others 2000, 2012; G¨um¨u¸s and others 2011), and a number of computer vision-based automated methods for measuring the length or estimating the weight of fish in order to sort/grade them have been proposed. Recently, those methods are also being used to estimate vaccine injection site of flatfish (Lee and others 2013). With regard to length, most of these methods measure the link length between the fish width midpoints (Strachan 1993; Ibrahim and Wang, 2009; Jeong and others 2013). With regard to weight estimation, the methods use a regression formula between the projected two-dimensional (2D) view area and the weight (G¨um¨u¸s and Balaban, 2010; Balaban and others 2010a, 2010b). However, these methods are subject to measurement inaccuracies because they disregard thickness information. Lee and others (2001) propose an oyster volume measurement algorithm that grades oysters automatically via a three-dimensional (3D) measurement method that considers thickness information. The algorithm combines a laser triangulation technique with 2D measurements to reconstruct a 3D surface for volume measurement. These 3D measurement methods facilitate accurate volume measurement, but their measurement systems are complex and the measurements are difficult to correct. Furthermore, they require complex processing procedures to collect the tomographic image data and reconstruct the surface (Lee and others 2003). This paper propose an automatic abalone grading algorithm on the basis of computer vision using 2D images that uses regression analysis of total length (TL), body length (BL), thickness (T), view MS 20140618 Submitted 4/13/2014, Accepted 12/4/2014. Authors are area (A), and actual volume of abalone against measured weight with Fisheries System Engineering Div., Natl. Fisheries Research & Development to surmount the challenges identified and accurately sort/grade Inst., Korea. Direct inquiries to author Yang (E-mail: [email protected]). abalone.
The value and price of marine products are dependent on the size of the products; therefore, accurate grading is required, and the grading process must be performed as quickly as possible in order to preserve the freshness of the product. However, because in most cases conventional methods manually measure the length and weight of the products, they are labor-intensive and time-consuming. Consequently, automatic grading technologies that can sort/grade marine products quickly and accurately, and thereby preserve their freshness, are required. As present, mechanical sorting/grading machines, such as those with increasing gap belts and roller graders, are used to sort/grade fish. These enable the processing of several fish in a short period of time; however, the resulting friction exerted on the fish may decrease their commercial value. Further, accurate grading cannot be expected because the weight and length of the fish are not measured during the processing (Veliyulin and others 2011). In South Korea, mechanical automatic abalone graders, which perform the grading process by comparing abalone weight to weights established according to grades, have been developed and are used in fish farms to grade the abalones automatically. In this method, abalone are placed, one at a time, on grading cups that are rotated on an oval-shaped rail and, as a result of the seesaw principle, abalone that are heavier than the weight set up on each grading cup fall off into a container. However, this equipment has a limitation: each grading cup and weight combination is designed to return to its original position after the abalone falls; however,
R C 2015 Institute of Food Technologists
doi: 10.1111/1750-3841.12799 Further reproduction without permission is prohibited
Vol. 80, Nr. 4, 2015 r Journal of Food Science E729
E: Food Engineering & Materials Science
Donggil Lee, Kyounghoon Lee, Seonghun Kim, and Yongsu Yang
An optimum abalone grading algorithm . . . Table 1–CCD camera specifications and test conditions.
Abalone features measurement and image processing In our study, the total length (TL) and body length (BL) of an CCD cameras abalone are measured simply by detecting the edge points of the Setting Base Side projected image from the bottom side of the abalone; in addition, Lens 16 (mm) 12 (mm) least squares fitting of ellipses with the detected edge points (Rosin CCD size 1/3 in, mono 1/3 in, mono 1993) was conducted to measure the longest and shortest lengths Pixels 640 × 480 640 × 480 of the ellipse. The view area of the abalones is easily calculated by Field of view (X) 223.36 (mm) 168.32 (mm) Resolution 0.349 (mm/pixel) 0.263 (mm/pixel) substituting the measured TL and BL of the abalones into the area equation of the ellipse (Eq. 1). Working distance 740 (mm) 400 (mm) Max. error 1 (pixel) 1 (pixel) The abalone’s thickness (T) is measured as the maximum height of the abalone’s side image, and the view area is measured using the abalone’s TL and BL values and the area equation of the ellipse, Materials and Methods as indicated by Eq. (1):
E: Food Engineering & Materials Science
Experimental animals In January 2014, 500 abalones, weighting in the range 16.51 π View Area A(cm2 ) = · T L · B L (1) to 128.01 g, and cultivated in the abalone hatchery of the South4 western Sea Research Center of the National Fisheries Research and Development Institute, were transferred to an indoor recirFigure 2 is the image processing flowchart for measuring the culation tank in the institute’s System Engineering Department abalone’s TL, BL, T, and view area: the preprocessing procedure for breeding and adaptation. All abalones were placed in water at 12.8 °C and were not fed for 24 h before the experiment to is shown in Figure 2(a) and the measuring procedure in Figure 2(b). As shown in Figure 2(a), during the preprocessing process, first, the minimize the effects of excretion on weight. image of the abalone’s base and the side projections are captured The experimental system simultaneously, and the brightness information of the images is To continuously measure various abalone sizes, a laboratory- converted to 8-bit grayscale. When the height of the abalone is scale system for measuring abalone features, shown in Figure 1, being measured, the closing computation is performed using 9 × 9 was manufactured. Abalones are measured by falling free from matrix structure images to flatten the extruding parts of the shell guide rail of an abalone feature measurement system with a grade (Gonzalez and Woods 2007). Next, in the binarization process, the pixel brightness of the 8-bit images is respectively converted of 35° and it is feasible to measure up to 3 abalones per second. The system consists of 0.3 megapixel charge-coupled device to 0 and 225 levels on the basis of 50 levels, to vividly show the (CCD) cameras (Panasonic Electric Works SUNX Co., Ltd., AN- edge points and the lines of base and side images of the projected PVC1040, Japan) installed at its base and sides, and a 30 W infrared abalone. After preprocessing and during the measuring process, as shown LED backlight installed across from the CCD camera to prevent worker’s dazzle and to generate a silhouette of the abalone. A trans- in Figure 2(b), first, the edge points are detected from the base mission photosensor (Autonics Co., Ltd., BYS500-TDT, Korea) image projected to measure the abalone’s TL, BL, and view area. is installed at the side to provide a trigger signal for capturing With the detected edge points, the external looped curve is dean image of the abalone for the vision system. A vision system tected from the base image of the projected abalone; the center (Panasonic Co., Ltd., PV200, Japan) and data collection system point and the angle of the area connected to the looped curve are (National Instruments, cRIO-9024, USA) were also present for extracted; and, while referring to the extracted center point and image processing, measurement, data collection, and analysis of the the angle, the coordinate is converted to a reference coordinate. abalones. Data collected from the vision system were transmitted Following coordinate conversion and by scanning from the outer over an Ethernet network and stored in a data collection system. edge to the direction of the center point with each circular pattern Table 1 lists the specifications of the CCD cameras and test and straight line pattern, 100 edge points are detected. Through conditions installed at the base and sides of the abalone feature least squares fitting of ellipses using the detected edge points, the TL and BL of the abalone are measured with the longest and measurement system.
Figure 1–Abalone features measurement system.
E730 Journal of Food Science r Vol. 80, Nr. 4, 2015
An optimum abalone grading algorithm . . . towel. The average value of 20 consecutive measurements was determined to be the weight of each abalone. Subsequently, each regression equation capable of weighing by measuring abalone features is extracted using a power curve regression model or linear regression analysis for each of the TL, BL, T, view area, and volume with respect to the measured abalone weight.
Abalone volume measurement and estimation The average of values measured 20 times repetitively using the density meter (Shinko Denshi Co., Ltd., DME-220, Japan) is used as to be the actual volume of the abalone. The abalone actual volume measurements are also obtained using a density meter Abalone weight measurement and estimation following removal of external water from the specimens by blotting The weight measurements of the abalones were obtained, using them on a paper towel. Using Archimedes’ principle, the actual a digital balance (A&D Co., Ltd., ER-180A, Japan), after external volume of each abalone is measured by substituting the weight of water was removed from the specimens by blotting them on a paper the abalone in the air, the abalone weight measured in distilled
Figure 2–Image processing flowchart for measuring TL, BL, T, and view area of the abalone: (a) preprocessing process, (b) measuring process.
Vol. 80, Nr. 4, 2015 r Journal of Food Science E731
E: Food Engineering & Materials Science
shortest sides of the ellipse, respectively, and the view area of the abalone is calculating Eq. (1). The abalone’s T is measured with the maximum thickness of the side image of the projected abalone via a method of that scants from brightness to darkness using the edge detection technique (Davis 1975). Here, because measurement variation may occur depending on the position of the CCD cameras and the abalone given the cameras’ perspective, using the algorithm proposed by Lee and others (2012), the center point coordinate of the measurement subject’s area is found and corrected by the amount of thickness variations, which occur according to measurement distance.
An optimum abalone grading algorithm . . . Table 2–Equation fit results for abalone weight compared to TL, sensor, which has a suitable property for rectilinear propagation, is BL, T, A, Cv , and Av . used instead of CCD cameras to measure abalone T, it might not Equation I y=
a·xb
TL
BL
T
A
a 0.071 0.319 8.029 0.202 b 3.285 3.141 2.440 1.635 R2 0.969 0.973 0.923 0.979
Equation II y = a·x
Cv
Av
a 1.222 1.224 R2 0.998 0.999
∗
Equation I: The y denotes weight (g), x represents the TL, BL, T (cm), and A (cm2 ), which represent view area. ∗ Equation II: The y denotes weight (g), x represents the Cv and Av (cm3 ), respectively. ∗ The a and b represents regression coefficients.
E: Food Engineering & Materials Science
water, and the density of the distilled water into Eq. (2). In our study, we considered the density of water to be 1 because the density of pure water at room temperature (25 °C) is 0.997 g/cm3 (Damar and others 2007) Actual volume Av (cm3 ) =
weight in air (g) − weight in water(g) density of water(g/cm3 ) (2)
To calculate the abalone volume using the simplified method, the abalones are assumed to have the shape of half-oblate ellipsoids (Wu and others 2009); consequently, using the measured TL, BL, and T of the abalone and Eq. (3) for irregular half-oblate ellipsoids (Du and Sun 2006), the volume of the abalone is calculated Calculated volume Cv (cm3 ) =
π · TL · BL · T 6
(3)
To compare and analyze the calculated volume and the actual volume of the abalone, and to predict the actual volume using Eq. (3), which comprises of the measurement parameters of TL, BL, and T, a linear regression analysis without the intercept was performed between the calculated volume (Cv ) and the actual volume (Av ), as shown in Eq. (4), in which a is regression coefficient: Av (c m 3 ) = a · Cv
Experimental Results
(4)
be necessary to correct T according to the location of the abalone, and a more accurate measurement might be obtained. The R2 value between the view area and the abalone weight is 0.979, which is similar to the R2 value between the view area and the fish weight from previous studies (Balaban and others 2010b). The experimental results indicate that the regression formulas between the weight and each of TL, BL, T, and view area are nonlinear distribution distributed, and that a more precise measurement technology is required to subdivide the grades by abalone weight. However, if there is no need to subdivide the grades by abalone weight and if there is no need to develop an automatic grading system at low cost, then the automatic grading method, which estimates weights through the conventional view area measurement, is more efficient (Balaban and others 1994; Eifert and others 2006). As listed in Table 2, compared to the R2 values between the weight and each abalone TL, BL, T, and view area, the R2 value between the actual volume and abalone weight, 0.999, is the highest. The next highest R2 value is 0.998, which is the value between the calculated volumes and the weights. From these results, it can be deduced that (1) the volume and the abalone weight have a linear relationship, (2) it is possible to subdivide the grades by abalone weight, and (3) accurate weight measurement is possible because, relative to the R2 values between the weight and each abalone TL, BL, T, and view area, the R2 value between the actual volume and the abalone weight is higher. Further, we ascertained that the slope is 1.224 g/cm3 which means the abalone’s density from regression analysis between the weight and actual volume of the abalone. Figure 3 shows the relationship between the calculated volumes and the actual volumes as indicated in Eq. (4), regression analysis results relationship between the show that the regression coefficient a and R2 value are 0.999 and 0.998, respectively. In the experimental results, the slope of the regression formula between the calculated volume of the abalone and the actual volume is close to 1. By means of this basic experiment, we ascertained that our method for estimating weight via volume measurement of the abalone is effective. Furthermore, we discovered that the actual volume of the abalone can be estimated using the volume
The relationship between the measured weight and each of TL, BL, T, and view area shows that the nonlinear distribution and the regression analysis were performed using the power curve regression mode, respectively. The relationship between measured weight and each of calculated volume and actual volume show that the linear distributed, and regression analysis was performed using the linear regression analysis without the intercept. The results of the regression analysis between the measured weight and each abalone measurement factor are summarized in Table 2. Table 2 indicates that, the R2 value between the measured weight and each TL, BL, and the view area is more than 0.969, but the R2 value between T and weight is 0.923, which shows a relatively lower correlation. It is assumed that even a small change in the T of the abalone has a major impact on the weight because the degree of meat thickness varies by individual abalone pertaining to the T. In addition, the organisms attached to the surface impact the results. Consequently, we believe that a technology that can filter the images of meat and attached organisms is required in order to estimate the weight along with the thickness of the abalone. Furthermore, it appears that if a through-beam laser photoelectric Figure 3–Linear curve regression for abalone Cv compared to Av . E732 Journal of Food Science r Vol. 80, Nr. 4, 2015
An optimum abalone grading algorithm . . . weight of abalones, can be effectively applied for growth analysis of abalones.
Acknowledgment This work was supported by a grant from the National Fisheries Research and Development Institute (NFRDI), Republic of Korea. (RP-2015-FE-002).
Author Contributions D. Lee designed and performed the study. K. Lee and S. Kim collected test data and analyzed data. Y. Yang interpreted the results and drafted the manuscript.
Figure 4–Weight estimation error distribution evaluated via leave-one-out cross-validation of fresh abalone.
calculated under the assumption that abalones have the shape of half-oblate ellipsoids. From the results of our study, we derived the automatic abalone grading algorithm for abalone’s weight estimation indicated in Eq. (5) using the regression formula shown in Eq. (4) between the actual volumes and the abalone weights listed in Table 2: this algorithm facilitates automatic abalone grading based on computer vision π (5) Estimated weight (g) = · T L · B L · T 5 We evaluated the performance of this algorithm via leave-oneout cross-validation (Lachenbruch and Mickey 1968; Theodoridis and Koutroumbas 1999) and thereby ascertained that the root mean square error and the weight prediction error spread are 2.8 and ±8 g, for a sample (range of abalones weight: 16.51 to 128.01 g), respectively, as shown in Figure 4. These results show improvements over the results obtained from the methods used to estimate fish weight via the 3D volume measurements in previous studies (Storbeck and Daan 1991; Mathiassen and others 2011).
Conclusion In this paper, we showed that abalone volume can be calculated by measuring the abalone’s TL, BT, and T via a simple method that assumes that the shape of the abalones is a half-oblate ellipsoid, without using conventional expensive equipment or complex formulas. Further, we showed that estimating the abalone’s weight by measuring its volume is more effective than estimating the weight by measuring the TL, BL, T, and view area. From the experimental results, in the range of abalones weighting from 16.51 to 128.01 g, the results of evaluation of the performance of algorithm via cross-validation indicate root mean square and worst-case prediction errors of are 2.8 and ±8 g, respectively. We also found that estimating the abalone’s weight using its view area is also useful, and is similar to conventional methods that estimate fish weight using a regression formula between view areas and fish weight, and therefore can be used to minimize development cost. We expect that these finding will enable abalone farmers to provide fresh abalones to consumers and generate profits. Furthermore, we believe that in the field of marine research, this algorithm, developed to estimate the volume and
Balaban MO, Bergmann Y, Yeralan S, Otwell WS. 1994. Determination of count and uniformity ratio of shrimp by machine vision. J Aquat Food Prod Technol 3(3):43–58. doi: 10.1300/J030v03n03_04 ¨ Balaban MO, Unal-S ¸ eng¨or GF, Soriano MG, Guill´en Ruiz E. 2010a. Using image analysis to predict the weight of Alaskan salmon of different species. J Food Sci 75(3):E157–62. doi: 10.1111/j.1750-841.2010.01522.x Balaban MO, Chombeau M, Cırban D, G¨um¨u¸s B. 2010b. Prediction of the weight of Alaskan Pollock using image analysis. J Food Sci 75(8):E552–6. doi: 10.1111/j.17503841.2010.01813.x Damar S, Yagiz Y, Balaban MO, Ural S, Oliveira ACM, Crapo CA. 2007. Prediction of oyster volume and weight using machine vision. J Aquat Food Prod Technol 15(4):3–15. doi: 10.1300/J030v15n04_02 Davis LS. 1975. A survey of edge detection techniques. Comput Vision Graph 4(3):248–70. doi: http://dx.doi.org/ 10.1016/0146-664X(75)90012-X Du CJ, Sun DW. 2006. Estimating the surface area and volume of ellipsoidal ham using computer vision. J Food Eng 73(3):260–8. doi:http://dx.doi.org/ 10.1016/j.jfoodeng.2005.01.029 Eifert JD, Sanglay GC, Lee D-J, Summer SS, Pierson MD. 2006. Prediction of raw produce surface area from weight measurement. J Food Eng 74(4):552–6. doi:http://dx.doi.org/ 10.1016/j.jfoodeng.2005.02.030 Gonzalez R, Woods R. 2007. Digital image processing. Upper Saddle River, N.J.: Prentice Hall. G¨um¨u¸s B, Balaban MO. 2010. Prediction of the weight of aquacultured rainbow trout (Oncorhynchusmykiss) by image analysis. J Aquat Food Prod Technol 19(3–4):227–37. doi: 10.1080/10498850.2010.508869 ¨ Unl¨ ¨ usayın M. 2011. Machine vision applications to aquatic foods: a G¨um¨u¸s B, Balaban MO, review. Turkish J Fish Aqua. Sci 11:71–81. doi: 10.4194/trjfas.2011.0124 Ibrahim MY, Wang J. 2009. Mechatronics applications to fish sorting Part 1: fish size identification. IEEE International Symposium on Industrial Electronics. ISIE 2009:1978–83. doi: 10.1109/ISIE.2009.5215660 Jeong SJ, Yang YS, Lee K, Kang JG, Lee DG. 2013. Vision-based automatic system for non-contact measurement of morphometric characteristics of flatfish. J Electr Eng Technol 8(5):1194–201. Lachenbruch PA, Mickey RM. 1968. Estimation of error rates in discriminant analysis. Technometrics 10:1–11. doi: 10.1080/00401706.1968.10490530 Lee DG, Yang YS, Kim S, Choi JH, Kang JG, Kim HJ. 2012. A study on system for measuring morphometric characteristic of fish using morphological image processing. J Kor Soc Fish Technol 48:469–78. Lee DG, Yang YS, Kang JG. 2013. Conveyor belt automatic vaccine injection system (AVIS) for flatfish based on computer vision analysis of fish shape. Aquacult Eng. 57:54–62. doi: 10.1016/j.aquaeng.2013.08.004 Lee DJ, Lane RM, Chang GH. 2001. Three-dimensional reconstruction for highspeed volume measurement. In: Intelligent systems and smart manufacturing. International Society for Optics and Photonics. p 258–67. http://proceedings.spiedigitallibrary. org/proceeding.aspx?articleid=926941. doi: 10.1117/12.417201 Lee DJ, Eifert J, Zhan P, Westover B. 2003. Fast surface approximation for volume and surface area measurements using distance transform. Opt Eng 42(10):2947–55. doi:10.1117/1.1605737 Mathiassen JR, Misimi E, Toldnes B, Bondø M, Østvik SO. 2011. High-speed weight estimation of whole herring (Clupeaharengus) using 3D machine vision. J Food Sci 76(6):E458–64. doi: 10.1111/j.1750-3841.2011.02226.x Rosin PL. 1993. A note on the least squares fitting of ellipses. Pattern Recogn Lett 14(10):799– 808. Storbeck F, Daan B. 1991. Weight estimation of flatfish by means of structured light and image analysis. Fish Res 11:99–108. doi:http://dx.doi.org/ 10.106/0165-7836(91)90101-K Strachan NJC. 1993. Length measurement of fish by computer vision. Comput Electron Agric 8(2):93–104. doi: http://dx.doi.org/ 10.1016/0168-1699(93)90009-P Strachan NJC. 1994. Sea trials of a computer vision based fish species sorting and size grading machine. Mechatronics 4(8):773–83. doi: http://dx.doi.org/ 10.1016/0957-4158(94)900523 Theodoridis S, Koutroumbas K. 1999. Pattern recognition. San Diego, Calif.: Academic Press. Veliyulin E, Misimi E, Bondø M, Vebenstad PA, Østvik SO. 2011. A simple method for weight estimation of whole herring (Clupeaharengus) using planar X-ray imaging. J Food Sci 76(3):E328–31. doi: 10.1111/j.1750-3841.2011.02093.x Wu F, Liu X, Zhang G, Wang C. 2009. Effects of the initial size, stocking density and sorting on the growth of juvenile Pacific abalone, Haliotis discus hannaiIno. Aquac Res 40(10):1103–10. doi: 10.1111/j.1365-2109.2009.02205.x Zion B. 2012. The use of computer vision technologies in aquaculture—a review. Comput Electron Agric 88:125–32. doi: http://dx.doi.org/ 10.1016/j.compag.2012.07.010 Zion B, Shklyar A, Karplus I. 2000. In-vivo fish sorting by computer vision. Aquacult Eng 22:165–79. doi: http://dx.doi.org/ 10.1016/S0144-8609(99)00037-0
Vol. 80, Nr. 4, 2015 r Journal of Food Science E733
E: Food Engineering & Materials Science
References
