Journal of Cranio-Maxillo-Facial Surgery 43 (2015) 144e148
Contents lists available at ScienceDirect
Journal of Cranio-Maxillo-Facial Surgery journal homepage: www.jcmfs.com
Gender differences in posed smiles using principal component analysis Katsuaki Mishima*, Asuka Nakano, Hirotsugu Umeda, Ruriko Shiraishi, Yoshiya Ueyama Department of Oral and Maxillofacial Surgery, Graduate School of Medicine, Yamaguchi University, Minami-kogushi 1-1-1, Ube City, Yamaguchi 755-8505, Japan
a r t i c l e i n f o
a b s t r a c t
Article history: Paper received 17 May 2014 Accepted 22 October 2014 Available online 15 November 2014
Objectives: The purpose of this study was to clarify gender differences in posed smiles using principal component analysis (PCA). Materials and methods: Fourteen adult volunteers, 7 males and 7 females, were enrolled. Using the motion analyzing system we developed, range images and 5 5 virtual grids were produced across the whole sequence while the volunteers were asked to smile. Two sets of all intersections of the virtual grids captured while the subject was smiling were regarded as PCA variables. Discriminate analysis was then applied to compare the males and females. Results: The first and second principal component scores (PCSs) were plotted on the x-axis and y-axis, respectively. The center of gravity of the PCSs is shown by the plus on the x-axis and minus on the y-axis for the males and by the minus on the x-axis and the plus on the y-axis for the females. Discriminate analyses of the PCSs revealed a correct gender classification rate of 74.4% for posed smiles. Conclusions: While the sample size is too small to extrapolate from these results, we can conclude that PCA can be used to identify gender differences while smiling. © 2014 European Association for Cranio-Maxillo-Facial Surgery. Published by Elsevier Ltd. All rights reserved.
Keywords: Discriminate analysis Gender difference Motion analysis Posed smile Principal component analysis
1. Introduction A nice smile is extremely important to people. Therefore, improving the smile is an important goal of orthodontic treatment (Meyer et al., 2014) and an important outcome of maxillofacial surgery (Popat et al., 2012a). To analyze smiles, both qualitative and quantitative methods are indispensable. If we can determine parameters to analyze that reflect the characteristics of a good smile, quantitative approaches would become more dominant. In other words, if characteristics associated with a balanced smile or healthy smile, or if characteristics that would allow gender differentiation could be identified, significant progress could be made in the study of facial expressions. Several recent articles have discussed the use of principal component analysis (PCA) for this purpose (Valentin et al., 1997; Popat et al., 2010). On the other hand, there have been reports that both spontaneous smiles (Van der Geld et al., 2007) and posed smiles
* Corresponding author. Tel.: þ81 836 22 2299; fax: þ81 836 22 2298. E-mail address: [email protected] (K. Mishima).
(Ackerman et al., 1998) could be measured reproducibly. In our previous article, both the three-dimensional intra-rater reliability and the inter-rater reliability of measurements taken for a posed smile were considered to be relatively high, and sufficient reliability could be expected by calculating the average of values measured twice (Mishima et al., 2014). Therefore, in the present study, PCA was applied to two sets of data for a posed smile, and gender differences were analyzed. 2. Materials and methods 2.1. Subjects Fourteen adult volunteers (7 males aged 24e34 and 7 females aged 24e29 years) who had no medical history related to lip movement were enrolled. The subjects were seated with no fixation of their heads. First they were asked to speak five Japanese vowels and to open their mouth wide. Then, they were asked to smile, and their lip motion was measured twice while smiling. This research was approved by the Institutional Review Board of Yamaguchi University Hospital.
http://dx.doi.org/10.1016/j.jcms.2014.10.026 1010-5182/© 2014 European Association for Cranio-Maxillo-Facial Surgery. Published by Elsevier Ltd. All rights reserved.
K. Mishima et al. / Journal of Cranio-Maxillo-Facial Surgery 43 (2015) 144e148
2.2. Production of range images Motion images obtained by three infrared digital video cameras (IR camera, Sony XC-E150, Tokyo) and one color digital video camera (Sony DXC-390, Tokyo), controlled by a synchronizing signal generator (Imagenics SG-701, Tokyo), were recorded with an infrared pattern projected onto the face at a sampling rate of 30 frames per second. These image sequences were captured directly on a personal computer via IEEE1394. Each frame was digitized at a horizontal and vertical resolution of 720 480 pixels. The computer programs described below were based on 3D Video™ software developed by OGIS Research Institute Co. Ltd. (Osaka), but were improved for the needs of this specific application. The image processing has been described in detail in a previous article (Mishima et al., 2006) and is explained briefly here. The intrinsic and extrinsic parameters for the four cameras, 1 color and 3 IR, were obtained using a known object, which was a cube in which a checkerboard was printed. An infrared pattern was projected and the images were recorded through three IR cameras. To find a match, two different techniques, a multiple-baseline stereo technique (Okutomi and Kanade, 1991) and a template-matching technique (Tamura, 2002), were applied. After stereo-matching, the disparity was then calculated, and a range image was produced across the whole image sequences. The subjects' heads were not fixed in order not to disturb spontaneous movements of the face. The movement of the head during recording was canceled using a sun visor on which a checkerboard pattern was printed, and which was worn on the head. The intersections of the checkerboard pattern were automatically tracked across image sequences by applying the LucasKanade algorithm (Lucas and Kanade, 1981). The head position was compensated for as follows: the distances between designated immobile points from one frame to the next would be minimized using a least-squares method within the constraints of the orthogonal matrix.
145
were produced. The intersections were projected perspectively onto the curved surfaces of the range images, and the threedimensional coordinates of these intersections were computed. Lip motions when making a posed smile were analyzed using PCA. The program was based on Body Shape Browser™ (Ergovision, Osaka) and customized for the needs of this specific application. Before PCA, distances between both mouth corners at rest were normalized for all subjects. Two sets of three-dimensional coordinates of all intersections of the virtual grid were regarded as variables of PCA for all frames from the starting to the end frame while making a posed smile. The first to the 12th principal component scores (PCSs) were then calculated. Lip configurations corresponding to specific PCSs could be observed by adjusting each PCS available in the present program using a sliding scale. The relationship between lip configurations and the PCSs were investigated. It was found that the first and second scores were parameters that almost completely described anterior-posterior movement and superior-inferior movement of the lips, respectively (Fig. 2). Therefore, the first and second scores were plotted (first on the x-axis and second on the y-axis), and discriminate analysis was then applied to compare the difference in the posed smiles of males and females (IBM SPSS ver. 22). 3. Results The PCSs plotted in two dimensions (first PCS on the x-axis and second PCS on the y-axis) are shown in Fig. 3. The center of gravity of the PCSs is shown by the plus sign on the x-axis and by a minus sign on the y-axis for the males, and by a minus sign on the x-axis and a plus sign on the y-axis for the females. Discriminate analysis of the points (PCSs) plotted in two dimensions for the male and female subjects revealed that Wilk's lambda and significant probability were 0.676 and 0.000, respectively, and that the measurements were able to correctly discriminate males and females for 74.4% of the posed smiles.
2.3. Analysis of lip motion
4. Discussion
The lips were divided into eight areas, which consisted of four areas in the upper white lip and four areas in the upper and lower vermilions. Each area consisted of 15 straight lines connecting ten landmarks and three Bezier curves. In these eight areas, the following 5 5 virtual grids were produced (Fig. 1). A threedimensional curved line on the range image was divided into several segments. The divided points were projected in two dimensions (XY planes), and intersections between these gridlines
In studies on smiles, two types of smiles, a spontaneous smile and a posed smile, are often described. Ackerman et al. (1998) have stated that a posed smile can be repeated reliably. On the other hand, Van der Geld et al. (2007) reported that the reproducibility of a spontaneous smile is also high, and indicated that a spontaneous smile is a person's authentic smile expression. However, until now there have been no reports comparing the reproducibility of a spontaneous smile and a posed smile using the same measurement
Fig. 1. Eight areas and virtual grids applied to the lip. Grids (5 5) were applied to eight areas, as determined by ten landmarks and Bezier curved lines (a). Two sets of threedimensional coordinates of the virtual grid intersections were utilized for PCA (b). Blue dots indicate the intersections of the virtual grid (c).
146
K. Mishima et al. / Journal of Cranio-Maxillo-Facial Surgery 43 (2015) 144e148
Fig. 2. Lip shapes corresponding to the first and second PCSs. The first and second PCSs almost completely described anterior-posterior movement and superior-inferior movement of the lips, respectively. In the left of each figure, sliding scales of each PCS available in the present program, by which lip configurations corresponding PCSs can be observed, are shown; (a) minus of the first PCS, (b) plus of the first PCS, (c) minus of the second PCS, and (d) plus of the second PCS.
system. Generally speaking, recording a spontaneous smile is time consuming and resource intensive. The main purpose of our study was to evaluate lip movement, and we did not seek to evaluate the quality of the smile or the emotion behind it. Therefore, a posed smile was sufficient for our purposes. Measuring surface shift is considered to be able to better reflect real lip movement compared to measuring points. Therefore, virtual grids were applied to the lips, preventing the loss of surface information. As for the reproducibility of measurement of the posed smiles using the virtual grids, it was reported that both ICC (1,1) and ICC (2,1) are high, ranging from 0.71 to 0.83 and from 0.77 to 0.99, respectively, in our previous manuscript (Mishima et al., 2014). Furthermore, reliability can be improved by calculating the average of duplicate measurements (Mishima et al., 2014). Therefore, two measurements were made for each person in the present study. The central idea of PCA is to reduce the dimensionality of a data set consisting of a large number of interrelated variables, while retaining as much as possible of the variation present in the data set (Jolliffe, 2002). In other words, it is hoped that PCA can be used to extract important characteristics from huge data sets. The first principal component is the score having the highest rate of explained variance (Jolliffe, 2002). There have been some reports on the PCSs derived by application of PCA to lip motion. Popat et al. (2010) have stated that the first PCS represents the largest variation in lip movement for a particular facial gesture, and that the first PCS for the verbal gesture, the word “puppy,” is lip opening. Popat et al.
(2012b) have noted that the first four PCSs describe the majority of variation in lip movement during articulation involving a complex interaction of lip movements in three dimensions. In our previous manuscript, PCA applied to three-dimensional lip movement during phonation of five Japanese vowels using the present videobased measuring system revealed that the fourth and fifth PCSs are the parameters that control the mouth opening and the retraction of the corners of the mouth, respectively (Mishima et al., 2011). In the present study, the first and second PCSs were revealed to be the parameters that almost completely describe anteriorposterior movement and superior-inferior movement of the lips, respectively. Because only a posed smile was targeted for PCA, it is easy to understand why anterior-posterior movement is the largest variation. There are several reports that study gender differences when smiling. Kawamura and Kageyama (2006) have reported that Japanese university students rate smiling faces as being more feminine than serious faces. Kawamura et al. (2008) have also suggested that smiling significantly reduces the perceived masculinity of men's faces. To date, gender differences in smiling have mostly been evaluated qualitatively and subjectively. There was one study that measured the relationship between the lips and the teeth or the gingiva while smiling (Singh et al., 2013) The measurements in this study were done in two dimensions. In the article, significant observations included increased resting upper lip length for females and decreased upper lip thickness, maxillary incisor exposure, and lip elevation for males. However, there have been no articles
K. Mishima et al. / Journal of Cranio-Maxillo-Facial Surgery 43 (2015) 144e148
147
Fig. 3. Graphs of the first and second PCSs. The PCSs are plotted in two dimensions (first PCS on x-axis and second PCS on y-axis). Red and green dots indicate PCSs and centers of gravity of the PCSs, respectively, and the red lines indicate temporal trajectories of the PCSs. The center of gravity of the PCSs is shown by the plus on the x-axis and minus on the yaxis for the males (a) and by the minus on the x-axis and the plus on the y-axis for the females (b).
examining differences in smiling by measuring lip movements three-dimensionally to date. In the present study, gender could be correctly identified while the subject was smiling in 74.4% of cases using the first and second PCSs. While the sample size is too small to extrapolate from these results, we can conclude that PCA can be used to identify gender differences while smiling. In addition, it has been reported that there are cultural or ethnic differences in the perception of smiles and dynamic characteristics of smiles (Sharma et al., 2012; Liang et al., 2013). In extrapolation from the present study results, it is necessary to consider cultural and ethnic backgrounds carefully.
5. Conclusions Gender differences in a posed smile were investigated using PCA. Fourteen adult volunteers, 7 males and 7 females, were enrolled. Using our developed motion analyzing system, range images and virtual grids of 5 5 were produced across the whole sequence while each subjects was making a posed smile. Two sets of all intersections of the virtual grids during a posed smile were regarded as PCA variables. Gender could be determined while the subject was smiling, with a correct classification rate of 74.4% using the first and second PCSs. While the sample size is too small to
148
K. Mishima et al. / Journal of Cranio-Maxillo-Facial Surgery 43 (2015) 144e148
extrapolate from these results, we can conclude that PCA can be used to identify gender differences while smiling. Conflicts of interest The authors have no conflicts of interest in relation to the current investigation. Acknowledgments The authors thank Mr. Shozo Hirose from Ivis Co. Ltd. (Osaka) and Mr. Yuji Uchida from Ergovision Co. Ltd. (Osaka) for their advice. References Ackerman JL, Ackerman MB, Brensinger CM, Landis JR: A morphometric analysis of the posed smile. Clin Orthod Res 1: 2e11, 1998 Jolliffe IT: Principal component analysis, 2nd edn. New York: Springer, 1e6, 2002 Kawamura S, Kageyama K: Smiling faces rated more feminine than serious faces in Japan. Percept Mot Skills 103: 210e214, 2006 Kawamura S, Komori M, Miyamoto Y: Smiling reduces masculinity: principal component analysis applied to facial images. Perception 37: 1637e1648, 2008 Liang LZ, Hu WJ, Zhang YL, Chung KH: Analysis of dynamic smile and upper lip curvature in young Chinese. Int J Oral Sci 5: 49e53, 2013 Lucas BD, Kanade T: An iterative image registration technique with an application to stereo vision. In: Proceedings of the International Joint Conference on Artificial Intelligence; 1981, 674e679, 1981 Meyer AH, Woods MG, Manton DJ: Maxillary arch width and buccal corridor changes with orthodontic treatment. Part 2: attractiveness of the frontal facial smile in extraction and nonextraction outcomes. Am J Orthod Dentofacial Orthop 145: 296e304, 2014
Mishima K, Yamada T, Ohura A, Sugahara T: Production of a range image for facial motion analysis: a method for analyzing lip motion. Comput Med Imaging Graph 30: 53e59, 2006 Mishima K, Yamada T, Matsumura T, Moritani N: Analysis of lip motion using principal component analyses. J Craniomaxillofac Surg 39: 232e236, 2011 Mishima K, Umeda H, Nakano A, Shiraishi R, Hori S, Ueyama Y: Three-dimensional intra-rater and inter-rater reliability during a posed smile using a video-based motion analyzing system. J Craniomaxillofac Surg 42: 428e431, 2014 Okutomi M, Kanade T: A multiple baseline stereo. In: Proceedings of the 1991 IEEE conference on computer vision and pattern recognition; 1991, 63e69, 1991 Popat H, Henley E, Richmond S, Benedikt L, Marshall D, Rosin PL: A comparison of the reproducibility of verbal and nonverbal facial gestures using threedimensional motion analysis. Otolaryngol Head Neck Surg 142: 867e872, 2010 Popat H, Richmond S, Marshall D, Rosin PL: Three-dimensional assessment of functional change following Class 3 orthognathic correctionea preliminary report. J Craniomaxillofac Surg 40: 36e42, 2012a Popat H, Zhurov AI, Toma AM, Richmond S, Marshall D, Rosin PL: Statistical modelling of lip movement in the clinical context. Orthod Craniofac Res 15: 92e102, 2012b Sharma N, Rosenstiel SF, Fields HW, Beck FM: Smile characterization by U.S. white, U.S. Asian Indian, and Indian populations. J Prosthet Dent 107: 327e335, 2012 Singh B, Ahluwalia R, Verma D, Grewal SB, Goel R, Kumar PS: Perioral age-related changes in smile dynamics along the vertical plane: a videographic crosssectional study. Angle Orthod 83: 468e475, 2013 Tamura H: Computer image processing. Tokyo: Ohmsha Publishing, 184e197 and 252-265 [in Japanese], 2002 Valentin D, Abdi H, Edelman B, O'Toole AJ: Principal component and neural network analyses of face images: what can be generalized in gender classification? J Math Psychol 41: 398e413, 1997 Van der Geld PA, Oosterveld P, van Waas MA, Kuijpers-Jagtman AM: Digital videographic measurement of tooth display and lip position in smiling and speech: reliability and clinical application. Am J Orthod Dentofacial Orthop 131, 2007 301.e1-8
Contents lists available at ScienceDirect
Journal of Cranio-Maxillo-Facial Surgery journal homepage: www.jcmfs.com
Gender differences in posed smiles using principal component analysis Katsuaki Mishima*, Asuka Nakano, Hirotsugu Umeda, Ruriko Shiraishi, Yoshiya Ueyama Department of Oral and Maxillofacial Surgery, Graduate School of Medicine, Yamaguchi University, Minami-kogushi 1-1-1, Ube City, Yamaguchi 755-8505, Japan
a r t i c l e i n f o
a b s t r a c t
Article history: Paper received 17 May 2014 Accepted 22 October 2014 Available online 15 November 2014
Objectives: The purpose of this study was to clarify gender differences in posed smiles using principal component analysis (PCA). Materials and methods: Fourteen adult volunteers, 7 males and 7 females, were enrolled. Using the motion analyzing system we developed, range images and 5 5 virtual grids were produced across the whole sequence while the volunteers were asked to smile. Two sets of all intersections of the virtual grids captured while the subject was smiling were regarded as PCA variables. Discriminate analysis was then applied to compare the males and females. Results: The first and second principal component scores (PCSs) were plotted on the x-axis and y-axis, respectively. The center of gravity of the PCSs is shown by the plus on the x-axis and minus on the y-axis for the males and by the minus on the x-axis and the plus on the y-axis for the females. Discriminate analyses of the PCSs revealed a correct gender classification rate of 74.4% for posed smiles. Conclusions: While the sample size is too small to extrapolate from these results, we can conclude that PCA can be used to identify gender differences while smiling. © 2014 European Association for Cranio-Maxillo-Facial Surgery. Published by Elsevier Ltd. All rights reserved.
Keywords: Discriminate analysis Gender difference Motion analysis Posed smile Principal component analysis
1. Introduction A nice smile is extremely important to people. Therefore, improving the smile is an important goal of orthodontic treatment (Meyer et al., 2014) and an important outcome of maxillofacial surgery (Popat et al., 2012a). To analyze smiles, both qualitative and quantitative methods are indispensable. If we can determine parameters to analyze that reflect the characteristics of a good smile, quantitative approaches would become more dominant. In other words, if characteristics associated with a balanced smile or healthy smile, or if characteristics that would allow gender differentiation could be identified, significant progress could be made in the study of facial expressions. Several recent articles have discussed the use of principal component analysis (PCA) for this purpose (Valentin et al., 1997; Popat et al., 2010). On the other hand, there have been reports that both spontaneous smiles (Van der Geld et al., 2007) and posed smiles
* Corresponding author. Tel.: þ81 836 22 2299; fax: þ81 836 22 2298. E-mail address: [email protected] (K. Mishima).
(Ackerman et al., 1998) could be measured reproducibly. In our previous article, both the three-dimensional intra-rater reliability and the inter-rater reliability of measurements taken for a posed smile were considered to be relatively high, and sufficient reliability could be expected by calculating the average of values measured twice (Mishima et al., 2014). Therefore, in the present study, PCA was applied to two sets of data for a posed smile, and gender differences were analyzed. 2. Materials and methods 2.1. Subjects Fourteen adult volunteers (7 males aged 24e34 and 7 females aged 24e29 years) who had no medical history related to lip movement were enrolled. The subjects were seated with no fixation of their heads. First they were asked to speak five Japanese vowels and to open their mouth wide. Then, they were asked to smile, and their lip motion was measured twice while smiling. This research was approved by the Institutional Review Board of Yamaguchi University Hospital.
http://dx.doi.org/10.1016/j.jcms.2014.10.026 1010-5182/© 2014 European Association for Cranio-Maxillo-Facial Surgery. Published by Elsevier Ltd. All rights reserved.
K. Mishima et al. / Journal of Cranio-Maxillo-Facial Surgery 43 (2015) 144e148
2.2. Production of range images Motion images obtained by three infrared digital video cameras (IR camera, Sony XC-E150, Tokyo) and one color digital video camera (Sony DXC-390, Tokyo), controlled by a synchronizing signal generator (Imagenics SG-701, Tokyo), were recorded with an infrared pattern projected onto the face at a sampling rate of 30 frames per second. These image sequences were captured directly on a personal computer via IEEE1394. Each frame was digitized at a horizontal and vertical resolution of 720 480 pixels. The computer programs described below were based on 3D Video™ software developed by OGIS Research Institute Co. Ltd. (Osaka), but were improved for the needs of this specific application. The image processing has been described in detail in a previous article (Mishima et al., 2006) and is explained briefly here. The intrinsic and extrinsic parameters for the four cameras, 1 color and 3 IR, were obtained using a known object, which was a cube in which a checkerboard was printed. An infrared pattern was projected and the images were recorded through three IR cameras. To find a match, two different techniques, a multiple-baseline stereo technique (Okutomi and Kanade, 1991) and a template-matching technique (Tamura, 2002), were applied. After stereo-matching, the disparity was then calculated, and a range image was produced across the whole image sequences. The subjects' heads were not fixed in order not to disturb spontaneous movements of the face. The movement of the head during recording was canceled using a sun visor on which a checkerboard pattern was printed, and which was worn on the head. The intersections of the checkerboard pattern were automatically tracked across image sequences by applying the LucasKanade algorithm (Lucas and Kanade, 1981). The head position was compensated for as follows: the distances between designated immobile points from one frame to the next would be minimized using a least-squares method within the constraints of the orthogonal matrix.
145
were produced. The intersections were projected perspectively onto the curved surfaces of the range images, and the threedimensional coordinates of these intersections were computed. Lip motions when making a posed smile were analyzed using PCA. The program was based on Body Shape Browser™ (Ergovision, Osaka) and customized for the needs of this specific application. Before PCA, distances between both mouth corners at rest were normalized for all subjects. Two sets of three-dimensional coordinates of all intersections of the virtual grid were regarded as variables of PCA for all frames from the starting to the end frame while making a posed smile. The first to the 12th principal component scores (PCSs) were then calculated. Lip configurations corresponding to specific PCSs could be observed by adjusting each PCS available in the present program using a sliding scale. The relationship between lip configurations and the PCSs were investigated. It was found that the first and second scores were parameters that almost completely described anterior-posterior movement and superior-inferior movement of the lips, respectively (Fig. 2). Therefore, the first and second scores were plotted (first on the x-axis and second on the y-axis), and discriminate analysis was then applied to compare the difference in the posed smiles of males and females (IBM SPSS ver. 22). 3. Results The PCSs plotted in two dimensions (first PCS on the x-axis and second PCS on the y-axis) are shown in Fig. 3. The center of gravity of the PCSs is shown by the plus sign on the x-axis and by a minus sign on the y-axis for the males, and by a minus sign on the x-axis and a plus sign on the y-axis for the females. Discriminate analysis of the points (PCSs) plotted in two dimensions for the male and female subjects revealed that Wilk's lambda and significant probability were 0.676 and 0.000, respectively, and that the measurements were able to correctly discriminate males and females for 74.4% of the posed smiles.
2.3. Analysis of lip motion
4. Discussion
The lips were divided into eight areas, which consisted of four areas in the upper white lip and four areas in the upper and lower vermilions. Each area consisted of 15 straight lines connecting ten landmarks and three Bezier curves. In these eight areas, the following 5 5 virtual grids were produced (Fig. 1). A threedimensional curved line on the range image was divided into several segments. The divided points were projected in two dimensions (XY planes), and intersections between these gridlines
In studies on smiles, two types of smiles, a spontaneous smile and a posed smile, are often described. Ackerman et al. (1998) have stated that a posed smile can be repeated reliably. On the other hand, Van der Geld et al. (2007) reported that the reproducibility of a spontaneous smile is also high, and indicated that a spontaneous smile is a person's authentic smile expression. However, until now there have been no reports comparing the reproducibility of a spontaneous smile and a posed smile using the same measurement
Fig. 1. Eight areas and virtual grids applied to the lip. Grids (5 5) were applied to eight areas, as determined by ten landmarks and Bezier curved lines (a). Two sets of threedimensional coordinates of the virtual grid intersections were utilized for PCA (b). Blue dots indicate the intersections of the virtual grid (c).
146
K. Mishima et al. / Journal of Cranio-Maxillo-Facial Surgery 43 (2015) 144e148
Fig. 2. Lip shapes corresponding to the first and second PCSs. The first and second PCSs almost completely described anterior-posterior movement and superior-inferior movement of the lips, respectively. In the left of each figure, sliding scales of each PCS available in the present program, by which lip configurations corresponding PCSs can be observed, are shown; (a) minus of the first PCS, (b) plus of the first PCS, (c) minus of the second PCS, and (d) plus of the second PCS.
system. Generally speaking, recording a spontaneous smile is time consuming and resource intensive. The main purpose of our study was to evaluate lip movement, and we did not seek to evaluate the quality of the smile or the emotion behind it. Therefore, a posed smile was sufficient for our purposes. Measuring surface shift is considered to be able to better reflect real lip movement compared to measuring points. Therefore, virtual grids were applied to the lips, preventing the loss of surface information. As for the reproducibility of measurement of the posed smiles using the virtual grids, it was reported that both ICC (1,1) and ICC (2,1) are high, ranging from 0.71 to 0.83 and from 0.77 to 0.99, respectively, in our previous manuscript (Mishima et al., 2014). Furthermore, reliability can be improved by calculating the average of duplicate measurements (Mishima et al., 2014). Therefore, two measurements were made for each person in the present study. The central idea of PCA is to reduce the dimensionality of a data set consisting of a large number of interrelated variables, while retaining as much as possible of the variation present in the data set (Jolliffe, 2002). In other words, it is hoped that PCA can be used to extract important characteristics from huge data sets. The first principal component is the score having the highest rate of explained variance (Jolliffe, 2002). There have been some reports on the PCSs derived by application of PCA to lip motion. Popat et al. (2010) have stated that the first PCS represents the largest variation in lip movement for a particular facial gesture, and that the first PCS for the verbal gesture, the word “puppy,” is lip opening. Popat et al.
(2012b) have noted that the first four PCSs describe the majority of variation in lip movement during articulation involving a complex interaction of lip movements in three dimensions. In our previous manuscript, PCA applied to three-dimensional lip movement during phonation of five Japanese vowels using the present videobased measuring system revealed that the fourth and fifth PCSs are the parameters that control the mouth opening and the retraction of the corners of the mouth, respectively (Mishima et al., 2011). In the present study, the first and second PCSs were revealed to be the parameters that almost completely describe anteriorposterior movement and superior-inferior movement of the lips, respectively. Because only a posed smile was targeted for PCA, it is easy to understand why anterior-posterior movement is the largest variation. There are several reports that study gender differences when smiling. Kawamura and Kageyama (2006) have reported that Japanese university students rate smiling faces as being more feminine than serious faces. Kawamura et al. (2008) have also suggested that smiling significantly reduces the perceived masculinity of men's faces. To date, gender differences in smiling have mostly been evaluated qualitatively and subjectively. There was one study that measured the relationship between the lips and the teeth or the gingiva while smiling (Singh et al., 2013) The measurements in this study were done in two dimensions. In the article, significant observations included increased resting upper lip length for females and decreased upper lip thickness, maxillary incisor exposure, and lip elevation for males. However, there have been no articles
K. Mishima et al. / Journal of Cranio-Maxillo-Facial Surgery 43 (2015) 144e148
147
Fig. 3. Graphs of the first and second PCSs. The PCSs are plotted in two dimensions (first PCS on x-axis and second PCS on y-axis). Red and green dots indicate PCSs and centers of gravity of the PCSs, respectively, and the red lines indicate temporal trajectories of the PCSs. The center of gravity of the PCSs is shown by the plus on the x-axis and minus on the yaxis for the males (a) and by the minus on the x-axis and the plus on the y-axis for the females (b).
examining differences in smiling by measuring lip movements three-dimensionally to date. In the present study, gender could be correctly identified while the subject was smiling in 74.4% of cases using the first and second PCSs. While the sample size is too small to extrapolate from these results, we can conclude that PCA can be used to identify gender differences while smiling. In addition, it has been reported that there are cultural or ethnic differences in the perception of smiles and dynamic characteristics of smiles (Sharma et al., 2012; Liang et al., 2013). In extrapolation from the present study results, it is necessary to consider cultural and ethnic backgrounds carefully.
5. Conclusions Gender differences in a posed smile were investigated using PCA. Fourteen adult volunteers, 7 males and 7 females, were enrolled. Using our developed motion analyzing system, range images and virtual grids of 5 5 were produced across the whole sequence while each subjects was making a posed smile. Two sets of all intersections of the virtual grids during a posed smile were regarded as PCA variables. Gender could be determined while the subject was smiling, with a correct classification rate of 74.4% using the first and second PCSs. While the sample size is too small to
148
K. Mishima et al. / Journal of Cranio-Maxillo-Facial Surgery 43 (2015) 144e148
extrapolate from these results, we can conclude that PCA can be used to identify gender differences while smiling. Conflicts of interest The authors have no conflicts of interest in relation to the current investigation. Acknowledgments The authors thank Mr. Shozo Hirose from Ivis Co. Ltd. (Osaka) and Mr. Yuji Uchida from Ergovision Co. Ltd. (Osaka) for their advice. References Ackerman JL, Ackerman MB, Brensinger CM, Landis JR: A morphometric analysis of the posed smile. Clin Orthod Res 1: 2e11, 1998 Jolliffe IT: Principal component analysis, 2nd edn. New York: Springer, 1e6, 2002 Kawamura S, Kageyama K: Smiling faces rated more feminine than serious faces in Japan. Percept Mot Skills 103: 210e214, 2006 Kawamura S, Komori M, Miyamoto Y: Smiling reduces masculinity: principal component analysis applied to facial images. Perception 37: 1637e1648, 2008 Liang LZ, Hu WJ, Zhang YL, Chung KH: Analysis of dynamic smile and upper lip curvature in young Chinese. Int J Oral Sci 5: 49e53, 2013 Lucas BD, Kanade T: An iterative image registration technique with an application to stereo vision. In: Proceedings of the International Joint Conference on Artificial Intelligence; 1981, 674e679, 1981 Meyer AH, Woods MG, Manton DJ: Maxillary arch width and buccal corridor changes with orthodontic treatment. Part 2: attractiveness of the frontal facial smile in extraction and nonextraction outcomes. Am J Orthod Dentofacial Orthop 145: 296e304, 2014
Mishima K, Yamada T, Ohura A, Sugahara T: Production of a range image for facial motion analysis: a method for analyzing lip motion. Comput Med Imaging Graph 30: 53e59, 2006 Mishima K, Yamada T, Matsumura T, Moritani N: Analysis of lip motion using principal component analyses. J Craniomaxillofac Surg 39: 232e236, 2011 Mishima K, Umeda H, Nakano A, Shiraishi R, Hori S, Ueyama Y: Three-dimensional intra-rater and inter-rater reliability during a posed smile using a video-based motion analyzing system. J Craniomaxillofac Surg 42: 428e431, 2014 Okutomi M, Kanade T: A multiple baseline stereo. In: Proceedings of the 1991 IEEE conference on computer vision and pattern recognition; 1991, 63e69, 1991 Popat H, Henley E, Richmond S, Benedikt L, Marshall D, Rosin PL: A comparison of the reproducibility of verbal and nonverbal facial gestures using threedimensional motion analysis. Otolaryngol Head Neck Surg 142: 867e872, 2010 Popat H, Richmond S, Marshall D, Rosin PL: Three-dimensional assessment of functional change following Class 3 orthognathic correctionea preliminary report. J Craniomaxillofac Surg 40: 36e42, 2012a Popat H, Zhurov AI, Toma AM, Richmond S, Marshall D, Rosin PL: Statistical modelling of lip movement in the clinical context. Orthod Craniofac Res 15: 92e102, 2012b Sharma N, Rosenstiel SF, Fields HW, Beck FM: Smile characterization by U.S. white, U.S. Asian Indian, and Indian populations. J Prosthet Dent 107: 327e335, 2012 Singh B, Ahluwalia R, Verma D, Grewal SB, Goel R, Kumar PS: Perioral age-related changes in smile dynamics along the vertical plane: a videographic crosssectional study. Angle Orthod 83: 468e475, 2013 Tamura H: Computer image processing. Tokyo: Ohmsha Publishing, 184e197 and 252-265 [in Japanese], 2002 Valentin D, Abdi H, Edelman B, O'Toole AJ: Principal component and neural network analyses of face images: what can be generalized in gender classification? J Math Psychol 41: 398e413, 1997 Van der Geld PA, Oosterveld P, van Waas MA, Kuijpers-Jagtman AM: Digital videographic measurement of tooth display and lip position in smiling and speech: reliability and clinical application. Am J Orthod Dentofacial Orthop 131, 2007 301.e1-8
