Perception, 2014, volume 43, pages 1393 – 1399
doi:10.1068/p7838
SHORT REPORT
Adaptation aftereffects may decipher Ophelia’s facial expression Nozomi Takahashi1, Chang Hong Liu2, Hiroshi Yamada1 1
Graduate School of Literature and Social Science, Nihon University, 3-25-40, Setagaya-ku, Sakurajosui, Tokyo, Japan; 2 Department of Psychology, Bournemouth University, Poole, UK; e‑mail: [email protected] Received 18 August 2014, in revised form 7 November 2014 Abstract. Ophelia is a 19th century painting by John Everett Millais. It shows Ophelia with a blank look to encourage the viewer’s own imagination (Rosenfeld & Smith, 2007, Millais. London: Tate Publishing). Using the face adaptation paradigm, we attempted to identify the subtle emotion a viewer might perceive from Ophelia’s expression. Since adapting to an expression is known to lower the viewer’s subsequent sensitivity to that expression, we hypothesized that adaptation to Ophelia would impair identification of a similar expression. Participants adapted to Ophelia’s face before identifying expression of a schematic face that was variably morphed between a neutral expression and each of the six basic expressions. Results showed a selective impairment of identification for sadness, suggesting that sadness was what participants perceived. The study demonstrates that high-level adaptation can reveal aesthetic experience and its neural mechanisms. Keywords: adaptation aftereffects, facial expression recognition, facial expression in portrait paintings
1 Introduction There has been a growing interest among scientists in the relationship between art and vision. Art has always been a source of knowledge for science, which has in turn advanced understanding of art (eg Marr & Hildreth, 1980; Zeki, 1999). In this paper we explore how the recently developed face adaptation paradigm may help reveal subtle emotions perceived from an enigmatic facial expression in a masterpiece painting. Human faces are one of the main motifs in paintings. Facial expression is often carefully rendered in portraits. However, some artists choose a neutral or a rather subtle expression at times to engage the viewer’s imagination. John Everett Millais (1829–1896) frequently adopted this option. He pioneered a type of female portrait in which the heroine’s expression appears blank or enigmatic, which allows the viewer to draw meaning from the surrounding scene (Rosenfeld & Smith, 2007). His masterpiece Ophelia sets a perfect example. Completed between 1851 and 1852, the painting describes a scene in Shakespeare’s Hamlet, in which Ophelia is mad with grief when she learns about her father’s murder by her lover Hamlet and subsequently drowns herself in a stream. Rosenfeld and Smith (2007) point out that Millais faithfully depicted the botanical detail to render Ophelia’s presence insignificant. Her face was considered unemotional (Rusche, 2003) or in no way suggestive of her own distress (Fine Arts, 1852). She turned toward the sky with a seemingly neutral expression. The setup may be intended to encourage the viewer’s free imagination and empathy. However, is Ophelia’s face really perceived as neutrally blank or even meaningless? To address this question, we employed the face adaptation paradigm. Adaptation temporarily weakens the sensitivity of a specific neural population, which is commonly observed in perception of low-level features such as colour, motion, and spatial frequency. However, recent research has revealed that adaptation can also be demonstrated in perception of complex stimuli such as faces (Leopold, O’Toole, Vetter, & Blanz, 2001; Webster & MacLin, 1999). A wide range of adaptation effects has been reported in connection to facial
1394
N Takahashi, C H Liu, H Yamada
5000 ms adapting stimulus
OR
100 ms blank
50 ms test stimulus
until response
(a) Happiness
Surprise
Fear
Sadness
Anger
Disgust 10
20
30 40 50 Expression strength of test face/%
60
(b) Figure 1. [In colour online, see http://dx.doi.org/10.1068/p7838] (a) Illustration of a trial procedure. The response options on the response screen are the names of the six emotional expressions in Japanese, which correspond to happiness, surprise, fear, sadness, anger, and disgust in this example. (b) Test faces of varying emotional strengths for each expression.
Decipher Ophelia’s expression
1395
identity, gender, race, and attractiveness (eg Fox & Barton, 2007; Leopold et al., 2001; Rhodes, Jeffery, Watson, Clifford, & Nakayama, 2003; Webster, Kaping, Mizokami, & Duhamel, 2004). Of particular relevance to this study, the literature has shown that high-level face adaptation can reveal the neural mechanisms of expression processing. Hsu and Young (2004) demonstrate that exposure to an emotional face for a few seconds can lower subsequent sensitivity to faces showing a similar expression. The effect manifests a temporary fatigue of a neural population tuned to that specific facial expression. The authors also report that adapting to a sad face facilitated discrimination of a happy expression and vice versa. This suggests a cross-talk among neural representations tuned for antagonistic expressions. The main objective of this study was to identify the kind of facial expressions a viewer might perceive from Ophelia’s face. Using the procedure illustrated in figure 1a, Ophelia’s face was adapted for a few seconds, followed by a test stimulus. To minimize adaptation effects based on low-level features, we used schematic faces as test stimuli (figure 1b). Each schematic face showed one of the six basic emotional expressions with a range of intensities. Participants were asked to identify the emotion of the test stimulus. The results were compared with a baseline condition, where the adapting stimulus was a random-dot pattern. On the basis of an established knowledge of impaired discrimination for adapted facial expression, we predicted a poorer discrimination of expression when the test face showed a similar expression to that of Ophelia. 2 Results Trials that were 2 SD above or below the mean response time were excluded from data analyses. As a result 0% to 6% of trials per participant were excluded. The proportion-correct identification for each test face was computed within participant. For each expression the mean identification rates across participants in the two adapting conditions are plotted in figure 2 as a function of emotional strength of the test face. As the figure shows, the sensitivities were markedly expression dependent. For the happy expression, for instance, even 10% of happy strength was sufficient to produce around 80% identification rate, and this reached a plateau at around 30% happy strength. In contrast, 10% of emotional strength for most other expressions produced only a chance-level performance. Similar patterns of psychometric profiles are also found in Hsu and Young (2004). To explore whether the psychometric curve of each test emotion differed following adaptation to Ophelia’s face image relative to the control stimulus, we performed a repeatedmeasures analysis of variance (ANOVA) on arcsine-transformed data for each test expression. All p‑values were Geisser–Greenhouse adjusted when the sphericity assumption was violated. For the sadness condition, the analysis revealed a statistically significant interaction between adapting stimulus and emotional strength (F5, 95 = 2.72, p
doi:10.1068/p7838
SHORT REPORT
Adaptation aftereffects may decipher Ophelia’s facial expression Nozomi Takahashi1, Chang Hong Liu2, Hiroshi Yamada1 1
Graduate School of Literature and Social Science, Nihon University, 3-25-40, Setagaya-ku, Sakurajosui, Tokyo, Japan; 2 Department of Psychology, Bournemouth University, Poole, UK; e‑mail: [email protected] Received 18 August 2014, in revised form 7 November 2014 Abstract. Ophelia is a 19th century painting by John Everett Millais. It shows Ophelia with a blank look to encourage the viewer’s own imagination (Rosenfeld & Smith, 2007, Millais. London: Tate Publishing). Using the face adaptation paradigm, we attempted to identify the subtle emotion a viewer might perceive from Ophelia’s expression. Since adapting to an expression is known to lower the viewer’s subsequent sensitivity to that expression, we hypothesized that adaptation to Ophelia would impair identification of a similar expression. Participants adapted to Ophelia’s face before identifying expression of a schematic face that was variably morphed between a neutral expression and each of the six basic expressions. Results showed a selective impairment of identification for sadness, suggesting that sadness was what participants perceived. The study demonstrates that high-level adaptation can reveal aesthetic experience and its neural mechanisms. Keywords: adaptation aftereffects, facial expression recognition, facial expression in portrait paintings
1 Introduction There has been a growing interest among scientists in the relationship between art and vision. Art has always been a source of knowledge for science, which has in turn advanced understanding of art (eg Marr & Hildreth, 1980; Zeki, 1999). In this paper we explore how the recently developed face adaptation paradigm may help reveal subtle emotions perceived from an enigmatic facial expression in a masterpiece painting. Human faces are one of the main motifs in paintings. Facial expression is often carefully rendered in portraits. However, some artists choose a neutral or a rather subtle expression at times to engage the viewer’s imagination. John Everett Millais (1829–1896) frequently adopted this option. He pioneered a type of female portrait in which the heroine’s expression appears blank or enigmatic, which allows the viewer to draw meaning from the surrounding scene (Rosenfeld & Smith, 2007). His masterpiece Ophelia sets a perfect example. Completed between 1851 and 1852, the painting describes a scene in Shakespeare’s Hamlet, in which Ophelia is mad with grief when she learns about her father’s murder by her lover Hamlet and subsequently drowns herself in a stream. Rosenfeld and Smith (2007) point out that Millais faithfully depicted the botanical detail to render Ophelia’s presence insignificant. Her face was considered unemotional (Rusche, 2003) or in no way suggestive of her own distress (Fine Arts, 1852). She turned toward the sky with a seemingly neutral expression. The setup may be intended to encourage the viewer’s free imagination and empathy. However, is Ophelia’s face really perceived as neutrally blank or even meaningless? To address this question, we employed the face adaptation paradigm. Adaptation temporarily weakens the sensitivity of a specific neural population, which is commonly observed in perception of low-level features such as colour, motion, and spatial frequency. However, recent research has revealed that adaptation can also be demonstrated in perception of complex stimuli such as faces (Leopold, O’Toole, Vetter, & Blanz, 2001; Webster & MacLin, 1999). A wide range of adaptation effects has been reported in connection to facial
1394
N Takahashi, C H Liu, H Yamada
5000 ms adapting stimulus
OR
100 ms blank
50 ms test stimulus
until response
(a) Happiness
Surprise
Fear
Sadness
Anger
Disgust 10
20
30 40 50 Expression strength of test face/%
60
(b) Figure 1. [In colour online, see http://dx.doi.org/10.1068/p7838] (a) Illustration of a trial procedure. The response options on the response screen are the names of the six emotional expressions in Japanese, which correspond to happiness, surprise, fear, sadness, anger, and disgust in this example. (b) Test faces of varying emotional strengths for each expression.
Decipher Ophelia’s expression
1395
identity, gender, race, and attractiveness (eg Fox & Barton, 2007; Leopold et al., 2001; Rhodes, Jeffery, Watson, Clifford, & Nakayama, 2003; Webster, Kaping, Mizokami, & Duhamel, 2004). Of particular relevance to this study, the literature has shown that high-level face adaptation can reveal the neural mechanisms of expression processing. Hsu and Young (2004) demonstrate that exposure to an emotional face for a few seconds can lower subsequent sensitivity to faces showing a similar expression. The effect manifests a temporary fatigue of a neural population tuned to that specific facial expression. The authors also report that adapting to a sad face facilitated discrimination of a happy expression and vice versa. This suggests a cross-talk among neural representations tuned for antagonistic expressions. The main objective of this study was to identify the kind of facial expressions a viewer might perceive from Ophelia’s face. Using the procedure illustrated in figure 1a, Ophelia’s face was adapted for a few seconds, followed by a test stimulus. To minimize adaptation effects based on low-level features, we used schematic faces as test stimuli (figure 1b). Each schematic face showed one of the six basic emotional expressions with a range of intensities. Participants were asked to identify the emotion of the test stimulus. The results were compared with a baseline condition, where the adapting stimulus was a random-dot pattern. On the basis of an established knowledge of impaired discrimination for adapted facial expression, we predicted a poorer discrimination of expression when the test face showed a similar expression to that of Ophelia. 2 Results Trials that were 2 SD above or below the mean response time were excluded from data analyses. As a result 0% to 6% of trials per participant were excluded. The proportion-correct identification for each test face was computed within participant. For each expression the mean identification rates across participants in the two adapting conditions are plotted in figure 2 as a function of emotional strength of the test face. As the figure shows, the sensitivities were markedly expression dependent. For the happy expression, for instance, even 10% of happy strength was sufficient to produce around 80% identification rate, and this reached a plateau at around 30% happy strength. In contrast, 10% of emotional strength for most other expressions produced only a chance-level performance. Similar patterns of psychometric profiles are also found in Hsu and Young (2004). To explore whether the psychometric curve of each test emotion differed following adaptation to Ophelia’s face image relative to the control stimulus, we performed a repeatedmeasures analysis of variance (ANOVA) on arcsine-transformed data for each test expression. All p‑values were Geisser–Greenhouse adjusted when the sphericity assumption was violated. For the sadness condition, the analysis revealed a statistically significant interaction between adapting stimulus and emotional strength (F5, 95 = 2.72, p
