Cognition and Emotion
ISSN: 0269-9931 (Print) 1464-0600 (Online) Journal homepage: http://www.tandfonline.com/loi/pcem20
Emotional influences on perception and working memory Juyoen Hur, Alexandru D. Iordan, Florin Dolcos & Howard Berenbaum To cite this article: Juyoen Hur, Alexandru D. Iordan, Florin Dolcos & Howard Berenbaum (2016): Emotional influences on perception and working memory, Cognition and Emotion, DOI: 10.1080/02699931.2016.1213703 To link to this article: http://dx.doi.org/10.1080/02699931.2016.1213703
Published online: 16 Aug 2016.
Submit your article to this journal
View related articles
View Crossmark data
Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=pcem20 Download by: [Cornell University Library]
Date: 17 August 2016, At: 05:02
COGNITION AND EMOTION, 2016 http://dx.doi.org/10.1080/02699931.2016.1213703
BRIEF ARTICLE
Emotional influences on perception and working memory Juyoen Hura, Alexandru D. Iordanb, Florin Dolcosa,c,d and Howard Berenbauma a
Department of Psychology, University of Illinois at Urbana-Champaign, Champaign, IL, USA; bDepartment of Psychology, University of Michigan, Ann Arbor, MI, USA; cNeuroscience Program, University of Illinois at Urbana-Champaign, Champaign, IL, USA; dBeckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Champaign, IL, USA ABSTRACT
ARTICLE HISTORY
Although there has been steady progress elucidating the influence of emotion on cognition, it remains unclear precisely when and why emotion impairs or facilitates cognition. The present study investigated the mechanisms involved in the influence of emotion on perception and working memory (WM), using modified 0-back and 2-back tasks, respectively. First, results showed that attentional focus modulated the impact of emotion on perception. Specifically, emotion facilitated perceptual task performance when it was relevant to the task, but it impaired performance when it was irrelevant to the task. The differential behavioural effect of emotion on perception as a function of attentional focus diminished under high WM load. Second, attentional focus did not directly modulate the impact of emotion on WM, but rather its influence depended on the dynamic relationship between internal representations. Specifically, WM performance was worse when the material already being held online and the new input were of matching emotions (e.g. both were negative), compared to when they were not. We propose that the competition between “bottom-up” and “top-down” processing for limited cognitive resources explains the nature of the influence of emotion on both perception and WM.
Received 24 December 2015 Revised 27 June 2016 Accepted 29 June 2016
Emotion influences the way we perceive and remember things. The prioritised and relatively automatised processing of emotional content facilitates the perceptual and mnemonic processing of relevant information (Dolcos, Iordan, & Dolcos, 2011). However, in certain circumstances, emotional content impairs task performance (Dolcos & McCarthy, 2006; Hur et al., 2015). Although there has been steady progress elucidating the influence of emotion on cognition, it remains unclear precisely when and why emotion impairs or facilitates cognition. The present study investigated the mechanisms involved in the influence of emotion on perception and working memory (WM), focusing on potential modulation by the attentional focus. A top-down factor that biases the competition for processing resources is whether task requirements involve focusing attention on emotional vs. nonemotional aspects of the stimuli (i.e. attentional focus). Attentional focus concerns how top-down CONTACT Juyoen Hur
[email protected]
© 2016 Informa UK Limited, trading as Taylor & Francis Group
KEYWORDS
Emotion; cognition; attentional focus; biased competition model; congruence interference effect
(e.g. goal of the task) and bottom-up (e.g. emotional salience) influences interact to bias the competition among items in the visual field (Desimone & Duncan, 1995). Studies show that emotion impairs performance on a perception task when it is task-irrelevant, whereas emotion facilitates performance when it is task-relevant (Dodd, Vogt, Turkileri, & Notebaert, 2016; Lichtenstein-Vidne, Henik, & Safadi, 2012). For example, in an Emotional Stroop task, response times (RTs) for naming the ink colour of words were longer for negative words than neutral words, suggesting that the emotional content of words can divert attention from the primary colour-naming task (Compton et al., 2003). On the other hand, there is evidence that emotion facilitates the likelihood and the degree to which arousing information is processed when it is relevant to the task. For example, in a rapid serial visual presentation task, the likelihood of missing the second of two stimuli presented in rapid succession was reduced when the stimulus was
2
J. HUR ET AL.
emotionally arousing (Anderson & Phelps, 2001). However, the task-relevant or irrelevant character of emotion was intrinsic to the tasks in these previous investigations, and hence it is not known how manipulating the focus of attention on the emotional vs. non-emotional aspects influences perception. Thus, the first goal of our study was to investigate how the effect of emotion on perception is influenced by attentional focus. Another top-down factor to consider is the attentional load of the task. Research suggests that the emotional impact on perception is limited to situations in which the attentional load is low, during which attentional resources remain available for processing emotional information (Okon-Singer et al., 2014; Pessoa, McKenna, Gutierrez, & Ungerleider, 2002; Van Dillen & Derks, 2012). For example, the behavioural interference and neural responses (amygdala activation) to negative distractor stimuli (compared with neutral stimuli) have been observed only under low perceptual load, but not high perceptual load (Okon-Singer et al., 2014; Pessoa et al., 2002; Shafer et al., 2012). Similarly, the relatively slower RT to angry faces (compared to happy faces) in the gender-naming task was eliminated under high WM load, suggesting that WM load interferes with the typically facilitated processing of threating stimuli (Hur, Iordan, Berenbaum, & Dolcos, 2015; Van Dillen & Derks, 2012). However, most of these studies investigated the effect of attentional load when emotion is task-irrelevant, leaving the combined effects of attentional focus and attentional load an open question. Thus, in addition to examining the modulating effect of attentional focus, we investigated the impact of attentional load and the potential interaction of these two factors. As with perception, attentional focus may also explain the contradictory results regarding the effect of emotion on WM, as both impairing and facilitating effects have been reported. Studies show that in the presence of emotional distractors, WM performance tends to be impaired (Dolcos et al., 2011). As with perception, the bottom-up salience of the emotional distractor biases attention away from the task-relevant information in WM (Desimone & Duncan, 1995). For example, negative stimuli presented as irrelevant distractors during a delayed-response WM task led to worse cognitive performance compared to neutral stimuli (Dolcos & McCarthy, 2006). On the other hand, when emotion is task-relevant, the opposite effects were observed. For example, on a modified
recency-probes paradigm, proactive interference (interference from preceding but no longer relevant materials) involved in WM was reduced when emotional stimuli were used as probes (Levens & Phelps, 2008), suggesting that emotion facilitates interference resolution processes within WM (Lindström & Bohlin, 2011). Despite the evidence suggesting a potential modulating effect of attentional focus on the influence of emotion on WM, investigations of this matter using manipulations of attentional focus are lacking. Thus, the second goal of our study was to investigate whether the effect of emotion on WM is influenced by attentional focus. Another important question pertains to the congruence between the emotional content of material already being held on-line in WM and new input. Although not investigated in the context of WM, the memory literature suggests a congruence interference effect (Sison & Mather, 2007). According to this account, emotional memories compete with memories that evoke similar emotions because resolving the competition among memories of the same emotion requires greater inhibition at retrieval than resolving the competition among items that are not of the same emotion (Mather, 2009). The congruence interference effect of emotion held only when participants’ attention was explicitly driven to the emotional aspects of stimuli (Sison & Mather, 2007). Although this theory has been proposed in relation to the competition in recall between two memories learned in the past, it is possible that such competition is not limited to long-term memories but may also apply to a broader range of memory processes, including WM. When updating the representations online (e.g. n-back task), the to-be maintained and new input stimuli in WM are likely to compete for limited cognitive resources since newly learned information can be vulnerable to interference from the information kept online (Jonides & Nee, 2006). Based on the extant literature, we made the following predictions. Regarding the effect of emotion on perception, we hypothesised that emotion would facilitate or interfere with perception depending on its relevance to the task (attentional focus). Specifically, we hypothesised that emotion would improve task performance when it is relevant to the task requirements, but would impair task performance when it is irrelevant to the task requirements. We expected that the differential behavioural effect of emotion on perception as a function of attentional focus would diminish under high WM load. In
COGNITION AND EMOTION
addition, we hypothesised that the effect of emotion on WM would be influenced by both attentional focus and the dynamic relationship between the emotional contents of the to-be maintained and new input stimuli. If the congruence interference effect holds, the performance would be impaired when both stimuli are from the same emotional category, relative to when they are not, and only when attention was explicitly drawn to the emotional properties of the stimuli.
Methods Participants One hundred and thirty-one undergraduate students (64% female; mean age = 19.3) participated in the study for course credit. Only individuals, who reported having normal or corrected-to-normal vision, including normal colour vision, were permitted to participate. All participants provided informed consent. The experimental protocol was approved for ethical treatment to human participants by the Institutional Review Board at the University of Illinois at UrbanaChampaign.
Design and procedures Picture stimuli were selected from the International Affective Picture System (Lang, Bradley, & Cuthbert, 2008). Sixteen negative pictures were selected to be relatively high in arousal and negative in valence, and 16 neutral pictures were selected that were rated as non-arousing and of neither negative nor
3
positive valence. Ratings ranged from 1 (low pleasure, low arousal) to 9 (high pleasure, high arousal). The mean valence and arousal scores were 2.1 (SD = 0.4) and 6.4 (SD = 0.6), respectively, for the negative stimuli, and 5.2 (SD = 0.2) and 3.3 (SD = 0.4), respectively, for the neutral stimuli. In each valence category (negative, neutral), two different stimulus types were included: pictures with people and pictures without people. The pictures included were: (a) eight negative pictures with people (e.g. injured), (b) eight negative pictures without people (e.g. crashed cars), (c) eight neutral pictures with people (e.g. answering phone), and (d) eight neutral pictures without people (e.g. buildings). Each picture appeared four times during the task and they were randomly coloured in yellow or blue hues (see Figure 1). We used a between-subject design to minimise the likelihood of fatigue and confusion regarding task instructions. Participants were randomly assigned to complete one of four tasks. Specifically, participants were randomly assigned to complete either a perception task (0-back task) or a WM task (2-back task). Participants were also randomly assigned to one of two attentional focus conditions; they were asked to either report the colour of the picture (COLOR) or report the emotion of the picture (EMO). In the perception task, participants had to indicate either the colour or the emotion of the current trial. In the WM task, participants had to indicate whether the colour (or emotion) of the picture in the present trial was the same as the one two trials before (see Appendix 1). Thus, there were four conditions: (1) EMO 0-back; (2) COLOR 0-back; (3) EMO 2-back; and (4) COLOR 2back. Each task consisted of 128 trials, each of which
Figure 1. Task design. Participants were randomly assigned to complete one of four tasks. First, participants completed either a perception task (i.e. 0-back task) or a WM task (i.e. 2-back task). In addition, participants were randomly assigned to one of two attentional focus conditions – that is, they were asked to either report the colour (i.e. COLOR) or the emotion of the picture (i.e. EMO).
4
J. HUR ET AL.
consisted of a target for 1500 ms (which did not disappear upon reaction), followed by a fixation cross for 3000 ms. As targets, negative and neutral pictures that were coloured in yellow or blue hues were presented in a quasi-randomised order such that the same emotion type (negative, neutral) or colour type (yellow, blue) was never presented on more than three consecutive trials. Luminance values of the tinted pictures were not significantly different across emotion types [F(1, 15) = 0.12, p > .7, h2p = 0.008]. Before the experimental task was administered, participants completed a short practice task after receiving instructions. Responses were made using arrow keys (left or right) on the keyboard. In every condition, participants were asked to respond as quickly and accurately as possible.
Analytic strategy First, a 2 (Emotion: Negative vs. Neutral) × 2 (Task Type: 0-back vs. 2-back) × 2 (Attentional Focus: EMO vs. COLOR) ANOVA was conducted on RT and Accuracy data. Emotion was entered as a within-subject variable and Task Type and Attentional Focus were dichotomous variables entered as between-subject variables. Significant interactions were followed up by conducting 2 (Emotion: Negative vs. Neutral) × 2 (Attentional Focus: EMO vs. COLOR) ANOVAs separately for the perception task (i.e. 0-back) and for the WM task (i.e. 2-back). For WM, an additional 2 (2back Type: Negative vs. Neutral) × 2 (Current Type: Negative vs. Neutral) × 2 (Attentional Focus: EMO vs. COLOR) ANOVA was conducted on RT and Accuracy data to examine the possibility of a congruence interference effect.
Results Descriptive statistics Out of 131 participants, the accuracy of 5 participants was 2 SDs below the mean in at least one condition (EMO 0-back: 1, COLOR 0-back: 1, EMO 2-back: 2, COLOR 2-back: 1 participant), and thus they were excluded from the analyses. Thus, a total of 126 participants were included in the following analyses (EMO 0back: n = 32, COLOR 0-back: n = 30, EMO 2-back: n = 33, COLOR 2-back: n = 31). For RT analyses, we only considered trials in which participants made correct responses. Trials with extreme RT scores (below 200 and above 2000 ms) were considered outliers and
eliminated from the analyses. Age and gender were balanced across conditions, showing neither gender × condition, χ 2(3, 126) = 1.28, p > .7, ф = 0.10, nor age × condition, F(3, 122) = 0.49, p > .6, h2p = 0.01 effects. Descriptive statistics for each task are reported in Appendix 2.
Overall RT and accuracy differences in perceptual vs. WM tasks We first examined differences in RT and accuracy between the perceptual task (0-back) and the WM task (2-back). As expected, there were significant differences in both RT [t(124) = −10.86, p < .01, d = 1.95] and accuracy [t(124) = 12.48, p < .01, d = 2.24]; RTs were greater in the WM task than in the perception task, and accuracy was lower in the WM task than in the perception task. In the perception (0-back) task, there was a ceiling effect in accuracy (M = 97%, SD = 0.02%). More meaningful variation was found in RTs (M = 693.5, SD = 150.9 in milliseconds). Thus, for the perception task, although both RT and accuracy data were analysed, we focused on interpreting the RT results. This approach is consistent with previous literature because RT is the dependent measure most frequently used in assessing the behavioural effects regarding selective attention and distractor interference in perceptual tasks (Nelson, Crisostomo, Khericha, Russo, & Thorne, 2012). On the other hand, there was more variability in accuracy in the WM (2-back) task (M = 79%, SD = 11%) than in the perception task (coefficients of variability were 0.14 and 0.03, respectively), whereas there was less variability in RT in the WM task (M = 1031.5, SD = 194.8 in milliseconds) than in the perception task (coefficients of variability were 0.18 and 0.22, respectively). In this type of WM task, participants’ efforts are generally focused more on performing the task accurately than responding as fast as they can. Thus, for the WM task, although both RT and accuracy data were analysed, we focused on interpreting the accuracy results.
The effect of emotion on perception and WM as a function of attentional focus A 2 (Emotion: Negative vs. Neutral) × 2 (Task Type: 0back vs. 2-back) × 2 (Attentional Focus: EMO vs. COLOR) ANOVA on RT data showed a significant three-way interaction [F(2, 122) = 8.44, p = .004, h2p =
COGNITION AND EMOTION
0.07]. As expected, follow-up ANOVAs revealed a significant Emotion × Attentional Focus interaction in the perception (0-back) task [F(1, 60) = 11.54, p = .001, h2p = 0.16], but not in the WM (2-back) task [F(1, 62) = 0.56, p > .4, h2p = 0.009]. As shown in Figure 2, during the perception (0-back) task, RTs were faster for negative than neutral stimuli in the EMO condition [t(31) = −2.12, p = .042, d = 0.38], whereas RTs were slower for negative than neutral stimuli in the COLOR condition [t(29) = 2.88, p = .007, d = 0.53]. A parallel analysis was conducted on Accuracy data. A 2 (Emotion: Negative vs. Neutral) × 2 (Task Type: 0-back vs. 2-back) × 2 (Attentional Focus: EMO vs. COLOR) ANOVA showed that there was a significant main effect of Emotion [F(1, 122) = 8.06, p = .005, h2p = 0.06], such that accuracy for negative stimuli was worse than for neutral stimuli.
Test of congruence interference effect in WM For WM, additional 2 (2-back Type: Negative vs. Neutral) × 2 (Current Type: Negative vs. Neutral) × 2 (Attentional Focus: EMO vs. COLOR) ANOVAs were conducted for accuracy and RT data. For Accuracy, although there were no main effects of 2-back Type or Current Type and no significant 2-way interactions, a significant 3-way interaction of 2-back Type × Current Type × Attentional Focus [F(1, 62) = 4.64, p = .035, h2p = 0.07] was found. A follow-up analysis revealed that a 2-back Type × Current Type interaction
5
was significant under the EMO condition [F(1, 32) = 6.12, p = .019, h2p = 0.16], but not under the COLOR condition [F(1, 30) = 0.03, p > .8, h2p = 0.001]. As shown in Figure 3, under the EMO condition, we found the expected congruence effect. Specifically, accuracy for matching conditions (i.e. NEG:NEG, NEU:NEU) was lower than for mismatch conditions (i.e. NEG:NEU, NEU:NEG). An additional 2 (Target Type: Negative vs. Neutral) × 2 (Match Type: Match vs. Mismatch) ANOVA in the EMO condition confirmed a significant Match Type main effect [F(1, 32) = 6.12, p = .019, h2p = 0.16], but the Target Type × Match Type interaction was not significant [F(1, 32) = 0.12, p > .7, h2p = 0.004]. This result suggests that the magnitude of accuracy difference in match vs. mismatch trials is not statistically different for the NEG (target) trials than for the NEU (target) trials. A parallel analysis was conducted on RT data. A 2 (2-back Type: Negative vs. Neutral) × 2 (Current Type: Negative vs. Neutral) × 2 (Attentional Focus: EMO vs. COLOR) ANOVA showed a significant 2-way interaction of 2-back Type × Attentional Focus [F(1, 62) = 5.29, p = .025, h2p = 0.08]. No significant main effects or other interactions were observed. In the EMO condition, RTs were faster when the 2-back stimulus was a negative picture, compared to when it was a neutral picture. In the COLOR condition, however, the pattern was reversed, such that RTs were slower when the 2-back stimulus was a negative picture, compared to when it was a neutral picture. Other effects were not significant. The additional analyses performed based on colour confirmed that the congruence interference effect was specific to when updating emotions, but not when updating colour (see Appendix 3). In addition, an analysis performed on 0-back tasks confirmed that the congruence interference effect held only for WM but not for perceptual tasks (see Appendix 4).
Discussion The effect of emotion on perception is modulated by attentional focus
Figure 2. The effect of emotion on perception and WM as a function of attentional focus. The interactive effect of Emotion and Attentional Focus on RT was only evident in the perception (0-back) task, not in the WM (2-back) task. During the perception task, emotion facilitated performance when it was task-relevant (EMO 0-back), whereas it impaired performance when it was task-irrelevant (COLOR 0-back). NEG: Negative, NEU: Neutral. *p < .05, two-tailed.
Consistent with previous findings (Anderson & Phelps, 2001; Compton et al., 2003), attentional focus modulated the impact of emotion on perception. Specifically, emotion facilitated perceptual task performance when it was relevant to the task, whereas it impaired performance when it was irrelevant to the task. This is consistent with the proposal that the impact of emotion on performance depends on the task relevance of the
6
J. HUR ET AL.
Figure 3. Congruence interference effect in WM. Under the EMO 2-back task, performance was worse when the material already being held online and the new input were of matching emotions (i.e. both were negative), compared to when they were not. No such effect was observed under the COLOR 2-back task. NEG: Negative, NEU: Neutral.
emotional information (Vuilleumier, 2005). Previous studies investigated either the distracting effect of emotional stimuli (that were spatially segregated from the target) or the facilitating effect of emotion (as part of the main task). Thus, this is the first study to assess the potentially facilitating and impairing effect of emotion on perception in the same study using the same paradigm. We were able to show that the nature of the emotional influence on perception changes even by simply changing the focus of attention to the emotional vs. non-emotional aspects of the same stimuli. These results confirm that the nature of the emotional influence on perception depends on top-down attention, specifically, whether task requirements involve the processing of emotional information. The modulation effect of attentional focus on the impact of emotion on perception can be explained using the biased competition model framework (Desimone & Duncan, 1995). In a limited processing capacity system like perception, emotional stimuli hold a competitive advantage conferred by their affective significance (Vuilleumier, 2005), biasing attention in a bottom-up fashion. Depending on the task relevance of emotion as determined by attentional focus, the bottom-up (i.e. emotional salience) and the top-down biases (i.e. goal of the task) attract attentional resources in the same or opposite direction, which in turn influences performance. That is, when
emotion is task-relevant, the bottom-up and topdown biases work in the same direction of pulling resources, which in turn facilitates performance. When emotion is task-irrelevant, the bottom-up and top-down biases are opposed such that the high salience of bottom-up attention drawn by emotion needs to be overcome by top-down attention (Anderson & Phelps, 2001; Lichtenstein-Vidne et al., 2012), which in turn may impair performance. The effect of attentional focus was further modulated by WM load, such that the differential behavioural effect of emotion on perception as a function of attentional focus was eliminated under high-WM load. This result is consistent with previous behavioural and brain imaging evidence showing diminished impact of emotion under high-attentional load, during which available attentional resources are limited for processing emotional information (OkonSinger et al., 2014; Pessoa et al., 2002; Shafer et al., 2012). The current findings suggest that the emotional impact on perception is determined by both attentional resources and attentional focus.
The effect of emotion on WM depends on emotional congruence of the materials online When it comes to the impact of emotion on WM, our results show that attentional focus itself does not facilitate or impair performance. Rather, our results
COGNITION AND EMOTION
suggest that the nature of the influence of emotion on WM depends on the dynamic relationship between internal representations. Specifically, we found that WM performance was worse when the material already being held on-line and the new input belonged to the same emotional category, compared to when they were not. This result shows that the congruence interference effect (Sison & Mather, 2007) holds in the context of WM, although it remains unclear whether the same mechanisms hold for WM as for long-term memory. A possible explanation could be that changes in emotion (here, changes in emotional categories) may be easier to detect due to their saliency. This explanation is in line with the repetition blindness (RB) effect, which refers to difficulty in detecting or reporting the repetition of an item in a stream displayed in a rapid serial visual presentation (RSVP) paradigm (Kanwisher, Kim, & Wickens, 1996). Although our WM paradigm was not an RSVP task per se, RB could potentially explain why participants’ performance on matching trials is worse than mismatch trials in the emotional 2-back task. Research suggests that a stronger stimulus representation (e.g. representing the emotional value of a stimulus) tends to induce a longer, increased RB (Kanwisher et al., 1996); the emotional content of a stimulus is an example of an important semantic component. In line with this explanation, it is important to note that the impaired performance in matching trials (compared to mismatch trials) was only found in the EMO 2-back task but not in the COLOR 2-back task. In real life contexts, noticing changes in emotions has greater survival and evolutionary value than does noticing the same emotion (Anderson & Phelps, 2001). The so-called pop-out or snake in the grass effect (e.g. the more rapid and accurate detection of threatening faces among non-emotional faces) is well established in visual perception (Öhman, Lundqvist, & Esteves, 2001). It is possible that the adaptive value of detecting changes in emotions is at least part of the reason why updating changes in emotions in WM is easier than updating sameness in emotions. The current result can be also explained in the context of the biased competition model. According to this model, items held in WM function as a search template that facilitates the processing of matching items (Desimone & Duncan, 1995). This is generally true when it comes to neutral items (e.g. shape) (Bichot, Rossi, & Desimone, 2005), but to our knowledge, no studies thus far have tested whether this
7
applies to updating emotional contents. Our result in the EMO task suggests that in the context of updating emotional contents (EMO 2-back), under matching trials, the bottom-up attention, which is recruited by “difference” or “change” in emotional categories, may be in direct conflict with the top-down strategy of imposing an attentional template (looking for “similarity”), which in turn impairs performance. This explanation is consistent with the idea that the salience network (anterior insula, dorsal cingulate cortex) plays a critical role in mediating the interaction between bottom-up and top-down attention in the context of emotional WM processing (Luo et al., 2014).
Conclusion Our results suggest that (1) the impact of emotion on perception depends on both attentional focus and attentional resources and (2) the impact of emotion on WM depends on the dynamic relationship between internal representations. Although we hypothesise that the competition between bottomup and top-down processing for limited cognitive resources may explain the nature of the influence of emotion on both perception and WM, additional research is needed to examine whether the working mechanisms truly overlap between the influence of emotion on perception and WM. Future research should continue to examine whether the emotional category congruence interference effect in WM holds with another emotional valence category (i.e. positive). It would be also valuable to explore whether the congruence interference effect still holds when the task becomes even more difficult. Furthermore, additional research regarding the precise mechanisms involved in the congruence interference effect is needed, as difficulties in adaptively updating emotional content in WM may contribute to the development and maintenance of emotional disorders, such as anxiety and depression.
Disclosure statement No potential conflict of interest was reported by the authors.
References Anderson, A. K., & Phelps, E. A. (2001). Lesions of the human amygdala impair enhanced perception of emotionally salient events. Nature, 411(6835), 305–309.
8
J. HUR ET AL.
Bichot, N. P., Rossi, A. F., & Desimone, R. (2005). Parallel and serial neural mechanisms for visual search in macaque area V4. Science, 308(5721), 529–534. Compton, R. J., Banich, M. T., Mohanty, A., Milham, M. P., Herrington, J., Miller, G. A., … Heller, W. (2003). Paying attention to emotion. Cognitive, Affective, & Behavioral Neuroscience, 3(2), 81–96. Desimone, R., & Duncan, J. (1995). Neural mechanisms of selective visual attention. Annual Review of Neuroscience, 18(1), 193–222. Dodd, H. F., Vogt, J., Turkileri, N., & Notebaert, L. (2016). Task relevance of emotional information affects anxiety-linked attention bias in visual search. Biological Psychology. Advance online publication. doi:10.1016/j.biopsycho.2016.01.017 Dolcos, F., Iordan, A. D., & Dolcos, S. (2011). Neural correlates of emotion-cognition interactions: A review of evidence from brain imaging investigations. Journal of Cognitive Psychology, 23(6), 669–694. Dolcos, F., & McCarthy, G. (2006). Brain systems mediating cognitive interference by emotional distraction. The Journal of Neuroscience, 26(7), 2072–2079. Hur, J., Iordan, A. D., Berenbaum, H., & Dolcos, F. (2015). Emotion– attention interactions in fear conditioning: Moderation by executive load, neuroticism, and awareness. Biological Psychology. Advance online publication. doi:10.1016/j.biopsycho.2015.10. 007 Hur, J., Miller, G. A., McDavitt, J. R., Spielberg, J. M., Crocker, L. D., Infantolino, Z. P., … Heller, W. (2015). Interactive effects of trait and state affect on top-down control of attention. Social Cognitive and Affective Neuroscience, 10(8), 1128–1136. Jonides, J., & Nee, D. E. (2006). Brain mechanisms of proactive interference in working memory. Neuroscience, 139(1), 181–193. Kanwisher, N. G., Kim, J. W., & Wickens, T. D. (1996). Signal detection analyses of repetition blindness. Journal of Experimental Psychology: Human Perception and Performance, 22(5), 1249– 1260. Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (2008). International Affective Picture System (IAPS): Affective ratings of pictures and instruction manual (Technical Report A-8). Levens, S. M., & Phelps, E. A. (2008). Emotion processing effects on interference resolution in working memory. Emotion, 8 (2), 267–280. Lichtenstein-Vidne, L., Henik, A., & Safadi, Z. (2012). Task relevance modulates processing of distracting emotional stimuli. Cognition & Emotion, 26(1), 42–52. Lindström, B. R., & Bohlin, G. (2011). Emotion processing facilitates working memory performance. Cognition & Emotion, 25(7), 1196–1204. Luo, Y., Qin, S., Fernandez, G., Zhang, Y., Klumpers, F., & Li, H. (2014). Emotion perception and executive control interact in the salience network during emotionally charged working memory processing. Human Brain Mapping, 35(11), 5606–5616.
Mather, M. (2009). When emotion intensifies memory interference. Psychology of Learning and Motivation, 51, 101–120. Nelson, M. D., Crisostomo, M., Khericha, A., Russo, F., & Thorne, G. L. (2012). Classic debates in selective attention: Early vs. late, perceptual load vs. dilution, mean RT vs. measures of capacity. Perception, 41(8), 997–1000. Öhman, A., Lundqvist, D., & Esteves, F. (2001). The face in the crowd revisited: A threat advantage with schematic stimuli. Journal of Personality and Social Psychology, 80(3), 381–396. Okon-Singer, H., Mehnert, J., Hoyer, J., Hellrung, L., Schaare, H. L., Dukart, J., & Villringer, A. (2014). Neural control of vascular reactions: Impact of emotion and attention. The Journal of Neuroscience, 34(12), 4251–4259. Pessoa, L., McKenna, M., Gutierrez, E., & Ungerleider, L. (2002). Neural processing of emotional faces requires attention. Proceedings of the National Academy of Sciences, 99(17), 11458–11463. Shafer, A. T., Matveychuk, D., Penney, T., O’Hare, A. J., Stokes, J., & Dolcos, F. (2012). Processing of emotional distraction is both automatic and modulated by attention: Evidence from an event-related fMRI investigation. Journal of Cognitive Neuroscience, 24(5), 1233–1252. Sison, J. A. G., & Mather, M. (2007). Does remembering emotional items impair recall of same-emotion items? Psychonomic Bulletin & Review, 14(2), 282–287. Van Dillen, L. F., & Derks, B. (2012). Working memory load reduces facilitated processing of threatening faces: An ERP study. Emotion, 12(6), 1340–1349. Vuilleumier, P. (2005). How brains beware: Neural mechanisms of emotional attention. Trends in Cognitive Sciences, 9(12), 585– 594.
Appendix 1. Instructions for four different conditions
Perception Task (0-back)
WM Task (2-back)
Attention is on EMOTION EMO 0-back task: Indicate the EMOTION of the current picture (i.e. Negative or Neutral), while ignoring the colour EMO 2-back task: Indicate whether the current EMOTION of the picture is the same as the one presented 2trials back (i.e. same or different), while ignoring the colour
Attention is on COLOR COLOR 0-back task: Indicate the COLOR of the current picture (i.e. yellow or blue), while ignoring the content COLOR 2-back task: Indicate whether the current COLOR is the same as the one presented 2-trials back (i.e. same or different), while ignoring the content
COGNITION AND EMOTION
9
Appendix 2. Descriptive statistics for the four different conditions
EMO 0-back (N = 32) M Gender Age Accuracy (%) RT (ms)
SD
COLOR 0-back (N = 30) M
65% 19.2 96.6 754.5
SD
EMO 2-back (N = 33) M
62% 1.0 0.03 110.3
19.2 97.2 633.1
Appendix 3. Results showing that the congruence interference effect is specific to emotion In order to examine whether the observed congruence interference effect is specific to emotion, an additional 2 (2-back Type: Yellow vs. Blue) × 2 (Current Type: Yellow vs. Blue) × 2 (Attentional Focus: EMO vs. COLOR) ANOVA was performed for accuracy and reaction time as a negative control. For accuracy, neither a 2-back Type × Current Type interaction [F(1, 62) = 0.99, p > .3, h2p = 0.02] nor a 2-back Type × Current Type × Attentional Focus interaction [F(1, 62) = 0.08, p > .7, h2p = 0.001] was significant. However, for RT data, a significant 2-back Type × Current Type × Attentional Focus interaction [F(1, 62) = 12.37, p = .01, h2p = 0.17] was observed. When these effects were decomposed, a significant 2-back Type × Current Type interaction was found in the COLOR condition [F(1, 30) = 22.99, p < .001, h2p = 0.43], but not in the EMO condition [F(1, 32) = 0.001, p > .9, h2p < 0.001]. Under the COLOR condition (i.e. COLOR 2-back task), RT was slower for mismatch conditions (i.e. yellow:blue, blue:yellow) than for matching conditions (i.e. yellow:yellow, blue:blue). This result confirms that the interfering
SD
COLOR 2-back (N = 31) M
62% 1.3 0.03 165.2
19.2 78.1 1103.3
SD 65%
1.3 0.11 163.8
19.4 79.9 983.4
1.3 0.11 205.6
congruence effect (i.e. impaired performance in match trials compared to mismatch trials) is specific to when updating emotions, but not when updating colour.
Appendix 4. Results showing that the congruence interference effect holds only for WM but not for perceptual tasks In order to confirm that the congruence interference effect is specific to WM (but not to perception), we tested whether each trial in the 0-back task influenced the subsequent trial by performing a 2 (1-back Type: Negative vs. Neutral) × 2 (Current Type: Negative vs. Neutral) × 2 (Attentional Focus: EMO vs. COLOR) ANOVA in 0-back tasks for accuracy and RT. The results show neither a 1-back Type × Current Type interaction [F(1, 60) = 0.75, p > .3, h2p = 0.01] nor a 1-back Type × Current Type × Condition interaction [F(1, 60) = 1.58, p > .2, h2p = 0.03] for accuracy. No significant interaction effect was found in the RT analysis, either. Thus, we can conclude that the sequential effect holds only for WM but not for perceptual tasks.
ISSN: 0269-9931 (Print) 1464-0600 (Online) Journal homepage: http://www.tandfonline.com/loi/pcem20
Emotional influences on perception and working memory Juyoen Hur, Alexandru D. Iordan, Florin Dolcos & Howard Berenbaum To cite this article: Juyoen Hur, Alexandru D. Iordan, Florin Dolcos & Howard Berenbaum (2016): Emotional influences on perception and working memory, Cognition and Emotion, DOI: 10.1080/02699931.2016.1213703 To link to this article: http://dx.doi.org/10.1080/02699931.2016.1213703
Published online: 16 Aug 2016.
Submit your article to this journal
View related articles
View Crossmark data
Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=pcem20 Download by: [Cornell University Library]
Date: 17 August 2016, At: 05:02
COGNITION AND EMOTION, 2016 http://dx.doi.org/10.1080/02699931.2016.1213703
BRIEF ARTICLE
Emotional influences on perception and working memory Juyoen Hura, Alexandru D. Iordanb, Florin Dolcosa,c,d and Howard Berenbauma a
Department of Psychology, University of Illinois at Urbana-Champaign, Champaign, IL, USA; bDepartment of Psychology, University of Michigan, Ann Arbor, MI, USA; cNeuroscience Program, University of Illinois at Urbana-Champaign, Champaign, IL, USA; dBeckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Champaign, IL, USA ABSTRACT
ARTICLE HISTORY
Although there has been steady progress elucidating the influence of emotion on cognition, it remains unclear precisely when and why emotion impairs or facilitates cognition. The present study investigated the mechanisms involved in the influence of emotion on perception and working memory (WM), using modified 0-back and 2-back tasks, respectively. First, results showed that attentional focus modulated the impact of emotion on perception. Specifically, emotion facilitated perceptual task performance when it was relevant to the task, but it impaired performance when it was irrelevant to the task. The differential behavioural effect of emotion on perception as a function of attentional focus diminished under high WM load. Second, attentional focus did not directly modulate the impact of emotion on WM, but rather its influence depended on the dynamic relationship between internal representations. Specifically, WM performance was worse when the material already being held online and the new input were of matching emotions (e.g. both were negative), compared to when they were not. We propose that the competition between “bottom-up” and “top-down” processing for limited cognitive resources explains the nature of the influence of emotion on both perception and WM.
Received 24 December 2015 Revised 27 June 2016 Accepted 29 June 2016
Emotion influences the way we perceive and remember things. The prioritised and relatively automatised processing of emotional content facilitates the perceptual and mnemonic processing of relevant information (Dolcos, Iordan, & Dolcos, 2011). However, in certain circumstances, emotional content impairs task performance (Dolcos & McCarthy, 2006; Hur et al., 2015). Although there has been steady progress elucidating the influence of emotion on cognition, it remains unclear precisely when and why emotion impairs or facilitates cognition. The present study investigated the mechanisms involved in the influence of emotion on perception and working memory (WM), focusing on potential modulation by the attentional focus. A top-down factor that biases the competition for processing resources is whether task requirements involve focusing attention on emotional vs. nonemotional aspects of the stimuli (i.e. attentional focus). Attentional focus concerns how top-down CONTACT Juyoen Hur
[email protected]
© 2016 Informa UK Limited, trading as Taylor & Francis Group
KEYWORDS
Emotion; cognition; attentional focus; biased competition model; congruence interference effect
(e.g. goal of the task) and bottom-up (e.g. emotional salience) influences interact to bias the competition among items in the visual field (Desimone & Duncan, 1995). Studies show that emotion impairs performance on a perception task when it is task-irrelevant, whereas emotion facilitates performance when it is task-relevant (Dodd, Vogt, Turkileri, & Notebaert, 2016; Lichtenstein-Vidne, Henik, & Safadi, 2012). For example, in an Emotional Stroop task, response times (RTs) for naming the ink colour of words were longer for negative words than neutral words, suggesting that the emotional content of words can divert attention from the primary colour-naming task (Compton et al., 2003). On the other hand, there is evidence that emotion facilitates the likelihood and the degree to which arousing information is processed when it is relevant to the task. For example, in a rapid serial visual presentation task, the likelihood of missing the second of two stimuli presented in rapid succession was reduced when the stimulus was
2
J. HUR ET AL.
emotionally arousing (Anderson & Phelps, 2001). However, the task-relevant or irrelevant character of emotion was intrinsic to the tasks in these previous investigations, and hence it is not known how manipulating the focus of attention on the emotional vs. non-emotional aspects influences perception. Thus, the first goal of our study was to investigate how the effect of emotion on perception is influenced by attentional focus. Another top-down factor to consider is the attentional load of the task. Research suggests that the emotional impact on perception is limited to situations in which the attentional load is low, during which attentional resources remain available for processing emotional information (Okon-Singer et al., 2014; Pessoa, McKenna, Gutierrez, & Ungerleider, 2002; Van Dillen & Derks, 2012). For example, the behavioural interference and neural responses (amygdala activation) to negative distractor stimuli (compared with neutral stimuli) have been observed only under low perceptual load, but not high perceptual load (Okon-Singer et al., 2014; Pessoa et al., 2002; Shafer et al., 2012). Similarly, the relatively slower RT to angry faces (compared to happy faces) in the gender-naming task was eliminated under high WM load, suggesting that WM load interferes with the typically facilitated processing of threating stimuli (Hur, Iordan, Berenbaum, & Dolcos, 2015; Van Dillen & Derks, 2012). However, most of these studies investigated the effect of attentional load when emotion is task-irrelevant, leaving the combined effects of attentional focus and attentional load an open question. Thus, in addition to examining the modulating effect of attentional focus, we investigated the impact of attentional load and the potential interaction of these two factors. As with perception, attentional focus may also explain the contradictory results regarding the effect of emotion on WM, as both impairing and facilitating effects have been reported. Studies show that in the presence of emotional distractors, WM performance tends to be impaired (Dolcos et al., 2011). As with perception, the bottom-up salience of the emotional distractor biases attention away from the task-relevant information in WM (Desimone & Duncan, 1995). For example, negative stimuli presented as irrelevant distractors during a delayed-response WM task led to worse cognitive performance compared to neutral stimuli (Dolcos & McCarthy, 2006). On the other hand, when emotion is task-relevant, the opposite effects were observed. For example, on a modified
recency-probes paradigm, proactive interference (interference from preceding but no longer relevant materials) involved in WM was reduced when emotional stimuli were used as probes (Levens & Phelps, 2008), suggesting that emotion facilitates interference resolution processes within WM (Lindström & Bohlin, 2011). Despite the evidence suggesting a potential modulating effect of attentional focus on the influence of emotion on WM, investigations of this matter using manipulations of attentional focus are lacking. Thus, the second goal of our study was to investigate whether the effect of emotion on WM is influenced by attentional focus. Another important question pertains to the congruence between the emotional content of material already being held on-line in WM and new input. Although not investigated in the context of WM, the memory literature suggests a congruence interference effect (Sison & Mather, 2007). According to this account, emotional memories compete with memories that evoke similar emotions because resolving the competition among memories of the same emotion requires greater inhibition at retrieval than resolving the competition among items that are not of the same emotion (Mather, 2009). The congruence interference effect of emotion held only when participants’ attention was explicitly driven to the emotional aspects of stimuli (Sison & Mather, 2007). Although this theory has been proposed in relation to the competition in recall between two memories learned in the past, it is possible that such competition is not limited to long-term memories but may also apply to a broader range of memory processes, including WM. When updating the representations online (e.g. n-back task), the to-be maintained and new input stimuli in WM are likely to compete for limited cognitive resources since newly learned information can be vulnerable to interference from the information kept online (Jonides & Nee, 2006). Based on the extant literature, we made the following predictions. Regarding the effect of emotion on perception, we hypothesised that emotion would facilitate or interfere with perception depending on its relevance to the task (attentional focus). Specifically, we hypothesised that emotion would improve task performance when it is relevant to the task requirements, but would impair task performance when it is irrelevant to the task requirements. We expected that the differential behavioural effect of emotion on perception as a function of attentional focus would diminish under high WM load. In
COGNITION AND EMOTION
addition, we hypothesised that the effect of emotion on WM would be influenced by both attentional focus and the dynamic relationship between the emotional contents of the to-be maintained and new input stimuli. If the congruence interference effect holds, the performance would be impaired when both stimuli are from the same emotional category, relative to when they are not, and only when attention was explicitly drawn to the emotional properties of the stimuli.
Methods Participants One hundred and thirty-one undergraduate students (64% female; mean age = 19.3) participated in the study for course credit. Only individuals, who reported having normal or corrected-to-normal vision, including normal colour vision, were permitted to participate. All participants provided informed consent. The experimental protocol was approved for ethical treatment to human participants by the Institutional Review Board at the University of Illinois at UrbanaChampaign.
Design and procedures Picture stimuli were selected from the International Affective Picture System (Lang, Bradley, & Cuthbert, 2008). Sixteen negative pictures were selected to be relatively high in arousal and negative in valence, and 16 neutral pictures were selected that were rated as non-arousing and of neither negative nor
3
positive valence. Ratings ranged from 1 (low pleasure, low arousal) to 9 (high pleasure, high arousal). The mean valence and arousal scores were 2.1 (SD = 0.4) and 6.4 (SD = 0.6), respectively, for the negative stimuli, and 5.2 (SD = 0.2) and 3.3 (SD = 0.4), respectively, for the neutral stimuli. In each valence category (negative, neutral), two different stimulus types were included: pictures with people and pictures without people. The pictures included were: (a) eight negative pictures with people (e.g. injured), (b) eight negative pictures without people (e.g. crashed cars), (c) eight neutral pictures with people (e.g. answering phone), and (d) eight neutral pictures without people (e.g. buildings). Each picture appeared four times during the task and they were randomly coloured in yellow or blue hues (see Figure 1). We used a between-subject design to minimise the likelihood of fatigue and confusion regarding task instructions. Participants were randomly assigned to complete one of four tasks. Specifically, participants were randomly assigned to complete either a perception task (0-back task) or a WM task (2-back task). Participants were also randomly assigned to one of two attentional focus conditions; they were asked to either report the colour of the picture (COLOR) or report the emotion of the picture (EMO). In the perception task, participants had to indicate either the colour or the emotion of the current trial. In the WM task, participants had to indicate whether the colour (or emotion) of the picture in the present trial was the same as the one two trials before (see Appendix 1). Thus, there were four conditions: (1) EMO 0-back; (2) COLOR 0-back; (3) EMO 2-back; and (4) COLOR 2back. Each task consisted of 128 trials, each of which
Figure 1. Task design. Participants were randomly assigned to complete one of four tasks. First, participants completed either a perception task (i.e. 0-back task) or a WM task (i.e. 2-back task). In addition, participants were randomly assigned to one of two attentional focus conditions – that is, they were asked to either report the colour (i.e. COLOR) or the emotion of the picture (i.e. EMO).
4
J. HUR ET AL.
consisted of a target for 1500 ms (which did not disappear upon reaction), followed by a fixation cross for 3000 ms. As targets, negative and neutral pictures that were coloured in yellow or blue hues were presented in a quasi-randomised order such that the same emotion type (negative, neutral) or colour type (yellow, blue) was never presented on more than three consecutive trials. Luminance values of the tinted pictures were not significantly different across emotion types [F(1, 15) = 0.12, p > .7, h2p = 0.008]. Before the experimental task was administered, participants completed a short practice task after receiving instructions. Responses were made using arrow keys (left or right) on the keyboard. In every condition, participants were asked to respond as quickly and accurately as possible.
Analytic strategy First, a 2 (Emotion: Negative vs. Neutral) × 2 (Task Type: 0-back vs. 2-back) × 2 (Attentional Focus: EMO vs. COLOR) ANOVA was conducted on RT and Accuracy data. Emotion was entered as a within-subject variable and Task Type and Attentional Focus were dichotomous variables entered as between-subject variables. Significant interactions were followed up by conducting 2 (Emotion: Negative vs. Neutral) × 2 (Attentional Focus: EMO vs. COLOR) ANOVAs separately for the perception task (i.e. 0-back) and for the WM task (i.e. 2-back). For WM, an additional 2 (2back Type: Negative vs. Neutral) × 2 (Current Type: Negative vs. Neutral) × 2 (Attentional Focus: EMO vs. COLOR) ANOVA was conducted on RT and Accuracy data to examine the possibility of a congruence interference effect.
Results Descriptive statistics Out of 131 participants, the accuracy of 5 participants was 2 SDs below the mean in at least one condition (EMO 0-back: 1, COLOR 0-back: 1, EMO 2-back: 2, COLOR 2-back: 1 participant), and thus they were excluded from the analyses. Thus, a total of 126 participants were included in the following analyses (EMO 0back: n = 32, COLOR 0-back: n = 30, EMO 2-back: n = 33, COLOR 2-back: n = 31). For RT analyses, we only considered trials in which participants made correct responses. Trials with extreme RT scores (below 200 and above 2000 ms) were considered outliers and
eliminated from the analyses. Age and gender were balanced across conditions, showing neither gender × condition, χ 2(3, 126) = 1.28, p > .7, ф = 0.10, nor age × condition, F(3, 122) = 0.49, p > .6, h2p = 0.01 effects. Descriptive statistics for each task are reported in Appendix 2.
Overall RT and accuracy differences in perceptual vs. WM tasks We first examined differences in RT and accuracy between the perceptual task (0-back) and the WM task (2-back). As expected, there were significant differences in both RT [t(124) = −10.86, p < .01, d = 1.95] and accuracy [t(124) = 12.48, p < .01, d = 2.24]; RTs were greater in the WM task than in the perception task, and accuracy was lower in the WM task than in the perception task. In the perception (0-back) task, there was a ceiling effect in accuracy (M = 97%, SD = 0.02%). More meaningful variation was found in RTs (M = 693.5, SD = 150.9 in milliseconds). Thus, for the perception task, although both RT and accuracy data were analysed, we focused on interpreting the RT results. This approach is consistent with previous literature because RT is the dependent measure most frequently used in assessing the behavioural effects regarding selective attention and distractor interference in perceptual tasks (Nelson, Crisostomo, Khericha, Russo, & Thorne, 2012). On the other hand, there was more variability in accuracy in the WM (2-back) task (M = 79%, SD = 11%) than in the perception task (coefficients of variability were 0.14 and 0.03, respectively), whereas there was less variability in RT in the WM task (M = 1031.5, SD = 194.8 in milliseconds) than in the perception task (coefficients of variability were 0.18 and 0.22, respectively). In this type of WM task, participants’ efforts are generally focused more on performing the task accurately than responding as fast as they can. Thus, for the WM task, although both RT and accuracy data were analysed, we focused on interpreting the accuracy results.
The effect of emotion on perception and WM as a function of attentional focus A 2 (Emotion: Negative vs. Neutral) × 2 (Task Type: 0back vs. 2-back) × 2 (Attentional Focus: EMO vs. COLOR) ANOVA on RT data showed a significant three-way interaction [F(2, 122) = 8.44, p = .004, h2p =
COGNITION AND EMOTION
0.07]. As expected, follow-up ANOVAs revealed a significant Emotion × Attentional Focus interaction in the perception (0-back) task [F(1, 60) = 11.54, p = .001, h2p = 0.16], but not in the WM (2-back) task [F(1, 62) = 0.56, p > .4, h2p = 0.009]. As shown in Figure 2, during the perception (0-back) task, RTs were faster for negative than neutral stimuli in the EMO condition [t(31) = −2.12, p = .042, d = 0.38], whereas RTs were slower for negative than neutral stimuli in the COLOR condition [t(29) = 2.88, p = .007, d = 0.53]. A parallel analysis was conducted on Accuracy data. A 2 (Emotion: Negative vs. Neutral) × 2 (Task Type: 0-back vs. 2-back) × 2 (Attentional Focus: EMO vs. COLOR) ANOVA showed that there was a significant main effect of Emotion [F(1, 122) = 8.06, p = .005, h2p = 0.06], such that accuracy for negative stimuli was worse than for neutral stimuli.
Test of congruence interference effect in WM For WM, additional 2 (2-back Type: Negative vs. Neutral) × 2 (Current Type: Negative vs. Neutral) × 2 (Attentional Focus: EMO vs. COLOR) ANOVAs were conducted for accuracy and RT data. For Accuracy, although there were no main effects of 2-back Type or Current Type and no significant 2-way interactions, a significant 3-way interaction of 2-back Type × Current Type × Attentional Focus [F(1, 62) = 4.64, p = .035, h2p = 0.07] was found. A follow-up analysis revealed that a 2-back Type × Current Type interaction
5
was significant under the EMO condition [F(1, 32) = 6.12, p = .019, h2p = 0.16], but not under the COLOR condition [F(1, 30) = 0.03, p > .8, h2p = 0.001]. As shown in Figure 3, under the EMO condition, we found the expected congruence effect. Specifically, accuracy for matching conditions (i.e. NEG:NEG, NEU:NEU) was lower than for mismatch conditions (i.e. NEG:NEU, NEU:NEG). An additional 2 (Target Type: Negative vs. Neutral) × 2 (Match Type: Match vs. Mismatch) ANOVA in the EMO condition confirmed a significant Match Type main effect [F(1, 32) = 6.12, p = .019, h2p = 0.16], but the Target Type × Match Type interaction was not significant [F(1, 32) = 0.12, p > .7, h2p = 0.004]. This result suggests that the magnitude of accuracy difference in match vs. mismatch trials is not statistically different for the NEG (target) trials than for the NEU (target) trials. A parallel analysis was conducted on RT data. A 2 (2-back Type: Negative vs. Neutral) × 2 (Current Type: Negative vs. Neutral) × 2 (Attentional Focus: EMO vs. COLOR) ANOVA showed a significant 2-way interaction of 2-back Type × Attentional Focus [F(1, 62) = 5.29, p = .025, h2p = 0.08]. No significant main effects or other interactions were observed. In the EMO condition, RTs were faster when the 2-back stimulus was a negative picture, compared to when it was a neutral picture. In the COLOR condition, however, the pattern was reversed, such that RTs were slower when the 2-back stimulus was a negative picture, compared to when it was a neutral picture. Other effects were not significant. The additional analyses performed based on colour confirmed that the congruence interference effect was specific to when updating emotions, but not when updating colour (see Appendix 3). In addition, an analysis performed on 0-back tasks confirmed that the congruence interference effect held only for WM but not for perceptual tasks (see Appendix 4).
Discussion The effect of emotion on perception is modulated by attentional focus
Figure 2. The effect of emotion on perception and WM as a function of attentional focus. The interactive effect of Emotion and Attentional Focus on RT was only evident in the perception (0-back) task, not in the WM (2-back) task. During the perception task, emotion facilitated performance when it was task-relevant (EMO 0-back), whereas it impaired performance when it was task-irrelevant (COLOR 0-back). NEG: Negative, NEU: Neutral. *p < .05, two-tailed.
Consistent with previous findings (Anderson & Phelps, 2001; Compton et al., 2003), attentional focus modulated the impact of emotion on perception. Specifically, emotion facilitated perceptual task performance when it was relevant to the task, whereas it impaired performance when it was irrelevant to the task. This is consistent with the proposal that the impact of emotion on performance depends on the task relevance of the
6
J. HUR ET AL.
Figure 3. Congruence interference effect in WM. Under the EMO 2-back task, performance was worse when the material already being held online and the new input were of matching emotions (i.e. both were negative), compared to when they were not. No such effect was observed under the COLOR 2-back task. NEG: Negative, NEU: Neutral.
emotional information (Vuilleumier, 2005). Previous studies investigated either the distracting effect of emotional stimuli (that were spatially segregated from the target) or the facilitating effect of emotion (as part of the main task). Thus, this is the first study to assess the potentially facilitating and impairing effect of emotion on perception in the same study using the same paradigm. We were able to show that the nature of the emotional influence on perception changes even by simply changing the focus of attention to the emotional vs. non-emotional aspects of the same stimuli. These results confirm that the nature of the emotional influence on perception depends on top-down attention, specifically, whether task requirements involve the processing of emotional information. The modulation effect of attentional focus on the impact of emotion on perception can be explained using the biased competition model framework (Desimone & Duncan, 1995). In a limited processing capacity system like perception, emotional stimuli hold a competitive advantage conferred by their affective significance (Vuilleumier, 2005), biasing attention in a bottom-up fashion. Depending on the task relevance of emotion as determined by attentional focus, the bottom-up (i.e. emotional salience) and the top-down biases (i.e. goal of the task) attract attentional resources in the same or opposite direction, which in turn influences performance. That is, when
emotion is task-relevant, the bottom-up and topdown biases work in the same direction of pulling resources, which in turn facilitates performance. When emotion is task-irrelevant, the bottom-up and top-down biases are opposed such that the high salience of bottom-up attention drawn by emotion needs to be overcome by top-down attention (Anderson & Phelps, 2001; Lichtenstein-Vidne et al., 2012), which in turn may impair performance. The effect of attentional focus was further modulated by WM load, such that the differential behavioural effect of emotion on perception as a function of attentional focus was eliminated under high-WM load. This result is consistent with previous behavioural and brain imaging evidence showing diminished impact of emotion under high-attentional load, during which available attentional resources are limited for processing emotional information (OkonSinger et al., 2014; Pessoa et al., 2002; Shafer et al., 2012). The current findings suggest that the emotional impact on perception is determined by both attentional resources and attentional focus.
The effect of emotion on WM depends on emotional congruence of the materials online When it comes to the impact of emotion on WM, our results show that attentional focus itself does not facilitate or impair performance. Rather, our results
COGNITION AND EMOTION
suggest that the nature of the influence of emotion on WM depends on the dynamic relationship between internal representations. Specifically, we found that WM performance was worse when the material already being held on-line and the new input belonged to the same emotional category, compared to when they were not. This result shows that the congruence interference effect (Sison & Mather, 2007) holds in the context of WM, although it remains unclear whether the same mechanisms hold for WM as for long-term memory. A possible explanation could be that changes in emotion (here, changes in emotional categories) may be easier to detect due to their saliency. This explanation is in line with the repetition blindness (RB) effect, which refers to difficulty in detecting or reporting the repetition of an item in a stream displayed in a rapid serial visual presentation (RSVP) paradigm (Kanwisher, Kim, & Wickens, 1996). Although our WM paradigm was not an RSVP task per se, RB could potentially explain why participants’ performance on matching trials is worse than mismatch trials in the emotional 2-back task. Research suggests that a stronger stimulus representation (e.g. representing the emotional value of a stimulus) tends to induce a longer, increased RB (Kanwisher et al., 1996); the emotional content of a stimulus is an example of an important semantic component. In line with this explanation, it is important to note that the impaired performance in matching trials (compared to mismatch trials) was only found in the EMO 2-back task but not in the COLOR 2-back task. In real life contexts, noticing changes in emotions has greater survival and evolutionary value than does noticing the same emotion (Anderson & Phelps, 2001). The so-called pop-out or snake in the grass effect (e.g. the more rapid and accurate detection of threatening faces among non-emotional faces) is well established in visual perception (Öhman, Lundqvist, & Esteves, 2001). It is possible that the adaptive value of detecting changes in emotions is at least part of the reason why updating changes in emotions in WM is easier than updating sameness in emotions. The current result can be also explained in the context of the biased competition model. According to this model, items held in WM function as a search template that facilitates the processing of matching items (Desimone & Duncan, 1995). This is generally true when it comes to neutral items (e.g. shape) (Bichot, Rossi, & Desimone, 2005), but to our knowledge, no studies thus far have tested whether this
7
applies to updating emotional contents. Our result in the EMO task suggests that in the context of updating emotional contents (EMO 2-back), under matching trials, the bottom-up attention, which is recruited by “difference” or “change” in emotional categories, may be in direct conflict with the top-down strategy of imposing an attentional template (looking for “similarity”), which in turn impairs performance. This explanation is consistent with the idea that the salience network (anterior insula, dorsal cingulate cortex) plays a critical role in mediating the interaction between bottom-up and top-down attention in the context of emotional WM processing (Luo et al., 2014).
Conclusion Our results suggest that (1) the impact of emotion on perception depends on both attentional focus and attentional resources and (2) the impact of emotion on WM depends on the dynamic relationship between internal representations. Although we hypothesise that the competition between bottomup and top-down processing for limited cognitive resources may explain the nature of the influence of emotion on both perception and WM, additional research is needed to examine whether the working mechanisms truly overlap between the influence of emotion on perception and WM. Future research should continue to examine whether the emotional category congruence interference effect in WM holds with another emotional valence category (i.e. positive). It would be also valuable to explore whether the congruence interference effect still holds when the task becomes even more difficult. Furthermore, additional research regarding the precise mechanisms involved in the congruence interference effect is needed, as difficulties in adaptively updating emotional content in WM may contribute to the development and maintenance of emotional disorders, such as anxiety and depression.
Disclosure statement No potential conflict of interest was reported by the authors.
References Anderson, A. K., & Phelps, E. A. (2001). Lesions of the human amygdala impair enhanced perception of emotionally salient events. Nature, 411(6835), 305–309.
8
J. HUR ET AL.
Bichot, N. P., Rossi, A. F., & Desimone, R. (2005). Parallel and serial neural mechanisms for visual search in macaque area V4. Science, 308(5721), 529–534. Compton, R. J., Banich, M. T., Mohanty, A., Milham, M. P., Herrington, J., Miller, G. A., … Heller, W. (2003). Paying attention to emotion. Cognitive, Affective, & Behavioral Neuroscience, 3(2), 81–96. Desimone, R., & Duncan, J. (1995). Neural mechanisms of selective visual attention. Annual Review of Neuroscience, 18(1), 193–222. Dodd, H. F., Vogt, J., Turkileri, N., & Notebaert, L. (2016). Task relevance of emotional information affects anxiety-linked attention bias in visual search. Biological Psychology. Advance online publication. doi:10.1016/j.biopsycho.2016.01.017 Dolcos, F., Iordan, A. D., & Dolcos, S. (2011). Neural correlates of emotion-cognition interactions: A review of evidence from brain imaging investigations. Journal of Cognitive Psychology, 23(6), 669–694. Dolcos, F., & McCarthy, G. (2006). Brain systems mediating cognitive interference by emotional distraction. The Journal of Neuroscience, 26(7), 2072–2079. Hur, J., Iordan, A. D., Berenbaum, H., & Dolcos, F. (2015). Emotion– attention interactions in fear conditioning: Moderation by executive load, neuroticism, and awareness. Biological Psychology. Advance online publication. doi:10.1016/j.biopsycho.2015.10. 007 Hur, J., Miller, G. A., McDavitt, J. R., Spielberg, J. M., Crocker, L. D., Infantolino, Z. P., … Heller, W. (2015). Interactive effects of trait and state affect on top-down control of attention. Social Cognitive and Affective Neuroscience, 10(8), 1128–1136. Jonides, J., & Nee, D. E. (2006). Brain mechanisms of proactive interference in working memory. Neuroscience, 139(1), 181–193. Kanwisher, N. G., Kim, J. W., & Wickens, T. D. (1996). Signal detection analyses of repetition blindness. Journal of Experimental Psychology: Human Perception and Performance, 22(5), 1249– 1260. Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (2008). International Affective Picture System (IAPS): Affective ratings of pictures and instruction manual (Technical Report A-8). Levens, S. M., & Phelps, E. A. (2008). Emotion processing effects on interference resolution in working memory. Emotion, 8 (2), 267–280. Lichtenstein-Vidne, L., Henik, A., & Safadi, Z. (2012). Task relevance modulates processing of distracting emotional stimuli. Cognition & Emotion, 26(1), 42–52. Lindström, B. R., & Bohlin, G. (2011). Emotion processing facilitates working memory performance. Cognition & Emotion, 25(7), 1196–1204. Luo, Y., Qin, S., Fernandez, G., Zhang, Y., Klumpers, F., & Li, H. (2014). Emotion perception and executive control interact in the salience network during emotionally charged working memory processing. Human Brain Mapping, 35(11), 5606–5616.
Mather, M. (2009). When emotion intensifies memory interference. Psychology of Learning and Motivation, 51, 101–120. Nelson, M. D., Crisostomo, M., Khericha, A., Russo, F., & Thorne, G. L. (2012). Classic debates in selective attention: Early vs. late, perceptual load vs. dilution, mean RT vs. measures of capacity. Perception, 41(8), 997–1000. Öhman, A., Lundqvist, D., & Esteves, F. (2001). The face in the crowd revisited: A threat advantage with schematic stimuli. Journal of Personality and Social Psychology, 80(3), 381–396. Okon-Singer, H., Mehnert, J., Hoyer, J., Hellrung, L., Schaare, H. L., Dukart, J., & Villringer, A. (2014). Neural control of vascular reactions: Impact of emotion and attention. The Journal of Neuroscience, 34(12), 4251–4259. Pessoa, L., McKenna, M., Gutierrez, E., & Ungerleider, L. (2002). Neural processing of emotional faces requires attention. Proceedings of the National Academy of Sciences, 99(17), 11458–11463. Shafer, A. T., Matveychuk, D., Penney, T., O’Hare, A. J., Stokes, J., & Dolcos, F. (2012). Processing of emotional distraction is both automatic and modulated by attention: Evidence from an event-related fMRI investigation. Journal of Cognitive Neuroscience, 24(5), 1233–1252. Sison, J. A. G., & Mather, M. (2007). Does remembering emotional items impair recall of same-emotion items? Psychonomic Bulletin & Review, 14(2), 282–287. Van Dillen, L. F., & Derks, B. (2012). Working memory load reduces facilitated processing of threatening faces: An ERP study. Emotion, 12(6), 1340–1349. Vuilleumier, P. (2005). How brains beware: Neural mechanisms of emotional attention. Trends in Cognitive Sciences, 9(12), 585– 594.
Appendix 1. Instructions for four different conditions
Perception Task (0-back)
WM Task (2-back)
Attention is on EMOTION EMO 0-back task: Indicate the EMOTION of the current picture (i.e. Negative or Neutral), while ignoring the colour EMO 2-back task: Indicate whether the current EMOTION of the picture is the same as the one presented 2trials back (i.e. same or different), while ignoring the colour
Attention is on COLOR COLOR 0-back task: Indicate the COLOR of the current picture (i.e. yellow or blue), while ignoring the content COLOR 2-back task: Indicate whether the current COLOR is the same as the one presented 2-trials back (i.e. same or different), while ignoring the content
COGNITION AND EMOTION
9
Appendix 2. Descriptive statistics for the four different conditions
EMO 0-back (N = 32) M Gender Age Accuracy (%) RT (ms)
SD
COLOR 0-back (N = 30) M
65% 19.2 96.6 754.5
SD
EMO 2-back (N = 33) M
62% 1.0 0.03 110.3
19.2 97.2 633.1
Appendix 3. Results showing that the congruence interference effect is specific to emotion In order to examine whether the observed congruence interference effect is specific to emotion, an additional 2 (2-back Type: Yellow vs. Blue) × 2 (Current Type: Yellow vs. Blue) × 2 (Attentional Focus: EMO vs. COLOR) ANOVA was performed for accuracy and reaction time as a negative control. For accuracy, neither a 2-back Type × Current Type interaction [F(1, 62) = 0.99, p > .3, h2p = 0.02] nor a 2-back Type × Current Type × Attentional Focus interaction [F(1, 62) = 0.08, p > .7, h2p = 0.001] was significant. However, for RT data, a significant 2-back Type × Current Type × Attentional Focus interaction [F(1, 62) = 12.37, p = .01, h2p = 0.17] was observed. When these effects were decomposed, a significant 2-back Type × Current Type interaction was found in the COLOR condition [F(1, 30) = 22.99, p < .001, h2p = 0.43], but not in the EMO condition [F(1, 32) = 0.001, p > .9, h2p < 0.001]. Under the COLOR condition (i.e. COLOR 2-back task), RT was slower for mismatch conditions (i.e. yellow:blue, blue:yellow) than for matching conditions (i.e. yellow:yellow, blue:blue). This result confirms that the interfering
SD
COLOR 2-back (N = 31) M
62% 1.3 0.03 165.2
19.2 78.1 1103.3
SD 65%
1.3 0.11 163.8
19.4 79.9 983.4
1.3 0.11 205.6
congruence effect (i.e. impaired performance in match trials compared to mismatch trials) is specific to when updating emotions, but not when updating colour.
Appendix 4. Results showing that the congruence interference effect holds only for WM but not for perceptual tasks In order to confirm that the congruence interference effect is specific to WM (but not to perception), we tested whether each trial in the 0-back task influenced the subsequent trial by performing a 2 (1-back Type: Negative vs. Neutral) × 2 (Current Type: Negative vs. Neutral) × 2 (Attentional Focus: EMO vs. COLOR) ANOVA in 0-back tasks for accuracy and RT. The results show neither a 1-back Type × Current Type interaction [F(1, 60) = 0.75, p > .3, h2p = 0.01] nor a 1-back Type × Current Type × Condition interaction [F(1, 60) = 1.58, p > .2, h2p = 0.03] for accuracy. No significant interaction effect was found in the RT analysis, either. Thus, we can conclude that the sequential effect holds only for WM but not for perceptual tasks.
