Reward- and Attention-related Biasing of Sensory Selection in Visual Cortex Antje Buschschulte1, Carsten N. Boehler2, Hendrik Strumpf1, Christian Stoppel1, Hans-Jochen Heinze1,3, Mircea A. Schoenfeld1,3, and Jens-Max Hopf1,3
Abstract
oo
f
not the reward-relevant color produced a selective enhancement of the neuromagnetic response between 180 and 280 msec in ventral extrastriate visual cortex. Increasing reward prospect caused a delayed attenuation (220–250 msec) of the response to reward probes, which followed a prior (160–180 msec) response enhancement in dorsal ACC. Notably, shorter latency responses in dorsal ACC were associated with stronger attenuation in extrastriate visual cortex. Finally, an analysis of the brain response to the search frames revealed that the presence of the reward-relevant color in search distractors elicited an enhanced response that was abolished after increasing reward size. The present data together indicate that when top–down definitions of reward relevance and attention are separated, the behavioral significance of reward-associated features is still rapidly coded in higher-level cortex areas, thereby commanding effective top– down inhibitory control to counter a selection bias for those features in extrastriate visual cortex. ■
INTRODUCTION
or
re
ct
ed
selection in extrastriate cortex. Features signaling reward seem to produce a similar bias, but how modulatory effects due to reward and attention relate to each other is largely unexplored. To address this issue, it is critical to separate top–down settings defining reward relevance from those defining attention. To this end, we used a visual search paradigm in which the targetʼs definition (attention to color) was dissociated from reward relevance by delivering monetary reward on search frames where a certain task-irrelevant color was combined with the targetdefining color to form the target object. We assessed the state of neural biasing for the attended and reward-relevant color by analyzing the neuromagnetic brain response to asynchronously presented irrelevant distractor probes drawn in the target-defining color, the reward-relevant color, and a completely irrelevant color as a reference. We observed that for the prospect of moderate rewards, the target-defining color but
Pr
■ Attention to task-relevant features lead to a biasing of sensory
U nc
Considerable experimental evidence suggests that the sudden onset of an unexpected visual stimulus can draw attention in an automatic bottom–up fashion—an effect referred to as attentional capture (Theeuwes, 2010; Theeuwes & Godljn, 2001; Yantis, 1996). Such capture, however, turns out to be not entirely automatic, as it has been shown to depend on the degree to which features of the capturing stimulus match the set of top– down defined descriptions of the task-relevant input (contingent involuntary orienting; Eimer & Kiss, 2008; Lamy, Leber, & Egeth, 2004; Gibson & Kelsey, 1998; Bacon & Egeth, 1994; Folk, Remington, & Wright, 1994; Folk, Remington, & Johnston, 1992). Top–down descriptions may entail a selection bias for one or a set of simple target-defining features (e.g., color, orientation) or refer to a selection bias contingent on the more general behavioral relevance of a target object, for example, its emotional significance, or association with reward.
1
Otto-von-Guericke University Magdeburg, 2Ghent University, Leibniz Institute for Neurobiology, Magdeburg, Germany
3
© Massachusetts Institute of Technology
Attentional mechanisms of top–down biasing of feature selection in visual cortex have been extensively investigated, with data from different methodologies indicating that attention exerts its effect by modulating the gain of neural processing in sensory visual areas (Hopf, Heinze, Schoenfeld, & Hillyard, 2009; Maunsell & Treue, 2006; Treue, 2001; Kastner & Ungerleider, 2000; Hillyard, Vogel, & Luck, 1998). This may come about either by increasing the gain of neural representations that match the attended feature (Maunsell & Treue, 2006; Treue & Martinez Trujillo, 1999; Motter, 1994) or by attenuating neural activity of irrelevant feature representations (Lennert & Martinez-Trujillo, 2011; Chelazzi, Duncan, Miller, & Desimone, 1998; Chelazzi & Desimone, 1994). In addition, more recent research has mounted evidence suggesting that value-based factors like the reward relevance of stimuli may have an analogous modulatory influence on feature-selective processing in visual cortex (Hickey, Chelazzi, & Theeuwes, 2010; Franko, Seitz, & Vogels, 2009; Serences, 2008; Shuler & Bear, 2006) and recent behavioral and neurophysiological observations imply an influence on even earlier processing stages (Hickey & van Zoest, 2012; Ikeda & Hikosaka, 2003). Journal of Cognitive Neuroscience X:Y, pp. 1–17 doi:10.1162/jocn_a_00539
Pr
oo
f
ing from extrastriate visual cortex areas (Hopf, Boelmans, Schoenfeld, Luck, & Heinze, 2004; Hopf, Vogel, Woodman, Heinze, & Luck, 2002; Hopf et al., 2000; Martinez et al., 1999; Heinze et al., 1994). Moreover, several studies have reported an N2pc component in response to attentional capture contingent on distractors sharing task-relevant features with the target. The N2pc is known to index attentional focusing (Ansorge, Kiss, & Eimer, 2009; Eimer & Kiss, 2008; Kiss, Jolicoeur, Dellʼacqua, & Eimer, 2008; Leblanc et al., 2008) and has been shown to arise from source activity in ventral extrastriate cortex based on event-related magnetic field (ERMF) recordings (Hopf et al., 2000, 2002, 2004). Here we will focus on the ERMF response elicited by task-irrelevant color probes and analyze whether modulations in visual extrastriate visual cortex differs when comparing target and reward probes relative to control probes. We will further investigate whether above reviewed signatures of attentional capture can be identified in the ERMF response to search frames where the rewardrelevant color is presented in the distractor.
METHODS
Participants (Experiment 1)
ed
Twenty participants (mean age = 26.1 years, range = 21– 32, 14 women) took part in Experiment 1. The experiment was undertaken with the understanding and written consent of the participants. All participants were students of the Otto-von-Guericke University Magdeburg, were right-handed, had normal color vision, and had normal or corrected-to-normal visual acuity. Participants were paid for participation in addition to the payoff that could be gained on rewarded trials. All experiments were approved by the ethics review board of the Otto-vonGuericke University Magdeburg.
U nc
or
re
ct
These observations raise the question how reward relevance imposes its effect on visual sensory processing. Despite the currently proliferating knowledge about brain circuits and functions mediating reward relevance in general (Haber & Knutson, 2010; Hikosaka, Bromberg-Martin, Hong, & Matsumoto, 2008; Schultz, 2000), the mechanism by which reward ultimately influences visual sensory selection and whether they dissociate from modulatory effects because of attention is not well understood. In fact, Maunsell (2004) emphasized the importance to clarify whether neural correlates of attention and reward dissociate in their impact on early visual processing because in animal experiments attention is typically tied to the application of reward and therefore inherently at risk to be confounded by reward (and vice versa). A few recent studies have explicitly aimed at a dissociation of reward effects from attention in humans and monkeys, with mixed results ranging from modest (Weil et al., 2010) to clear-cut effects attributable to reward proper (Arsenault, Nelissen, Jarraya, & Vanduffel, 2013; Hickey et al., 2010). Importantly, in humans, top–down settings for task-relevant stimulus properties can be set by plain verbal instruction without resorting to the direct application of reward, which makes it easier to separate taskand reward-relevant top–down settings (Boehler et al., 2011). With the following two experiments, we aim at such paradigmatic separation between reward relevance and feature-based attention. Specifically, we ask how top– down settings that define reward relevance (a rewarddefining feature) but not task relevance (a target-defining feature) influence sensory processing in extrastriate cortex. To this end, we will use a simple visual search task requiring participants to search for a target defined by color (top–down settings for a task-relevant feature), whereas the combined occurrence of a specific taskirrelevant color with the target color will be associated with reward (top–down settings for reward relevance). To assess the degree of passive sensory biasing for the selection of those colors in visual cortex, we analyzed the neuromagnetic brain response elicited by the sudden onset of a task-irrelevant color probe at performanceirrelevant time points between search frame presentations. The probe will be drawn either in the target color, the reward color, or a control color. Additionally, we will also assess the biasing effect of reward color during target selection by analyzing the brain response elicited by the search frames as a function of the presence/absence of the reward color in the search distractor. Electrophysiological signatures of attentional capture have been documented to appear as amplitude modulations of early sensory visual components (Fukuda & Vogel, 2009; Eimer & Kiss, 2008; Leblanc, Prime, & Jolicoeur, 2008; Lien, Ruthruff, Goodin, & Remington, 2008; Hickey, McDonald, & Theeuwes, 2006; Hopfinger & Ries, 2005; Arnott, Pratt, Shore, & Alain, 2001; Hopfinger & Mangun, 1998), with the underlying current source activity originat2
Journal of Cognitive Neuroscience
Stimuli and Procedure (Experiment 1) Search arrays, probe stimuli, and the stimulation protocol are illustrated in Figure 1. Each search array contained a fixation cross and two double-colored 3-D spheres (diameter 2.7° visual angle), one in the left and one in the right visual field (VF; 5° lateral, 2.7° downward from fixation), presented on a homogenous gray background (luminance: 24 cd/m2). Each sphere was composed of two colors taken from a set of five colors (green, red, blue, yellow, gray), which were randomly assigned to the left and right half of the spheres. On each trial, one of the four half-spheres was drawn in the target color (green in Figure 1), with three of the other four colors being assigned to the remaining half-spheres in the following way. Although all half-spheres of a given search array were always drawn in a different color, on one third of the trials the target color was combined (randomly) with one of the three control colors (red in top search frame of Figure 1) with the distractor sphere containing Volume X, Number Y
U nc
or
re
ct
Figure 1. Structure of an experimental trial block of Experiment 1. An instruction frame informed participants about the target-defining color (here green) and the color associated with reward (here blue) when combined with the target color. On each search frame (the frames containing two double-colored spheres), participants had to discriminate the orientation of the convexity of the target half-sphere (left, right). Participants had to ignore the colored squares (color probes) flashed randomly between the search frames. The probes appeared randomly in the left and right VF and could be drawn in the target-defining color (target probe), the color associated with reward (reward probe), or one of the irrelevant colors (control probe (here gray, yellow, and red).
ed
Pr
oo
f
definition would clearly undermine the present aim to dissociate task and reward relevance. The participantsʼ task was to report whether the target color appeared on the left or right side of the target sphere with a two-alternative button press of the right hand (index finger: left, middle finger: right) as fast and accurate as possible. Search arrays were presented for 700 msec. On rewarded trials, a feedback frame was shown for 400 msec, 1000–1500 msec after search frame offset, informing the participant whether reward was gained or not. The ISI between an unrewarded search frame or the feedback frame and the next stimulus onset was randomized between 1000 and 1500 msec (rectangular distribution). Randomly on 50% of the trials, this subsequent stimulus was a single square (the probe, 1.8° × 1.8°, ISI 1000– 1500 msec) that was flashed for 50 msec in the left or right VF (centered at the location of the sphere). After such trials, the next search array appeared 600–900 msec after the probe. A probe was never directly followed by another probe, that is, a probe appeared on average on every second trial of an experimental block. Probe color and the VF of probe presentation were randomly assigned, such that the probe appeared as often in the VF of the subsequent search target or in the opposite VF. Experimental blocks were designed with target and reward color counterbalanced. That is, each color served as the target in four blocks, with every other color serving as reward color on one of the four blocks. Throughout the entire experimental session, each color served four times as reward color. During one experimental block (duration = 5 min, one break), 72 search frames and 36 probes were presented, yielding a session total of 60 trials per probe condition (target, reward, control) and VF (left, right) for each participant. The luminance of the different probe colors was matched using heterochromatic flicker photometry (Lee, Martin, & Valberg, 1988) in three selected participants. Respective luminance values were used for all participants that participated in Experiments 1 and 2.
a combination of the remaining control colors (yellow, gray). On another one third of the trials, the target color was combined with one of the control colors (yellow in middle search frame of Figure 1), with one half-sphere of the distractor sphere being drawn in the reward color (blue). On the last one third of the trials, the target color was combined with the reward color (green and blue in bottom search frame of Figure 1), and the distractor sphere contained a combination of two distractor colors (red and yellow). Importantly, reward was delivered only on the one third of the trials when the reward color was combined with the target color. When the reward color was combined with a control color in the VF opposite to the target (middle search frame in Figure 1), no reward was given. Note that the number of rewarded trials was set to a comparably low proportion of one third of the trials. This was done to de-emphasize the tendency that with an increasing proportion of rewarded trials participants might take the specific combination of target and reward color as defining the target. Such combined target
Reward Schedule The payoff for a rewarded trial was 5 Euro Cent, which amounted to a total bonus payment of approximately 10.70–12.00A per participant. Reward could be earned upon correct target discrimination on trials where the target sphere also contained a reward color. Participants (Experiment 2) Sixteen participants (mean age = 26.5 years, range = 23– 41 years, 14 women) participated in Experiment 2. All participants were students of the Otto-von-Guericke University Magdeburg, were right-handed, had normal color vision, and had normal or corrected-to-normal visual acuity. Participants were paid for participation in addition to the payoff that could be gained on rewarded trials. All Buschschulte et al.
3
experiments were approved by the ethics committee of the Otto-von-Guericke University Magdeburg.
U nc
Data Recording and Analysis
or
oo
Pr
ed
re
ct
Stimuli and procedure were similar to Experiment 1, except for the following modifications. (1) As in Experiment 1, each color served as the target color with all other colors serving one time as the reward color, however, under two different reward conditions (high-reward blocks: 10 Euro Cent, low-reward blocks: 5 Euro Cent per rewarded trial). (2) As this doubled the number of experimental blocks per set and participant in comparison with Experiment 1, each participant performed two experimental sessions on different days (maximum delay between sessions 40 days). During the first session, half of the participants performed high-reward blocks on uneven block numbers and low-reward blocks on even blocks numbers. The other half of the participants was assigned to high- and low-reward blocks in the reverse order. During the second session, the assignment of reward magnitude to block number was reversed in each participant, such that—with both sessions taken together—participants performed every target–reward combination under high- and low-reward conditions. Again, across both sessions, each participant performed 60 trials per probe condition (target, reward, control), VF (left, right), and reward magnitude (high, low). Participants were informed at the start of each block about whether high or low reward will be delivered. The range of payoff gained from rewarded trials amounted to 15.85– 18.00A per participant.
f
Stimuli and Procedure (Experiment 2)
generated by five coils fixated on defined positions on the EEG cap. Primary MEG data analysis included off-line artifact rejection applied to epochs of interest with peak-to-peak amplitudes exceeding a threshold of 2–3 pT. For each participant, epochs of interest ranging 200 msec before (baseline) to 750 msec after probe onset were extracted according to the relevant experimental conditions. Epochs containing peak-to-peak EOG amplitudes exceeding 150 μV were considered to contain eye motion artifacts and were rejected from further data analysis. This resulted in an average rejection rate of 18.0% for Experiment 1 and 13.2% for Experiment 2, with no significant difference between experimental conditions. For Experiment 1, averages were computed for the three different probe types (target, reward, control probes) in the left and right VF, thereby collapsing data across the different colors. In addition, selective averages were computed for ERMF responses elicited by the search frames (spheres) as a function of whether the reward color was present or absent in the distractor opposite to the search target. For this analysis, only nonrewarded trials were considered. For Experiment 2, the same averages were computed, but separately for high- and low-reward blocks, and after collapsing the data from the two experimental sessions of a given participant. For subsequent data analysis and visualization, the ERP software ERPSS (Event-Related Potential Laboratory, University of California, San Diego, La Jolla, CA) was used. The statistical validation of waveform differences was performed using a repeated-measures ANOVA (rANOVA) approach. If necessary, violations of data sphericity were corrected (Greenhouse–Geisser epsilon). Respective data will be reported with the original degrees of freedom, but with an adjusted level of significance ( p value). The statistical validation of onset latency differences was performed using a sliding t-test approach applied to subsequent time samples (window width of 20 msec; Guthrie & Buchwald, 1991). The first sample showing a significant difference in a sequence of at least three subsequently significant time samples was taken to mark the onset latency.
The magnetoencephalography (MEG) was recorded in a magnetically shielded chamber with a BTI Magnes 3600 WH whole-head magnetometer system (4D Neuroimaging, San Diego, CA) with 248 sensors. For controlling eye movements, the EOG was recorded with a horizontal and vertical montage using bipolar electrode placements at the temples (horizontal) as well as a unipolar electrode below the left eye (vertical). Impedances were kept below 5 kΩ (EEG cap and Abralyt light gel, Easycap, Herrsching, Germany). EOG signals were amplified with an EPA-6 amplifier (Sensorium, Inc., Charlotte, VT). The MEG and EOG signals were band-pass filtered (0.01–50 Hz) and digitized with a sampling rate of 254.31 Hz. In addition to recording the EOG, fixation performance was continuously monitored with a custommade zoom lens infrared camera system. To coregister anatomical and functional data, anatomical landmarks (nasion and left and right preauricular points) were digitized with a Polhemus 3Space Fastrak system (Polhemus Inc., Colchester, VT). These landmarks were then brought into reference with magnetic marker fields 4
Journal of Cognitive Neuroscience
Source Localization Analysis Current source localization was performed by computing minimum norm least squares estimates (MNLS, sLORETA) as implemented in the CURRY V6.0 neuroimaging software (Compumedics Neuroscan, Charlotte, NC). Estimates for the grand averaged data were computed by using realistic anatomical models of the source compartment (3-D surface segmentation of the cortical gray matter space) and the volume conductor (3-D surface segmentation of the cerebrospinal fluid space) of the standard MNI brain (Montreal Neurological Institute). Before averaging waveforms across participants, sensor data of each participant were coregistered to a reference subject based on the participantʼs individual landmarks. Volume X, Number Y
The grand averaged sensor data were then brought into reference with the anatomical data of the MNI brain.
RESULTS Experiment 1 Behavioral Performance as a Function of Probe Color and Probe Location
U nc
or
re
ct
ed
Pr
oo
f
Participants were to ignore the probes, so that no behavioral performance measure directly reflecting probe color processing can be obtained. However, the effect of probe color can be analyzed indirectly by determining the amount of exogenous spatial cuing (for the performance
on a subsequent search frame) created by the probes. That is, the amount of cuing produced by the different probe types will provide an index of the degree to which spatial attention was captured. To assess the effect of exogenous cuing, target discrimination performance was analyzed as a function of whether a given probe appeared on the side of the subsequently appearing target sphere (valid probes) versus on the opposite side (invalid probe). Given that the probe target SOA varied between 600 and 900 msec, the expected cuing effect is one of inhibition of return (IOR; Klein, 2000; Maylor & Hockey, 1985; Posner & Cohen, 1984), that is, a relative slowing of the response for valid probes, which was in fact observed for all three probe conditions. As shown in Figure 2A, valid probes led to an average of 11 msec slower response than invalid probes. Importantly, IOR did not vary with probe type indicating that spatial attention was captured by the probe to the same extent in all three probe conditions. A two-way rANOVA with the factors probe condition (target, reward, control) and probe validity (valid, invalid) confirmed this impression. There was a significant main effect of probe validity, F(1, 19) = 19.9, p < .0001, but no Probe Validity × Probe Condition interaction, F(2, 38) = 0.56. RTs were slightly slower for targets following reward probes versus target or control probes, but the main effect of probe condition did not reach significance, F(2, 38) = 2.76, p = .09. Figure 2B summarizes response accuracy (RA; % correct responses). Consistent with IOR, accuracy was generally lower for valid than for invalid trials, with this effect not being different for the three probe conditions. A respective rANOVA yielded a significant main effect of probe validity, F(1, 19) = 4.87, p < .05, but no interaction of Probe Validity × Probe Condition, F(2, 38) = 0.03. Again, there was a trend toward a main effect of probe condition, F(2, 38) = 2.83, p = .08, reflecting that performance accuracy was slightly reduced after reward probes relative to target and control probes. It should be noted that—although not significant—the general pattern of responses hints at a possible effect of reward on performance in form of an increased IOR effect. However, given that the overall IOR effect is already small and not significant, strong interpretations may not be warranted.
MEG Responses to the Color Probes
Figure 2. Target discrimination performance as a function of probelocation in Experiments 1 and 2. (A) Mean RT and (B) percent correct values for targets following color probes in the same VF as the probe (valid) or in the opposite VF (invalid) in Experiment 1. Data are shown separately for probes drawn in the target color (target), the reward color (reward), and the control color (control). (C) Mean RT and (D) percent correct responses of Experiment 2. Data are shown separately for the different probe types as well as the low- (low) and the highreward condition (high). White bars index the standard error of mean.
Figure 3A shows ERMF waveforms and distributions elicited by the target probes, reward probes, and control probes at representative selected sensor sites (arrows) over the left and right occipito-temporal cortex contralateral to the VF of probe presentation. Shown are waveforms and field distributions separately for right (top row) and left VF probes (bottom row). As visible, target probes (red traces) led to a response enhancement relative to control probes (black traces) between ∼180 and 280 msec after probe onset. No such enhancement Buschschulte et al.
5
f oo Pr ed ct re
U nc
or
Figure 3. MEG response to color probes of Experiment 1. (A) ERMF waveforms and distributions elicited by the different probe types in the right (top row) and left VF (bottom row). The shown waveforms were recorded at the sensor sites indicated by the small arrows. The black ellipses highlight the efflux–influx field configuration representing the response elicited by the probe. (B) Source waveforms and CSD distributions estimated for the data shown in A. Source waveforms represent the time course of source density estimates in the cortical regions centered at the source density maximum highlighted by white the small circles.
appeared for reward probes (green traces), for which the response remained almost indistinguishable from the response to control probes. This difference in response strength is further illustrated in the topographical maps (right) showing the average ERMF response between 180 and 280 msec for the three probe conditions. The response in visual cortex to the probe appears as efflux–influx configuration (encircled by black ellipses) over the lateral occipito-temporal cortex contralateral to the VF of probe presentation. Note that the efflux and influx component are shown in red and blue, respectively. Both together form the magnetic field effect that is generated by source activity (see Figure 3B) located under the transition zone (asterisks) between the efflux and influx component. The field effect is stronger for target probes than reward and control probes, with the latter two showing effects of comparable size. For statistical validation, a three-way rANOVA with the factor
6
Journal of Cognitive Neuroscience
probe condition (target, reward, control) was performed on the mean ERMF response between 180 and 280 msec. Data from right and left VF and VF probes were analyzed separately at corresponding contralateral sensor sites over the left and right hemisphere, respectively (arrows in Figure 3A). This analysis yielded a significant main effect of probe condition for both left and right VF probes (right VF probes (sensor A196): F(2, 38) = 5.2, p < .01; left VF probes (sensor A245): F(2, 38) = 4.66, p < .05). Subsequent pairwise comparisons (paired-samples t tests) showed that target probes differed significantly from control probes (right VF probes: p = .0048, left VF probes: p = .014) and from reward probes (right VF probes: p = .018, left VF probes: p = .041), whereas reward and control probes did not differ (right VF probes: p = .68, left VF probes: p = .29). Figure 3B summarizes the results of the current source density (CSD) analysis of the ERMF response to the three
Volume X, Number Y
probe types separately for probes appearing in the right (top row) and left VF (bottom row). The topographical maps show the CSD distribution (MNLS estimates, see Methods) overlaid onto a 3-D surface segmentation of the cortical gray matter layer of the MNI brain (average of 152 T1-weighted stereotaxic volumes of the ICBM project, ICBM152), which served as source compartment for the computation. CSD maxima of the three probe types appear in similar regions of the inferior occipito-temporal cortex contralateral to the VF of probe presentation. In line with what is visible in field distributions (Figure 3A), maximum CSD estimates are seen for target probes, whereas reward and control probes display comparable but smaller effect sizes. This is further illustrated by the time course of CSD estimates obtained from ROIs centered at the CSD maximum of all three probe conditions (white ellipses in Figure 3B). Starting at ∼180 msec after probe onset, the CSD of target probes (red) increases beyond that of reward (green) and control probes (black) until ∼280 msec.
Pr
oo
f
to when it was presented with the target or was absent. A rANOVA with a three-level factor reward location validates this impression by showing a significant effect, F(2, 38) = 30.7, p < .0001. Post hoc pairwise comparisons confirm that the effect is because of increased RT for the reward color presented with the distractor relative to the other conditions (reward color in target vs. in distractor: p < .0001; reward color absent vs. in distractor: p < .0001). Notably, participants were not faster for trials where the reward color was combined with the target than for reward color absent trials ( p = .112). Reward color did also influence RA in a significant way, F(2, 38) = 11.3, p < .005. Although RA was generally high, the presence of reward color caused a small decrease in performance relative to reward color absent trials. Post hoc pairwise comparisons confirm that performance on reward color absent trials was better than when the reward color was with the target ( p < .005) or with the distractor ( p < .0001). Participants were also more accurate when the reward color appeared in the target versus in the distractor, but this effect was not significant ( p = .102). MEG Responses to the Search Frames (Spheres)
Figure 4 (A, B) summarizes the RT and RA measures in Experiment 1 as a function of where the reward color was presented in the search frames (in the target, in the distractor, absent). Apparently, RT is slowed when the reward color appeared with the distractor relative
Figure 5A shows the response to the spheres of nonrewarded trials as a function of whether the reward color was presented in the nontarget VF (green traces) or not (black traces). Displayed are waveforms from a selected sensor site showing the maximum modulatory effect over
or
U nc
Figure 4. Target discrimination performance as a function of reward color location in the search frames of Experiments 1 and 2. (A) Mean RT and (B) percent correct responses for the reward color presented in the target or in the distractor or not presented at all in Experiment 1. (C) Mean RT and (D) percent correct responses of Experiment 2. Data are separately shown for the low- (low) and the high-reward condition (high). White bars index the standard error of mean.
re
ct
ed
Behavioral Performance as a Function of Reward Color Location in the Search Frames
Buschschulte et al.
7
difference reward color present minus absent in the unattended VF. Current source maxima appear in ventral lateral extrastriate visual cortex, contralateral to the side of reward color presentation. Hence, in contrast to the presentation of task-irrelevant probes where reward color had no effect on extrastriate processing, presenting the reward-relevant color in a task-irrelevant distractor during target selection in visual search produced enhanced responses. This indicates that reward relevance did in fact bias feature processing during the selection of task-relevant features when actually performing the visual search task. Experiment 2
or
re
ct
ed
Pr
oo
f
Experiment 1 failed to detect any difference in the ventral extrastriate response to color probes when they were associated with reward as compared with control probes. However, the amount of monetary reward to be gained on a given trial of Experiment 1 was moderate (5 Euro Cent), and it is possible that the absence of modulatory effects reflects the fact that reward was not of adequate significance to the observer to generate the effect of interest. Experiment 2 addressed this possibility by doubling the amount of money (10 Euro Cent) to be gained on rewarded trials. Specifically, participants performed two types of trial blocks in which the amount of reward was either as high as in Experiment 1 (lowreward blocks) or doubled (high-reward blocks). With respect to stimulation, trial structure, and experimental task, Experiment 2 was identical to Experiment 1.
U nc
Figure 5. MEG response to search frames of Experiments 1 and 2. (A) Experiment 1: ERMF waveforms elicited by nonrewarded search frames, with the reward color being present (green) or absent (black) in the distractor sphere. The waveforms are shown for selected sensors (RH/LH sensor = right/left hemisphere sensor) contralateral to the VF of distractor presentation. The topographical maps above display the corresponding CSD distributions estimated for the reward-minuscontrol difference. (B) Experiment 2: ERMF waveforms for reward color present (green) and absent in the distractor VF (black), separately shown for the low- (top) and high-reward condition (bottom). The topographical maps display the CSD distributions estimated for the reward-minus-control difference of the low-reward condition.
the hemisphere contralateral to the nontarget VF. Apparently, the reward color elicits a response enhancement starting around 200 msec after search frame onset. rANOVAs with the factor reward color (present/absent in nontarget VF) on mean amplitude measures between 200 and 280 msec confirm this observation by yielding significant effects for the right VF, F(1, 19) = 10.09, p < .005, as well as the left VF, F(1, 19) = 8.3, p < .01. The topographical maps show the results of the current source localization analysis computed for the ERMF
8
Journal of Cognitive Neuroscience
Behavioral Performance as a Function of Probe Color and Probe Location Figure 2C shows that, for all probe types and reward conditions, valid probes were associated with IOR, that is, with a slowing of RT relative to invalid probes. RT was also generally faster in high-reward blocks than in lowreward blocks. A three-way rANOVA with the factors probe validity (valid, invalid), probe condition (target, reward, control), and reward size (low, high) confirmed that there were significant main effects of probe validity, F(1, 15) = 43.2, p < .0001, and reward size, F(1, 15) = 13.8, p < .005, but no main effect of probe condition, F(2, 30) = 1.1, p < .4. There were no significant interactions (Reward Size × Probe Condition, F(2, 30) = 1.4, p < .3; Probe Validity × Probe Condition, F(2, 30) = 2.0, p < .15; and Reward Size × Probe Validity, F(1, 15) = 0.54, p < .5). Inspection of the RA in Figure 2D revealed a minimal variation because of the experimental conditions without any systematic effect of probe validity, probe condition, or reward size. This is confirmed by a rANOVA showing that neither of the respective main effects was significant (probe validity: F(1, 15) = 0.29, p < .6; probe condition: F(2, 30) = 1.03, p < .4; reward size: F(1, 15) = 0.49, p < .5). Volume X, Number Y
MEG Responses to the Color Probes
probe types of the low-reward condition at 200 msec (Figure 6A, right), as well as the high-reward condition at 200 msec (top row maps in Figure 6B) and at 230 msec (bottom row maps in Figure 5B). At 200 msec after probe onset, the response to the three probe types shows a pretty similar field distribution and amplitude pattern for the low- and high-reward condition. That is, the efflux– influx configuration (ellipses) elicited by target probes over the left lateral occipito-temporal cortex is stronger than the ones elicited by reward and control probes. Notably, the pattern changes around 220 msec after probe onset, where target and control probes show a field response of similar size while the response to reward probes is significantly reduced. Figure 7 shows the results of the source localization analysis (source density estimates, right VF probes only) performed on the mean target-minus-control difference between 190 and 220 msec of the low- (top) and highreward condition (bottom). Apparently, current source maxima (highlighted by white and green dots) appear over the left ventral-lateral occipito-temporal cortex consistent with the localization of current source maxima in Experiment 1 (cf. Figure 3B, top row). The reward-minus-control
U nc
or
re
ct
ed
Pr
oo
f
Figure 6 shows waveforms and ERMF distributions of the three different probe conditions of the low- (a) and high-reward condition (b) for probes in the right VF (qualitatively similar effects were observed for the left VF). For the low-reward condition, the ERMF response to the three probe types perfectly reproduces the pattern seen in Experiment 1. The response to target probes is increased in ventral extrastriate cortex between ∼180 and 270 msec relative to reward and control probes, with the latter being nearly indistinguishable. A similar picture is seen for the high-reward condition between 190 and 220 msec (Figure 6B). Although the size of the response to the probes is generally smaller than under low-reward conditions, target probes, again, show an increased response relative to reward and control probes, whereas the response elicited by reward probes does not differ from control probes. In a subsequent time range (∼220– 250 msec), however, the response to reward probes is reduced relative to target and control probes. This is also visible in the corresponding ERMF maps showing the distribution of the mean probe response for the three
Figure 6. MEG response to color probes of Experiment 2. Waveforms and distributions elicited by the three different probe types in the right VF under low- (A) and high-reward conditions (B). The presented waveforms were recorded at the sensor sites indicated by the small arrow in A and correspond with the sensor site shown in Figure 3A. The topographical maps on the right side show the distribution of the ERMF response at the time points indicated by the black arrowheads in the corresponding waveform displays. The black ellipses highlight the efflux–influx field configuration representing the probe-elicited response.
Buschschulte et al.
9
Pr
Effects to Color Probes outside the Visual Cortex
ed
In contrast to Experiment 1, a significant enhancement of the ERMF response to reward and target probes was found outside visual areas, which appeared as efflux– influx field configuration over left lateral-frontal (efflux, red) and central-parietal regions (influx, blue). Respective modulation was seen under high- (Figure 8A) but not under low-reward conditions (Figure 8B). Figure 8C shows that there was also no such effect in Experiment 1, where the reward magnitude was the same as the lowreward condition of Experiment 2. Notably, the response to reward probes under high-reward conditions of Experiment 2 showed its maximum at ∼170 msec after probe onset (arrowhead), that is, roughly 20 msec before the earliest modulation effects in response to target probes and 40 msec before the attenuation effect to reward probes in ventral extrastriate cortex. As visible, reward probes elicit a clear response enhancement starting approximately around 160 msec, whereas the enhancement for target probes arises later between ∼200 and 270 msec (maximum at 230 msec). Under low-reward conditions (Figure 8B), no such enhancement is present for either reward or target probes. A rANOVA with the factors probe condition (target, reward, control) and reward size (low, high) on the mean ERMF response between 160 and 180 msec yielded a significant Probe Condition × Reward Size interaction, F(2, 30) = 3.52, p < .05, validating the observation that the enhanced response to reward probes was only evident under high-reward conditions. The main effects of reward size ( p = .34) and probe condition ( p = .11) were not significant. In the later time range between 200 and 270 msec again a significant Probe Condition × Reward Size interaction, F(2, 30) = 5.0, p < .05, was seen confirming the observation that the response enhancement to target probes was only seen under high-reward conditions. The
U nc
or
re
ct
difference of the high-reward condition between 220 and 250 msec is shown on the right of Figure 7. Because the map shows the CSD distribution of the reduced response to reward probes relative to control probes, we show this map as blue scale distribution. The CSD maximum of the reward-minus-control difference locates to the ventral occipital cortex at a site more posterior than the maxima of the target-minus-control differences. For statistical validation of the ERMF effects, rANOVAs with the factors probe condition (target, reward, control) and reward size (low, high) were computed for mean responses in two time ranges between 190–220 and 220– 250 msec. In the 190–220 msec time range, a significant main effect of probe condition, F(2, 30) = 3.9, p < .05, but no interaction of Probe Condition × Reward Size, F(2, 30) = 0.69, was observed, which validates the enhanced response to target probes relative to reward and control probes under both, low- and high-reward conditions. Subsequent pairwise comparisons confirm this observation by showing a significant main effect of probe condition for the target versus control comparison, F(1, 15) = 9.45, p < .01, but no effect of the reward versus control comparison, F(1, 15) = 0.27, p = .61. There was also a main effect of reward size, F(1, 15) = 7.07, p < .05, indicating that the ERMF response to the probes was generally smaller under high- than under low-reward conditions. Between 220 and 250 msec, no effect of probe condition, F(2, 30) = 0.29, but a significant interaction of Probe Condition × Reward Size, F(2, 30) = 3.7, p < .05, appeared, which validates the attenuated response to reward probes under high- but not under low-reward conditions. Although the response to the probes was generally smaller under high- versus low-reward conditions in this time range, the main effect of reward size, F(2, 30) = 2.73, p = .14, was not significant.
oo
f
Figure 7. Results current source analysis of Experiment 2. On the left: CSD (CSD) distributions (minimum norm least squares estimates) estimated for the response difference target probes minus control probes (Target-minus-Control) of the low- (top) and high-reward condition (bottom). The small circles (white and green) highlight the CSD maxima. On the right: CSD distributions estimated for the response difference reward probes minus control probes (Reward-minus-Control). The red circle highlights the source density maximum in relation to the maxima of the other comparisons.
10
Journal of Cognitive Neuroscience
Volume X, Number Y
Pr
oo
f
ventral extrastriate visual cortex, it is reasonable to ask whether the latter is linked to activity changes in dACC. Such a direct modulatory influence of frontal lobe activity on processes of attentional selection in visual cortex has been has been repeatedly documented (Noudoost & Moore, 2011; Cohen, Heitz, Schall, & Woodman, 2009; Moore & Armstrong, 2003; Moore, Armstrong, & Fallah, 2003). To address this possibility, we analyzed the amount of ERMF attenuation as a function of the amplitude and latency variation in dACC across participants. Specifically, we assessed the correlation between peak amplitude/peak latency measures of the response (reward-minus-control ERMF difference) to reward probes in dACC and the mean response attenuation between 220 and 250 msec in ventral extrastriate cortex. This analysis revealed no correlation between amplitude measures (r = 0.213, t(16) = 0.816), but we found a significant correlation between response latency in dACC and the amplitude reduction in ventral extrastriate cortex (r = 0.426, t(16) = 1.76, p < .05). The scatterplot in Figure 9 illustrates this relation. It shows that participants with progressively shorter dACC latencies displayed an increasing effect of attenuation of the ERMF response to reward probes in extrastriate visual cortex.
ed
Behavioral Performance as a Function of Reward Color Location in the Search Frames Figure 4 (C, D) shows RT and RA measures as a function of the location of the reward color in the search array separately for the low- and high-reward condition of
re
ct
corresponding main effects were not significant (probe condition: p = .28, reward size: p = .37). The prior response enhancement to reward probes relative to target probes visible in Figure 8A is notable and warrants further validation. To this end, we computed sliding window t tests (see Methods) on the reward-minus-control and the target-minus-control differences, which revealed response onsets for reward and target probes at 155 and 190 msec, respectively. Figure 8D shows results of the source localization analysis (LORETA estimates, see Methods) computed for the reward-minus-control ERMF difference at 170 msec and overlaid onto transsections of the MNI brain. The corresponding source density maximum is located to a medial frontal cortex area of the dorsal ACC (dACC). Source waveforms (Figure 8E) of the reward-minus-control (blue trace) and the target-minus-control difference (red trace) taken from the CSD maximum (dACC) show that the modulation in response to reward probes appears earlier than the response to target probes. Furthermore, in line with the ERMF waveform effects, the response to reward probes in dACC arises prior (∼20 msec) to the activity enhancement because of target probes in ventral extrastriate cortex shown in Figure 8F (see difference between dashed lines). In contrast, the anterior cingulate response enhancement to target probes (red trace in Figure 8E) did not arise before the response in ventral extrastriate cortex (Figure 8F). Given that the response in dACC to reward probes under high-reward conditions appeared before the attenuation of the ERMF response in
U nc
or
Figure 8. MEG response to color probes outside the visual cortex (Experiments 1 and 2). ERMF waveforms elicited by the three different probe types outside the visual cortex for the (A) high-reward condition of Experiment 2, (B) the low-reward condition of Experiment 2, and (C) of Experiment 1. The small arrow in A highlights the sensor over the parietal influx maximum—the site where the shown waveforms were recorded from. The colored horizontal bars index the time range of significant amplitude differences between reward and control probes (blue) and between target and control probes (red). (D) CSD estimates (LORETA) of the reward-minus-control difference at the time point highlighted in (A; black arrowhead). (E) Normalized source waveforms measured from the source density maximum shown in D of the reward-minus-control ERMF difference (blue) and the target-minus-control difference (red). (F) Normalized source waveform corresponding with the ventral extrastriate CSD maximum of the target-minus-control difference of the high-reward condition (green circle in Figure 5).
Buschschulte et al.
11
for the reward in distractor condition relative to the reward in target, F(1, 15) = 5.5, p < .05, and the reward absent condition, F(1, 15) = 9.5, p < .01, with the small decrement for the reward in target condition versus the reward absent condition being also significant, F(1, 15) = 6.5, p < .05. MEG Responses to the Search Frames (Spheres)
U nc
or
oo
Pr
ed
re
ct
Experiment 2. Consistent with the observation in Experiment 1, RTs are slowed when presenting the reward color with the distractor relative to when reward is absent or presented with the target. In addition, RT is slightly increased when the reward feature is presented together with target relative to the reward absent condition. This pattern is seen for both the low- and high-reward condition. A two-way rANOVA with the factors reward location (in target, in distractor, absent) and reward size (low, high) validates these observations by yielding a significant main effect of reward location, F(2, 30) = 23.2, p < .0001, but no interaction of Reward Location × Reward Size, F(2, 30) = 0.49, p = .58. There was, however, a significant main effect of reward size, F(1, 15) = 12.3, p < .005, reflecting the fact that RT was generally faster on high-reward blocks. Pairwise post hoc comparisons reveal that participants were significantly slower on the reward in distractor condition relative to the reward in target, F(1, 15) = 19.2, p < .005, and reward absent condition, F(1, 15) = 27.9, p < .0001. The slight response slowing for the reward in target versus the reward absent condition was also significant, F(1, 15) = 13.6, p < .005, indicating that the presence of a reward color had generally a distracting effect, which was largest when appearing in the search distractor. This increased distracting effect of reward color was also reflected by the RA pattern, as the reward in distractor condition showed a performance decrement relative to conditions where the reward color appeared in the target or was absent. This effect again did not vary with reward size. A respective two-way rANOVA confirms this by showing a significant main effect of reward location, F(2, 30) = 7.6, p < .01, without showing a Reward Location × Reward Size interaction, F(2, 30) = 0.86, p = .43. There was also no main effect of reward size, F(1, 15) = 1.2, p = .29. Pairwise post hoc comparisons revealed that RA was reduced
f
Figure 9. Scatter diagram plotting the mean amplitude reduction between 220 and 250 msec in ventral extrastriate cortex against the peak response latency in dACC of each participant (n = 16) observed for the high-reward condition of Experiment 2. The line shows the linear regression between measures.
Figure 5B shows waveforms elicited by (nonrewarded) search frames with the reward color present (green traces) or absent (black traces) in the nontarget VF. Results are separately shown for low- and high-reward blocks. As in Experiment 1, on low-reward blocks the reward color elicited an enhanced response between 200 and 260 msec after search frame onset. The corresponding topographical map again reveals a current source maximum in ventral extrastriate cortex contralateral to the VF of reward color presentation. Most notably, on high-reward blocks this modulatory effect is abolished, with the response to the reward color being indistinguishable from the response to the control color. For statistical validation, a two-way rANOVA with the factors reward color (present/absent in nontarget VF) and reward size (low, high) was computed on mean amplitude measures between 200a and 260 msec after search frame onset. This yielded a significant main effect of reward color, F(1, 15) = 6.09, p < .05, and a significant interaction between reward color and reward size, F(1, 15) = 4.62, p < .05, but no main effect of reward size, F(1, 15) = 0.08. Subsequent pairwise comparisons revealed a significant effect of reward color for low-reward trials, F(1, 15) = 7.49, p < .05, but no such effect for high-reward trials, F(1, 15) = 0.11. In summary, after doubling the amount of money to be gained on rewarded trials, reward probes did not elicit an increased response in ventral extrastriate cortex as seen for target probes. Instead, a delayed attenuation of the response to reward probes was observed in a more posterior ventral extrastriate region. In addition, reward and target probes elicited an increased response in medialfrontal/ACC under high-reward conditions, with the effect to reward probes arising before the effect to target probes as well as before the attenuation effect in ventral extrastriate cortex. A correlation analysis revealed that the amount of delayed attenuation to reward probes in posterior extrastriate visual cortex increased with shorter latencies of the response in ACC, suggesting a functional link between the speed of reward-representation in frontal cortex and the subsequent attenuation of sensory processing in visual areas. Finally, this observation dovetails with the differential effect of reward size on the response elicited by search arrays. The presentation of a reward color in nontarget spheres led to a response enhancement under low-reward conditions as in Experiment 1. Doubling the amount of reward, however, eliminated this enhancement effect.
12
Journal of Cognitive Neuroscience
Volume X, Number Y
U nc
or
oo
Pr
ed
re
ct
The reported experiments revealed that task-irrelevant color probes drawn in a target-defining color led to activity enhancements between 180 and 280 msec in ventral extrastriate cortex contralateral to probe presentation. Probes drawn in a reward-relevant color, in contrast, did not show such response enhancement. Instead, increasing reward relevance by doubling the amount of money to be gained on rewarded trials (Experiment 2) resulted in a delayed reduction (∼220-250 msec) of the response relative to control probes. Analyzing the effect of reward color presented during the visual search task revealed that presenting the reward-relevant color in the distractor caused an activity enhancement starting around 200 msec in ventral extrastriate cortex. This enhancement was, however, abolished under high-reward conditions of Experiment 2. These observations together suggest that rewardcontingent modulatory effects in extrastriate visual cortex are under effective top–down control even when the behavioral significance of reward is separated from taskrelevant feature descriptions defining the target. Under the specific settings of the present experiments, rewardrelated biases of sensory processing in extrastriate visual cortex were eliminated or even suppressed by top–down inhibitory influences, with top–down inhibition becoming stronger with larger reward prospect. Such strong inhibitory control of distractor processing is consistent with recent evidence from ERP recordings (Sawaki & Luck, 2010, 2011). Respective studies showed that salient singletons may cause an automatic (bottom–up) “attend-to-me” signal, but that the actual consequence of attentional capture may be counteracted by active suppression. The existence of such “attend-to-me” signal was inferred from the presence of a component called distractor positivity, an ERP modulation shown to reflect the suppression of distractors (Hickey, Di Lollo, & McDonald, 2009). The general attenuation of the probe-elicited response under high-reward conditions of Experiment 2 suggests that such counteracting suppression appeared to some extent for all probe types. The delayed attenuation to reward probes under high-reward conditions—presumably corresponding with an increased distractor positivity component— may then reflect extra suppression, which was of smaller amplitude under low-reward conditions, such that a positive modulation bias, as seen for target probes, was just cancelled. Other observations in the present experiments as well as in the literature lend support to this interpretation. First, the mere presence of the reward color led to a performance decrement during target selection, indicating that the reward color had a generally distracting effect, even when combined with the target color. Second, RTs were overall faster under high- than low-reward conditions of Experiment 2. Third, under high-reward conditions an early response enhancement was seen in
dACC, which appeared before the response attenuation in ventral extrastriate cortex. A quantification of this effect revealed that the latency of the dACC response correlated with the amount of attenuation in extrastriate cortex (Figure 9)—a relation consistent with dACC being involved in the top–down (inhibitory) control of extrastriate sensory processing. In fact, fMRI research in humans revealed that error-related activity modulations in posterior medial frontal cortex correlate with the suppression of activity in visual cortex areas coding distracting features of the target input (Danielmeier, Eichele, Forstmann, Tittgemeyer, & Ullsperger, 2011). Furthermore, it has been reported that reward-dependent changes of stimulus saliency as well as the detection and interpretation of reward are mediated in ACC (Hickey et al., 2010; Hayden, Pearson, & Platt, 2009). Finally, direct top–down influences on extrastriate visual processing have been documented in the monkey using different methodologies (Noudoost & Moore, 2011; Cohen et al., 2009; Moore & Fallah, 2004; Moore & Armstrong, 2003; Moore et al., 2003). How then does the attenuation or absence of effects to task-irrelevant reward probes observed here reconcile with previous studies showing that reward-dependent biasing of sensory processing does appear in visual cortex and earlier stages even when the reward-defining feature is not task relevant (subject to discrimination), (Hickey et al., 2010; Franko et al., 2009; Kiss, Driver, & Eimer, 2009; Serences, 2008; Shuler & Bear, 2006; Ikeda & Hikosaka, 2003). Serences (2008) observed with fMRI in human observers that a history of color gratings being associated with reward that is independent of subjective valuation led to larger BOLD responses in early visual cortex areas ( V1–V4). Franko et al. (2009) recorded local field potentials from macaque V4 while the monkey passively viewed gratings of different orientations. Orientation was task-irrelevant, but the orientation consistently paired with the subsequent delivery of reward led to increased LFP responses. Although these studies show positive biasing effects of reward on visual selection for task-irrelevant reward-defining features, they differ from the present experiments in a critical way, namely that the occurrence of a reward feature was consistently associated with subsequent reward delivery or the valuation of reward. In other words, the reward feature directed attention toward the subsequent event that delivered or signaled the gain of reward without distracting performance in a negative way—a situation not giving explicit or implicit incentive to inhibit modulatory effects because of the reward feature. In contrast, in the present experiments, the presentation of reward probes was completely nonpredictive of whether the subsequent search target would be combined with a reward color. Furthermore, the present experiments were designed to dissociate top–down definitions of reward and task relevance, with the biasing effect of reward being probed by distracting events during task-irrelevant phases of the
f
DISCUSSION
Buschschulte et al.
13
ed
Pr
oo
f
by trial-by-trial adjustments, because the color–reward association as well as reward size varied constantly from trial to trial. In the present experiments, the color–reward association and reward size were both fixed within trial blocks—conditions allowing for more consistent control settings, thereby permitting participants to adopt a stronger and more specific top–down inhibitory scheme to counter the distracting effect of the reward color—hence the general effect of response attenuation. Although the behavioral performance pattern is generally consistent with the conclusion that the effects of reward color receive strong top–down inhibitory control in extrastriate cortex, there is one observation in the behavioral data that warrants further consideration. Increasing reward prospect had a generally inhibitory impact on the extrastriate modulations, reward size, however, had no corresponding effect on search performance (in Experiment 2). The presence of the reward-relevant color caused a clear decrement of performance when presented with the distractor as well as when presented with the target. However, the amount of decrement in terms of RT and accuracy was uninfluenced by reward size. Increased top–down inhibitory control under highreward conditions would predict reward color to have a less distracting effect on target selection. We think that the unchanged performance level reflects the consequence of increased top–down distractor attenuation under highreward conditions, an interpretation that can of course not be ultimately decided based on the present data. One could object that defining reward relevance independent of task relevance may have motivated participants to ignore the reward feature completely and focus exclusively on the target-defining feature. In other words, reward probes may simply not have captured attention—a situation that would preempt top–down modulatory influences to counter the distracting effect of the reward color in visual cortex areas. However, this interpretation is unlikely given that participants showed a clear performance decrement when presenting the reward color in the search distractor under low- and high-reward conditions. Furthermore, this possibility conflicts with the observation that under high-reward conditions reward probes elicited a stronger and earlier response than target probes in dACC. ACC is believed to subserve the detection and interpretation of reward and in particular to code reward-dependent changes of stimulus saliency (Hickey et al., 2010; Hayden et al., 2009). Hence, the rewardrelevant color was not only registered but also assigned a higher significance and temporal priority than the targetdefining feature. Although reward probes were subject to effective top– down inhibition, the target probes showed a significant activity enhancement relative to control probes under both low- and high-reward conditions—an observation predicted by the contingent involuntary orienting account (Remington, Folk, & McLean, 2001; Folk et al., 1992). According to this account, the attention-capturing effect
U nc
or
re
ct
experiment. These experimental conditions rendered any occurrence of the reward color a distracting event, even when the reward color was combined with the target color. In fact, the reward color impaired target discrimination performance not only when presented in the search distractor but also when presented in the target. Hence, participants likely adopted a general set toward deemphasizing reward feature selection, which is overall consistent with the absence or the suppressive nature of effects elicited by reward probes. Notable in this respect is recent research, which has documented increased responses to task-irrelevant features even after dissociating their reward association from endogenous attentional settings (Hickey et al., 2010). That is, positive neural biasing effects were found under conditions that should—according to the considerations above—deemphasize reward feature selection. For example, Hickey et al. (2010) had participants search for a shape singleton among similarly colored items and one color singleton distractor. Participants were given high or low reward upon correct target discrimination on every trial. From trial to trial, color assignment to the target and distractor changed or remained constant, such that, on a given trial, the target or the distractor was either presented in the previously rewarded or the nonrewarded color. Importantly, color was completely task irrelevant, and color discrimination was in fact counterproductive for performing the task. The critical observation was that when color distractors appeared in the color that was combined with high-reward on a previous trial, an enhanced P1 response was seen contralateral to the location of the color distractor—an enhancement not seen when the same color was associated with low reward on the previous trial. A P1 enhancement typically reflects an effect of location selection associated with a gain amplification of neural processing in retinotopically corresponding extrastriate visual areas (Hillyard et al., 1998). This suggests that reward relevance biased processing in extrastriate cortex, although color processing was generally task-irrelevant and hindering target selection (see Anderson, Laurent, & Yantis, 2011; Kristjansson, Sigurjonsdottir, & Driver, 2010, for converging observations at the behavioral level). These results run counter to the present observations. Hickey et al. (2010) observed effects of positive reward priming during target selection in search arrays containing a distractor that carried the color associated with reward on the previous trial. That is, the priming effect in the neural response (P1 enhancement) emerged with increasing reward. We likewise observed an enhanced response to the reward-relevant color distractor in visual search, but the effect was instead eliminated after increasing reward. We have no definitive explanation of the opposite response pattern, but there are significant differences in experimental design presumably causing different modes of top–down control. The strategic effects controlling the effect of reward assignment in Hickey et al. (2010) were mainly reflecting 14
Journal of Cognitive Neuroscience
Volume X, Number Y
ed
Pr
oo
f
effect appeared for either reward or target probes. Consistently, a modulation in the dACC was also not seen in Experiment 1 where the magnitude of reward delivered on rewarded trials was the same as that deliverd on lowreward trials of Experiment 2. The fact that activity enhancements in dACC were present for both reward and target probes is generally consistent with the role the dACC is supposed to play in executive control (Mansouri, Tanaka, & Buckley, 2009; Dosenbach et al., 2006; Botvinick, Cohen, & Carter, 2004; Schall, Stuphorn, & Brown, 2002). Our observation particularly aligns with the more recently advanced proposal that the dACC is a key part of a cortical system maintaining task sets (Dosenbach et al., 2006), which operates upon cues that entail increased demands on attentional control to keep focused on relevant input (Weissman, Gopalakrishnan, Hazlett, & Woldorff, 2005; Weissman, Warner, & Woldorff, 2004). Doubling the amount of monetary reward may have increased the general significance of the task- and reward-associated color, thereby rendering corresponding probes more potent in capturing attention. The activity enhancements in dACC may then serve to block the distracting effect of probes drawn in these colors to maintain the performance focus on subsequent task-relevant search frames. Conclusion
The reported experiments show that when top–down settings defining task and reward relevance are dissociated, the presentation of reward-relevant probes produced no (low reward) or an attenuated sensory response (high reward) in extrastriate visual cortex relative to control probes. Furthermore, increasing reward prospect caused reward and target probes to elicit response enhancements in dACC—a frontal cortex structure known to mediate the selection of stimuli with particular behavioral salience. Importantly, the dACC response to reward probes appeared significantly before that to target probes, with shorter response latencies to the former being followed by stronger attenuation in extrastriate cortex. Finally, an analysis of the brain response to the search frames revealed a response enhancement to the reward color when presented in the distractor—an enhancement that disappeared when increasing the reward relevance of the reward color. These observations together suggest that the selection of rewardrelevant features is under effective top–down inhibitory control in extrastriate visual cortex even when top–down definitions of reward relevance attention are separated.
U nc
or
re
ct
of an onset stimulus depends on the degree to which properties of that stimulus meet top–down defined target descriptions (but see recent debate; Theeuwes, 2010). Previous studies have documented ERP/ERMF correlates of such contingent attentional capture in visual areas. Arnott et al. (2001) observed with ERPs that attentional capture contingent on a relevant color led to an enhanced occipital N1 response contralateral to the capturing probe stimulus. Hopf et al. (2004)report that the presentation of a target-defining orientation feature at a nontarget location in visual search was associated with a retinotopically consistent enhancement of neural activity in ventral extrastriate cortex before spatial focusing onto the target. The present data show that top–down settings for a task-relevant color translate into a color-selective bias of neural processing in ventral extrastriate cortex that is attenuated but still present under high-reward conditions where task settings emphasize distractor attenuation. It should be noted that although target probes produced increased extrastriate activations, they were not associated with an IOR effect different from control and reward probes—an observation that appears to conflict with the notion that the stronger extrastriate modulation to target probes relates to attentional capture. However, there are data suggesting that the neural operations giving rise to IOR do not depend on the extrastriate processing bias for target color observed here. IOR has been suggested to be strictly location/object related, and research has typically failed to demonstrate to IOR in the color domain (Kwak & Egeth, 1992). Although Law, Pratt, and Abrams (1995) reported some effects compatible with a color-based IOR (Law et al., 1995), they were only observable when a nontarget distractor color was presented between the color cue and the color target—a situation not matching the present experimental conditions. Hence, IOR may have been a consequence of spatial orienting in response to the mere onset of the probes, without being further influenced by an additional color-selective bias. Indeed, Busse, Katzner, and Treue (2006) investigated effects of exogenous cuing from combined feature cues (color/motion direction) and observed that for long cue-target SOAs, location-valid versus −invalid cues produced the typical IOR effect, whereas the validity of motion direction cues did not give rise to any IOR. Finally, it is possible that target probes may have caused a stronger location bias with the consequence of facilitated target selection after valid target probes. However, such facilitating effect would be expected to appear for SOAs much shorter than the ones used in the present experiments (600–900 msec; Hopfinger & Mangun, 1998). Under high-reward conditions of Experiment 2, both reward probes and target probes elicited an enhanced response in dACC, with the enhancement for reward probes appearing earlier (between 160 and 200 msec) than the enhancement for target probes (between 200 and 260 msec). Under low-reward conditions, no dACC
Acknowledgments J.-M. H. and M. A. S. were supported by Grant SFB779-TPA1. Reprint requests should be sent to Jens-Max Hopf, Leibniz Institute for Neurobiology and Otto-von-Guericke-University of Magdeburg, Leipziger Strasse 44, 39120 Magdeburg, Germany, or via e-mail: [email protected].
Buschschulte et al.
15
U nc
or
oo
Pr
ed
re
ct
Anderson, B. A., Laurent, P. A., & Yantis, S. (2011). Value-driven attentional capture. Proceedings of the National Academy of Sciences, U.S.A. Ansorge, U., Kiss, M., & Eimer, M. (2009). Goal-driven attentional capture by invisible colors: Evidence from event-related potentials. Psychonomic Bulletin & Review, 16, 648–653. Arnott, S. R., Pratt, J., Shore, D. I., & Alain, C. (2001). Attentional set modulates visual areas: An event-related potential study of attentional capture. Brain Research, Cognitive Brain Research, 12, 383–395. Arsenault, J. T., Nelissen, K., Jarraya, B., & Vanduffel, W. (2013). Dopaminergic reward signals selectively decrease fMRI activity in primate visual cortex. Neuron, 77, 1174–1186. Bacon, W. F., & Egeth, H. E. (1994). Overriding stimulus-driven attentional capture. Perception & Psychophysics, 55, 485–496. Boehler, C. N., Hopf, J. M., Krebs, R. M., Stoppel, C. M., Schoenfeld, M. A., Heinze, H. J., et al. (2011). Task-loaddependent activation of dopaminergic midbrain areas in the absence of reward. Journal of Neuroscience, 31, 4955–4961. Botvinick, M. M., Cohen, J. D., & Carter, C. S. (2004). Conflict monitoring and anterior cingulate cortex: An update. Trends in Cognitive Sciences, 8, 539–546. Busse, L., Katzner, S., & Treue, S. (2006). Spatial and featurebased effects of exogenous cueing on visual motion processing. Vision Research, 46, 2019–2027. Chelazzi, L., & Desimone, R. (1994). Responses of V4 neurons during visual search. Society for Neuroscience Abstract, 20, 1054. Chelazzi, L., Duncan, J., Miller, E. K., & Desimone, R. (1998). Response of neurons in inferior temporal cortex during memory-guided visual search. Journal of Neurophysiology, 80, 2918–2940. Cohen, J. Y., Heitz, R. P., Schall, J. D., & Woodman, G. F. (2009). On the origin of event-related potentials indexing covert attentional selection during visual search. Journal of Neurophysiology, 102, 2375–2386. Danielmeier, C., Eichele, T., Forstmann, B. U., Tittgemeyer, M., & Ullsperger, M. (2011). Posterior medial frontal cortex activity predicts post-error adaptations in task-related visual and motor areas. The Journal of Neuroscience, 31, 1780–1789. Dosenbach, N. U., Visscher, K. M., Palmer, E. D., Miezin, F. M., Wenger, K. K., Kang, H. C., et al. (2006). A core system for the implementation of task sets. Neuron, 50, 799–812. Eimer, M., & Kiss, M. (2008). Involuntary attentional capture is determined by task set: Evidence from event-related brain potentials. Journal of Cognitive Neuroscience, 20, 1423–1433. Folk, C. L., Remington, R. W., & Johnston, J. C. (1992). Involuntary covert orienting is contingent on attentional control settings. Journal of Experimental Psychology: Human Perception and Performance, 18, 1030–1044. Folk, C. L., Remington, R. W., & Wright, J. H. (1994). The structure of attentional control: Contingent attentional capture by apparent motion, abrupt onset, and color. Journal of Experimental Psychology: Human Perception and Performance, 20, 317–329. Franko, E., Seitz, A. R., & Vogels, R. (2009). Dissociable neural effects of long-term stimulus-reward pairing in macaque visual cortex. Journal of Cognitive Neuroscience, 22, 1425–1439. Fukuda, K., & Vogel, E. K. (2009). Human variation in overriding attentional capture. Journal of Neuroscience, 29, 8726–8733. Gibson, B. S., & Kelsey, E. M. (1998). Stimulus-driven attentional capture is contingent on attentional set for displaywide visual features. Journal of Experimental
Psychology: Human Perception and Performance, 24, 966–706. Guthrie, D., & Buchwald, J. S. (1991). Significance testing of difference potentials. Psychophysiology, 28, 240–244. Haber, S. N., & Knutson, B. (2010). The reward circuit: Linking primate anatomy and human imaging. Neuropsychopharmacology, 35, 4–26. Hayden, B. Y., Pearson, J. M., & Platt, M. L. (2009). Fictive reward signals in the anterior cingulate cortex. Science, 324, 948–950. Heinze, H. J., Mangun, G. R., Burchert, W., Hinrichs, H., Scholz, M., Münte, T. F., et al. (1994). Combined spatial and temporal imaging of brain activity during visual selective attention in humans. Nature, 372, 543–546. Hickey, C., Chelazzi, L., & Theeuwes, J. (2010). Reward changes salience in human vision via the anterior cingulate. Journal of Neuroscience, 30, 11096–11103. Hickey, C., Di Lollo, V., & McDonald, J. J. (2009). Electrophysiological indices of target and distractor processing in visual search. Journal of Cognitive Neuroscience, 21, 760–775. Hickey, C., McDonald, J. J., & Theeuwes, J. (2006). Electrophysiological evidence of the capture of visual attention. Journal of Cognitive Neuroscience, 18, 604–613. Hickey, C., & van Zoest, W. (2012). Reward creates oculomotor salience. Current Biology, 22, R219–R220. Hikosaka, O., Bromberg-Martin, E., Hong, S., & Matsumoto, M. (2008). New insights on the subcortical representation of reward. Current Opinion in Neurobiology, 18, 203–208. Hillyard, S. A., Vogel, E. K., & Luck, S. J. (1998). Sensory gain control (amplification) as a mechanism of selective attention: Electrophysiological and neuroimaging evidence. Philosophical Transactions of the Royal Society of London, Series B, 353, 1257–1270. Hopf, J.-M., Boelmans, K., Schoenfeld, A., Luck, S. J., & Heinze, H.-J. (2004). Attention to features precedes attention to locations in visual search: Evidence from electromagnetic brain responses in humans. Journal of Neuroscience, 24, 1822–1832. Hopf, J.-M., Heinze, H. J., Schoenfeld, M. A., & Hillyard, S. A. (2009). Spatio-temporal analysis of visual attention. In M. S. Gazzaniga (Ed.), The cognitive neurosciences IV (pp. 235–250). Cambridge, MA: MIT Press. Hopf, J.-M., Luck, S. J., Girelli, M., Hagner, T., Mangun, G. R., Scheich, H., et al. (2000). Neural sources of focused attention in visual search. Cerebral Cortex, 10, 1233–1241. Hopf, J.-M., Vogel, E., Woodman, G., Heinze, H. J., & Luck, S. J. (2002). Localizing visual discrimination processes in time and space. Journal of Neurophysiology, 88, 2088–2095. Hopfinger, J. B., & Mangun, G. R. (1998). Reflexive attention modulates processing of visual stimuli in human extrastriate cortex. Psychological Science, 9, 441–447. Hopfinger, J. B., & Ries, A. J. (2005). Automatic versus contingent mechanisms of sensory-driven neural biasing and reflexive attention. Journal of Cognitive Neuroscience, 17, 1341–1352. Ikeda, T., & Hikosaka, O. (2003). Reward-dependent gain and bias of visual responses in primate superior colliculus. Neuron, 39, 693–700. Kastner, S., & Ungerleider, L. G. (2000). Mechanisms of visual attention in the human cortex. Annual Review of Neuroscience, 23, 315–341. Kiss, M., Driver, J., & Eimer, M. (2009). Reward priority of visual target singletons modulates event-related potential signatures of attentional selection. Psychological Science, 20, 245–251. Kiss, M., Jolicoeur, P., Dellʼacqua, R., & Eimer, M. (2008). Attentional capture by visual singletons is mediated by
f
REFERENCES
16
Journal of Cognitive Neuroscience
Volume X, Number Y
ed
Pr
oo
f
Motter, B. C. (1994). Neural correlates of feature selective memory and pop-out in extrastriate area V4. Journal of Neuroscience, 14, 2190–2199. Noudoost, B., & Moore, T. (2011). Control of visual cortical signals by prefrontal dopamine. Nature. Posner, M. I., & Cohen, Y. (1984). Components of visual orienting. In H. Bouma & D. Bowhuis (Eds.), Attention and performance X (pp. 531–556). Hillsdale, NJ: Erlbaum. Remington, R. W., Folk, C. L., & McLean, J. P. (2001). Contingent attentional capture or delayed allocation of attention? Perception & Psychophysics, 63, 298–307. Sawaki, R., & Luck, S. J. (2010). Capture versus suppression of attention by salient singletons: Electrophysiological evidence for an automatic attend-to-me signal. Attention, Perception & Psychophysics, 72, 1455–1470. Sawaki, R., & Luck, S. J. (2011). Active suppression of distractors that match the contents of visual working memory. Visual Cognition, 19, 956–972. Schall, J. D., Stuphorn, V., & Brown, J. W. (2002). Monitoring and control of action by the frontal lobes. Neuron, 36, 309–322. Schultz, W. (2000). Multiple reward signals in the brain. Nature Reviews Neuroscience, 1, 199–207. Serences, J. T. (2008). Value-based modulations in human visual cortex. Neuron, 60, 1169–1181. Shuler, M. G., & Bear, M. F. (2006). Reward timing in the primary visual cortex. Science, 311, 1606–1609. Theeuwes, J. (2010). Top–down and bottom–up control of visual selection. Acta Psychologica (Amsterdam), 135, 77–99. Theeuwes, J., & Godljn, R. (2001). Attentional and oculomotor capture. In C. L. Folk & B. S. Gibson (Eds.), Attraction, distraction, and action: Multiple perspectives on attentional capture (pp. 121–150). Amsterdam: Elsevier. Treue, S. (2001). Neural correlates of attention in primate visual cortex. Trends in Neurosciences, 24, 295–300. Treue, S., & Martinez Trujillo, J. C. (1999). Feature-based attention influences motion processing gain in macaque visual cortex. Nature, 399, 575–579. Weil, R. S., Furl, N., Ruff, C. C., Symmonds, M., Flandin, G., Dolan, R. J., et al. (2010). Rewarding feedback after correct visual discriminations has both general and specific influences on visual cortex. Journal of Neurophysiology, 104, 1746–1757. Weissman, D. H., Gopalakrishnan, A., Hazlett, C. J., & Woldorff, M. G. (2005). Dorsal anterior cingulate cortex resolves conflict from distracting stimuli by boosting attention toward relevant events. Cerebral Cortex, 15, 229–237. Weissman, D. H., Warner, L. M., & Woldorff, M. G. (2004). The neural mechanisms for minimizing cross-modal distraction. Journal of Neuroscience, 24, 10941–10949. Yantis, S. (1996). Attentional capture in vision. In A. Kramer, M. Coles, & G. Logan (Eds.), Converging operations in the study of selective visual attention (pp. 45–76). Washington, DC: American Psychological Association.
U nc
or
re
ct
top–down task set: New evidence from the N2pc component. Psychophysiology, 45, 1013–1024. Klein, R. M. (2000). Inhibition of return. Trends in Cognitive Sciences, 4, 138–147. Kristjansson, A., Sigurjonsdottir, O., & Driver, J. (2010). Fortune and reversals of fortune in visual search: Reward contingencies for pop-out targets affect search efficiency and target repetition effects. Attention, Perception, & Psychophysics, 72, 1229–1236. Kwak, H. W., & Egeth, H. (1992). Consequences of allocating attention to locations and to other attributes. Perception & Psychophysics, 51, 455–464. Lamy, D., Leber, A., & Egeth, H. E. (2004). Effects of task relevance and stimulus-driven salience in feature-search mode. Journal of Experimental Psychology: Human Perception and Performance, 30, 1019–1031. Law, M. B., Pratt, J., & Abrams, R. A. (1995). Color-based inhibition of return. Perception & Psychophysics, 57, 402–408. Leblanc, E., Prime, D. J., & Jolicoeur, P. (2008). Tracking the location of visuospatial attention in a contingent capture paradigm. Journal of Cognitive Neuroscience, 20, 657–671. Lee, B. B., Martin, P. R., & Valberg, A. (1988). The physiological basis of heterochromatic flicker photometry demonstrated in the ganglion cells of the macaque retina. Journal of Physiology, 404, 323–347. Lennert, T., & Martinez-Trujillo, J. (2011). Strength of response suppression to distracter stimuli determines attentionalfiltering performance in primate prefrontal neurons. Neuron, 70, 141–152. Lien, M. C., Ruthruff, E., Goodin, Z., & Remington, R. W. (2008). Contingent attentional capture by top–down control settings: Converging evidence from event-related potentials. Journal of Experimental Psychology: Human Perception and Performance, 34, 509–530. Mansouri, F. A., Tanaka, K., & Buckley, M. J. (2009). Conflictinduced behavioural adjustment: A clue to the executive functions of the prefrontal cortex. Nature Reviews Neuroscience, 10, 141–152. Martinez, A., Anllo-Vento, L., Sereno, M. I., Frank, L. R., Buxton, R. B., Dubowitz, D. J., et al. (1999). Involvement of striate and extrastriate visual cortical areas in spatial attention. Nature Neuroscience, 2, 364–369. Maunsell, J. H., & Treue, S. (2006). Feature-based attention in visual cortex. Trends in Neurosciences, 29, 317–322. Maylor, E. A., & Hockey, R. (1985). Inhibitory component of externally controlled covert orienting in visual space. Journal of Experimental Psychology: Human Perception and Performance, 11, 777–787. Moore, T., & Armstrong, K. M. (2003). Selective gating of visual signals by microstimulation of frontal cortex. Nature, 421, 370–373. Moore, T., Armstrong, K. M., & Fallah, M. (2003). Visuomotor origins of covert spatial attention. Neuron, 40, 671–683. Moore, T., & Fallah, M. (2004). Microstimulation of the frontal eye field and its effects on covert spatial attention. Journal of Neurophysiology, 91, 152–162.
Buschschulte et al.
17
Abstract
oo
f
not the reward-relevant color produced a selective enhancement of the neuromagnetic response between 180 and 280 msec in ventral extrastriate visual cortex. Increasing reward prospect caused a delayed attenuation (220–250 msec) of the response to reward probes, which followed a prior (160–180 msec) response enhancement in dorsal ACC. Notably, shorter latency responses in dorsal ACC were associated with stronger attenuation in extrastriate visual cortex. Finally, an analysis of the brain response to the search frames revealed that the presence of the reward-relevant color in search distractors elicited an enhanced response that was abolished after increasing reward size. The present data together indicate that when top–down definitions of reward relevance and attention are separated, the behavioral significance of reward-associated features is still rapidly coded in higher-level cortex areas, thereby commanding effective top– down inhibitory control to counter a selection bias for those features in extrastriate visual cortex. ■
INTRODUCTION
or
re
ct
ed
selection in extrastriate cortex. Features signaling reward seem to produce a similar bias, but how modulatory effects due to reward and attention relate to each other is largely unexplored. To address this issue, it is critical to separate top–down settings defining reward relevance from those defining attention. To this end, we used a visual search paradigm in which the targetʼs definition (attention to color) was dissociated from reward relevance by delivering monetary reward on search frames where a certain task-irrelevant color was combined with the targetdefining color to form the target object. We assessed the state of neural biasing for the attended and reward-relevant color by analyzing the neuromagnetic brain response to asynchronously presented irrelevant distractor probes drawn in the target-defining color, the reward-relevant color, and a completely irrelevant color as a reference. We observed that for the prospect of moderate rewards, the target-defining color but
Pr
■ Attention to task-relevant features lead to a biasing of sensory
U nc
Considerable experimental evidence suggests that the sudden onset of an unexpected visual stimulus can draw attention in an automatic bottom–up fashion—an effect referred to as attentional capture (Theeuwes, 2010; Theeuwes & Godljn, 2001; Yantis, 1996). Such capture, however, turns out to be not entirely automatic, as it has been shown to depend on the degree to which features of the capturing stimulus match the set of top– down defined descriptions of the task-relevant input (contingent involuntary orienting; Eimer & Kiss, 2008; Lamy, Leber, & Egeth, 2004; Gibson & Kelsey, 1998; Bacon & Egeth, 1994; Folk, Remington, & Wright, 1994; Folk, Remington, & Johnston, 1992). Top–down descriptions may entail a selection bias for one or a set of simple target-defining features (e.g., color, orientation) or refer to a selection bias contingent on the more general behavioral relevance of a target object, for example, its emotional significance, or association with reward.
1
Otto-von-Guericke University Magdeburg, 2Ghent University, Leibniz Institute for Neurobiology, Magdeburg, Germany
3
© Massachusetts Institute of Technology
Attentional mechanisms of top–down biasing of feature selection in visual cortex have been extensively investigated, with data from different methodologies indicating that attention exerts its effect by modulating the gain of neural processing in sensory visual areas (Hopf, Heinze, Schoenfeld, & Hillyard, 2009; Maunsell & Treue, 2006; Treue, 2001; Kastner & Ungerleider, 2000; Hillyard, Vogel, & Luck, 1998). This may come about either by increasing the gain of neural representations that match the attended feature (Maunsell & Treue, 2006; Treue & Martinez Trujillo, 1999; Motter, 1994) or by attenuating neural activity of irrelevant feature representations (Lennert & Martinez-Trujillo, 2011; Chelazzi, Duncan, Miller, & Desimone, 1998; Chelazzi & Desimone, 1994). In addition, more recent research has mounted evidence suggesting that value-based factors like the reward relevance of stimuli may have an analogous modulatory influence on feature-selective processing in visual cortex (Hickey, Chelazzi, & Theeuwes, 2010; Franko, Seitz, & Vogels, 2009; Serences, 2008; Shuler & Bear, 2006) and recent behavioral and neurophysiological observations imply an influence on even earlier processing stages (Hickey & van Zoest, 2012; Ikeda & Hikosaka, 2003). Journal of Cognitive Neuroscience X:Y, pp. 1–17 doi:10.1162/jocn_a_00539
Pr
oo
f
ing from extrastriate visual cortex areas (Hopf, Boelmans, Schoenfeld, Luck, & Heinze, 2004; Hopf, Vogel, Woodman, Heinze, & Luck, 2002; Hopf et al., 2000; Martinez et al., 1999; Heinze et al., 1994). Moreover, several studies have reported an N2pc component in response to attentional capture contingent on distractors sharing task-relevant features with the target. The N2pc is known to index attentional focusing (Ansorge, Kiss, & Eimer, 2009; Eimer & Kiss, 2008; Kiss, Jolicoeur, Dellʼacqua, & Eimer, 2008; Leblanc et al., 2008) and has been shown to arise from source activity in ventral extrastriate cortex based on event-related magnetic field (ERMF) recordings (Hopf et al., 2000, 2002, 2004). Here we will focus on the ERMF response elicited by task-irrelevant color probes and analyze whether modulations in visual extrastriate visual cortex differs when comparing target and reward probes relative to control probes. We will further investigate whether above reviewed signatures of attentional capture can be identified in the ERMF response to search frames where the rewardrelevant color is presented in the distractor.
METHODS
Participants (Experiment 1)
ed
Twenty participants (mean age = 26.1 years, range = 21– 32, 14 women) took part in Experiment 1. The experiment was undertaken with the understanding and written consent of the participants. All participants were students of the Otto-von-Guericke University Magdeburg, were right-handed, had normal color vision, and had normal or corrected-to-normal visual acuity. Participants were paid for participation in addition to the payoff that could be gained on rewarded trials. All experiments were approved by the ethics review board of the Otto-vonGuericke University Magdeburg.
U nc
or
re
ct
These observations raise the question how reward relevance imposes its effect on visual sensory processing. Despite the currently proliferating knowledge about brain circuits and functions mediating reward relevance in general (Haber & Knutson, 2010; Hikosaka, Bromberg-Martin, Hong, & Matsumoto, 2008; Schultz, 2000), the mechanism by which reward ultimately influences visual sensory selection and whether they dissociate from modulatory effects because of attention is not well understood. In fact, Maunsell (2004) emphasized the importance to clarify whether neural correlates of attention and reward dissociate in their impact on early visual processing because in animal experiments attention is typically tied to the application of reward and therefore inherently at risk to be confounded by reward (and vice versa). A few recent studies have explicitly aimed at a dissociation of reward effects from attention in humans and monkeys, with mixed results ranging from modest (Weil et al., 2010) to clear-cut effects attributable to reward proper (Arsenault, Nelissen, Jarraya, & Vanduffel, 2013; Hickey et al., 2010). Importantly, in humans, top–down settings for task-relevant stimulus properties can be set by plain verbal instruction without resorting to the direct application of reward, which makes it easier to separate taskand reward-relevant top–down settings (Boehler et al., 2011). With the following two experiments, we aim at such paradigmatic separation between reward relevance and feature-based attention. Specifically, we ask how top– down settings that define reward relevance (a rewarddefining feature) but not task relevance (a target-defining feature) influence sensory processing in extrastriate cortex. To this end, we will use a simple visual search task requiring participants to search for a target defined by color (top–down settings for a task-relevant feature), whereas the combined occurrence of a specific taskirrelevant color with the target color will be associated with reward (top–down settings for reward relevance). To assess the degree of passive sensory biasing for the selection of those colors in visual cortex, we analyzed the neuromagnetic brain response elicited by the sudden onset of a task-irrelevant color probe at performanceirrelevant time points between search frame presentations. The probe will be drawn either in the target color, the reward color, or a control color. Additionally, we will also assess the biasing effect of reward color during target selection by analyzing the brain response elicited by the search frames as a function of the presence/absence of the reward color in the search distractor. Electrophysiological signatures of attentional capture have been documented to appear as amplitude modulations of early sensory visual components (Fukuda & Vogel, 2009; Eimer & Kiss, 2008; Leblanc, Prime, & Jolicoeur, 2008; Lien, Ruthruff, Goodin, & Remington, 2008; Hickey, McDonald, & Theeuwes, 2006; Hopfinger & Ries, 2005; Arnott, Pratt, Shore, & Alain, 2001; Hopfinger & Mangun, 1998), with the underlying current source activity originat2
Journal of Cognitive Neuroscience
Stimuli and Procedure (Experiment 1) Search arrays, probe stimuli, and the stimulation protocol are illustrated in Figure 1. Each search array contained a fixation cross and two double-colored 3-D spheres (diameter 2.7° visual angle), one in the left and one in the right visual field (VF; 5° lateral, 2.7° downward from fixation), presented on a homogenous gray background (luminance: 24 cd/m2). Each sphere was composed of two colors taken from a set of five colors (green, red, blue, yellow, gray), which were randomly assigned to the left and right half of the spheres. On each trial, one of the four half-spheres was drawn in the target color (green in Figure 1), with three of the other four colors being assigned to the remaining half-spheres in the following way. Although all half-spheres of a given search array were always drawn in a different color, on one third of the trials the target color was combined (randomly) with one of the three control colors (red in top search frame of Figure 1) with the distractor sphere containing Volume X, Number Y
U nc
or
re
ct
Figure 1. Structure of an experimental trial block of Experiment 1. An instruction frame informed participants about the target-defining color (here green) and the color associated with reward (here blue) when combined with the target color. On each search frame (the frames containing two double-colored spheres), participants had to discriminate the orientation of the convexity of the target half-sphere (left, right). Participants had to ignore the colored squares (color probes) flashed randomly between the search frames. The probes appeared randomly in the left and right VF and could be drawn in the target-defining color (target probe), the color associated with reward (reward probe), or one of the irrelevant colors (control probe (here gray, yellow, and red).
ed
Pr
oo
f
definition would clearly undermine the present aim to dissociate task and reward relevance. The participantsʼ task was to report whether the target color appeared on the left or right side of the target sphere with a two-alternative button press of the right hand (index finger: left, middle finger: right) as fast and accurate as possible. Search arrays were presented for 700 msec. On rewarded trials, a feedback frame was shown for 400 msec, 1000–1500 msec after search frame offset, informing the participant whether reward was gained or not. The ISI between an unrewarded search frame or the feedback frame and the next stimulus onset was randomized between 1000 and 1500 msec (rectangular distribution). Randomly on 50% of the trials, this subsequent stimulus was a single square (the probe, 1.8° × 1.8°, ISI 1000– 1500 msec) that was flashed for 50 msec in the left or right VF (centered at the location of the sphere). After such trials, the next search array appeared 600–900 msec after the probe. A probe was never directly followed by another probe, that is, a probe appeared on average on every second trial of an experimental block. Probe color and the VF of probe presentation were randomly assigned, such that the probe appeared as often in the VF of the subsequent search target or in the opposite VF. Experimental blocks were designed with target and reward color counterbalanced. That is, each color served as the target in four blocks, with every other color serving as reward color on one of the four blocks. Throughout the entire experimental session, each color served four times as reward color. During one experimental block (duration = 5 min, one break), 72 search frames and 36 probes were presented, yielding a session total of 60 trials per probe condition (target, reward, control) and VF (left, right) for each participant. The luminance of the different probe colors was matched using heterochromatic flicker photometry (Lee, Martin, & Valberg, 1988) in three selected participants. Respective luminance values were used for all participants that participated in Experiments 1 and 2.
a combination of the remaining control colors (yellow, gray). On another one third of the trials, the target color was combined with one of the control colors (yellow in middle search frame of Figure 1), with one half-sphere of the distractor sphere being drawn in the reward color (blue). On the last one third of the trials, the target color was combined with the reward color (green and blue in bottom search frame of Figure 1), and the distractor sphere contained a combination of two distractor colors (red and yellow). Importantly, reward was delivered only on the one third of the trials when the reward color was combined with the target color. When the reward color was combined with a control color in the VF opposite to the target (middle search frame in Figure 1), no reward was given. Note that the number of rewarded trials was set to a comparably low proportion of one third of the trials. This was done to de-emphasize the tendency that with an increasing proportion of rewarded trials participants might take the specific combination of target and reward color as defining the target. Such combined target
Reward Schedule The payoff for a rewarded trial was 5 Euro Cent, which amounted to a total bonus payment of approximately 10.70–12.00A per participant. Reward could be earned upon correct target discrimination on trials where the target sphere also contained a reward color. Participants (Experiment 2) Sixteen participants (mean age = 26.5 years, range = 23– 41 years, 14 women) participated in Experiment 2. All participants were students of the Otto-von-Guericke University Magdeburg, were right-handed, had normal color vision, and had normal or corrected-to-normal visual acuity. Participants were paid for participation in addition to the payoff that could be gained on rewarded trials. All Buschschulte et al.
3
experiments were approved by the ethics committee of the Otto-von-Guericke University Magdeburg.
U nc
Data Recording and Analysis
or
oo
Pr
ed
re
ct
Stimuli and procedure were similar to Experiment 1, except for the following modifications. (1) As in Experiment 1, each color served as the target color with all other colors serving one time as the reward color, however, under two different reward conditions (high-reward blocks: 10 Euro Cent, low-reward blocks: 5 Euro Cent per rewarded trial). (2) As this doubled the number of experimental blocks per set and participant in comparison with Experiment 1, each participant performed two experimental sessions on different days (maximum delay between sessions 40 days). During the first session, half of the participants performed high-reward blocks on uneven block numbers and low-reward blocks on even blocks numbers. The other half of the participants was assigned to high- and low-reward blocks in the reverse order. During the second session, the assignment of reward magnitude to block number was reversed in each participant, such that—with both sessions taken together—participants performed every target–reward combination under high- and low-reward conditions. Again, across both sessions, each participant performed 60 trials per probe condition (target, reward, control), VF (left, right), and reward magnitude (high, low). Participants were informed at the start of each block about whether high or low reward will be delivered. The range of payoff gained from rewarded trials amounted to 15.85– 18.00A per participant.
f
Stimuli and Procedure (Experiment 2)
generated by five coils fixated on defined positions on the EEG cap. Primary MEG data analysis included off-line artifact rejection applied to epochs of interest with peak-to-peak amplitudes exceeding a threshold of 2–3 pT. For each participant, epochs of interest ranging 200 msec before (baseline) to 750 msec after probe onset were extracted according to the relevant experimental conditions. Epochs containing peak-to-peak EOG amplitudes exceeding 150 μV were considered to contain eye motion artifacts and were rejected from further data analysis. This resulted in an average rejection rate of 18.0% for Experiment 1 and 13.2% for Experiment 2, with no significant difference between experimental conditions. For Experiment 1, averages were computed for the three different probe types (target, reward, control probes) in the left and right VF, thereby collapsing data across the different colors. In addition, selective averages were computed for ERMF responses elicited by the search frames (spheres) as a function of whether the reward color was present or absent in the distractor opposite to the search target. For this analysis, only nonrewarded trials were considered. For Experiment 2, the same averages were computed, but separately for high- and low-reward blocks, and after collapsing the data from the two experimental sessions of a given participant. For subsequent data analysis and visualization, the ERP software ERPSS (Event-Related Potential Laboratory, University of California, San Diego, La Jolla, CA) was used. The statistical validation of waveform differences was performed using a repeated-measures ANOVA (rANOVA) approach. If necessary, violations of data sphericity were corrected (Greenhouse–Geisser epsilon). Respective data will be reported with the original degrees of freedom, but with an adjusted level of significance ( p value). The statistical validation of onset latency differences was performed using a sliding t-test approach applied to subsequent time samples (window width of 20 msec; Guthrie & Buchwald, 1991). The first sample showing a significant difference in a sequence of at least three subsequently significant time samples was taken to mark the onset latency.
The magnetoencephalography (MEG) was recorded in a magnetically shielded chamber with a BTI Magnes 3600 WH whole-head magnetometer system (4D Neuroimaging, San Diego, CA) with 248 sensors. For controlling eye movements, the EOG was recorded with a horizontal and vertical montage using bipolar electrode placements at the temples (horizontal) as well as a unipolar electrode below the left eye (vertical). Impedances were kept below 5 kΩ (EEG cap and Abralyt light gel, Easycap, Herrsching, Germany). EOG signals were amplified with an EPA-6 amplifier (Sensorium, Inc., Charlotte, VT). The MEG and EOG signals were band-pass filtered (0.01–50 Hz) and digitized with a sampling rate of 254.31 Hz. In addition to recording the EOG, fixation performance was continuously monitored with a custommade zoom lens infrared camera system. To coregister anatomical and functional data, anatomical landmarks (nasion and left and right preauricular points) were digitized with a Polhemus 3Space Fastrak system (Polhemus Inc., Colchester, VT). These landmarks were then brought into reference with magnetic marker fields 4
Journal of Cognitive Neuroscience
Source Localization Analysis Current source localization was performed by computing minimum norm least squares estimates (MNLS, sLORETA) as implemented in the CURRY V6.0 neuroimaging software (Compumedics Neuroscan, Charlotte, NC). Estimates for the grand averaged data were computed by using realistic anatomical models of the source compartment (3-D surface segmentation of the cortical gray matter space) and the volume conductor (3-D surface segmentation of the cerebrospinal fluid space) of the standard MNI brain (Montreal Neurological Institute). Before averaging waveforms across participants, sensor data of each participant were coregistered to a reference subject based on the participantʼs individual landmarks. Volume X, Number Y
The grand averaged sensor data were then brought into reference with the anatomical data of the MNI brain.
RESULTS Experiment 1 Behavioral Performance as a Function of Probe Color and Probe Location
U nc
or
re
ct
ed
Pr
oo
f
Participants were to ignore the probes, so that no behavioral performance measure directly reflecting probe color processing can be obtained. However, the effect of probe color can be analyzed indirectly by determining the amount of exogenous spatial cuing (for the performance
on a subsequent search frame) created by the probes. That is, the amount of cuing produced by the different probe types will provide an index of the degree to which spatial attention was captured. To assess the effect of exogenous cuing, target discrimination performance was analyzed as a function of whether a given probe appeared on the side of the subsequently appearing target sphere (valid probes) versus on the opposite side (invalid probe). Given that the probe target SOA varied between 600 and 900 msec, the expected cuing effect is one of inhibition of return (IOR; Klein, 2000; Maylor & Hockey, 1985; Posner & Cohen, 1984), that is, a relative slowing of the response for valid probes, which was in fact observed for all three probe conditions. As shown in Figure 2A, valid probes led to an average of 11 msec slower response than invalid probes. Importantly, IOR did not vary with probe type indicating that spatial attention was captured by the probe to the same extent in all three probe conditions. A two-way rANOVA with the factors probe condition (target, reward, control) and probe validity (valid, invalid) confirmed this impression. There was a significant main effect of probe validity, F(1, 19) = 19.9, p < .0001, but no Probe Validity × Probe Condition interaction, F(2, 38) = 0.56. RTs were slightly slower for targets following reward probes versus target or control probes, but the main effect of probe condition did not reach significance, F(2, 38) = 2.76, p = .09. Figure 2B summarizes response accuracy (RA; % correct responses). Consistent with IOR, accuracy was generally lower for valid than for invalid trials, with this effect not being different for the three probe conditions. A respective rANOVA yielded a significant main effect of probe validity, F(1, 19) = 4.87, p < .05, but no interaction of Probe Validity × Probe Condition, F(2, 38) = 0.03. Again, there was a trend toward a main effect of probe condition, F(2, 38) = 2.83, p = .08, reflecting that performance accuracy was slightly reduced after reward probes relative to target and control probes. It should be noted that—although not significant—the general pattern of responses hints at a possible effect of reward on performance in form of an increased IOR effect. However, given that the overall IOR effect is already small and not significant, strong interpretations may not be warranted.
MEG Responses to the Color Probes
Figure 2. Target discrimination performance as a function of probelocation in Experiments 1 and 2. (A) Mean RT and (B) percent correct values for targets following color probes in the same VF as the probe (valid) or in the opposite VF (invalid) in Experiment 1. Data are shown separately for probes drawn in the target color (target), the reward color (reward), and the control color (control). (C) Mean RT and (D) percent correct responses of Experiment 2. Data are shown separately for the different probe types as well as the low- (low) and the highreward condition (high). White bars index the standard error of mean.
Figure 3A shows ERMF waveforms and distributions elicited by the target probes, reward probes, and control probes at representative selected sensor sites (arrows) over the left and right occipito-temporal cortex contralateral to the VF of probe presentation. Shown are waveforms and field distributions separately for right (top row) and left VF probes (bottom row). As visible, target probes (red traces) led to a response enhancement relative to control probes (black traces) between ∼180 and 280 msec after probe onset. No such enhancement Buschschulte et al.
5
f oo Pr ed ct re
U nc
or
Figure 3. MEG response to color probes of Experiment 1. (A) ERMF waveforms and distributions elicited by the different probe types in the right (top row) and left VF (bottom row). The shown waveforms were recorded at the sensor sites indicated by the small arrows. The black ellipses highlight the efflux–influx field configuration representing the response elicited by the probe. (B) Source waveforms and CSD distributions estimated for the data shown in A. Source waveforms represent the time course of source density estimates in the cortical regions centered at the source density maximum highlighted by white the small circles.
appeared for reward probes (green traces), for which the response remained almost indistinguishable from the response to control probes. This difference in response strength is further illustrated in the topographical maps (right) showing the average ERMF response between 180 and 280 msec for the three probe conditions. The response in visual cortex to the probe appears as efflux–influx configuration (encircled by black ellipses) over the lateral occipito-temporal cortex contralateral to the VF of probe presentation. Note that the efflux and influx component are shown in red and blue, respectively. Both together form the magnetic field effect that is generated by source activity (see Figure 3B) located under the transition zone (asterisks) between the efflux and influx component. The field effect is stronger for target probes than reward and control probes, with the latter two showing effects of comparable size. For statistical validation, a three-way rANOVA with the factor
6
Journal of Cognitive Neuroscience
probe condition (target, reward, control) was performed on the mean ERMF response between 180 and 280 msec. Data from right and left VF and VF probes were analyzed separately at corresponding contralateral sensor sites over the left and right hemisphere, respectively (arrows in Figure 3A). This analysis yielded a significant main effect of probe condition for both left and right VF probes (right VF probes (sensor A196): F(2, 38) = 5.2, p < .01; left VF probes (sensor A245): F(2, 38) = 4.66, p < .05). Subsequent pairwise comparisons (paired-samples t tests) showed that target probes differed significantly from control probes (right VF probes: p = .0048, left VF probes: p = .014) and from reward probes (right VF probes: p = .018, left VF probes: p = .041), whereas reward and control probes did not differ (right VF probes: p = .68, left VF probes: p = .29). Figure 3B summarizes the results of the current source density (CSD) analysis of the ERMF response to the three
Volume X, Number Y
probe types separately for probes appearing in the right (top row) and left VF (bottom row). The topographical maps show the CSD distribution (MNLS estimates, see Methods) overlaid onto a 3-D surface segmentation of the cortical gray matter layer of the MNI brain (average of 152 T1-weighted stereotaxic volumes of the ICBM project, ICBM152), which served as source compartment for the computation. CSD maxima of the three probe types appear in similar regions of the inferior occipito-temporal cortex contralateral to the VF of probe presentation. In line with what is visible in field distributions (Figure 3A), maximum CSD estimates are seen for target probes, whereas reward and control probes display comparable but smaller effect sizes. This is further illustrated by the time course of CSD estimates obtained from ROIs centered at the CSD maximum of all three probe conditions (white ellipses in Figure 3B). Starting at ∼180 msec after probe onset, the CSD of target probes (red) increases beyond that of reward (green) and control probes (black) until ∼280 msec.
Pr
oo
f
to when it was presented with the target or was absent. A rANOVA with a three-level factor reward location validates this impression by showing a significant effect, F(2, 38) = 30.7, p < .0001. Post hoc pairwise comparisons confirm that the effect is because of increased RT for the reward color presented with the distractor relative to the other conditions (reward color in target vs. in distractor: p < .0001; reward color absent vs. in distractor: p < .0001). Notably, participants were not faster for trials where the reward color was combined with the target than for reward color absent trials ( p = .112). Reward color did also influence RA in a significant way, F(2, 38) = 11.3, p < .005. Although RA was generally high, the presence of reward color caused a small decrease in performance relative to reward color absent trials. Post hoc pairwise comparisons confirm that performance on reward color absent trials was better than when the reward color was with the target ( p < .005) or with the distractor ( p < .0001). Participants were also more accurate when the reward color appeared in the target versus in the distractor, but this effect was not significant ( p = .102). MEG Responses to the Search Frames (Spheres)
Figure 4 (A, B) summarizes the RT and RA measures in Experiment 1 as a function of where the reward color was presented in the search frames (in the target, in the distractor, absent). Apparently, RT is slowed when the reward color appeared with the distractor relative
Figure 5A shows the response to the spheres of nonrewarded trials as a function of whether the reward color was presented in the nontarget VF (green traces) or not (black traces). Displayed are waveforms from a selected sensor site showing the maximum modulatory effect over
or
U nc
Figure 4. Target discrimination performance as a function of reward color location in the search frames of Experiments 1 and 2. (A) Mean RT and (B) percent correct responses for the reward color presented in the target or in the distractor or not presented at all in Experiment 1. (C) Mean RT and (D) percent correct responses of Experiment 2. Data are separately shown for the low- (low) and the high-reward condition (high). White bars index the standard error of mean.
re
ct
ed
Behavioral Performance as a Function of Reward Color Location in the Search Frames
Buschschulte et al.
7
difference reward color present minus absent in the unattended VF. Current source maxima appear in ventral lateral extrastriate visual cortex, contralateral to the side of reward color presentation. Hence, in contrast to the presentation of task-irrelevant probes where reward color had no effect on extrastriate processing, presenting the reward-relevant color in a task-irrelevant distractor during target selection in visual search produced enhanced responses. This indicates that reward relevance did in fact bias feature processing during the selection of task-relevant features when actually performing the visual search task. Experiment 2
or
re
ct
ed
Pr
oo
f
Experiment 1 failed to detect any difference in the ventral extrastriate response to color probes when they were associated with reward as compared with control probes. However, the amount of monetary reward to be gained on a given trial of Experiment 1 was moderate (5 Euro Cent), and it is possible that the absence of modulatory effects reflects the fact that reward was not of adequate significance to the observer to generate the effect of interest. Experiment 2 addressed this possibility by doubling the amount of money (10 Euro Cent) to be gained on rewarded trials. Specifically, participants performed two types of trial blocks in which the amount of reward was either as high as in Experiment 1 (lowreward blocks) or doubled (high-reward blocks). With respect to stimulation, trial structure, and experimental task, Experiment 2 was identical to Experiment 1.
U nc
Figure 5. MEG response to search frames of Experiments 1 and 2. (A) Experiment 1: ERMF waveforms elicited by nonrewarded search frames, with the reward color being present (green) or absent (black) in the distractor sphere. The waveforms are shown for selected sensors (RH/LH sensor = right/left hemisphere sensor) contralateral to the VF of distractor presentation. The topographical maps above display the corresponding CSD distributions estimated for the reward-minuscontrol difference. (B) Experiment 2: ERMF waveforms for reward color present (green) and absent in the distractor VF (black), separately shown for the low- (top) and high-reward condition (bottom). The topographical maps display the CSD distributions estimated for the reward-minus-control difference of the low-reward condition.
the hemisphere contralateral to the nontarget VF. Apparently, the reward color elicits a response enhancement starting around 200 msec after search frame onset. rANOVAs with the factor reward color (present/absent in nontarget VF) on mean amplitude measures between 200 and 280 msec confirm this observation by yielding significant effects for the right VF, F(1, 19) = 10.09, p < .005, as well as the left VF, F(1, 19) = 8.3, p < .01. The topographical maps show the results of the current source localization analysis computed for the ERMF
8
Journal of Cognitive Neuroscience
Behavioral Performance as a Function of Probe Color and Probe Location Figure 2C shows that, for all probe types and reward conditions, valid probes were associated with IOR, that is, with a slowing of RT relative to invalid probes. RT was also generally faster in high-reward blocks than in lowreward blocks. A three-way rANOVA with the factors probe validity (valid, invalid), probe condition (target, reward, control), and reward size (low, high) confirmed that there were significant main effects of probe validity, F(1, 15) = 43.2, p < .0001, and reward size, F(1, 15) = 13.8, p < .005, but no main effect of probe condition, F(2, 30) = 1.1, p < .4. There were no significant interactions (Reward Size × Probe Condition, F(2, 30) = 1.4, p < .3; Probe Validity × Probe Condition, F(2, 30) = 2.0, p < .15; and Reward Size × Probe Validity, F(1, 15) = 0.54, p < .5). Inspection of the RA in Figure 2D revealed a minimal variation because of the experimental conditions without any systematic effect of probe validity, probe condition, or reward size. This is confirmed by a rANOVA showing that neither of the respective main effects was significant (probe validity: F(1, 15) = 0.29, p < .6; probe condition: F(2, 30) = 1.03, p < .4; reward size: F(1, 15) = 0.49, p < .5). Volume X, Number Y
MEG Responses to the Color Probes
probe types of the low-reward condition at 200 msec (Figure 6A, right), as well as the high-reward condition at 200 msec (top row maps in Figure 6B) and at 230 msec (bottom row maps in Figure 5B). At 200 msec after probe onset, the response to the three probe types shows a pretty similar field distribution and amplitude pattern for the low- and high-reward condition. That is, the efflux– influx configuration (ellipses) elicited by target probes over the left lateral occipito-temporal cortex is stronger than the ones elicited by reward and control probes. Notably, the pattern changes around 220 msec after probe onset, where target and control probes show a field response of similar size while the response to reward probes is significantly reduced. Figure 7 shows the results of the source localization analysis (source density estimates, right VF probes only) performed on the mean target-minus-control difference between 190 and 220 msec of the low- (top) and highreward condition (bottom). Apparently, current source maxima (highlighted by white and green dots) appear over the left ventral-lateral occipito-temporal cortex consistent with the localization of current source maxima in Experiment 1 (cf. Figure 3B, top row). The reward-minus-control
U nc
or
re
ct
ed
Pr
oo
f
Figure 6 shows waveforms and ERMF distributions of the three different probe conditions of the low- (a) and high-reward condition (b) for probes in the right VF (qualitatively similar effects were observed for the left VF). For the low-reward condition, the ERMF response to the three probe types perfectly reproduces the pattern seen in Experiment 1. The response to target probes is increased in ventral extrastriate cortex between ∼180 and 270 msec relative to reward and control probes, with the latter being nearly indistinguishable. A similar picture is seen for the high-reward condition between 190 and 220 msec (Figure 6B). Although the size of the response to the probes is generally smaller than under low-reward conditions, target probes, again, show an increased response relative to reward and control probes, whereas the response elicited by reward probes does not differ from control probes. In a subsequent time range (∼220– 250 msec), however, the response to reward probes is reduced relative to target and control probes. This is also visible in the corresponding ERMF maps showing the distribution of the mean probe response for the three
Figure 6. MEG response to color probes of Experiment 2. Waveforms and distributions elicited by the three different probe types in the right VF under low- (A) and high-reward conditions (B). The presented waveforms were recorded at the sensor sites indicated by the small arrow in A and correspond with the sensor site shown in Figure 3A. The topographical maps on the right side show the distribution of the ERMF response at the time points indicated by the black arrowheads in the corresponding waveform displays. The black ellipses highlight the efflux–influx field configuration representing the probe-elicited response.
Buschschulte et al.
9
Pr
Effects to Color Probes outside the Visual Cortex
ed
In contrast to Experiment 1, a significant enhancement of the ERMF response to reward and target probes was found outside visual areas, which appeared as efflux– influx field configuration over left lateral-frontal (efflux, red) and central-parietal regions (influx, blue). Respective modulation was seen under high- (Figure 8A) but not under low-reward conditions (Figure 8B). Figure 8C shows that there was also no such effect in Experiment 1, where the reward magnitude was the same as the lowreward condition of Experiment 2. Notably, the response to reward probes under high-reward conditions of Experiment 2 showed its maximum at ∼170 msec after probe onset (arrowhead), that is, roughly 20 msec before the earliest modulation effects in response to target probes and 40 msec before the attenuation effect to reward probes in ventral extrastriate cortex. As visible, reward probes elicit a clear response enhancement starting approximately around 160 msec, whereas the enhancement for target probes arises later between ∼200 and 270 msec (maximum at 230 msec). Under low-reward conditions (Figure 8B), no such enhancement is present for either reward or target probes. A rANOVA with the factors probe condition (target, reward, control) and reward size (low, high) on the mean ERMF response between 160 and 180 msec yielded a significant Probe Condition × Reward Size interaction, F(2, 30) = 3.52, p < .05, validating the observation that the enhanced response to reward probes was only evident under high-reward conditions. The main effects of reward size ( p = .34) and probe condition ( p = .11) were not significant. In the later time range between 200 and 270 msec again a significant Probe Condition × Reward Size interaction, F(2, 30) = 5.0, p < .05, was seen confirming the observation that the response enhancement to target probes was only seen under high-reward conditions. The
U nc
or
re
ct
difference of the high-reward condition between 220 and 250 msec is shown on the right of Figure 7. Because the map shows the CSD distribution of the reduced response to reward probes relative to control probes, we show this map as blue scale distribution. The CSD maximum of the reward-minus-control difference locates to the ventral occipital cortex at a site more posterior than the maxima of the target-minus-control differences. For statistical validation of the ERMF effects, rANOVAs with the factors probe condition (target, reward, control) and reward size (low, high) were computed for mean responses in two time ranges between 190–220 and 220– 250 msec. In the 190–220 msec time range, a significant main effect of probe condition, F(2, 30) = 3.9, p < .05, but no interaction of Probe Condition × Reward Size, F(2, 30) = 0.69, was observed, which validates the enhanced response to target probes relative to reward and control probes under both, low- and high-reward conditions. Subsequent pairwise comparisons confirm this observation by showing a significant main effect of probe condition for the target versus control comparison, F(1, 15) = 9.45, p < .01, but no effect of the reward versus control comparison, F(1, 15) = 0.27, p = .61. There was also a main effect of reward size, F(1, 15) = 7.07, p < .05, indicating that the ERMF response to the probes was generally smaller under high- than under low-reward conditions. Between 220 and 250 msec, no effect of probe condition, F(2, 30) = 0.29, but a significant interaction of Probe Condition × Reward Size, F(2, 30) = 3.7, p < .05, appeared, which validates the attenuated response to reward probes under high- but not under low-reward conditions. Although the response to the probes was generally smaller under high- versus low-reward conditions in this time range, the main effect of reward size, F(2, 30) = 2.73, p = .14, was not significant.
oo
f
Figure 7. Results current source analysis of Experiment 2. On the left: CSD (CSD) distributions (minimum norm least squares estimates) estimated for the response difference target probes minus control probes (Target-minus-Control) of the low- (top) and high-reward condition (bottom). The small circles (white and green) highlight the CSD maxima. On the right: CSD distributions estimated for the response difference reward probes minus control probes (Reward-minus-Control). The red circle highlights the source density maximum in relation to the maxima of the other comparisons.
10
Journal of Cognitive Neuroscience
Volume X, Number Y
Pr
oo
f
ventral extrastriate visual cortex, it is reasonable to ask whether the latter is linked to activity changes in dACC. Such a direct modulatory influence of frontal lobe activity on processes of attentional selection in visual cortex has been has been repeatedly documented (Noudoost & Moore, 2011; Cohen, Heitz, Schall, & Woodman, 2009; Moore & Armstrong, 2003; Moore, Armstrong, & Fallah, 2003). To address this possibility, we analyzed the amount of ERMF attenuation as a function of the amplitude and latency variation in dACC across participants. Specifically, we assessed the correlation between peak amplitude/peak latency measures of the response (reward-minus-control ERMF difference) to reward probes in dACC and the mean response attenuation between 220 and 250 msec in ventral extrastriate cortex. This analysis revealed no correlation between amplitude measures (r = 0.213, t(16) = 0.816), but we found a significant correlation between response latency in dACC and the amplitude reduction in ventral extrastriate cortex (r = 0.426, t(16) = 1.76, p < .05). The scatterplot in Figure 9 illustrates this relation. It shows that participants with progressively shorter dACC latencies displayed an increasing effect of attenuation of the ERMF response to reward probes in extrastriate visual cortex.
ed
Behavioral Performance as a Function of Reward Color Location in the Search Frames Figure 4 (C, D) shows RT and RA measures as a function of the location of the reward color in the search array separately for the low- and high-reward condition of
re
ct
corresponding main effects were not significant (probe condition: p = .28, reward size: p = .37). The prior response enhancement to reward probes relative to target probes visible in Figure 8A is notable and warrants further validation. To this end, we computed sliding window t tests (see Methods) on the reward-minus-control and the target-minus-control differences, which revealed response onsets for reward and target probes at 155 and 190 msec, respectively. Figure 8D shows results of the source localization analysis (LORETA estimates, see Methods) computed for the reward-minus-control ERMF difference at 170 msec and overlaid onto transsections of the MNI brain. The corresponding source density maximum is located to a medial frontal cortex area of the dorsal ACC (dACC). Source waveforms (Figure 8E) of the reward-minus-control (blue trace) and the target-minus-control difference (red trace) taken from the CSD maximum (dACC) show that the modulation in response to reward probes appears earlier than the response to target probes. Furthermore, in line with the ERMF waveform effects, the response to reward probes in dACC arises prior (∼20 msec) to the activity enhancement because of target probes in ventral extrastriate cortex shown in Figure 8F (see difference between dashed lines). In contrast, the anterior cingulate response enhancement to target probes (red trace in Figure 8E) did not arise before the response in ventral extrastriate cortex (Figure 8F). Given that the response in dACC to reward probes under high-reward conditions appeared before the attenuation of the ERMF response in
U nc
or
Figure 8. MEG response to color probes outside the visual cortex (Experiments 1 and 2). ERMF waveforms elicited by the three different probe types outside the visual cortex for the (A) high-reward condition of Experiment 2, (B) the low-reward condition of Experiment 2, and (C) of Experiment 1. The small arrow in A highlights the sensor over the parietal influx maximum—the site where the shown waveforms were recorded from. The colored horizontal bars index the time range of significant amplitude differences between reward and control probes (blue) and between target and control probes (red). (D) CSD estimates (LORETA) of the reward-minus-control difference at the time point highlighted in (A; black arrowhead). (E) Normalized source waveforms measured from the source density maximum shown in D of the reward-minus-control ERMF difference (blue) and the target-minus-control difference (red). (F) Normalized source waveform corresponding with the ventral extrastriate CSD maximum of the target-minus-control difference of the high-reward condition (green circle in Figure 5).
Buschschulte et al.
11
for the reward in distractor condition relative to the reward in target, F(1, 15) = 5.5, p < .05, and the reward absent condition, F(1, 15) = 9.5, p < .01, with the small decrement for the reward in target condition versus the reward absent condition being also significant, F(1, 15) = 6.5, p < .05. MEG Responses to the Search Frames (Spheres)
U nc
or
oo
Pr
ed
re
ct
Experiment 2. Consistent with the observation in Experiment 1, RTs are slowed when presenting the reward color with the distractor relative to when reward is absent or presented with the target. In addition, RT is slightly increased when the reward feature is presented together with target relative to the reward absent condition. This pattern is seen for both the low- and high-reward condition. A two-way rANOVA with the factors reward location (in target, in distractor, absent) and reward size (low, high) validates these observations by yielding a significant main effect of reward location, F(2, 30) = 23.2, p < .0001, but no interaction of Reward Location × Reward Size, F(2, 30) = 0.49, p = .58. There was, however, a significant main effect of reward size, F(1, 15) = 12.3, p < .005, reflecting the fact that RT was generally faster on high-reward blocks. Pairwise post hoc comparisons reveal that participants were significantly slower on the reward in distractor condition relative to the reward in target, F(1, 15) = 19.2, p < .005, and reward absent condition, F(1, 15) = 27.9, p < .0001. The slight response slowing for the reward in target versus the reward absent condition was also significant, F(1, 15) = 13.6, p < .005, indicating that the presence of a reward color had generally a distracting effect, which was largest when appearing in the search distractor. This increased distracting effect of reward color was also reflected by the RA pattern, as the reward in distractor condition showed a performance decrement relative to conditions where the reward color appeared in the target or was absent. This effect again did not vary with reward size. A respective two-way rANOVA confirms this by showing a significant main effect of reward location, F(2, 30) = 7.6, p < .01, without showing a Reward Location × Reward Size interaction, F(2, 30) = 0.86, p = .43. There was also no main effect of reward size, F(1, 15) = 1.2, p = .29. Pairwise post hoc comparisons revealed that RA was reduced
f
Figure 9. Scatter diagram plotting the mean amplitude reduction between 220 and 250 msec in ventral extrastriate cortex against the peak response latency in dACC of each participant (n = 16) observed for the high-reward condition of Experiment 2. The line shows the linear regression between measures.
Figure 5B shows waveforms elicited by (nonrewarded) search frames with the reward color present (green traces) or absent (black traces) in the nontarget VF. Results are separately shown for low- and high-reward blocks. As in Experiment 1, on low-reward blocks the reward color elicited an enhanced response between 200 and 260 msec after search frame onset. The corresponding topographical map again reveals a current source maximum in ventral extrastriate cortex contralateral to the VF of reward color presentation. Most notably, on high-reward blocks this modulatory effect is abolished, with the response to the reward color being indistinguishable from the response to the control color. For statistical validation, a two-way rANOVA with the factors reward color (present/absent in nontarget VF) and reward size (low, high) was computed on mean amplitude measures between 200a and 260 msec after search frame onset. This yielded a significant main effect of reward color, F(1, 15) = 6.09, p < .05, and a significant interaction between reward color and reward size, F(1, 15) = 4.62, p < .05, but no main effect of reward size, F(1, 15) = 0.08. Subsequent pairwise comparisons revealed a significant effect of reward color for low-reward trials, F(1, 15) = 7.49, p < .05, but no such effect for high-reward trials, F(1, 15) = 0.11. In summary, after doubling the amount of money to be gained on rewarded trials, reward probes did not elicit an increased response in ventral extrastriate cortex as seen for target probes. Instead, a delayed attenuation of the response to reward probes was observed in a more posterior ventral extrastriate region. In addition, reward and target probes elicited an increased response in medialfrontal/ACC under high-reward conditions, with the effect to reward probes arising before the effect to target probes as well as before the attenuation effect in ventral extrastriate cortex. A correlation analysis revealed that the amount of delayed attenuation to reward probes in posterior extrastriate visual cortex increased with shorter latencies of the response in ACC, suggesting a functional link between the speed of reward-representation in frontal cortex and the subsequent attenuation of sensory processing in visual areas. Finally, this observation dovetails with the differential effect of reward size on the response elicited by search arrays. The presentation of a reward color in nontarget spheres led to a response enhancement under low-reward conditions as in Experiment 1. Doubling the amount of reward, however, eliminated this enhancement effect.
12
Journal of Cognitive Neuroscience
Volume X, Number Y
U nc
or
oo
Pr
ed
re
ct
The reported experiments revealed that task-irrelevant color probes drawn in a target-defining color led to activity enhancements between 180 and 280 msec in ventral extrastriate cortex contralateral to probe presentation. Probes drawn in a reward-relevant color, in contrast, did not show such response enhancement. Instead, increasing reward relevance by doubling the amount of money to be gained on rewarded trials (Experiment 2) resulted in a delayed reduction (∼220-250 msec) of the response relative to control probes. Analyzing the effect of reward color presented during the visual search task revealed that presenting the reward-relevant color in the distractor caused an activity enhancement starting around 200 msec in ventral extrastriate cortex. This enhancement was, however, abolished under high-reward conditions of Experiment 2. These observations together suggest that rewardcontingent modulatory effects in extrastriate visual cortex are under effective top–down control even when the behavioral significance of reward is separated from taskrelevant feature descriptions defining the target. Under the specific settings of the present experiments, rewardrelated biases of sensory processing in extrastriate visual cortex were eliminated or even suppressed by top–down inhibitory influences, with top–down inhibition becoming stronger with larger reward prospect. Such strong inhibitory control of distractor processing is consistent with recent evidence from ERP recordings (Sawaki & Luck, 2010, 2011). Respective studies showed that salient singletons may cause an automatic (bottom–up) “attend-to-me” signal, but that the actual consequence of attentional capture may be counteracted by active suppression. The existence of such “attend-to-me” signal was inferred from the presence of a component called distractor positivity, an ERP modulation shown to reflect the suppression of distractors (Hickey, Di Lollo, & McDonald, 2009). The general attenuation of the probe-elicited response under high-reward conditions of Experiment 2 suggests that such counteracting suppression appeared to some extent for all probe types. The delayed attenuation to reward probes under high-reward conditions—presumably corresponding with an increased distractor positivity component— may then reflect extra suppression, which was of smaller amplitude under low-reward conditions, such that a positive modulation bias, as seen for target probes, was just cancelled. Other observations in the present experiments as well as in the literature lend support to this interpretation. First, the mere presence of the reward color led to a performance decrement during target selection, indicating that the reward color had a generally distracting effect, even when combined with the target color. Second, RTs were overall faster under high- than low-reward conditions of Experiment 2. Third, under high-reward conditions an early response enhancement was seen in
dACC, which appeared before the response attenuation in ventral extrastriate cortex. A quantification of this effect revealed that the latency of the dACC response correlated with the amount of attenuation in extrastriate cortex (Figure 9)—a relation consistent with dACC being involved in the top–down (inhibitory) control of extrastriate sensory processing. In fact, fMRI research in humans revealed that error-related activity modulations in posterior medial frontal cortex correlate with the suppression of activity in visual cortex areas coding distracting features of the target input (Danielmeier, Eichele, Forstmann, Tittgemeyer, & Ullsperger, 2011). Furthermore, it has been reported that reward-dependent changes of stimulus saliency as well as the detection and interpretation of reward are mediated in ACC (Hickey et al., 2010; Hayden, Pearson, & Platt, 2009). Finally, direct top–down influences on extrastriate visual processing have been documented in the monkey using different methodologies (Noudoost & Moore, 2011; Cohen et al., 2009; Moore & Fallah, 2004; Moore & Armstrong, 2003; Moore et al., 2003). How then does the attenuation or absence of effects to task-irrelevant reward probes observed here reconcile with previous studies showing that reward-dependent biasing of sensory processing does appear in visual cortex and earlier stages even when the reward-defining feature is not task relevant (subject to discrimination), (Hickey et al., 2010; Franko et al., 2009; Kiss, Driver, & Eimer, 2009; Serences, 2008; Shuler & Bear, 2006; Ikeda & Hikosaka, 2003). Serences (2008) observed with fMRI in human observers that a history of color gratings being associated with reward that is independent of subjective valuation led to larger BOLD responses in early visual cortex areas ( V1–V4). Franko et al. (2009) recorded local field potentials from macaque V4 while the monkey passively viewed gratings of different orientations. Orientation was task-irrelevant, but the orientation consistently paired with the subsequent delivery of reward led to increased LFP responses. Although these studies show positive biasing effects of reward on visual selection for task-irrelevant reward-defining features, they differ from the present experiments in a critical way, namely that the occurrence of a reward feature was consistently associated with subsequent reward delivery or the valuation of reward. In other words, the reward feature directed attention toward the subsequent event that delivered or signaled the gain of reward without distracting performance in a negative way—a situation not giving explicit or implicit incentive to inhibit modulatory effects because of the reward feature. In contrast, in the present experiments, the presentation of reward probes was completely nonpredictive of whether the subsequent search target would be combined with a reward color. Furthermore, the present experiments were designed to dissociate top–down definitions of reward and task relevance, with the biasing effect of reward being probed by distracting events during task-irrelevant phases of the
f
DISCUSSION
Buschschulte et al.
13
ed
Pr
oo
f
by trial-by-trial adjustments, because the color–reward association as well as reward size varied constantly from trial to trial. In the present experiments, the color–reward association and reward size were both fixed within trial blocks—conditions allowing for more consistent control settings, thereby permitting participants to adopt a stronger and more specific top–down inhibitory scheme to counter the distracting effect of the reward color—hence the general effect of response attenuation. Although the behavioral performance pattern is generally consistent with the conclusion that the effects of reward color receive strong top–down inhibitory control in extrastriate cortex, there is one observation in the behavioral data that warrants further consideration. Increasing reward prospect had a generally inhibitory impact on the extrastriate modulations, reward size, however, had no corresponding effect on search performance (in Experiment 2). The presence of the reward-relevant color caused a clear decrement of performance when presented with the distractor as well as when presented with the target. However, the amount of decrement in terms of RT and accuracy was uninfluenced by reward size. Increased top–down inhibitory control under highreward conditions would predict reward color to have a less distracting effect on target selection. We think that the unchanged performance level reflects the consequence of increased top–down distractor attenuation under highreward conditions, an interpretation that can of course not be ultimately decided based on the present data. One could object that defining reward relevance independent of task relevance may have motivated participants to ignore the reward feature completely and focus exclusively on the target-defining feature. In other words, reward probes may simply not have captured attention—a situation that would preempt top–down modulatory influences to counter the distracting effect of the reward color in visual cortex areas. However, this interpretation is unlikely given that participants showed a clear performance decrement when presenting the reward color in the search distractor under low- and high-reward conditions. Furthermore, this possibility conflicts with the observation that under high-reward conditions reward probes elicited a stronger and earlier response than target probes in dACC. ACC is believed to subserve the detection and interpretation of reward and in particular to code reward-dependent changes of stimulus saliency (Hickey et al., 2010; Hayden et al., 2009). Hence, the rewardrelevant color was not only registered but also assigned a higher significance and temporal priority than the targetdefining feature. Although reward probes were subject to effective top– down inhibition, the target probes showed a significant activity enhancement relative to control probes under both low- and high-reward conditions—an observation predicted by the contingent involuntary orienting account (Remington, Folk, & McLean, 2001; Folk et al., 1992). According to this account, the attention-capturing effect
U nc
or
re
ct
experiment. These experimental conditions rendered any occurrence of the reward color a distracting event, even when the reward color was combined with the target color. In fact, the reward color impaired target discrimination performance not only when presented in the search distractor but also when presented in the target. Hence, participants likely adopted a general set toward deemphasizing reward feature selection, which is overall consistent with the absence or the suppressive nature of effects elicited by reward probes. Notable in this respect is recent research, which has documented increased responses to task-irrelevant features even after dissociating their reward association from endogenous attentional settings (Hickey et al., 2010). That is, positive neural biasing effects were found under conditions that should—according to the considerations above—deemphasize reward feature selection. For example, Hickey et al. (2010) had participants search for a shape singleton among similarly colored items and one color singleton distractor. Participants were given high or low reward upon correct target discrimination on every trial. From trial to trial, color assignment to the target and distractor changed or remained constant, such that, on a given trial, the target or the distractor was either presented in the previously rewarded or the nonrewarded color. Importantly, color was completely task irrelevant, and color discrimination was in fact counterproductive for performing the task. The critical observation was that when color distractors appeared in the color that was combined with high-reward on a previous trial, an enhanced P1 response was seen contralateral to the location of the color distractor—an enhancement not seen when the same color was associated with low reward on the previous trial. A P1 enhancement typically reflects an effect of location selection associated with a gain amplification of neural processing in retinotopically corresponding extrastriate visual areas (Hillyard et al., 1998). This suggests that reward relevance biased processing in extrastriate cortex, although color processing was generally task-irrelevant and hindering target selection (see Anderson, Laurent, & Yantis, 2011; Kristjansson, Sigurjonsdottir, & Driver, 2010, for converging observations at the behavioral level). These results run counter to the present observations. Hickey et al. (2010) observed effects of positive reward priming during target selection in search arrays containing a distractor that carried the color associated with reward on the previous trial. That is, the priming effect in the neural response (P1 enhancement) emerged with increasing reward. We likewise observed an enhanced response to the reward-relevant color distractor in visual search, but the effect was instead eliminated after increasing reward. We have no definitive explanation of the opposite response pattern, but there are significant differences in experimental design presumably causing different modes of top–down control. The strategic effects controlling the effect of reward assignment in Hickey et al. (2010) were mainly reflecting 14
Journal of Cognitive Neuroscience
Volume X, Number Y
ed
Pr
oo
f
effect appeared for either reward or target probes. Consistently, a modulation in the dACC was also not seen in Experiment 1 where the magnitude of reward delivered on rewarded trials was the same as that deliverd on lowreward trials of Experiment 2. The fact that activity enhancements in dACC were present for both reward and target probes is generally consistent with the role the dACC is supposed to play in executive control (Mansouri, Tanaka, & Buckley, 2009; Dosenbach et al., 2006; Botvinick, Cohen, & Carter, 2004; Schall, Stuphorn, & Brown, 2002). Our observation particularly aligns with the more recently advanced proposal that the dACC is a key part of a cortical system maintaining task sets (Dosenbach et al., 2006), which operates upon cues that entail increased demands on attentional control to keep focused on relevant input (Weissman, Gopalakrishnan, Hazlett, & Woldorff, 2005; Weissman, Warner, & Woldorff, 2004). Doubling the amount of monetary reward may have increased the general significance of the task- and reward-associated color, thereby rendering corresponding probes more potent in capturing attention. The activity enhancements in dACC may then serve to block the distracting effect of probes drawn in these colors to maintain the performance focus on subsequent task-relevant search frames. Conclusion
The reported experiments show that when top–down settings defining task and reward relevance are dissociated, the presentation of reward-relevant probes produced no (low reward) or an attenuated sensory response (high reward) in extrastriate visual cortex relative to control probes. Furthermore, increasing reward prospect caused reward and target probes to elicit response enhancements in dACC—a frontal cortex structure known to mediate the selection of stimuli with particular behavioral salience. Importantly, the dACC response to reward probes appeared significantly before that to target probes, with shorter response latencies to the former being followed by stronger attenuation in extrastriate cortex. Finally, an analysis of the brain response to the search frames revealed a response enhancement to the reward color when presented in the distractor—an enhancement that disappeared when increasing the reward relevance of the reward color. These observations together suggest that the selection of rewardrelevant features is under effective top–down inhibitory control in extrastriate visual cortex even when top–down definitions of reward relevance attention are separated.
U nc
or
re
ct
of an onset stimulus depends on the degree to which properties of that stimulus meet top–down defined target descriptions (but see recent debate; Theeuwes, 2010). Previous studies have documented ERP/ERMF correlates of such contingent attentional capture in visual areas. Arnott et al. (2001) observed with ERPs that attentional capture contingent on a relevant color led to an enhanced occipital N1 response contralateral to the capturing probe stimulus. Hopf et al. (2004)report that the presentation of a target-defining orientation feature at a nontarget location in visual search was associated with a retinotopically consistent enhancement of neural activity in ventral extrastriate cortex before spatial focusing onto the target. The present data show that top–down settings for a task-relevant color translate into a color-selective bias of neural processing in ventral extrastriate cortex that is attenuated but still present under high-reward conditions where task settings emphasize distractor attenuation. It should be noted that although target probes produced increased extrastriate activations, they were not associated with an IOR effect different from control and reward probes—an observation that appears to conflict with the notion that the stronger extrastriate modulation to target probes relates to attentional capture. However, there are data suggesting that the neural operations giving rise to IOR do not depend on the extrastriate processing bias for target color observed here. IOR has been suggested to be strictly location/object related, and research has typically failed to demonstrate to IOR in the color domain (Kwak & Egeth, 1992). Although Law, Pratt, and Abrams (1995) reported some effects compatible with a color-based IOR (Law et al., 1995), they were only observable when a nontarget distractor color was presented between the color cue and the color target—a situation not matching the present experimental conditions. Hence, IOR may have been a consequence of spatial orienting in response to the mere onset of the probes, without being further influenced by an additional color-selective bias. Indeed, Busse, Katzner, and Treue (2006) investigated effects of exogenous cuing from combined feature cues (color/motion direction) and observed that for long cue-target SOAs, location-valid versus −invalid cues produced the typical IOR effect, whereas the validity of motion direction cues did not give rise to any IOR. Finally, it is possible that target probes may have caused a stronger location bias with the consequence of facilitated target selection after valid target probes. However, such facilitating effect would be expected to appear for SOAs much shorter than the ones used in the present experiments (600–900 msec; Hopfinger & Mangun, 1998). Under high-reward conditions of Experiment 2, both reward probes and target probes elicited an enhanced response in dACC, with the enhancement for reward probes appearing earlier (between 160 and 200 msec) than the enhancement for target probes (between 200 and 260 msec). Under low-reward conditions, no dACC
Acknowledgments J.-M. H. and M. A. S. were supported by Grant SFB779-TPA1. Reprint requests should be sent to Jens-Max Hopf, Leibniz Institute for Neurobiology and Otto-von-Guericke-University of Magdeburg, Leipziger Strasse 44, 39120 Magdeburg, Germany, or via e-mail: [email protected].
Buschschulte et al.
15
U nc
or
oo
Pr
ed
re
ct
Anderson, B. A., Laurent, P. A., & Yantis, S. (2011). Value-driven attentional capture. Proceedings of the National Academy of Sciences, U.S.A. Ansorge, U., Kiss, M., & Eimer, M. (2009). Goal-driven attentional capture by invisible colors: Evidence from event-related potentials. Psychonomic Bulletin & Review, 16, 648–653. Arnott, S. R., Pratt, J., Shore, D. I., & Alain, C. (2001). Attentional set modulates visual areas: An event-related potential study of attentional capture. Brain Research, Cognitive Brain Research, 12, 383–395. Arsenault, J. T., Nelissen, K., Jarraya, B., & Vanduffel, W. (2013). Dopaminergic reward signals selectively decrease fMRI activity in primate visual cortex. Neuron, 77, 1174–1186. Bacon, W. F., & Egeth, H. E. (1994). Overriding stimulus-driven attentional capture. Perception & Psychophysics, 55, 485–496. Boehler, C. N., Hopf, J. M., Krebs, R. M., Stoppel, C. M., Schoenfeld, M. A., Heinze, H. J., et al. (2011). Task-loaddependent activation of dopaminergic midbrain areas in the absence of reward. Journal of Neuroscience, 31, 4955–4961. Botvinick, M. M., Cohen, J. D., & Carter, C. S. (2004). Conflict monitoring and anterior cingulate cortex: An update. Trends in Cognitive Sciences, 8, 539–546. Busse, L., Katzner, S., & Treue, S. (2006). Spatial and featurebased effects of exogenous cueing on visual motion processing. Vision Research, 46, 2019–2027. Chelazzi, L., & Desimone, R. (1994). Responses of V4 neurons during visual search. Society for Neuroscience Abstract, 20, 1054. Chelazzi, L., Duncan, J., Miller, E. K., & Desimone, R. (1998). Response of neurons in inferior temporal cortex during memory-guided visual search. Journal of Neurophysiology, 80, 2918–2940. Cohen, J. Y., Heitz, R. P., Schall, J. D., & Woodman, G. F. (2009). On the origin of event-related potentials indexing covert attentional selection during visual search. Journal of Neurophysiology, 102, 2375–2386. Danielmeier, C., Eichele, T., Forstmann, B. U., Tittgemeyer, M., & Ullsperger, M. (2011). Posterior medial frontal cortex activity predicts post-error adaptations in task-related visual and motor areas. The Journal of Neuroscience, 31, 1780–1789. Dosenbach, N. U., Visscher, K. M., Palmer, E. D., Miezin, F. M., Wenger, K. K., Kang, H. C., et al. (2006). A core system for the implementation of task sets. Neuron, 50, 799–812. Eimer, M., & Kiss, M. (2008). Involuntary attentional capture is determined by task set: Evidence from event-related brain potentials. Journal of Cognitive Neuroscience, 20, 1423–1433. Folk, C. L., Remington, R. W., & Johnston, J. C. (1992). Involuntary covert orienting is contingent on attentional control settings. Journal of Experimental Psychology: Human Perception and Performance, 18, 1030–1044. Folk, C. L., Remington, R. W., & Wright, J. H. (1994). The structure of attentional control: Contingent attentional capture by apparent motion, abrupt onset, and color. Journal of Experimental Psychology: Human Perception and Performance, 20, 317–329. Franko, E., Seitz, A. R., & Vogels, R. (2009). Dissociable neural effects of long-term stimulus-reward pairing in macaque visual cortex. Journal of Cognitive Neuroscience, 22, 1425–1439. Fukuda, K., & Vogel, E. K. (2009). Human variation in overriding attentional capture. Journal of Neuroscience, 29, 8726–8733. Gibson, B. S., & Kelsey, E. M. (1998). Stimulus-driven attentional capture is contingent on attentional set for displaywide visual features. Journal of Experimental
Psychology: Human Perception and Performance, 24, 966–706. Guthrie, D., & Buchwald, J. S. (1991). Significance testing of difference potentials. Psychophysiology, 28, 240–244. Haber, S. N., & Knutson, B. (2010). The reward circuit: Linking primate anatomy and human imaging. Neuropsychopharmacology, 35, 4–26. Hayden, B. Y., Pearson, J. M., & Platt, M. L. (2009). Fictive reward signals in the anterior cingulate cortex. Science, 324, 948–950. Heinze, H. J., Mangun, G. R., Burchert, W., Hinrichs, H., Scholz, M., Münte, T. F., et al. (1994). Combined spatial and temporal imaging of brain activity during visual selective attention in humans. Nature, 372, 543–546. Hickey, C., Chelazzi, L., & Theeuwes, J. (2010). Reward changes salience in human vision via the anterior cingulate. Journal of Neuroscience, 30, 11096–11103. Hickey, C., Di Lollo, V., & McDonald, J. J. (2009). Electrophysiological indices of target and distractor processing in visual search. Journal of Cognitive Neuroscience, 21, 760–775. Hickey, C., McDonald, J. J., & Theeuwes, J. (2006). Electrophysiological evidence of the capture of visual attention. Journal of Cognitive Neuroscience, 18, 604–613. Hickey, C., & van Zoest, W. (2012). Reward creates oculomotor salience. Current Biology, 22, R219–R220. Hikosaka, O., Bromberg-Martin, E., Hong, S., & Matsumoto, M. (2008). New insights on the subcortical representation of reward. Current Opinion in Neurobiology, 18, 203–208. Hillyard, S. A., Vogel, E. K., & Luck, S. J. (1998). Sensory gain control (amplification) as a mechanism of selective attention: Electrophysiological and neuroimaging evidence. Philosophical Transactions of the Royal Society of London, Series B, 353, 1257–1270. Hopf, J.-M., Boelmans, K., Schoenfeld, A., Luck, S. J., & Heinze, H.-J. (2004). Attention to features precedes attention to locations in visual search: Evidence from electromagnetic brain responses in humans. Journal of Neuroscience, 24, 1822–1832. Hopf, J.-M., Heinze, H. J., Schoenfeld, M. A., & Hillyard, S. A. (2009). Spatio-temporal analysis of visual attention. In M. S. Gazzaniga (Ed.), The cognitive neurosciences IV (pp. 235–250). Cambridge, MA: MIT Press. Hopf, J.-M., Luck, S. J., Girelli, M., Hagner, T., Mangun, G. R., Scheich, H., et al. (2000). Neural sources of focused attention in visual search. Cerebral Cortex, 10, 1233–1241. Hopf, J.-M., Vogel, E., Woodman, G., Heinze, H. J., & Luck, S. J. (2002). Localizing visual discrimination processes in time and space. Journal of Neurophysiology, 88, 2088–2095. Hopfinger, J. B., & Mangun, G. R. (1998). Reflexive attention modulates processing of visual stimuli in human extrastriate cortex. Psychological Science, 9, 441–447. Hopfinger, J. B., & Ries, A. J. (2005). Automatic versus contingent mechanisms of sensory-driven neural biasing and reflexive attention. Journal of Cognitive Neuroscience, 17, 1341–1352. Ikeda, T., & Hikosaka, O. (2003). Reward-dependent gain and bias of visual responses in primate superior colliculus. Neuron, 39, 693–700. Kastner, S., & Ungerleider, L. G. (2000). Mechanisms of visual attention in the human cortex. Annual Review of Neuroscience, 23, 315–341. Kiss, M., Driver, J., & Eimer, M. (2009). Reward priority of visual target singletons modulates event-related potential signatures of attentional selection. Psychological Science, 20, 245–251. Kiss, M., Jolicoeur, P., Dellʼacqua, R., & Eimer, M. (2008). Attentional capture by visual singletons is mediated by
f
REFERENCES
16
Journal of Cognitive Neuroscience
Volume X, Number Y
ed
Pr
oo
f
Motter, B. C. (1994). Neural correlates of feature selective memory and pop-out in extrastriate area V4. Journal of Neuroscience, 14, 2190–2199. Noudoost, B., & Moore, T. (2011). Control of visual cortical signals by prefrontal dopamine. Nature. Posner, M. I., & Cohen, Y. (1984). Components of visual orienting. In H. Bouma & D. Bowhuis (Eds.), Attention and performance X (pp. 531–556). Hillsdale, NJ: Erlbaum. Remington, R. W., Folk, C. L., & McLean, J. P. (2001). Contingent attentional capture or delayed allocation of attention? Perception & Psychophysics, 63, 298–307. Sawaki, R., & Luck, S. J. (2010). Capture versus suppression of attention by salient singletons: Electrophysiological evidence for an automatic attend-to-me signal. Attention, Perception & Psychophysics, 72, 1455–1470. Sawaki, R., & Luck, S. J. (2011). Active suppression of distractors that match the contents of visual working memory. Visual Cognition, 19, 956–972. Schall, J. D., Stuphorn, V., & Brown, J. W. (2002). Monitoring and control of action by the frontal lobes. Neuron, 36, 309–322. Schultz, W. (2000). Multiple reward signals in the brain. Nature Reviews Neuroscience, 1, 199–207. Serences, J. T. (2008). Value-based modulations in human visual cortex. Neuron, 60, 1169–1181. Shuler, M. G., & Bear, M. F. (2006). Reward timing in the primary visual cortex. Science, 311, 1606–1609. Theeuwes, J. (2010). Top–down and bottom–up control of visual selection. Acta Psychologica (Amsterdam), 135, 77–99. Theeuwes, J., & Godljn, R. (2001). Attentional and oculomotor capture. In C. L. Folk & B. S. Gibson (Eds.), Attraction, distraction, and action: Multiple perspectives on attentional capture (pp. 121–150). Amsterdam: Elsevier. Treue, S. (2001). Neural correlates of attention in primate visual cortex. Trends in Neurosciences, 24, 295–300. Treue, S., & Martinez Trujillo, J. C. (1999). Feature-based attention influences motion processing gain in macaque visual cortex. Nature, 399, 575–579. Weil, R. S., Furl, N., Ruff, C. C., Symmonds, M., Flandin, G., Dolan, R. J., et al. (2010). Rewarding feedback after correct visual discriminations has both general and specific influences on visual cortex. Journal of Neurophysiology, 104, 1746–1757. Weissman, D. H., Gopalakrishnan, A., Hazlett, C. J., & Woldorff, M. G. (2005). Dorsal anterior cingulate cortex resolves conflict from distracting stimuli by boosting attention toward relevant events. Cerebral Cortex, 15, 229–237. Weissman, D. H., Warner, L. M., & Woldorff, M. G. (2004). The neural mechanisms for minimizing cross-modal distraction. Journal of Neuroscience, 24, 10941–10949. Yantis, S. (1996). Attentional capture in vision. In A. Kramer, M. Coles, & G. Logan (Eds.), Converging operations in the study of selective visual attention (pp. 45–76). Washington, DC: American Psychological Association.
U nc
or
re
ct
top–down task set: New evidence from the N2pc component. Psychophysiology, 45, 1013–1024. Klein, R. M. (2000). Inhibition of return. Trends in Cognitive Sciences, 4, 138–147. Kristjansson, A., Sigurjonsdottir, O., & Driver, J. (2010). Fortune and reversals of fortune in visual search: Reward contingencies for pop-out targets affect search efficiency and target repetition effects. Attention, Perception, & Psychophysics, 72, 1229–1236. Kwak, H. W., & Egeth, H. (1992). Consequences of allocating attention to locations and to other attributes. Perception & Psychophysics, 51, 455–464. Lamy, D., Leber, A., & Egeth, H. E. (2004). Effects of task relevance and stimulus-driven salience in feature-search mode. Journal of Experimental Psychology: Human Perception and Performance, 30, 1019–1031. Law, M. B., Pratt, J., & Abrams, R. A. (1995). Color-based inhibition of return. Perception & Psychophysics, 57, 402–408. Leblanc, E., Prime, D. J., & Jolicoeur, P. (2008). Tracking the location of visuospatial attention in a contingent capture paradigm. Journal of Cognitive Neuroscience, 20, 657–671. Lee, B. B., Martin, P. R., & Valberg, A. (1988). The physiological basis of heterochromatic flicker photometry demonstrated in the ganglion cells of the macaque retina. Journal of Physiology, 404, 323–347. Lennert, T., & Martinez-Trujillo, J. (2011). Strength of response suppression to distracter stimuli determines attentionalfiltering performance in primate prefrontal neurons. Neuron, 70, 141–152. Lien, M. C., Ruthruff, E., Goodin, Z., & Remington, R. W. (2008). Contingent attentional capture by top–down control settings: Converging evidence from event-related potentials. Journal of Experimental Psychology: Human Perception and Performance, 34, 509–530. Mansouri, F. A., Tanaka, K., & Buckley, M. J. (2009). Conflictinduced behavioural adjustment: A clue to the executive functions of the prefrontal cortex. Nature Reviews Neuroscience, 10, 141–152. Martinez, A., Anllo-Vento, L., Sereno, M. I., Frank, L. R., Buxton, R. B., Dubowitz, D. J., et al. (1999). Involvement of striate and extrastriate visual cortical areas in spatial attention. Nature Neuroscience, 2, 364–369. Maunsell, J. H., & Treue, S. (2006). Feature-based attention in visual cortex. Trends in Neurosciences, 29, 317–322. Maylor, E. A., & Hockey, R. (1985). Inhibitory component of externally controlled covert orienting in visual space. Journal of Experimental Psychology: Human Perception and Performance, 11, 777–787. Moore, T., & Armstrong, K. M. (2003). Selective gating of visual signals by microstimulation of frontal cortex. Nature, 421, 370–373. Moore, T., Armstrong, K. M., & Fallah, M. (2003). Visuomotor origins of covert spatial attention. Neuron, 40, 671–683. Moore, T., & Fallah, M. (2004). Microstimulation of the frontal eye field and its effects on covert spatial attention. Journal of Neurophysiology, 91, 152–162.
Buschschulte et al.
17
