Articles in PresS. J Neurophysiol (July 16, 2014). doi:10.1152/jn.00134.2014
Running head: Maintenance of Relational Information in WM 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47
Maintenance of Relational Information in Working Memory Leads to Suppression of the Sensory Cortex
Akiko Ikkai1, Kara J. Blacker1, Balaji M. Lakshmanan3, Joshua B. Ewen3,5, Susan M. Courtney1,2,4 1 Department of Psychological & Brain Sciences, Johns Hopkins University 2 Department of Neuroscience, Johns Hopkins University School of Medicine 3 Deptartment of Neurology and Developmental Medicine, Kennedy Krieger Institute 4 F.M. Kirby Research Center, Kennedy Krieger Institute 5 Department of Neurology, Johns Hopkins University School of Medicine
Corresponding Author: Susan M. Courtney, Ph.D. Psychological and Brain Sciences Johns Hopkins University 227 Ames Hall 3400 N Charles St. Baltimore, MD 21218 (410) 516-8894 [email protected]
Copyright © 2014 by the American Physiological Society.
Maintenance of Relational Information in WM
2
48
Abstract
49
Working memory (WM) for sensory-based information about individual objects and their
50
locations appears to involve interactions between lateral prefrontal and sensory
51
cortices. The mechanisms and representations for maintenance of more abstract, non-
52
sensory information in WM are unknown, particularly whether such actively maintained
53
information can become independent of the sensory information from which it was
54
derived. Previous studies of WM for individual visual items found increased
55
electroencephalogram (EEG) alpha (8-13Hz) power over posterior electrode sites,
56
which appears to correspond to the suppression of cortical areas that represent
57
irrelevant sensory information. Here, we recorded EEG while participants performed a
58
visual WM task that involved maintaining either concrete spatial coordinates or abstract
59
relational information. Maintenance of relational information resulted in higher alpha
60
power in posterior electrodes. Furthermore, lateralization of alpha power due to a covert
61
shift of attention to one visual hemifield was marginally weaker during storage of
62
relational information than during storage of concrete information. These results suggest
63
that abstract relational information is maintained in WM differently from concrete,
64
sensory representations and that during maintenance of abstract information, posterior
65
sensory regions become task-irrelevant and are thus suppressed.
66 67 68 69 70 71
Abstract Word Count: 185 Keywords: EEG, working memory, spatial
Maintenance of Relational Information in WM
3
72 73
Introduction Selectively maintaining relevant information in working memory (WM) is crucial to
74
our ability to flexibly make decisions and guide behavior. It has been proposed that WM
75
for sensory-specific features of individual objects is achieved via interactions between
76
prefrontal cortex (PFC) and sensory regions. On the other hand, during the
77
maintenance of abstract information, such as relationships, rules, and strategies, the
78
involvement of sensory regions may be limited. For example, in order to flexibly
79
maintain a spatial relation between objects, sensory codes (e.g., absolute coordinates)
80
might be transformed into codes that do not necessarily rely on the representation of the
81
past sensory events. What, then, is the degree of involvement of the posterior sensory
82
regions during the maintenance of abstract, non-sensory-based information? One
83
possibility is that abstract rules and relationships are maintained as a simple extension
84
of a maintained retrospective sensory code. If so, the representation of abstract
85
information would rely heavily on the activation of the same sensory neurons as those
86
that are active when sensory information is task-relevant. Alternatively, abstract
87
information may be treated as a distinct type of information and its representation may
88
become independent of the original sensory codes.
89
Previous single-cell recording studies in monkeys have suggested that the PFC
90
and parietal cortex contain neurons that represent abstract information of many different
91
types, including rules (Fuster, Bodner, & Kroger, 2000; Wallis, Anderson, & Miller,
92
2001), category membership (Freedman, Riesenhuber, Poggio, & Miller, 2001;
93
Swaminathan & Freedman, 2012), strategies (Genovesio, Brasted, Mitz, & Wise, 2005;
94
Tsujimoto, Genovesio, & Wise, 2012), and spatial relations (Chafee, Averbeck, &
95
Crowe, 2007), in addition to representing sensory information across WM delays
Maintenance of Relational Information in WM 96
(Funahashi, Bruce, & Goldman-Rakic, 1989; Gnadt & Andersen, 1988; Wilson,
97
Scalaidhe, & Goldman-Rakic, 1993). Recent fMRI evidence suggest that subregions of
98
the human PFC and parietal cortex are differentially active during maintenance and
99
updating of abstract information compared to object-specific information (Ackerman &
4
100
Courtney, 2012; Montojo & Courtney, 2008). While there is evidence from fMRI studies
101
that there may be some degree of domain specificity within the abstract WM systems,
102
such as for spatial versus non-spatial relationships, or for mathematical operations
103
versus magnitude comparisons, there remains a broader distinction in all of these cases
104
for concrete, sensory versus abstract information that is consistent across studies
105
(Montojo & Courtney, 2008; Ackerman & Courtney, 2012; Bahlmann, Blumenfeld,
106
D’Esposito, 2014). This dissociation generally involves more medial and posterior parts
107
of parietal and frontal cortex for sensory WM and more lateral and anterior areas for
108
abstract WM. For example, Ackerman and Courtney (2012) reported greater BOLD
109
fMRI signal in the anterior portions of both the PFC and the intraparietal sulcus (IPS)
110
during the updating and maintenance of spatial relational information (i.e., “left of”, or
111
“right of”) in WM compared to item-specific information (i.e., particular locations of
112
individual objects). On the other hand, when participants were required to process item-
113
specific spatial information in WM, more posterior PFC and IPS regions demonstrated
114
greater activity. Furthermore, while patterns of neural activity in retinotopic cortical
115
regions could be classified according to the memory for individual object locations, only
116
in anterior PFC could patterns of activity be classified according to the abstract spatial
117
relationship being held in memory. These results suggest that neural populations in
118
anterior PFC can maintain a code representing the particular relationship being held in
Maintenance of Relational Information in WM
5
119
WM, and that this representation is different from that of the original sensory information
120
in the sample stimulus.
121
It is unknown how these neural populations in PFC that represent abstract
122
information interact with domain-specific sensory regions to achieve task-relevant goals.
123
When a particular type of stimulus or stimulus feature is task-relevant, domain-specific
124
sensory and PFC regions increase activation (Ben-Shachar, Dougherty, Deutsch, &
125
Wandell, 2007; Courtney, Ungerleider, Keil, & Haxby, 1997; Grill-Spector, Kushnir,
126
Edelman, Itzchak, & Malach, 1998; Ikkai, Jerde, & Curtis, 2011; Kanwisher, McDermott,
127
& Chun, 1997; Malach et al., 1995; Puce, Allison, Asgari, Gore, & McCarthy, 1996;
128
Wojciulik, Kanwisher, & Driver, 1998). Further, recent studies have shown that
129
increased connectivity between PFC subregions and task-relevant sensory regions is
130
associated with enhanced task-switching performance (Sakai & Passingham, 2003,
131
2006; Stelzel, Basten, & Fiebach, 2011), and that greater connectivity between the
132
middle and inferior frontal gyri and extrastriate cortex correlates with better WM
133
encoding and maintenance (Cohen, Sreenivasan, & D'Esposito, 2012). Together, these
134
imaging studies suggest that when information about a particular stimulus needs to be
135
held in WM, its representation appears to be selectively enhanced via complex
136
interactions between PFC and sensory cortex. What remains unknown is the fate of the
137
sensory information and the role of sensory cortex during the maintenance of more
138
abstract relational information in WM, when the specific sensory information is
139
irrelevant. Thus, the current study aimed to investigate the possibility that sensory areas
140
may be functionally suppressed during the maintenance of abstract information,
141
suggesting a qualitatively distinct and independent system. Alternatively, abstract WM
Maintenance of Relational Information in WM 142
representations may be built off of sensory representations in a hierarchical manner
143
(e.g., Badre & D’Esposito, 2007) with maintenance of those sensory representations
144
being necessary during abstract relational WM, suggesting an additional level of the
145
system for abstract WM, but not a qualitatively distinct mechanism.
6
146
Accumulating evidence suggests that neural oscillations are important in both the
147
maintenance and selective suppression of sensory information, such as object locations
148
and features in WM (Medendorp et al., 2007; for a review see, Roux & Uhlhaas, 2013).
149
Specifically, oscillation in the alpha frequency band (8-13Hz) has been particularly well
150
studied in human EEG and maintenance of information in WM has been associated with
151
increases in alpha power in posterior regions, which is thought to reflect suppression of
152
incoming sensory input that would interfere with the currently maintained information
153
(Jensen, Gelfand, Kounios, & Lisman, 2002; Klimesch, Doppelmayr, Schwaiger,
154
Auinger, & Winkler, 1999; Krause, Lang, Laine, Kuusisto, & Porn, 1996). Although alpha
155
was originally considered to be a signature of cortical idling (Adrian & Matthews, 1934;
156
Pfurtscheller, Stancak, & Neuper, 1996), more recent evidence suggests that alpha
157
oscillations may have a more active role in selective attention and WM, particularly in
158
the functional suppression of task-irrelevant brain regions (Bengson, Mangun, &
159
Mazaheri, 2012; Fu et al., 2001; Jensen & Mazaheri, 2010; Jokisch & Jensen, 2007;
160
Kelly, Lalor, Reilly, & Foxe, 2006; Rihs, Michel, & Thut, 2007; Sauseng et al., 2009; van
161
Dijk, van der Werf, Mazaheri, Medendorp, & Jensen, 2010; Worden, Foxe, Wang, &
162
Simpson, 2000). For example, Jokisch and Jensen (2007) contrasted oscillatory power
163
during the maintenance of spatial information versus object identity information and
164
found increased alpha power in the sensors over the brain region that was task-
Maintenance of Relational Information in WM 165
irrelevant (i.e., regions along the dorsal pathway during the object identity task). In
166
another study, alpha power increased over visual cortex when participants anticipated
167
the delivery of an auditory stimulus, making the visual cortex task-irrelevant (Fu et al.,
168
2001). Thus, alpha oscillations appear to reflect a mechanism by which brain regions
169
that represent task-irrelevant information are suppressed in order to prioritize the
170
processing of task-relevant information (Jensen & Mazaheri, 2010; Kelly et al., 2006).
171
However, no study to date has investigated fluctuations of alpha power in tasks that
172
require the maintenance of non-sensory representations.
173
7
In the present study, using alpha power modulation as a marker of sensory
174
region’s involvement in WM storage and maintenance, we tested whether relational
175
information is maintained as a representation that is distinct from the sensory code. To
176
this end, we designed a WM task that required participants to transform sensory stimuli
177
into either concrete (item-specific) information or abstract (relational) information in the
178
spatial domain, and to maintain this information during a delay period. Based on
179
previous findings that alpha power is modulated in domain-specific sensory areas
180
according to task-relevance, we had two specific predictions regarding posterior alpha
181
modulation during the WM delay. First, we predicted that during the maintenance of
182
relational information the sensory regions of the brain would become task-irrelevant and
183
therefore be functionally suppressed as evidenced by higher bilateral posterior alpha
184
power, as compared to when concrete information was maintained. Second, it would be
185
expected that during the maintenance of concrete spatial information, retinotopically
186
specific (i.e., contralateral vs. ipsilateral) sensory regions would be involved in the
187
representation of sensory stimuli, whereas during the maintenance of abstract
Maintenance of Relational Information in WM
8
188
information, this retinotopic information is irrelevant. Thus, we examined alpha
189
lateralization in response to a covert shift of attention to the right or left visual field (e.g.,
190
Kelly et al., 2006; Sauseng et al., 2005; Worden et al., 2000) to examine the strength of
191
the retinotopic representation when maintaining concrete versus relational information in
192
WM. We predicted that the retinotopic representation evidenced by alpha lateralization
193
would be weaker when abstract relational information was maintained in WM compared
194
to when concrete, item-specific information was maintained. In summary, we aimed to
195
test the hypothesis that abstract, relational information is maintained in WM as a
196
representation that is distinct from concrete, sensory information. Method
197 198 199
Participants Eighteen neurologically healthy adults (9 female, 18-31 years of age) participated
200
for monetary compensation. All participants had normal or corrected-to-normal vision,
201
and gave written informed consent approved by the Institutional Review Boards of
202
Johns Hopkins University and the Johns Hopkins Medical Institutions.
203
Task and Procedures
204
Experimental stimuli were controlled by MATLAB (The MathWorks, Natick, MA)
205
using Psychophysics Toolbox extensions (Brainard, 1997; Pelli, 1997), and displayed
206
on a Dell LCD monitor. Participants were seated 92cm away from the monitor, and
207
given a Logitech game controller to enter responses.
208
As shown in Figure 1A/B, a trial began with a white fixation square (0.1° of visual
209
angle) appearing in the center of the display for 1000ms. A 200ms arrow cue (0.46°)
210
was presented, instructing participants to attend covertly to one visual hemifield,
Maintenance of Relational Information in WM
9
211
followed by a sample display of 2 circles on the left and 2 circles on the right that
212
appeared for 500ms. The circles appeared within rectangular regions from 1.3°-4.8°
213
horizontally and from 0°-4° vertically above or below the central fixation point. Within a
214
visual hemifield, the two circles in the sample and test displays were placed between
215
1.1°-1.3° horizontally and between 1.4°-1.6° vertically from each other. Next, a 2000ms
216
delay period consisting of only a white fixation square was then followed by a 1500ms
217
test array and response period. The inter-trial interval (ITI) was chosen randomly for
218
each trial between 1250-1500ms in 50ms increments, during which the fixation square
219
was black.
220
Participants performed two types of trials: Item and Relation. In Item trials, a
221
sample display consisted of 2 circles (0.4° visual angle each) with different shades of
222
grey. Participants were instructed to form an imaginary line between these circles during
223
the sample display, and to keep the location of the line in memory throughout the delay
224
period. A test display contained a circle of a different shade of grey either on the
225
imaginary line, or off the imaginary line. Participants made a button press to indicate
226
whether the test circle appeared on this remembered imaginary line or not. The fixation
227
square changed to green, red, or blue to indicate the response was correct, incorrect, or
228
too slow, respectively. There was a 50% chance of the test circle being on the line, and
229
a 50% chance of it being off the line.
230
In Relation trials, the sample stimuli were exactly the same as in the Item trials: 2
231
circles of different shades of grey. However, participants were instructed to extract a
232
horizontal spatial relationship between the two circles, independent of their particular
233
retinotopic locations. One of the circles, the anchor, had a red dot in the center and the
Maintenance of Relational Information in WM
10
234
participants’ task was to ascertain whether the other circle was on the left or right side of
235
the anchor, and maintain this spatial information throughout the delay period. In the test
236
display, 2 circles with different shades of grey were presented. As in the sample display,
237
a red dot in one of the circles acted as the anchor, and participants were instructed to
238
make a judgment whether the horizontal spatial relationship between the anchor and
239
the other circle was preserved. The spatial locations of the test circles were always
240
different from those of the sample circles. Participants made a button press to indicate
241
their response, and feedback was given as in the Item task. There was a 50% chance of
242
the test circles having the same relationship as the sample circles, and a 50% chance of
243
the relationship being different.
244
There are a few important additional points to emphasize about our task design.
245
First, in the Relation condition, the shades of grey of the 2 sample circles were different
246
from those of the test circles in each trial. This manipulation discouraged participants
247
from encoding and maintaining sensory information, and instead, encouraging them to
248
extract the relational information between the circles. Spatial coordinates of circles
249
could also be quite different between sample and test displays in Relation trials. In fact,
250
the vertical spatial relationship between circles could switch independently of the
251
horizontal relationship between the sample and test displays, and only the horizontal
252
spatial relationship was relevant. This aspect of the design also aimed to discourage
253
participants from simply maintaining sensory information.
254
Similarly, in the Item condition the test circle had a different shade of grey than
255
either of the sample circles. Moreover, the test circle never appeared at the exact same
256
coordinates as either of the sample circles. Instead, the test circle could appear
Maintenance of Relational Information in WM
11
257
between the sample circles, intersecting the imaginary line (“on the line,” at 25% or 50%
258
from one of the sample circles, but never beyond the sample circle end points of the
259
line) in half of the trials. The other half of the trials, the test circle appeared at a
260
coordinate perpendicular to the imaginary line (“off the line”). The distance between the
261
test circle and the imaginary line was between 1.5°-1.7° of visual angle. Here, again, the
262
test circle never appeared beyond the end points of the imaginary line. As in the
263
Relation condition, these manipulations discouraged participants from simply encoding
264
and maintaining a perceptual copy of the sample display. Instead, participants were
265
required to maintain the location of the imaginary line, using the coordinates of the
266
sample circles as the endpoints. Alignment of the sample circles in the Item trials was
267
either vertical or horizontal, but never diagonal. This constraint, along with the test circle
268
in the “off the line” condition appearing at coordinates perpendicular to the imaginary
269
line, was used to equate the difficulty between the Item and Relation trials. Thus, both
270
Item and Relation conditions required active conversion of the sample display. The
271
crucial difference was that, in the Item condition, participants maintained the concrete
272
retinotopic spatial information of the line, whereas in the Relation condition, participants
273
maintained the abstract relational spatial information of sample circles, independent of
274
their particular retinotopic locations.
275
In addition, by requiring participants to actively convert spatial information, we
276
were able to better equate the memory load between conditions. The Relation condition
277
required the maintenance of one piece of spatial relational information (“left/right of”).
278
The Item condition also required the maintenance of one object (i.e., the imaginary line).
279
Although the sensory information load at the test phase was different between the two
Maintenance of Relational Information in WM
12
280
conditions, 2 test circles for Relation trials and 1 for item trials, we analyzed our EEG
281
data only up to the end of the delay period. The encoding and maintenance demands
282
were designed to be as similar as possible across conditions during sample and delay.
283
Lastly, trial types were blocked, and each block contained 40 trials, with 20
284
“Attend Right” and 20 “Attend Left” trials. Participants completed 5 blocks of Relation
285
and 5 blocks of Item trials, for a total of 400 trials, 200 of each trial type. Only correct
286
trials were analyzed. All participants came in for a one-hour practice session a few days
287
before the EEG session.
288
Data Collection and Analysis Procedures
289
EEG Recording. The EEG data were recorded at 128 sites covering the whole
290
scalp with approximately uniform density using an elastic electrode cap [WaveGuard
291
cap with 128-channel Duke (equidistant electrode placement) layout, Advanced Neuro
292
Technology, The Netherlands], referenced to the average of all electrodes during
293
recording. Electrode impedance was kept below 5kΩ. All EEG electrodes were recorded
294
continuously in DC mode at a sampling rate of 512Hz using an anti-aliasing filter with a
295
138Hz cutoff and a high-impedance ANT WaveGuard amplifier.
296
Preprocessing. Data were analyzed using the Fieldtrip software package
297
(http://www.ru.nl/fcdonders/fieldtrip/), a MATLAB-based toolbox that has been
298
developed at the F.C. Donders Centre for Cognitive Neuroimaging (Nijmegen, The
299
Netherlands). Data were first high-pass filtered at 1Hz, and then segmented into epochs
300
between 1.25sec before and 3.0sec after the onset of the sample display. Independent
301
component analysis (ICA) was performed on the epoched data, and the eye blink
302
component was identified and removed from the data. We visually inspected EEG
Maintenance of Relational Information in WM
13
303
waveforms of frontal electrodes to identify voltage fluctuations typical of horizontal eye
304
movements. Trials containing horizontal eye movements were rejected entirely. To
305
maintain sufficient statistical power for each condition, participants with more than 30%
306
rejection in any single condition (e.g., Item-Attend Left) were discarded (n=2).
307
Spectral analysis. Time-frequency analysis was performed by calculating power
308
at frequencies between 1-30Hz with 0.5Hz increment using a hanning taper method
309
applied to short sliding time windows (Percival & Walden, 1993) every 100ms. We
310
applied an adaptive time window of five cycles for each frequency (T = 5/f).
311
Statistical Analysis. We obtained statistics corrected for multiple comparisons in
312
the following way: we used a nonparametric randomization test (Maris & Oostenveld,
313
2007; Nichols & Holmes, 2002) to statistically test the difference between conditions.
314
This procedure controls for Type I error by calculating the cluster-level statistics by
315
randomizing trial labels at each iteration. First, spectral data from each of our 128
316
electrodes across the scalp were averaged over the time and frequency range of
317
interest. Our time range of interest was the delay period (i.e., 0.5 – 2.5 seconds after the
318
onset of the sample), but we excluded the first 500ms of the delay period because this
319
time period likely contained sensory-evoked response activity from the sample stimuli
320
(e.g., van Gerven et al., 2009; for a more detailed discussion of this topic, also see
321
Bastiaansen, Mazaheri, & Jensen, 2012). Next, a t-value was calculated at each
322
electrode. For each iteration, clusters of electrodes where the alpha-level was .05, or of attended visual field, F(1,15)=0.125, p>.05, for accuracy
362
values (see Figure 1C). For response time (RT), there was a significant main effect of
363
trial type, F(1,15)=107.4, p
Running head: Maintenance of Relational Information in WM 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47
Maintenance of Relational Information in Working Memory Leads to Suppression of the Sensory Cortex
Akiko Ikkai1, Kara J. Blacker1, Balaji M. Lakshmanan3, Joshua B. Ewen3,5, Susan M. Courtney1,2,4 1 Department of Psychological & Brain Sciences, Johns Hopkins University 2 Department of Neuroscience, Johns Hopkins University School of Medicine 3 Deptartment of Neurology and Developmental Medicine, Kennedy Krieger Institute 4 F.M. Kirby Research Center, Kennedy Krieger Institute 5 Department of Neurology, Johns Hopkins University School of Medicine
Corresponding Author: Susan M. Courtney, Ph.D. Psychological and Brain Sciences Johns Hopkins University 227 Ames Hall 3400 N Charles St. Baltimore, MD 21218 (410) 516-8894 [email protected]
Copyright © 2014 by the American Physiological Society.
Maintenance of Relational Information in WM
2
48
Abstract
49
Working memory (WM) for sensory-based information about individual objects and their
50
locations appears to involve interactions between lateral prefrontal and sensory
51
cortices. The mechanisms and representations for maintenance of more abstract, non-
52
sensory information in WM are unknown, particularly whether such actively maintained
53
information can become independent of the sensory information from which it was
54
derived. Previous studies of WM for individual visual items found increased
55
electroencephalogram (EEG) alpha (8-13Hz) power over posterior electrode sites,
56
which appears to correspond to the suppression of cortical areas that represent
57
irrelevant sensory information. Here, we recorded EEG while participants performed a
58
visual WM task that involved maintaining either concrete spatial coordinates or abstract
59
relational information. Maintenance of relational information resulted in higher alpha
60
power in posterior electrodes. Furthermore, lateralization of alpha power due to a covert
61
shift of attention to one visual hemifield was marginally weaker during storage of
62
relational information than during storage of concrete information. These results suggest
63
that abstract relational information is maintained in WM differently from concrete,
64
sensory representations and that during maintenance of abstract information, posterior
65
sensory regions become task-irrelevant and are thus suppressed.
66 67 68 69 70 71
Abstract Word Count: 185 Keywords: EEG, working memory, spatial
Maintenance of Relational Information in WM
3
72 73
Introduction Selectively maintaining relevant information in working memory (WM) is crucial to
74
our ability to flexibly make decisions and guide behavior. It has been proposed that WM
75
for sensory-specific features of individual objects is achieved via interactions between
76
prefrontal cortex (PFC) and sensory regions. On the other hand, during the
77
maintenance of abstract information, such as relationships, rules, and strategies, the
78
involvement of sensory regions may be limited. For example, in order to flexibly
79
maintain a spatial relation between objects, sensory codes (e.g., absolute coordinates)
80
might be transformed into codes that do not necessarily rely on the representation of the
81
past sensory events. What, then, is the degree of involvement of the posterior sensory
82
regions during the maintenance of abstract, non-sensory-based information? One
83
possibility is that abstract rules and relationships are maintained as a simple extension
84
of a maintained retrospective sensory code. If so, the representation of abstract
85
information would rely heavily on the activation of the same sensory neurons as those
86
that are active when sensory information is task-relevant. Alternatively, abstract
87
information may be treated as a distinct type of information and its representation may
88
become independent of the original sensory codes.
89
Previous single-cell recording studies in monkeys have suggested that the PFC
90
and parietal cortex contain neurons that represent abstract information of many different
91
types, including rules (Fuster, Bodner, & Kroger, 2000; Wallis, Anderson, & Miller,
92
2001), category membership (Freedman, Riesenhuber, Poggio, & Miller, 2001;
93
Swaminathan & Freedman, 2012), strategies (Genovesio, Brasted, Mitz, & Wise, 2005;
94
Tsujimoto, Genovesio, & Wise, 2012), and spatial relations (Chafee, Averbeck, &
95
Crowe, 2007), in addition to representing sensory information across WM delays
Maintenance of Relational Information in WM 96
(Funahashi, Bruce, & Goldman-Rakic, 1989; Gnadt & Andersen, 1988; Wilson,
97
Scalaidhe, & Goldman-Rakic, 1993). Recent fMRI evidence suggest that subregions of
98
the human PFC and parietal cortex are differentially active during maintenance and
99
updating of abstract information compared to object-specific information (Ackerman &
4
100
Courtney, 2012; Montojo & Courtney, 2008). While there is evidence from fMRI studies
101
that there may be some degree of domain specificity within the abstract WM systems,
102
such as for spatial versus non-spatial relationships, or for mathematical operations
103
versus magnitude comparisons, there remains a broader distinction in all of these cases
104
for concrete, sensory versus abstract information that is consistent across studies
105
(Montojo & Courtney, 2008; Ackerman & Courtney, 2012; Bahlmann, Blumenfeld,
106
D’Esposito, 2014). This dissociation generally involves more medial and posterior parts
107
of parietal and frontal cortex for sensory WM and more lateral and anterior areas for
108
abstract WM. For example, Ackerman and Courtney (2012) reported greater BOLD
109
fMRI signal in the anterior portions of both the PFC and the intraparietal sulcus (IPS)
110
during the updating and maintenance of spatial relational information (i.e., “left of”, or
111
“right of”) in WM compared to item-specific information (i.e., particular locations of
112
individual objects). On the other hand, when participants were required to process item-
113
specific spatial information in WM, more posterior PFC and IPS regions demonstrated
114
greater activity. Furthermore, while patterns of neural activity in retinotopic cortical
115
regions could be classified according to the memory for individual object locations, only
116
in anterior PFC could patterns of activity be classified according to the abstract spatial
117
relationship being held in memory. These results suggest that neural populations in
118
anterior PFC can maintain a code representing the particular relationship being held in
Maintenance of Relational Information in WM
5
119
WM, and that this representation is different from that of the original sensory information
120
in the sample stimulus.
121
It is unknown how these neural populations in PFC that represent abstract
122
information interact with domain-specific sensory regions to achieve task-relevant goals.
123
When a particular type of stimulus or stimulus feature is task-relevant, domain-specific
124
sensory and PFC regions increase activation (Ben-Shachar, Dougherty, Deutsch, &
125
Wandell, 2007; Courtney, Ungerleider, Keil, & Haxby, 1997; Grill-Spector, Kushnir,
126
Edelman, Itzchak, & Malach, 1998; Ikkai, Jerde, & Curtis, 2011; Kanwisher, McDermott,
127
& Chun, 1997; Malach et al., 1995; Puce, Allison, Asgari, Gore, & McCarthy, 1996;
128
Wojciulik, Kanwisher, & Driver, 1998). Further, recent studies have shown that
129
increased connectivity between PFC subregions and task-relevant sensory regions is
130
associated with enhanced task-switching performance (Sakai & Passingham, 2003,
131
2006; Stelzel, Basten, & Fiebach, 2011), and that greater connectivity between the
132
middle and inferior frontal gyri and extrastriate cortex correlates with better WM
133
encoding and maintenance (Cohen, Sreenivasan, & D'Esposito, 2012). Together, these
134
imaging studies suggest that when information about a particular stimulus needs to be
135
held in WM, its representation appears to be selectively enhanced via complex
136
interactions between PFC and sensory cortex. What remains unknown is the fate of the
137
sensory information and the role of sensory cortex during the maintenance of more
138
abstract relational information in WM, when the specific sensory information is
139
irrelevant. Thus, the current study aimed to investigate the possibility that sensory areas
140
may be functionally suppressed during the maintenance of abstract information,
141
suggesting a qualitatively distinct and independent system. Alternatively, abstract WM
Maintenance of Relational Information in WM 142
representations may be built off of sensory representations in a hierarchical manner
143
(e.g., Badre & D’Esposito, 2007) with maintenance of those sensory representations
144
being necessary during abstract relational WM, suggesting an additional level of the
145
system for abstract WM, but not a qualitatively distinct mechanism.
6
146
Accumulating evidence suggests that neural oscillations are important in both the
147
maintenance and selective suppression of sensory information, such as object locations
148
and features in WM (Medendorp et al., 2007; for a review see, Roux & Uhlhaas, 2013).
149
Specifically, oscillation in the alpha frequency band (8-13Hz) has been particularly well
150
studied in human EEG and maintenance of information in WM has been associated with
151
increases in alpha power in posterior regions, which is thought to reflect suppression of
152
incoming sensory input that would interfere with the currently maintained information
153
(Jensen, Gelfand, Kounios, & Lisman, 2002; Klimesch, Doppelmayr, Schwaiger,
154
Auinger, & Winkler, 1999; Krause, Lang, Laine, Kuusisto, & Porn, 1996). Although alpha
155
was originally considered to be a signature of cortical idling (Adrian & Matthews, 1934;
156
Pfurtscheller, Stancak, & Neuper, 1996), more recent evidence suggests that alpha
157
oscillations may have a more active role in selective attention and WM, particularly in
158
the functional suppression of task-irrelevant brain regions (Bengson, Mangun, &
159
Mazaheri, 2012; Fu et al., 2001; Jensen & Mazaheri, 2010; Jokisch & Jensen, 2007;
160
Kelly, Lalor, Reilly, & Foxe, 2006; Rihs, Michel, & Thut, 2007; Sauseng et al., 2009; van
161
Dijk, van der Werf, Mazaheri, Medendorp, & Jensen, 2010; Worden, Foxe, Wang, &
162
Simpson, 2000). For example, Jokisch and Jensen (2007) contrasted oscillatory power
163
during the maintenance of spatial information versus object identity information and
164
found increased alpha power in the sensors over the brain region that was task-
Maintenance of Relational Information in WM 165
irrelevant (i.e., regions along the dorsal pathway during the object identity task). In
166
another study, alpha power increased over visual cortex when participants anticipated
167
the delivery of an auditory stimulus, making the visual cortex task-irrelevant (Fu et al.,
168
2001). Thus, alpha oscillations appear to reflect a mechanism by which brain regions
169
that represent task-irrelevant information are suppressed in order to prioritize the
170
processing of task-relevant information (Jensen & Mazaheri, 2010; Kelly et al., 2006).
171
However, no study to date has investigated fluctuations of alpha power in tasks that
172
require the maintenance of non-sensory representations.
173
7
In the present study, using alpha power modulation as a marker of sensory
174
region’s involvement in WM storage and maintenance, we tested whether relational
175
information is maintained as a representation that is distinct from the sensory code. To
176
this end, we designed a WM task that required participants to transform sensory stimuli
177
into either concrete (item-specific) information or abstract (relational) information in the
178
spatial domain, and to maintain this information during a delay period. Based on
179
previous findings that alpha power is modulated in domain-specific sensory areas
180
according to task-relevance, we had two specific predictions regarding posterior alpha
181
modulation during the WM delay. First, we predicted that during the maintenance of
182
relational information the sensory regions of the brain would become task-irrelevant and
183
therefore be functionally suppressed as evidenced by higher bilateral posterior alpha
184
power, as compared to when concrete information was maintained. Second, it would be
185
expected that during the maintenance of concrete spatial information, retinotopically
186
specific (i.e., contralateral vs. ipsilateral) sensory regions would be involved in the
187
representation of sensory stimuli, whereas during the maintenance of abstract
Maintenance of Relational Information in WM
8
188
information, this retinotopic information is irrelevant. Thus, we examined alpha
189
lateralization in response to a covert shift of attention to the right or left visual field (e.g.,
190
Kelly et al., 2006; Sauseng et al., 2005; Worden et al., 2000) to examine the strength of
191
the retinotopic representation when maintaining concrete versus relational information in
192
WM. We predicted that the retinotopic representation evidenced by alpha lateralization
193
would be weaker when abstract relational information was maintained in WM compared
194
to when concrete, item-specific information was maintained. In summary, we aimed to
195
test the hypothesis that abstract, relational information is maintained in WM as a
196
representation that is distinct from concrete, sensory information. Method
197 198 199
Participants Eighteen neurologically healthy adults (9 female, 18-31 years of age) participated
200
for monetary compensation. All participants had normal or corrected-to-normal vision,
201
and gave written informed consent approved by the Institutional Review Boards of
202
Johns Hopkins University and the Johns Hopkins Medical Institutions.
203
Task and Procedures
204
Experimental stimuli were controlled by MATLAB (The MathWorks, Natick, MA)
205
using Psychophysics Toolbox extensions (Brainard, 1997; Pelli, 1997), and displayed
206
on a Dell LCD monitor. Participants were seated 92cm away from the monitor, and
207
given a Logitech game controller to enter responses.
208
As shown in Figure 1A/B, a trial began with a white fixation square (0.1° of visual
209
angle) appearing in the center of the display for 1000ms. A 200ms arrow cue (0.46°)
210
was presented, instructing participants to attend covertly to one visual hemifield,
Maintenance of Relational Information in WM
9
211
followed by a sample display of 2 circles on the left and 2 circles on the right that
212
appeared for 500ms. The circles appeared within rectangular regions from 1.3°-4.8°
213
horizontally and from 0°-4° vertically above or below the central fixation point. Within a
214
visual hemifield, the two circles in the sample and test displays were placed between
215
1.1°-1.3° horizontally and between 1.4°-1.6° vertically from each other. Next, a 2000ms
216
delay period consisting of only a white fixation square was then followed by a 1500ms
217
test array and response period. The inter-trial interval (ITI) was chosen randomly for
218
each trial between 1250-1500ms in 50ms increments, during which the fixation square
219
was black.
220
Participants performed two types of trials: Item and Relation. In Item trials, a
221
sample display consisted of 2 circles (0.4° visual angle each) with different shades of
222
grey. Participants were instructed to form an imaginary line between these circles during
223
the sample display, and to keep the location of the line in memory throughout the delay
224
period. A test display contained a circle of a different shade of grey either on the
225
imaginary line, or off the imaginary line. Participants made a button press to indicate
226
whether the test circle appeared on this remembered imaginary line or not. The fixation
227
square changed to green, red, or blue to indicate the response was correct, incorrect, or
228
too slow, respectively. There was a 50% chance of the test circle being on the line, and
229
a 50% chance of it being off the line.
230
In Relation trials, the sample stimuli were exactly the same as in the Item trials: 2
231
circles of different shades of grey. However, participants were instructed to extract a
232
horizontal spatial relationship between the two circles, independent of their particular
233
retinotopic locations. One of the circles, the anchor, had a red dot in the center and the
Maintenance of Relational Information in WM
10
234
participants’ task was to ascertain whether the other circle was on the left or right side of
235
the anchor, and maintain this spatial information throughout the delay period. In the test
236
display, 2 circles with different shades of grey were presented. As in the sample display,
237
a red dot in one of the circles acted as the anchor, and participants were instructed to
238
make a judgment whether the horizontal spatial relationship between the anchor and
239
the other circle was preserved. The spatial locations of the test circles were always
240
different from those of the sample circles. Participants made a button press to indicate
241
their response, and feedback was given as in the Item task. There was a 50% chance of
242
the test circles having the same relationship as the sample circles, and a 50% chance of
243
the relationship being different.
244
There are a few important additional points to emphasize about our task design.
245
First, in the Relation condition, the shades of grey of the 2 sample circles were different
246
from those of the test circles in each trial. This manipulation discouraged participants
247
from encoding and maintaining sensory information, and instead, encouraging them to
248
extract the relational information between the circles. Spatial coordinates of circles
249
could also be quite different between sample and test displays in Relation trials. In fact,
250
the vertical spatial relationship between circles could switch independently of the
251
horizontal relationship between the sample and test displays, and only the horizontal
252
spatial relationship was relevant. This aspect of the design also aimed to discourage
253
participants from simply maintaining sensory information.
254
Similarly, in the Item condition the test circle had a different shade of grey than
255
either of the sample circles. Moreover, the test circle never appeared at the exact same
256
coordinates as either of the sample circles. Instead, the test circle could appear
Maintenance of Relational Information in WM
11
257
between the sample circles, intersecting the imaginary line (“on the line,” at 25% or 50%
258
from one of the sample circles, but never beyond the sample circle end points of the
259
line) in half of the trials. The other half of the trials, the test circle appeared at a
260
coordinate perpendicular to the imaginary line (“off the line”). The distance between the
261
test circle and the imaginary line was between 1.5°-1.7° of visual angle. Here, again, the
262
test circle never appeared beyond the end points of the imaginary line. As in the
263
Relation condition, these manipulations discouraged participants from simply encoding
264
and maintaining a perceptual copy of the sample display. Instead, participants were
265
required to maintain the location of the imaginary line, using the coordinates of the
266
sample circles as the endpoints. Alignment of the sample circles in the Item trials was
267
either vertical or horizontal, but never diagonal. This constraint, along with the test circle
268
in the “off the line” condition appearing at coordinates perpendicular to the imaginary
269
line, was used to equate the difficulty between the Item and Relation trials. Thus, both
270
Item and Relation conditions required active conversion of the sample display. The
271
crucial difference was that, in the Item condition, participants maintained the concrete
272
retinotopic spatial information of the line, whereas in the Relation condition, participants
273
maintained the abstract relational spatial information of sample circles, independent of
274
their particular retinotopic locations.
275
In addition, by requiring participants to actively convert spatial information, we
276
were able to better equate the memory load between conditions. The Relation condition
277
required the maintenance of one piece of spatial relational information (“left/right of”).
278
The Item condition also required the maintenance of one object (i.e., the imaginary line).
279
Although the sensory information load at the test phase was different between the two
Maintenance of Relational Information in WM
12
280
conditions, 2 test circles for Relation trials and 1 for item trials, we analyzed our EEG
281
data only up to the end of the delay period. The encoding and maintenance demands
282
were designed to be as similar as possible across conditions during sample and delay.
283
Lastly, trial types were blocked, and each block contained 40 trials, with 20
284
“Attend Right” and 20 “Attend Left” trials. Participants completed 5 blocks of Relation
285
and 5 blocks of Item trials, for a total of 400 trials, 200 of each trial type. Only correct
286
trials were analyzed. All participants came in for a one-hour practice session a few days
287
before the EEG session.
288
Data Collection and Analysis Procedures
289
EEG Recording. The EEG data were recorded at 128 sites covering the whole
290
scalp with approximately uniform density using an elastic electrode cap [WaveGuard
291
cap with 128-channel Duke (equidistant electrode placement) layout, Advanced Neuro
292
Technology, The Netherlands], referenced to the average of all electrodes during
293
recording. Electrode impedance was kept below 5kΩ. All EEG electrodes were recorded
294
continuously in DC mode at a sampling rate of 512Hz using an anti-aliasing filter with a
295
138Hz cutoff and a high-impedance ANT WaveGuard amplifier.
296
Preprocessing. Data were analyzed using the Fieldtrip software package
297
(http://www.ru.nl/fcdonders/fieldtrip/), a MATLAB-based toolbox that has been
298
developed at the F.C. Donders Centre for Cognitive Neuroimaging (Nijmegen, The
299
Netherlands). Data were first high-pass filtered at 1Hz, and then segmented into epochs
300
between 1.25sec before and 3.0sec after the onset of the sample display. Independent
301
component analysis (ICA) was performed on the epoched data, and the eye blink
302
component was identified and removed from the data. We visually inspected EEG
Maintenance of Relational Information in WM
13
303
waveforms of frontal electrodes to identify voltage fluctuations typical of horizontal eye
304
movements. Trials containing horizontal eye movements were rejected entirely. To
305
maintain sufficient statistical power for each condition, participants with more than 30%
306
rejection in any single condition (e.g., Item-Attend Left) were discarded (n=2).
307
Spectral analysis. Time-frequency analysis was performed by calculating power
308
at frequencies between 1-30Hz with 0.5Hz increment using a hanning taper method
309
applied to short sliding time windows (Percival & Walden, 1993) every 100ms. We
310
applied an adaptive time window of five cycles for each frequency (T = 5/f).
311
Statistical Analysis. We obtained statistics corrected for multiple comparisons in
312
the following way: we used a nonparametric randomization test (Maris & Oostenveld,
313
2007; Nichols & Holmes, 2002) to statistically test the difference between conditions.
314
This procedure controls for Type I error by calculating the cluster-level statistics by
315
randomizing trial labels at each iteration. First, spectral data from each of our 128
316
electrodes across the scalp were averaged over the time and frequency range of
317
interest. Our time range of interest was the delay period (i.e., 0.5 – 2.5 seconds after the
318
onset of the sample), but we excluded the first 500ms of the delay period because this
319
time period likely contained sensory-evoked response activity from the sample stimuli
320
(e.g., van Gerven et al., 2009; for a more detailed discussion of this topic, also see
321
Bastiaansen, Mazaheri, & Jensen, 2012). Next, a t-value was calculated at each
322
electrode. For each iteration, clusters of electrodes where the alpha-level was .05, or of attended visual field, F(1,15)=0.125, p>.05, for accuracy
362
values (see Figure 1C). For response time (RT), there was a significant main effect of
363
trial type, F(1,15)=107.4, p
