Atten Percept Psychophys (2015) 77:450–463 DOI 10.3758/s13414-014-0775-3

Different effects of color-based and location-based selection on visual working memory Qi Li & Jun Saiki

Published online: 24 October 2014 # The Psychonomic Society, Inc. 2014

Abstract In the present study, we investigated how featureand location-based selection influences visual working memory (VWM) encoding and maintenance. In Experiment 1, cue type (color, location) and cue timing (precue, retro-cue) were manipulated in a change detection task. The stimuli were color–location conjunction objects, and binding memory was tested. We found a significantly greater effect for color precues than for either color retro-cues or location precues, but no difference between location pre- and retro-cues, consistent with previous studies (e.g., Griffin & Nobre in Journal of Cognitive Neuroscience, 15, 1176–1194, 2003). We also found no difference between location and color retro-cues. Experiment 2 replicated the color precue advantage with more complex color–shape–location conjunction objects. Only one retro-cue effect was different from that in Experiment 1: Color retro-cues were significantly less effective than location retrocues in Experiment 2, which may relate to a structural property of multidimensional VWM representations. In Experiment 3, a visual search task was used, and the result of a greater location than color precue effect suggests that the color precue advantage in a memory task is related to the modulation of VWM encoding rather than of sensation and perception. Experiment 4, using a task that required only memory for individual features but not for feature bindings, further confirmed that the color precue advantage is specific to binding memory. Together, these findings reveal new aspects of the interaction between attention and VWM and provide potentially important implications for the structural properties of VWM representations. Q. Li Graduate School of Human and Environmental Studies, Kyoto University, Kyoto, Japan J. Saiki (*) Graduate School of Human and Environmental Studies, Yoshida-nihonmatsucho, Sakyo Kyoto 606-8501, Japan e-mail: [email protected]

Keywords Visual working memory . Feature-based attention . Location-based attention . Retro-cue

Visual working memory (VWM), the mental workspace that keeps visual information in an active state, is typically characterized by severe capacity limitations (Baddeley & Hitch, 1974; Phillips, 1974). Evidence from laboratory experiments has suggested that VWM can hold only three or four objects at a time (Luck & Vogel, 1997; Vogel, Woodman, & Luck, 2001; Wheeler & Treisman, 2002). The efficient use of VWM capacity largely depends on selective attention, which enables preferential encoding and maintenance of a subset of the available information (Chun, Golomb, & Turk-Brown, 2011). Moreover, systematic manipulations of attention in VWM encoding and maintenance can complement understanding of the relationship between attention and VWM, and also provide important new information regarding VWM structure and function. To this end, in the present study cue type (feature, location) and cue timing (precue, retro-cue) were systematically manipulated in a standard VWM task with multidimensional objects. The results extend previous findings in the literature. One major approach to studying the role of attention in VWM has been to combine a cueing procedure with VWM tasks. Following earlier studies that showed that a location cue given before (precue) or shortly after (iconic cue) a memory display improves memory at the cued location (Averbach & Coriell, 1961; Becker, Pashler, & Anstis, 2000; Rensink, O’Regan, & Clark, 1997; Schmidt, Vogel, Woodman, & Luck, 2002; Scholl, 2000; Sperling, 1960; Woodman, Vecera, & Luck, 2003), recent studies have demonstrated that location cues presented long after memory display offset (retro-cue) are also effective in biasing VWM performance (Berryhill, Richmond, Shay, & Olson, 2011; Griffin & Nobre, 2003; Landman, Spekreijse, & Lamme, 2003; Makovski,

Atten Percept Psychophys (2015) 77:450–463

Sussman, & Jiang, 2008; Matsukura, Luck, & Vecera, 2007; Nobre et al., 2004; Tanoue & Berryhill, 2012). Accordingly, it has become increasingly clear that selective attention effectively influences not only VWM encoding (the process of transforming perceptual representations into durable VWM representations), but also VWM maintenance. Because precues select perceptual representations in the external world, and retro-cues select internal representations held in VWM, several studies have directly compared pre- and retro-cues in order to investigate the relationship between external and internal attention (e.g., Griffin & Nobre, 2003; Nobre et al., 2004). Interestingly, the results from these studies revealed comparable behavioral effects and similar neural activation patterns for both pre- and retro-cues. Specifically, Griffin and Nobre recorded electroencephalographic activity and found that both pre- and retro-cues evoked similar eventrelated potentials, including spatially specific responses in posterior and anterior electrodes. The functional magnetic resonance imaging study of Nobre et al. further confirmed that the neutral substrates associated with pre- and retro-cues have a large overlap in the prefrontal and parietal cortices. The considerable behavioral and neural similarities between preand retro-cueing raise the possibility that selections of external and internal representations rely on a common spatial attention mechanism. This view is supported by other recent neuroimaging studies reporting overlapping brain networks for spatial attention in memory and perception (Katus, Andersen, & Müller, 2014; Kuo, Rao, Lepsien, & Nobre, 2009; Munneke, Heslenfeld, & Theeuwes, 2010). In addition to comparing external and internal attention, the cued change detection paradigm can be extended to address other important theoretical issues regarding the commonalities and differences between feature- and location-based attention, and the structure of VWM representations. Feature-based attention is another important type of attention. Including feature-based attention in a cueing paradigm would provide important implications on theories of attention. Some theories of attention treat spatial location as having privileged status during attentional selection. For example, feature integration theory (Treisman, 1988; Treisman & Gelade, 1980) postulates that stimulus location plays a unique role in feature binding. Alternatively, other theories hold that stimulus location is just another feature, and there is no difference between location and nonspatial features in guiding attentional selection. For example, the feature similarity gain model of attention (Treue & Martínez-Trujillo, 1999) claims that the effects of attention are determined by feature similarity gain, without any fundamental difference between spatial and non-spatial feature selection. The goal of the present study was to compare feature- and location-based attention in the context of VWM. Specifically, feature- and location-based attentional selection were compared during VWM encoding and maintenance by examining

451

interactions between cue type and cue timing. It is known that the efficiency for different types of cues depends on stimulus properties (e.g., Zhuang & Papathomas, 2011). Although the difference between feature and location cue conditions may indicate theoretically important differences between feature- and location-based attention, it may also merely indicate the fact that a feature or location cue becomes more salient due to the particular feature values used in the experiment. Therefore, we focused on the interaction between cue type and cue timing in order to control for confounds introduced by stimulus properties. If the effect of cue type reflects cue efficiency specific to the properties of the current stimuli, then this effect of cue type would not differ between the pre- and retro-cue conditions. In contrast, if the effect of cue type is modulated by cue timing (i.e., the relationship between feature and location cues changes across preand retro-cue conditions), it is likely that the effects would be due to differential roles of feature- and location-based attention. Thus, an interaction between cue type and cue timing would eliminate the theoretically unimportant effects of perceptual cue efficiency and would unambiguously demonstrate a difference between feature- and location-based attention. The present design also allowed us to probe the structural properties of VWM. This was accomplished by manipulating object complexity in addition to cue type and cue timing. In previous studies (e.g., Griffin & Nobre, 2003), participants had been required to remember color–location conjunctions and were given location cues to access the color at the cued location. Cueing effects in such a task presuppose a connection between the cued dimension (location) and the other dimension (color) of the cued object. The stronger the connection between dimensions, the more efficient the cueing effect. In the case of color–location conjunction objects, because attention can only affect the color–location connection, differences between color- and locationcueing effects should not be due to VWM structure, but should reflect attentional modulation. However, adding another feature dimension (e.g., shape) enables inferences about any connection between the dimensions (i.e., color– location, shape–location, color–shape) within a mental representation. If these three dimensions are connected with equal strengths, the color and location cue effects should be the same. In contrast, if, for example, the color–shape connection is weaker than the others, then the color cue effect would be reduced because the color cue would only weakly influence shape memory. Thus, comparing cueing effects between simple and complex objects helps to differentiate the effects of attention and those of VWM structure. Effects of attention are expected to be invariant across object complexity conditions, but those due to VWM structure would be sensitive to object complexity.

452

Experiment 1 This experiment provided an initial comparison between feature- and location-based selection in VWM encoding and maintenance. To disambiguate differences due to perceptual efficiency of cue stimuli, the cue timing manipulation (precue and retro-cue) used in Griffin and Nobre (2003) was combined with a cue type manipulation (color and location cue). We also attempted to replicate previous findings of comparable effects in pre- and retro-cue conditions with location cues by using the same stimuli (color–location conjunction objects) used by Griffin and Nobre. Furthermore, as in previous studies (Griffin & Nobre, 2003, Exps. 2 and 3; Nobre et al., 2004), participants performed a binding-memory task, in which they were required to remember the color–location conjunctions in the memory array and detect a conjunction change between the memory and probe displays. Of note is that such a bindingmemory task can be defined in terms of detecting a color change at the same location or a location change of the same color between the memory and probe displays. In order to avoid an instruction bias to a particular dimension (color or location), our participants were explicitly instructed to memorize the color–location combinations in the memory array and to make a decision as to whether the color–location combination of the probe matched one of the combinations in the memory array. If the previous findings of similar attentional effects between VWM encoding and VWM maintenance can be generalized to feature-based selection, the difference between the color and location cue conditions would be invariant across different cue timings, indicated by the absence of an interaction between cue type and cue timing. However, a significant interaction between cue type and cue timing would indicate differential effects of location and color cues, independent of cue efficiency.

Method Participants Sixteen students from Kyoto University (18– 28 years of age; 8 men, 8 women) participated in the experiment after giving informed consent. All participants reported normal color vision and normal or corrected-to-normal visual acuity. The experiment protocol was approved by the Institutional Review Board of Kyoto University. Apparatus The experiment was conducted in a darkened testing room. Participants sat 57 cm away from a 21-in. CRT monitor (75-Hz refresh rate; 1,024 × 768 resolution), with their heads immobilized by a chinrest, forehead rest, and temple stabilizers. Visual stimuli were generated using Psychophysics Toolbox (Brainard, 1997; Pelli, 1997), implemented in MATLAB.

Atten Percept Psychophys (2015) 77:450–463

Stimuli The stimuli were presented on a gray background (21.6 cd/m2). The memory display consisted of four colored crosses, each subtending 0.78°, which were centered 3.82° above, below, to the left, and to the right of the central fixation point. We used four equiluminant colors (26.5 cd/m2): red, blue, green, and yellow. The combinations of color and location for each memory display were randomly selected without repetition. The probe display contained only one colored cross at each of the four possible locations. The color cues were colored rings (0.78° diameter) that predicted the color of the probe stimulus. The location cues were arrows (0.78° height and width) that predicted the location of the probe stimulus. The neutral cue was a plus sign (“+,” 0.78° height and width). Design and procedure The color and location cue conditions were presented in two separate experimental sessions (color session, location session). In each session, participants completed ten blocks of 32 trials (128 precue, 128 retro-cue, and 64 neutral trials). The trial sequence in each condition is shown in Fig. 1. Trials were self-initiated and commenced with the onset of a central fixation point. In all conditions, a trial consisted of a 267-ms fixation point, a 107-ms precue display, a 107-ms memory array, a 107-ms retro-cue display, and a 107-ms probe display. The blank interstimulus intervals (ISIs) were always 1,067 ms, except that the first ISI between the fixation point and the precue display was 667 ms. In the neutral condition (Fig. 1a), both the pre- and retro-cue displays consisted of a neutral cue. In the color precue condition (Fig. 1b, Precue), a color cue appeared in the precue display and a neutral cue appeared in the retro-cue display. In the color retro-cue condition (Fig. 1b, Retro-cue), the precue display consisted of a neutral cue and the retro-cue display consisted of a color cue. The color cues could be either valid (correctly predicting the probe’s color) or invalid (incorrectly predicting the probe’s color). For example, in Fig. 1b, a green cue is given, and when a green probe appears, the cue is valid; when the probe appears in other colors, such as red, the cue is invalid. Color cue validity was 50 %. That is, if a green cue was given, the probe would appear in green with 50 % probability, and appear in red, blue, or yellow with approximately 17 % probability for each color. In the location precue condition (Fig. 1c, Precue), a location cue was presented in the precue display and a neutral cue in the retro-cue display. In the location retro-cue condition (Fig. 1c, Retro-cue), the precue display had a neutral cue and the retro-cue display had a location cue. As with the color cues, both valid (correctly indicating the probe’s location) and invalid (incorrectly indicating the probe’s location) location cues were presented. For example, in Fig. 1c, because an upward arrow cue is given, the cue is valid when the probe appears above the central fixation but is invalid when the probe appears elsewhere, such as below the central fixation. The location cues were also 50 % valid. Thus, if an upward arrow

Atten Percept Psychophys (2015) 77:450–463

453

Fig. 1 Trial sequences for each condition of Experiment 1. (a) Neutral cue condition. (b) Pre- and retro-cue conditions in the color cue session. (c) Preand retro-cue conditions in the location cue session

cue was given, the probe would appear above the central fixation with 50 % probability, and appear below, to the left, or to the right of the central fixation with approximately 17 % probability for each location. Participants’ task was to remember the color– location combinations in the memory array and decided whether the color–location combination of the probe matched one of these combinations in the initial memory display by pressing the left or right button of the response box. In the example of a same trial shown in Fig. 1a, the memory array includes a yellow cross to the left of the central fixation point (yellow–left), and the probe display shows a yellow cross at the same location (yellow–left), the correct response for this trial is “same.” But in the example of a different trial, the probe display shows a green cross to the left of the central fixation point (green–left), and thus in this case, the correct answer is “different.” On each trial, the color–location combination of the probe matched or mismatched the initial memory display with equal probabilities (50 %). All conditions occurred in a random order within each session. Session order and the assignment of “same” and “different” values to the response buttons were counterbalanced between participants. At the beginning of each session, participants received both oral and written instructions and were given printed images describing the task, stimuli, and experimental conditions. They were instructed to emphasize accuracy rather than speed and were encouraged to use the color and location cues to improve performance. Before the experimental run of each session, the participants performed one or two practice blocks to ensure that they understood the task. Data analysis The sensitivity index d' (Green & Swets,1966)was the primary measure of the change detection tasks in the present study. The d' scores were calculated as d' = z(hit rate) – z(false alarm rate), where the hit rate was the proportion of different trials to which participants responded “different,” and the false alarm rate was the proportion of same trials to which participants responded “different.”

Reaction times (RTs) were also analyzed to check for speed–accuracy trade-offs. Only correct trials were used in the RT analysis. RTs that fell outside three standard deviations from the individual mean for every combination of condition were removed as outliers. Results and discussion The results for mean d' scores, mean hit rates, mean false alarm rates, and mean RTs are summarized in Table 1. Overall, mean d' scores were higher on valid trials and lower on invalid trials than on neutral trials, in both the color and location cue conditions. The neutral condition was used as a within-subjects and within-session baseline for determining the d' benefits conferred by valid cues (valid d'– neutral d') and the costs incurred by invalid cues (neutral d'– invalid d'). Figure 2a and b show the patterns of d' benefits and costs. A three-way repeated measures analysis of variance (ANOVA) with Cue Type (color, location), Cue Timing (pre, retro), and Effectiveness (benefit, cost) as within-subjects factors yielded a significant interaction between cue type and cue timing [F(1, 15) = 7.83, p = .014, ηp2 = .34]. Post-hoc tests revealed that the color cueing effect was significantly greater on precue trials than on retro-cue trials [F(1, 15) = 5.98, p = .027, ηp2 = .29], indicating that color precues were more efficient in modulating VWM performance than were color retro-cues. In contrast, the difference between location pre- and retro-cues did not reached significance (F < 1). Surprisingly, the effect of color precues was significantly greater than that of the location precues [F(1, 15) = 7.16, p = .017, ηp2 = .32], indicating that precueing the color of a to-be-attended representation affected VWM performance more than did precueing the location. The difference between color and location retro-cues did not approach significance (F < 1). These patterns of effects seemed to be more obvious in the cost than in the benefit data, which might be due to a ceiling effect on the valid-cue trials (all hit rates > 96 %). We

454

Atten Percept Psychophys (2015) 77:450–463

Table 1 Mean d' scores, mean hit and false alarm rates (as percentages), and mean RTs (in milliseconds) for each condition in Experiment 1 (standard errors are shown in parentheses)

Precue

Color

Location

Variable

Valid

Invalid

Valid

Invalid

Neutral

d'

3.92 (0.12)

2.11 (0.24)

3.70 (0.17)

2.41 (0.21)

3.12 (0.20)

Hits

97.07 (0.71)

87.70 (2.56)

96.09 (0.88)

90.23 (1.78)

93.55 (1.89)

False alarms

3.22 (0.68)

24.41 (3.82)

4.59 (1.30)

18.16 (3.04)

10.35 (2.18)

RTs d'

578 (46) 3.76 (0.14)

1,060 (118) 2.53 (0.20)

552 (47) 3.75 (0.13)

1,101 (119) 2.44 (0.18)

865 (79) 3.01 (0.22)

Hits

96.97 (0.50)

89.06 (2.23)

96.00 (0.95)

92.09 (1.77)

92.87 (1.61)

False alarms

4.79 (1.20)

13.96 (2.38)

3.41 (0.54)

20.70 (2.71)

10.84 (2.14)

RTs

560 (43)

1,023 (97)

549 (53)

1,059 (94)

864 (74)

found no effect of effectiveness or interaction between effectiveness and the other factors (all Fs < 1), indicating similar patterns of cueing effects in both the benefit and cost data. Although our instructions specifically emphasized accuracy, we analyzed RTs in order to detect whether there was any reversed pattern of interaction. Participants responded faster on valid and slower on invalid trials than on neutral trials. RT benefits (neutral RT– valid RT) and costs (invalid RT– neutral RT) were calculated and entered into a Cue Type × Cue Timing × Effectiveness ANOVA. Neither the two-way (Cue Type × Cue Timing) nor the three-way (Cue Type × Cue Timing × Effectiveness) interaction approached significance (Fs < 1), suggesting that the interaction observed in d' benefits and costs cannot be explained by speed–accuracy trade-offs.

a

1.4

d' benefit

1.2 1 0.8

Pre

0.6

Retro

0.4 0.2 0

b

Color

Locaon

1.4 1.2

d' cost

1 0.8

Pre

0.6

The significant interaction between cue type and cue timing in the d' analysis is clearly inconsistent with the hypothesis that the comparable effects of location-based selection on VWM encoding and VWM maintenance can be generalized to feature-based selection. However, it is important to note that previous findings with location cueing were successfully replicated, suggesting that the observed interaction is unlikely to be due to some peculiarity of the present experiment. Location cues generated similar effects in the pre- and retro-cue conditions, consistent with the findings of previous studies (Griffin & Nobre, 2003; Nobre et al., 2004) and with the hypothesis that common mechanisms underlie spatial attention in memory and perception (e.g., Katus et al., 2014). The color cue effect was stronger for precues than for retrocues, indicating that color information produced a greater effect during VWM encoding than during maintenance. Surprisingly, the effect of color precues was stronger than that of location precues, suggesting that color information affected VWM encoding even more than did location information. This pattern conflicts with the results of previous studies that have shown greater effects of location precues than of nonspatial feature precues in perceptual tasks (Posner, Snyder, & Davidson, 1980; Theeuwes, 1989; Theeuwes & Van der Burg, 2007). Although the results in the present experiment demonstrate profound differences between color- and location-based selection in modulating VWM processes, it was important to determine whether these patterns observed with color–location conjunction stimuli could extend to more complex stimuli, such as color–shape–location conjunction stimuli. General attentional effects should be invariant across object complexity conditions, whereas effects due to VWM structure should be sensitive to object complexity.

Retro

0.4 0.2 0

Retro-Cue

Experiment 2 Color

Locaon

Fig. 2 Experiment 1: Mean d' benefits (a) and costs (b) in the color and location cue conditions, separated according to the Cue Timing factor. Error bars represent standard errors

In Experiment 2, the generality of the findings from Experiment 1 was tested. The objects in this experiment were more complex and varied in shape as well as color. If the

Atten Percept Psychophys (2015) 77:450–463

effects observed in Experiment 1 were due to general attentional mechanisms, the results would not differ greatly between experiments, whereas if the effects were related to the structural properties of VWM representations, we would observe substantially different effects in Experiment 2. The effect of shape cues was also examined to determine whether other types of nonspatial cues produce effects similar to those of color cues. Method Unless stated otherwise, the method in Experiment 2 was identical to that of Experiment 1. Participants A group of 24 new participants (18–26 years of age; 13 men, 11 women) took part in the experiment. Stimuli Each memory display consisted of four differently colored novel shapes (Makovski et al., 2008; Shuman & Kanwisher, 2004) subtending 1.18°, which were centered 3.84° above, below, to the left, and to the right of the central fixation point. Four colors and four shapes were used. The stimulus properties (color, shape, location) for each item in a memory display were randomly selected without repetition. The probe display consisted of one colored shape at one of the four possible locations. The color and location cues were the same as those used in Experiment 1. Shape cues were outlines of the stimuli that predicted the shape of the probe. All cues subtended 1.18° in height and width. Design and procedure There were three types of attentional cues (color, shape, and location). These cue conditions were presented in three separate experimental sessions. Each session consisted of ten blocks of 24 trials (96 precue, 96 retrocue, and 48 neutral trials). As is shown in Fig. 3, the stimulus durations and ISIs were identical to those in Experiment 1, except that the memory arrays were presented for 267 ms to ensure adequate perceptual encoding time. The task was to determine whether the color–shape–location combinations of the probe matched one of the combinations in the initial memory display. On different trials, three kinds of change types were presented (Fig. 3a): color change (same combination of shape and location, but with a different color), shape change (same combination of color and location, but with a different shape), and location change (same combination of color and shape, but with a different location). These change types occurred with equal possibilities. Figure 3b, c, and d show examples for the color, shape, and location cue conditions, respectively. The examples of different trials in these figures are location change trials (Fig. 3b, c, and d, “Different”). Color and shape change trials were also presented in each case. For example, in Fig. 3b (color cue), if a red, star-like probe appeared at the bottom,

455

that would be a valid cue, color change trial; if it appeared at right, that would be a valid cue, shape change trials. Results and discussion The results for mean d' scores, mean hit rates, mean false alarm rates, and mean RTs are summarized in Table 2. In general, performance was better on valid and worse on invalid trials than on neutral trials for both pre- and retro-cues in all the three cue type conditions. Figure 4a and b show the patterns of d' benefits and costs in different cue type and cue timing conditions. As in Experiment 1, color and location cueing were compared using a three-way ANOVA with the factors Cue Type (color, location), Cue Timing (pre, retro), and Effectiveness (benefit, cost). We found a significant interaction between cue type and cue timing [F(1, 23) = 9.92, p = .005, ηp2 = .30]. Post-hoc tests revealed a significantly greater cueing effects for color precues than for color retro-cues [F(1, 23) = 11.34, p = .003, ηp2 = .33], but equivalent location pre- and retro-cues (F < 1). In addition, the color precue effect was greater than the location precue effect [F(1, 23) = 5.60, p = .027, ηp2 = .20]. These results were highly consistent with those of our first experiment. However, unlike the finding of equivalent color and location retro-cue effects in Experiment 1, color retro-cues had significantly weaker effects than location retro-cues, indicating that color cues were less efficient than location cues in biasing the maintenance of triple conjunction representations. As before, a three-way ANOVA was performed on RT benefits and costs in order to detect reversed interactions. Neither the Cue Type × Cue Timing interaction nor the Cue Type × Cue Timing × Effectiveness interaction was significant (Fs < 2.7). Thus, it is difficult to explain the d' data on the basis of speed–accuracy trade-offs. Shape cues were compared with location cues using a 2 (shape, location) × 2 (pre, retro) × 2 (benefit, cost) ANOVA. Neither the two-way nor the three-way interaction involving cue type and cue timing reached significance (Fs < 1). Thus, we found no evidence for the difference between shape and location cueing. The three-way ANOVA performed on RT benefits and costs also failed to reveal any differences between shape and location cueing. No significant main effects or interactions were found (Fs < 2.2). Regarding the comparison between color and location cue effects, most effects were invariant across Experiments 1 and 2, and one was significantly modulated by object complexity. First, the comparable effects of the pre- and retro-cue conditions for location cues were replicated in Experiment 2, suggesting that similar location-based selection in VWM encoding and maintenance can be attributed to a general spatial attention mechanism. Second, the finding of a larger effect of color precues than of both color retro-cues and location precues was also replicated. Therefore, the relatively

456

Atten Percept Psychophys (2015) 77:450–463

Fig. 3 Trial sequences for each condition of Experiment 2. (a) Neutral cue condition . (b) Preand retro-cue conditions in the color cue session. (c) Pre- and retro-cue conditions in the shape cue session. (d) Pre- and retro-cue conditions in the location cue session

higher efficiency of color precues likely reflects a general property of feature-based selection in VWM encoding. In contrast, one retro-cue effect was significantly modulated by object complexity: The effect of color retro-cues was significantly smaller than that of location retro-cues in Experiment 2, whereas their effects had been comparable in Experiment 1. The disadvantage of color retro-cues relative to location retrocues is specific to complex objects. As will be discussed in Table 2 Mean d' scores, mean hit and false alarm rates (as percentages), and mean RTs (in milliseconds) for each condition in Experiment 2 (standard errors are shown in parentheses)

more detail in the General Discussion, this sensitivity to complexity may indicate that the color–shape connection in VWM representations is substantially weaker than the connections of location and nonspatial features (i.e., location– color, location–shape). On the other hand, we found no significant difference between shape and location cues. This could be because the shape cues used in this study were less effective feature cues

Precue

Color

Shape

Location

Retro-Cue

Variable

Valid

Invalid

Valid

Invalid

Neutral

d'

3.29 (0.16)

1.67 (0.20)

2.52 (0.18)

1.66 (0.16)

2.15 (0.17)

Hits

90.68 (1.74)

82.70 (2.47)

88.14 (1.74)

83.10 (2.17)

85.65 (2.21)

False alarms

5.27 (1.17)

32.18 (4.79)

14.53 (2.49)

31.31 (4.30)

21.76 (3.68)

RTs d'

544 (28) 3.02 (0.15)

866 (73) 1.67 (0.18)

567 (33) 2.63 (0.17)

906 (69) 1.46 (0.14)

736 (50) 2.15 (0.14)

Hits

93.75 (0.94)

81.66 (2.45)

91.96 (1.28)

78.94 (2.26)

87.56 (1.65)

False alarms

11.17 (1.61)

28.82 (4.10)

16.84 (2.58)

29.28 (3.18)

21.35 (2.83)

RTs d'

567 (33) 2.86 (0.16)

926 (63) 1.66 (0.16)

556 (37) 2.63 (0.17)

931 (62) 1.42 (0.14)

735 (41) 1.95 (0.16)

Hits

91.38 (1.40)

83.39 (2.59)

89.24 (1.80)

80.90 (2.20)

84.32 (2.09)

False alarms

11.23 (1.71)

30.67 (2.97)

13.14 (2.24)

33.33 (2.82)

22.74 (2.29)

RTs

490 (24)

878 (72)

483 (23)

897 (66)

698 (38)

Atten Percept Psychophys (2015) 77:450–463

a

457

1.4

d' benefit

1.2 1 0.8

Pre Retro

0.6 0.4 0.2 0

b

Color

Shape

Locaon

1.4 1.2

d' cost

1 0.8

Pre Retro

0.6 0.4 0.2 0

Color

Shape

Locaon

Fig. 4 Experiment 2: Mean d' benefits (a) and costs (b) in the color, shape, and location cue conditions, separated according to the Cue Timing factor. Error bars represent standard errors

modulations of processes related to VWM encoding rather than modulations of sensation and perception. However, before reaching this conclusion, it was necessary to rule out the possibility that the stimuli used in Experiments 1 and 2 might in general produce larger color precue effects. This could be tested by assessing the effectiveness of color and location precues in a perceptual task using the same stimuli that were used in the VWM task. As such, in Experiment 3, participants performed a visual search task using the same stimuli as in Experiment 2. In the visual search task, participants were required to identify the items in the array, but were not required to remember them. Color or location cues were presented before the array in order to guide attentional selection. If the cueing effect was smaller for color than for location cues, as previous studies have reported, we could rule out the possibility that the larger color than location precue effect in the VWM task occurred because the stimuli that we used had a tendency to produce larger color precue effects. This would support the idea that the larger precue effects in our pervious experiments reflect greater modulation of VWM encoding than of stimulus sensation and perception. Method

than were the color cues. Note that the shapes used in the present experiment were unfamiliar nonsense shapes. Outline contour cues can be more difficult to use than color cues, especially with a time limit. The lower cue efficiency of the shape cue could consequently prevent it from revealing the difference between feature- and location-based selection. Further studies with better controlled stimuli would be necessary to resolve the issue of whether color-cue effects can be generalizable to other nonspatial features, including shape.

Unless stated otherwise, the method in Experiment 3 was identical to that of Experiment 2. Participants Sixteen new participants (19–26 years of age; 12 men, 4 women) took part in this experiment.

Experiment 3

Stimuli The stimuli were generally the same as in Experiment 2. Because stimulus shapes were used to designate the target in the search task, four additional irregular shapes were added in order to provide sufficient choices on target-present and target-absent trials.

The observation of a specific, strong color precue effect in the first two experiments seems inconsistent with many previous studies that have reported bigger effects for location cues than for feature cues (Posner et al., 1980; Theeuwes, 1989; Theeuwes & Van der Burg, 2007). However, of note is that all of these previous studies used perceptual tasks, such as target detection and visual search, unlike Experiments 1 and 2, in which a VWM task was used. Indeed, it is generally thought that precues operate in a similar manner—by modulating sensory and perceptual processing—in VWM and perceptual tasks. However, to succeed in a VWM task, participants have to first perceive the visual stimuli and then transform them into durable memory representations. Thus, precues in VWM tasks may influence not only the initial sensory and perceptual processes, but also the memory-encoding processes. If this is true, the finding of a stronger effect of color than of location precues in our first two experiments might reflect attentional

Design and procedure Two types of cues were used: color and location. The color and location cue conditions were presented in two separate experimental sessions. In each session, 16 blocks of 36 trials were presented (a total of 576 trials: 384 precue and 192 neutral trials). As is shown in Fig. 5, a target shape, randomly selected from eight shapes, was presented in black at the start of each 18-trial block. Trials were self-initiated. Each trial started with a 267-ms fixation point. After a 667-ms ISI, either an attentional cue (color or location) or a neutral cue (plus sign) was presented for 107 ms. This was followed by a 1,067-ms ISI. Then a search display consisting of four different color shapes was presented until response. Participants were required to make a speeded response to the search array. They were instructed to respond “present” when they found a stimulus that matched the target shape, regardless of its color and location, and to respond “absent” if the target shape was not

458

Atten Percept Psychophys (2015) 77:450–463

a

120

RT benefit (ms)

100 80 60 40 20 0

Fig. 5 Schematic of the visual search task in Experiment 3

b

Data analysis Because it was not possible to define cue validity on target-absent trials, only target-present trials were used in the analyses of this experiment. RT was the primary measure of the speeded visual search task, and only the RTs from correctly performed trials were included. RTs that fell outside three standard deviations from the individual mean for every combination of conditions were removed as outliers. Results and discussion Table 3 shows the results for mean RTs with valid and invalid cues on target-present trials. Overall, RTs were shorter on valid and longer on invalid trials than on neutral trials in both the color and location cue conditions. RT benefits and costs were calculated (Fig. 6) and submitted to a 2 (color, location) × 2 (benefit, cost) within-subjects ANOVA. Only the main Table 3 Mean RTs (in milliseconds) and mean hit rates (as percentages) on target-present trials in Experiment 3 (standard errors are shown in parentheses) Precue

Color Location

Variable

Valid

Invalid

Neutral

RTs Hits RTs Hits

525 (25) 98.70 (0.29) 505 (34) 99.09 (0.35)

621 (36) 96.55 (0.55) 678 (42) 94.86 (0.64)

551 (25) 97.98 (0.55) 598 (35) 97.33 (0.57)

Locaon

Color

Locaon

120 100

RT cost (ms)

presented in the search display. Thus, only the shape dimension was task-relevant, whereas both the color and location dimensions were task-irrelevant. Targets appeared on 50 % of the trials. Only target-present trials were of interest, and targetabsent trials were included to regulate the rate of anticipation responses. All conditions were presented in a random order in each session. The session order and the assignment of “present” and “absent” values to the response buttons were counterbalanced among participants.

Color

80 60 40 20 0

Fig. 6 Experiment 3: Mean RT benefits and costs in the color and location cue conditions. Error bars represent standard errors

effect of cue type was significant [F(1, 15) = 8.32, p = .011, ηp2 = .36], indicating significantly weaker effects for color than for location cues. Because there was no way to define cue validity on targetabsent trials, we were not able to calculate valid and invalid d'scores. Thus, only the hit rates (the proportions of targetpresent trials to which participants responded “present”) for valid and invalid cues are reported (Table 3). Mean hit rates were uniformly high in all conditions (>94 %). Hit rate benefits (valid hit rate – neutral hit rate) and costs (neutral hit rate– invalid hit rate) were calculated and analyzed in a Cue Type (color, location) × Effectiveness (benefit, cost) ANOVA. Only the main effect of effectiveness was significant [F(1, 15) = 37.18, p < .001, ηp2 = .71], indicating more cueing benefits than costs. We found no significant main effect of cue type or interaction between the factors (Fs < 1), indicating equivalent effects for color and location cues. The lack of a difference between color and location cues could be due to a ceiling effect in accuracy. The results in the present experiment are consistent with previous studies showing that advanced knowledge of target location affected visual selection more than advanced knowledge of target features (Posner et al., 1980; Theeuwes, 1989; Theeuwes & Van der Burg, 2007). Because the stimuli used in this experiment were almost identical to those used in

Atten Percept Psychophys (2015) 77:450–463

Experiment 2, the different patterns of cueing effects between Experiments 2 and 3 were unlikely to be due to a stimulus peculiarity. Consequently, together with the results of Experiments 1 and 2, the present results indicate that the advantage of color over location precues in VWM tasks might mainly reflect greater modulation of processes related to VWM encoding, rather than sensory and perceptual processes. Although the present experiment has helped to clarify that color precues play a special role in biasing VWM encoding, it remains unclear what information encoding is biased to a larger extent by the color precues in the VWM task. Because the color and location cues in Experiments 1 and 2 can influence both individual features and their bindings, and the task actually tested memory for binding information, cueing effects in such a task should mainly reflect attentional modulation of feature bindings. This leaves open the possibility that the greater color precue effect was binding-memory-specific. We tested this in Experiment 4 by conducting a change detection task only requiring memory for individual features, but not requiring memory for feature bindings.

Experiment 4 Experiments 1 and 2 consistently revealed a greater effect of color precues than of either location precues or color retrocues. But since these experiments tested binding memory, it was important to clarify whether this effect was bindingmemory-specific. This would provide more insights into the role of color- and location-based selection in VWM. Experiment 4 was conducted to address this issue by adopting a change detection task that only required memory for individual features, but not memory for feature bindings. More specifically, in the present experiment, a change always consisted of new feature values (new color, new location, or both new color and new location). Thus, unlike our first two experiments, in which accurate binding memory was necessary to succeed in performance, successful performance in this experiment only required accurate memory for individual features (color and location). If attention plays a role in feature memory similar to its role in binding memory, the greater color precue effect should be replicated. However, if the greater color precue effect is binding-memory-specific, it would disappear in the feature memory task. Method The method of Experiment 3 was identical to Experiment 1, with the following exceptions. Participants A group of 24 new participants (18–26 years of age; 12 men, 12 women) took part in the experiment.

459

Stimuli The stimuli were presented on a gray background (28.6 cd/m2). The memory displays consisted of four squares (0.78° height and width) with their centers lying on an imaginary circle of radius 5.5°. The locations of the squares were randomly selected from eight fixed positions spaced equally on the circumference of a circle. The colors of the squares were assigned randomly without replacement from a set of eight colors: red, green, blue, yellow, pink, purple, orange, and brown. On the test displays, only one square was presented. Design and procedure In all, 15 blocks of 24 trials (144 precue, 144 retro-cue, and 72 neutral trials) were presented in both the color and location sessions. The sequence of events, stimulus duration, and ISI were identical to those used in Experiment 1 (Fig. 7). But unlike in Experiment 1, changes at test always occurred with new feature values (new color, new location, or both a new color and new location). The task was to decide whether a new color or location appeared in the probe display. As is shown in Fig. 7a, on a new-color trial, a new color (brown) appeared at an old location (left) at test; on a new-location trial, an old color (blue) appeared at a new location (lower right) at test; on a both-new trial, both the color and location of the probe were selected from the remaining values that had not been used in the initial memory display (yellow, upper left). Participants were informed in detail about these change types. Results and discussion Mean d' scores, mean hit rates, mean false alarm rates, and mean RTs are summarized in Table 4. Higher mean d' scores on valid and lower d' scores on invalid trials, relative to neutral trials, were observed in both the color and location cue conditions. Overall, performance was better on valid and worse on invalid trials than on neutral trials in both the color and location cue conditions. Figure 8 shows the pattern of d' benefits and costs. As before, a three-way ANOVA with Cue Type (color, location), Cue Timing (pre, retro), and Effectiveness (benefit, cost) as within-subjects factors was performed. Only a significant main effect of cue timing was observed [F(1, 23) = 5.38, p = .030, ηp2 = .19], indicating significantly greater effects for precues than for retro-cues. Thus, in the feature memory task, even the location cue condition showed a greater effect for precues than for retrocues, though in the binding task, location pre- and retro-cue effects were almost equivalent. However, this finding must be interpreted with caution. In Experiments 1 and 2, both location pre- and retro-cues had selected one location from four possible locations. However, in the present experiment, precues served to select one location from eight possible locations, because participants did not know where the memory stimuli would appear before the memory array onset. But retro-cues were given to the participants after they had viewed the

460

Atten Percept Psychophys (2015) 77:450–463

Fig. 7 Trial sequences for each condition of Experiment 4. (a) Neutral cue condition . (b) Pre- and retro-cue conditions in the color cue session. (c) Preand retro-cue conditions in the location cue session

memory array, and attention might have been narrowed to the four locations occupied by the memory stimuli before the retrocue display. Thus, retro-cues served to select one location from four possible locations. This difference in spatial probability might have affected pre- and retro-cue efficiency. But most importantly, neither the Cue Type × Cue Timing nor the Cue Type × Cue Timing × Effectiveness interaction was significant (Fs < 1). A three-way ANOVA performed on RT benefits and costs also revealed no significant interactions between the factors (Fs < 1.26). Thus, we did not find any evidence for a difference between color and location cueing in the feature memory task. This suggests that the greater color versus location precue effect was very likely binding-memory-specific.

General discussion In the present study, the relationship between attentional selection of perceptual and VWM representations for both spatial and nonspatial features was examined by investigating interactions between cue type (location, feature) and cue timing (precue, retro-cue) in a VWM task. There were four major findings. First, we found a significant interaction between cue type and cue timing in both Experiments 1 and 2. Table 4 Mean d' scores, mean hit and false alarm rates (as percentages), and mean RTs (in milliseconds) for each condition in Experiment 4 (standard errors are shown in parentheses)

Specifically, color precues had greater effects than did color retro-cues, whereas the location pre- and retro-cues had equivalent effects. This suggests that attentional modulations of memory encoding and maintenance share a common mechanism when the modulation is mediated via location, but when the modulation is mediated via color, it is more effective during memory encoding than during maintenance. Moreover, the color precue effect was always greater than the location precue effect. This finding is inconsistent with attention theories that postulate a privileged role of locationbased selection. Second, color retro-cues had a significantly smaller effect than did location retro-cues in Experiment 2, in which the stimuli were triple-conjunction objects. This may reflect structural properties of VWM representations, as discussed below. Third, in Experiment 3, the relationship between location and color cue effects in the visual search task was opposite to that observed in Experiments 1 and 2, suggesting that the precue effects observed in Experiments 1 and 2 reflect modulation not only of sensory and perceptual processing but also of VWM encoding processing, and that color-based selection is qualitatively different in these two cases. Finally, Experiment 4 revealed that the greater color than location precue effect is specific to binding memory, and therefore suggests that the encoding of binding information

Precue

Color

Location

Retro-Cue

Variable

Valid

Invalid

Valid

Invalid

Neutral

d'

3.69 (0.17)

2.45 (0.17)

3.49 (0.16)

2.40 (0.14)

2.82 (0.14)

Hits

96.12 (0.85)

94.39 (1.56)

96.18 (0.86)

95.72 (0.85)

95.31 (1.17)

False alarms

3.99 (0.83)

31.42 (4.08)

6.25 (1.20)

30.15 (3.40)

17.07 (2.35)

RTs d'

514 (28) 3.72 (0.14)

645 (27) 2.37 (0.21)

481 (32) 3.35 (0.18)

674 (35) 2.32 (0.18)

620 (30) 2.97 (0.16)

Hits

96.76 (0.66)

94.50 (0.86)

94.73 (1.01)

95.25 (0.97)

95.31 (1.43)

False alarms

5.32 (1.08)

34.38 (4.71)

6.48 (1.18)

32.87 (4.60)

16.49 (2.64)

RTs

544 (32)

665 (38)

496 (31)

684 (43)

639 (37)

Atten Percept Psychophys (2015) 77:450–463

a

461

1.4

d' benefit

1.2 1 0.8

Pre

0.6

Retro

0.4 0.2 0

b

Color

Locaon

1.4 1.2

d' cost

1 0.8

Pre

0.6

Retro

0.4 0.2 0

Color

Locaon

Fig. 8 Experiment 4: Mean d' benefits (a) and costs (b) in the color and location cue conditions, separated according to the Cue Timing factor. Error bars represent standard errors

rather than individual feature information is modulated to a greater degree by color precues than by location precues. These findings revealed important differences between color- and location-based selection during perceptual processing, VWM encoding, and VWM maintenance. However, the particular pattern of results that we obtained was difficult to predict beforehand using current theories, and some of the results were quite surprising. In the following sections, we attempt to interpret the major findings of the present study by considering other related work. Why does color-based selection become efficient in VWM encoding? Our results show that during perceptual processing (Exp. 3) and VWM maintenance (Exps. 1 and 2, retro-cue conditions), location cues were more effective than color cues, or they were at least comparable, but during VWM encoding (Exps. 1 and 2, precue conditions), color cues were more effective than location cues. One possible account for the greater color versus location precue effect is that location cue effects are mainly mediated by facilitation via cued location, whereas color cue effects are mediated both by facilitation via cued color, and by suppression via uncued color. This account is consistent with previous studies showing that feature-based attentional modulation reflects a combination of boosting sensory gain and sharpening neuronal tuning, whereas location-based attention operates mainly by boosting sensory gain (Ling, Liu, & Carrasco, 2009; Treue & Martínez-Trujillo, 1999). An important question is why this suppression effect was only evident in the

precue condition. This could be due to the time course of facilitation and suppression effects. For example, facilitation effects may occur immediately, whereas suppression effects may take time to emerge. Since there was a long SOA between cue onset and memory array onset (1,174 ms) in the precue condition, the suppression mechanism may have been functioning at the time of memory array onset. In contrast, in the retro-cue condition, because the representations of the memory array have already been held in VWM, the selection mechanism can begin at the time of cue onset, whereas the suppression mechanism may not have time to begin functioning. This timing proposal can be tested by manipulating the SOA between the precue and the memory array, and if it is right, the color precue advantage will be reduced with reduced SOA. The observation that color precues had a greater effect than location precues might also be consistent with an alternative account that assumes encoding the sequence color–location (color precue condition) is easier than encoding the sequence location–color (location precue condition). But why should that be the case? Perhaps encoding color is less efficient than encoding location, leading to participants relying on color cues to a greater extent than location cues. Thus, although at first glance our data seem to show an advantage of color precues over location precues, the data may indeed reflect an encoding advantage of location over color. Alternatively, color precues might be more efficient than location precues because the exact color values were presented as color cues, whereas location cues were indirect arrow cues. The greater color versus location precue effect might reflect an advantage of direct cueing over indirect cueing during VWM encoding. However, the three possible accounts discussed above cannot explain why a different pattern of results was observed in the perception task in Experiment 3. This is discussed below. Why is color-based selection during VWM maintenance disrupted for complex objects? Another novel finding in the present experiment was that cueing effects were significantly smaller for color than location retro-cues, but only when complex objects were used (Exp. 2). With shape-invariant stimuli (Exp. 1), the retro-cue effect was comparable between location and color cue conditions. Structural features of VWM representations may explain these results. Because retro-cue effects in Experiments 1 and 2 reflect biased feature bindings, the magnitude of these effects may be a function of the connection strength between features within a VWM representation. In the case of color–location conjunction stimuli in Experiment 1, color and location retro-cues can only bias VWM maintenance via the color–location connection. Therefore, it is reasonable that color and location bias each other to a comparable degree, leading to comparable retro-cue effects. However, in the case of the color–shape–

462

location stimuli in Experiment 2, the color retro-cue effect was significantly weaker than location retro-cue effect. This is clearly incompatible with a simple model in which all three feature dimensions are connected with equal strength; this model would predict comparable retro-cue effects. A simple explanation for the observed results is that the color–shape connection is substantially weaker than the other connections within a VWM representation. Accordingly, there are strong location–color and location–shape connections, leading to a substantial biasing benefit for location retro-cues. In contrast, although color–location connections may be strong, color– shape connections are weak, leading to a reduced benefit for color retro-cues. Some recent studies have suggested a location-based organization of VWM representations (Fougnie & Alvarez, 2011; Kondo & Saiki, 2012). Fougnie and Alvarez used a continuous report task to investigate the precision of color and orientation memories. Their results showed that errors in color and orientation reports are independent, suggesting that memory for color and orientation are not tightly bound. Kondo and Saiki asked participants to ignore one feature dimension (color, shape, or location) in a binding memory task with objects defined by color, shape, and location. They found that performance was significantly reduced when participants were asked to ignore location—that is, exclusively remember color–shape combinations. Although our data can be interpreted very well on the basis of the location-based organization of VWM representations, the results of our experiments can only provide indirect support for this hypothesis because our experiments did not directly test feature connections within VWM representations. Perhaps one important direction for future research will be to directly investigate the connection strength between spatial and nonspatial features within a VWM representation. Why do cueing effects differ between perceptual processing and VWM encoding? A comparison of Experiments 2 and 3 revealed that the relative effectiveness of location and color cues is reversed during perceptual processing versus VWM encoding. That is, the advantage for color cues was only observed in the VWM task, not in the perceptual task. Because both experiments used a precue design, the featurebased mechanism should have had enough time to engage suppression. Note that there were at least two critical differences between the binding-memory task and the visual search task. The first difference was whether processing of uncued objects was required. In the binding-memory task, even uncued objects were processed, because all items needed be encoded and maintained. In contrast, in the search task, uncued object might not be fully processed, because search terminates when the cued object is the target. The second difference was whether binding was tested. In the bindingmemory task, participants were tested on feature bindings, and cueing effects mainly reflected modulated sensitivity to

Atten Percept Psychophys (2015) 77:450–463

binding changes. But in the visual search task, participants searched for a particular shape, and cueing benefits in this case mainly reflected improved sensory processing sensitivity to the target shape. These differences in task requirements might induce a reversal of the relative effectiveness of color cues. Moreover, structural difference between the color–shape and location–shape perceptual associations might create an advantage for location cues in the search task. As with VWM maintenance, if the color–shape association was significantly weaker than the location–shape association, color cues should be less efficient than location cues when the search was based on target shape. Indeed, feature integration theory postulates this structural difference, claiming that color–shape binding is mediated by location information (Treisman, 1988). One important question for future research is whether the color cue advantage in VWM encoding is inherently attributable to the memory process. In other words, can the color cue advantage be obtained in a perceptual task? Feature-based attention studies showing feature-tuning modulation (Ling et al., 2009; Treue & Martínez-Trujillo, 1999) suggest that it should be. The visual search task in the present study was likely unsuitable for demonstrating such an effect, because participants could not be forced to process all items in speeded tasks, and the cue–target associations might not have been properly controlled between the color and location cue conditions. Systematically investigating this question will be informative for a theoretical understanding of attention, perception, and VWM mechanisms. Conclusion In the present study, the cued change detection paradigm, which had previously exclusively used location cues, was extended to explore the effects of feature-based and location-based selection on VWM processes. Our results showed that color-based and location-based selection in VWM encoding and maintenance differ in important ways. Specifically, color cues are likely to play a spatial role in modulating the encoding of feature bindings. In addition, we found that the relationship between color and location retro-cues were sensitive to object complexity. This finding has potentially important implications for the structural properties of VWM representations.

References Averbach, E., & Coriell, A. S. (1961). Short-term memory in vision. Bell System Technical Journal, 40, 309–328. Baddeley, A. D., & Hitch, G. J. (1974). Working memory. In G. H. Bower (Ed.), The psychology of learning and motivation: Advances in research and theory (Vol. 8, pp. 47–89). New York: Academic Press. Becker, M. W., Pashler, H., & Anstis, S. M. (2000). The role of iconic memory in change-detection tasks. Perception, 29, 273–286. doi:10. 1068/p3035

Atten Percept Psychophys (2015) 77:450–463 Berryhill, M. E., Richmond, L. L., Shay, C. S., & Olson, I. R. (2011). Shifting attention among working memory representations: Testing cue type, awareness, and strategic control. Quarterly Journal of Experimental Psychology, 65, 426–438. doi:10.1080/17470218. 2011.604786 Brainard, D. H. (1997). The Psychophysics Toolbox. Spatial Vision, 10, 433–436. doi:10.1163/156856897X00357 Chun, M. M., Golomb, J. D., & Turk-Browne, N. B. (2011). A taxonomy of external and internal attention. Annual Review of Psychology, 62, 73–101. doi:10.1146/annurev.psych.093008.100427 Fougnie, D., & Alvarez, G. A. (2011). Object features fail independently in visual working memory: Evidence for a probabilistic feature-store model. Journal of Vision, 11(12), 3:1–12. doi:10.1167/11.12.3 Green, D. M., & Swets, J. A. (1966). Signal detection theory and psychophysics. Huntington, NY: Robert E. Krieger Publishing Company Griffin, I. C., & Nobre, A. C. (2003). Orienting attention to locations in internal representations. Journal of Cognitive Neuroscience, 15, 1176–1194. doi:10.1162/089892903322598139 Katus, T., Andersen, S. K., & Müller, M. M. (2014). Common mechanisms of spatial attention in memory and perception: A tactile dual-task study. Cerebral Cortex, 24, 707–718. doi:10.1093/cercor/bhs350 Kondo, A., & Saiki, J. (2012). Feature-specific encoding flexibility in visual working memory. PLoS ONE, 7(e50962), 1–8. doi:10.1371/ journal.pone.0050962 Kuo, B.-C., Rao, A., Lepsien, J., & Nobre, A. C. (2009). Searching for targets within the spatial layout of visual short-term memory. Journal of Neuroscience, 29, 8032–8038. doi:10.1523/ JNEUROSCI.0952-09.2009 Landman, R., Spekreijse, H., & Lamme, V. A. F. (2003). Large capacity storage of integrated objects before change blindness. Vision Research, 43, 149–164. doi:10.1016/S0042-6989(02)00402-9 Ling, S., Liu, T., & Carrasco, M. (2009). How spatial and feature-based attention affect the gain and tuning of population responses. Vision Research, 49, 1194–1204. Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working memory for features and conjunctions. Nature, 390, 279–281. doi: 10.1038/36846 Makovski, T., Sussman, R., & Jiang, Y. V. (2008). Orienting attention in visual working memory reduces interference from memory probes. Journal of Experimental Psychology: Learning, Memory, and Cognition, 34, 369–380. doi:10.1037/0278-7393.34.2.369 Matsukura, M., Luck, S. J., & Vecera, S. P. (2007). Attention effects during visual short-term memory maintenance: Protection or prioritization? Perception & Psychophysics, 69, 1422–1434. doi:10. 3758/BF03192957 Munneke, J., Heslenfeld, D. J., & Theeuwes, J. (2010). Spatial working memory effects in early visual cortex. Brain and Cognition, 72, 368–377. doi:10.1016/j.bandc.2009.11.001 Nobre, A. C., Coull, J. T., Maquet, P., Frith, C. D., Vandenberghe, R., & Mesulam, M. M. (2004). Orienting attention to locations in perceptual versus mental representations. Journal of Cognitive Neuroscience, 16, 363–373. Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision, 10, 437–442. doi:10.1163/156856897X00366

463 Phillips, W. A. (1974). On the distinction between sensory storage and short-term visual memory. Perception & Psychophysics, 16, 283– 290. doi:10.3758/BF03203943 Posner, M. I., Snyder, C. R., & Davidson, B. J. (1980). Attention and the detection of signals. Journal of Experimental Psychology: General, 109, 160–174. doi:10.1037/0096-3445.109.2.160 Rensink, R. A., O’Regan, J. K., & Clark, J. J. (1997). To see or not to see: The need for attention to perceive changes in scenes. Psychological Science, 8, 368–373. doi:10.1111/j.1467-9280.1997.tb00427.x Schmidt, B. K., Vogel, E. K., Woodman, G. F., & Luck, S. J. (2002). Voluntary and automatic attentional control of visual working memory. Perception & Psychophysics, 64, 754–763. doi:10.3758/ BF03194742 Scholl, B. J. (2000). Attenuated change blindness for exogenously attended items in a flicker paradigm. Visual Cognition, 7, 377–396. Shuman, M., & Kanwisher, N. (2004). Numerical magnitude in the human parietal lobe: Tests of representational generality and domain specificity. Neuron, 44, 557–569. Sperling, G. (1960). The information available in brief visual presentations. Psychological Monographs: General and Applied, 74, 1–29. 11, Whole No. 498. Tanoue, R. T., & Berryhill, M. E. (2012). The mental wormhole: Internal attention shifts without regard for distance. Attention, Perception, & Psychophysics, 74, 1199–1215. doi:10.3758/s13414-012-0305-0 Theeuwes, J. (1989). Effects of location and form cuing on the allocation of attention in the visual field. Acta Psychologica, 72, 177–192. doi: 10.1016/0001-6918(89)90043-7 Theeuwes, J., & Van der Burg, E. (2007). The role of spatial and nonspatial information in visual selection. Journal of Experimental Psychology: Human Perception and Performance, 33, 1335–1351. doi:10.1037/0096-1523.33.6.1335 Treisman, A. (1988). Features and objects: The Fourteenth Bartlett Memorial Lecture. Quarterly Journal of Experimental Psychology, 40A, 201–237. doi:10.1080/02724988843000104 Treisman, A. M., & Gelade, G. (1980). A feature-integration theory of attention. Cognitive Psychology, 12, 97–136. doi:10.1016/00100285(80)90005-5 Treue, S., & Martínez-Trujillo, J. C. (1999). Feature-based attention influences motion processing gain in macaque visual cortex. Nature, 399, 575–579. doi:10.1038/21176 Vogel, E. K., Woodman, G. F., & Luck, S. J. (2001). Storage of features, conjunctions, and objects in visual working memory. Journal of Experimental Psychology: Human Perception and Performance, 27, 92–114. doi:10.1037/0096-1523.27.1.92 Wheeler, M. E., & Treisman, A. M. (2002). Binding in short-term visual memory. Journal of Experimental Psychology: General, 131, 48– 64. doi:10.1037/0096-3445.131.1.48 Woodman, G. F., Vecera, S. P., & Luck, S. J. (2003). Perceptual o r g a n i z a t i o n i n f l u e n c e s v i s u a l w o r k i n g m e m o r y. Psychonomic Bulletin & Review, 10, 80–87. doi:10.3758/ BF03196470 Zhuang, X., & Papathomas, T. V. (2011). Cue relevance effects in conjunctive visual search: Cueing for location, color, and orientation. Journal of Vision, 11(7), 6:1–13. doi:10.1167/11.7.6