Psychological Research DOI 10.1007/s00426-014-0545-9
ORIGINAL ARTICLE
Dissociable effects of auditory attention switching and stimulus–response compatibility Vera Lawo • Iring Koch
Received: 18 June 2013 / Accepted: 28 January 2014 Ó Springer-Verlag Berlin Heidelberg 2014
Abstract Using a task-switching variant of dichotic listening, we examined the ability to intentionally switch auditory attention between two speakers. We specifically focused on possible interactions with stimulus–response compatibility. In each trial, two words, one spoken by a male and another by a female, were presented dichotically via headphones. In one experimental group, two animal names were presented, and the relevant animal had to be judged as smaller or larger than a sheep by pressing a left or right response key. In another group, two number words were presented and had to be judged as smaller or larger than 5. In each trial, a visual cue indicated the gender of the relevant speaker. Performance was worse when the gender of the relevant speaker switched from trial to trial. These switch costs were larger for animal names than for number words, suggesting stronger interference with slower access to semantic categories. Responses were slower if the side of the target stimulus (as defined by the relevant gender) was spatially incompatible with the required response (as defined by the size judgment). This stimulus–response compatibility effect did not differ across stimulus material and did not interact with attentional switch costs. These results indicate that auditory switch costs and stimulus– response compatibility effects are dissociable, referring to target selection and response selection, respectively.
V. Lawo (&) I. Koch Institute of Psychology, RWTH Aachen University, Ja¨gerstr. 17-19, 52066 Aachen, Germany e-mail: [email protected]
Introduction The conversation at a cocktail party is one of the best reallife examples of auditory selective attention (Pashler, 1998), as a variety of acoustic information is simultaneously available for auditory processing, but only certain information is relevant at once. In such situations, selective attention enables us to extract and attend to this relevant information while ignoring the irrelevant information (e.g., Dalton, Santangelo, & Spence, 2009; Lachter, Forster, & Ruthruff, 2004; see Shinn-Cunningham, 2008, for a review). Early studies on selective attention used dichotic-listening procedures to examine the underlying mechanisms of this selection (Broadbent, 1958; Cherry, 1953; Moray, 1959). In dichotic listening, participants attend to information presented to one ear, while ignoring information presented simultaneously to the other ear. In these traditional studies, attention switches were not instructed, but could occur whenever the task-irrelevant auditory information was nevertheless (i.e., involuntarily) attended (Lachter et al., 2004; see Shinn-Cunningham, 2008, for a discussion). To examine intentional switching of auditory attention, Koch, Lawo, Fels, and Vorla¨nder (2011) introduced a paradigm that combined the methodologies of dichotic listening and task cuing (Meiran, 1996), which is a basic variant of the task-switching paradigm. In the task-cuing paradigm, a cue precedes the stimulus to indicate the upcoming task. Typically, performance costs, so-called switch costs, occur in task switches compared to task repetitions, (see, e.g., Kiesel, Steinhauser, Wendt, Falkenstein, Jost, Philipp, & Koch, 2010, for a review). In this initial auditory task-switching study by Koch et al. (2011), two spoken number words, spoken by a
123
Psychological Research
female and a male speaker, were presented dichotically via headphones. Participants categorized the relevant number word as smaller or larger than five. A visual cue that preceded the auditory stimuli indicated whether the number word spoken by the female or male speaker was relevant in the upcoming trial. The main finding was that a cued switch of the to-beattended speaker (as defined by gender) resulted in attention switch costs (see also Koch & Lawo, 2014; Lawo & Koch, 2012). These switch costs may be explained either by an active process that reconfigures stimulus selection criteria (‘‘attentional filter settings’’) in auditory perception or by inertia of the filter settings, as previously relevant filter settings can proactively interfere with the new filter setting (see Koch et al., 2011, for a discussion). Since it has been shown that immediate repetition of visual cues can result in priming effects at the level of perceptual cue encoding, which, in turn, can contribute to switch costs (e.g., Arrington, Logan, & Schneider, 2007; Mayr & Kliegl, 2003), Koch et al. (2011) used two cues for each gender to disentangle auditory switch costs from visual cue priming effects. Even though cuerepetition priming effects contributed to switch costs, substantial switch costs (‘‘pure’’ auditory attention switch costs; Monsell & Mizon, 2006) remained even if cue priming was excluded (see Koch & Lawo, 2014, for a discussion). In this study, we focused on these ‘‘pure’’ auditory attention switch costs by excluding any influence of visual cue repetitions. We used two cues for each gender and designed the procedure in a way that direct cue repetitions in consecutive trials could not occur. Therefore, the cue switched on all trials, but on some trials the switched cue indicated the same gender as being relevant, whereas on other trials the switched cue indicated a gender switch. The primary aim of this study was to explore the possible interaction of these auditory attention switch costs and a stimulus–response (S–R) compatibility effect. This specific correspondence effect refers to the relation of the side at which the relevant stimulus is presented and the spatial location of the required response key. The concept of S–R compatibility was introduced by Fitts and Seeger (1953; see Hommel, 2011; Proctor & Vu, 2006; Proctor, Yamaguchi, Dutt, & Gonzalez, 2013; for reviews of compatibility effects). If the stimulus location is irrelevant to the task itself, performance is usually nevertheless better in conditions, in which the location of the relevant stimulus corresponds with the response location. This specific S–R compatibility effect has been termed Simon effect (Simon, 1990). For example, in a study using tones as auditory stimuli, Simon, Hinrichs, and Craft (1970) showed auditory S–R compatibility effects, and also Roswarski and Proctor (2000) found performance to be worse in conditions in
123
which the location (i.e., ear) of the relevant stimulus did not correspond with the response location. In our paradigm, the location of the auditory target stimulus and the location of the required response (i.e., key press) could be either compatible (i.e., the relevant number was presented to the left ear and was smaller than five, or the relevant number was presented to the right ear and was larger than five) or incompatible (i.e., the relevant number was presented to the left ear, but was larger than five and therefore required a right key press, or the relevant number was presented to the right ear and was smaller than five). Note, however, that in our paradigm there are always two auditory stimuli presented simultaneously, whereas only one auditory stimulus was presented in the studies by Simon et al. (1970) and Roswarski and Proctor (2000). Hence, stimulus location was much more salient in those studies, because it was associated with an abrupt spatial onset, than in our study, where stimuli are presented simultaneously and bilaterally. Specifically, we examined whether an auditory S–R compatibility effect could also be observed in conditions of dichotic stimulus presentation. We aimed to explore whether the mechanisms underlying auditory attention switch costs are dissociable from those underlying the S–R compatibility effect. Whereas attention-related effects refer per definition more to inputrelated processes (stimulus selection), the S–R compatibility effect should rather be output or response related (response selection, Kornblum, Hasbroucq, & Osman, 1990). If the mechanisms underlying auditory attention switch costs and S–R compatibility effects are dissociable, these effects should be independent rather than interactive. To explore this issue, we examined auditory attention switching and auditory S–R compatibility using the magnitude-judgment task with number words (smaller vs. larger than a reference number) as used by Koch et al. (2011). We additionally introduced animal names as new auditory stimulus material to generalize our findings to a different kind of stimulus material. In line with the number task, participants were asked to categorize the physical size of an animal, named by the relevant speaker, according to a ‘‘reference animal’’ (smaller vs. larger than a sheep). Before performing the categorization task with respect to numbers or animals, the semantic information of the relevant word has to be retrieved. Part of the semantic content is that not only especially numbers, but also symbols are believed to be spatially represented according to a mental number line (Dehaene, Bossini, & Giraux, 1993; see also Moyer & Bayer, 1976; Moyer & Landauer, 1967; Restle, 1970) with a left-to-right and small-to-large horizontal layout (Di Bono & Zorzi, 2013). Even though number words refer to specific numerical quantities (e.g., Thioux, Pesenti, Costes, De Volder, & Seron, 2005) with a more pronounced size representation than it is the case for
Psychological Research
animal names, we did not expect a qualitatively different result pattern for animal names and number words. In summary, we examined auditory attention switch costs and auditory S–R compatibility effects in a context where two stimuli are presented simultaneously (dichotically). We expected auditory attention switch costs and S– R compatibility effects with number words and animal names. If these effects refer to dissociable mechanisms, the effects should show an additive result pattern, but if not the effects should rather show a multiplicative result pattern.
Table 1 Animal names and number words were used as stimulus material (English translation in parentheses) Stimulus material Animal names
Number words
Maus (mouse)
eins (one)
Igel (hedgehog)
zwei (two)
Ente (duck) Katze (cat)
drei (three) vier (four)
Schwein (pig)
sechs (six)
Esel (donkey)
sieben (seven)
Kuh (cow)
acht (eight)
Method
Pferd (horse)
neun (nine)
Participants
calibration was carried out for each individual number word and for all the different speakers. Furthermore, the duration for all the different speakers was adjusted across the set of animal names and number words to be the same. A time-stretching algorithm was used to shorten longer samples while the frequency was maintained (see Koch et al., 2011). The auditory stimuli were time-windowed (Butterworth), but no other filters were applied. The task was to judge the relevant auditory stimulus according to a constant reference. One experimental group judged the physical size of the animal, spoken by the relevant speaker as smaller vs. larger than a sheep, and the other group judged the magnitude of the number word, spoken by the relevant speaker, as smaller vs. larger than five. Participants were instructed to press the key left vs. right of the space key of the computer keyboard (QWERTZ) with their index fingers. The assignment to the categories (\ and [) was held constant according to the ‘‘mental number line’’ (i.e., smaller associated with left, and larger with right).
Thirty-six participants (24 women, 12 men) in the age range of 18–30 years (M = 22.00, SD = 2.66) took part and received partial course credit or 6 Euros. Participants were recruited through a participant pool at the Institute of Psychology at the RWTH Aachen University and reported no presence of hearing problems. Five participants were left handed. Stimuli and tasks The visual cues were presented in white at the center of a black 17-in. screen. The participant’s distance to the screen was about 60 cm. Two verbal cues, in Arial font, and two figurative cues (pictograms) were used. The German word ‘‘FRAU’’ (2.5 9 8.5 cm; angle of vision: 2.4° 9 8.1°) or a pictogram of a female (6.5 9 3.0 cm; 6.2° 9 2.9°) was presented to indicate the female speaker as being relevant. The German word ‘‘MANN’’ (2.5 9 9.5 cm; 2.4° 9 9.1°) or a pictogram of a male (6.5 9 2.5 cm; 6.2° 9 2.4°) was presented to indicate the male speaker as being relevant. In each trial, two auditory stimuli were presented dichotically via headphones (AKG K530 LTD).1 For one group, the auditory stimuli consisted of animal names, of which four animals were smaller and four were larger than a sheep (i.e., reference). For the other group, the auditory stimuli were number words from one to nine, without five (which served as reference). The German animal names and number words with their English translation are shown in Table 1. The auditory stimuli were spoken by three different female speakers and three male speakers. The speech was recorded in an anechoic chamber at the Institute of Technical Acoustics of RWTH Aachen University (sampling rate: 44.1 kHz, quantization: 24 bit). A subjective loudness 1
A level of 65 dB SPL on every ear measured with a calibrated artificial head (with ear coupler) by HEAD acoustics.
Procedure Before the actual experiment started, there was an online instruction. For the animal names, additionally drawings of the animals were shown in the correct order from small to large to be sure participants had the correct size representations of the animals. Participants performed a practice block with 32 trials and four experimental blocks with 128 trials each, separated by short breaks. In each trial, the visual cue was presented first. After a cue–stimulus interval (CSI) of 550 ms, the two auditory stimuli were simultaneously presented via headphones. The two auditory stimuli presented to the left and right ear were always different within each trial. The visual cue remained on the screen until participants responded to the relevant auditory stimulus. Participants responded according to the visual cue by pressing the associated key. The interval
123
Psychological Research
1200
Animal Names
Number Words Switch Repetition
1150 1100
RT (ms)
between response and next cue (response–cue interval, RCI) was also 550 ms. In case of an error, a visual feedback (‘‘Fehler!’’, German for ‘‘error’’) was displayed for 500 ms, delaying the onset of the next cue. The stimulus material varied across participants. The mapping of gender to ear (left vs. right), identity of speaker within each gender, and the identity of the relevant and irrelevant auditory stimulus varied randomly from trial to trial. Every cue occurred equally often (i.e., 25 %) and indicated in half of the trials that the male speaker was relevant and in the other half that the female speaker was relevant. In about half of the trials, the gender changed from trial to trial (i.e., equal number of gender switches and gender repetitions), but direct cue repetitions did not occur. In half of the trials (50 %), both auditory stimuli led to the same response (i.e., ‘‘congruent’’)2 as both were from the same ‘‘size category’’.
1050 1000 950 900 Incompatible
Compatible
Incompatible
Compatible
S-R Compatibility
Fig. 1 Mean reaction times (ms) as a function of transition (switch vs. repetition), S–R compatibility (incompatible vs. compatible), and stimulus material (animal names vs. number words). Error bars indicate the standard error of the mean
Design The independent variables were transition (switch vs. repetition), compatibility (incompatible vs. compatible), and stimulus material (animal names vs. number words). The dependent variables were reaction times (RTs; measured from auditory stimulus onset) and error rates.
Results The practice block (32 trials) and the first trial of each block were excluded from the analyses. In the RT analysis, we additionally excluded errors and trials after an error. As outliers, we excluded RTs below 100 ms (0.01 %) and RTs exceeding three standard deviations from the participant’s mean (1.9 %).3 In the analysis of the error rates, trials after an error and outliers were excluded. RTs and error rates were submitted to separate analyses of variance (ANOVAs) using transition, compatibility, and stimulus material (across participants) as independent variables. Mean RT as a function of transition, compatibility, and stimulus material (i.e., experimental group) are shown in Fig. 1. The 29292 (Transition [Switch, Repetition] 9 Compatibility [Incompatible, Compatible] 9 Stimulus Material [Animal Names vs. Number Words]) mixed ANOVA of the RTs yielded a significant main effect of transition, F(1,34) = 69.071, MSE = 2,703, p \ 0.001, g2p = 0.67, indicating longer RT in switches 2
Previous results indicated worse performance on response-incongruent trials (Koch et al., 2011). However, congruency was not the focus of the present study, and including congruency in the data analyses did not affect the interpretation of the present data. 3 Different RT-trimming procedures did not change the results in qualitative terms.
123
than in repetitions (1,076 vs. 1,004 ms) and thus switch costs of 72 ms. The main effect of stimulus material was not significant, F(1,34) = 1.588, p [ 0.05, g2p = 0.05, but the interaction of transition and stimulus material was significant, F(1,34) = 6.818, MSE = 2,703, p \ 0.05, g2p = 0.17, indicating that the switch costs were larger for animal names than for number words (95 vs. 50 ms).4 The main effect of compatibility was significant, F(1,34) = 5.742, MSE = 986, p \ 0.05, g2p = 0.14, indicating longer RT in incompatible than compatible trials (1,046 vs. 1,034 ms) and thus a compatibility effect of 12 ms. Importantly, the interaction of compatibility and stimulus material was not significant, just like the interaction of compatibility and transition, and the three-way interaction, Fs \ 1, suggesting that S–R compatibility is quite independent from attention switching. The same analysis of the error rates (see Table 2) yielded a significant main effect of transition, F(1,34) = 7.151, MSE = 0.001, p \ 0.05, g2p = 0.17, indicating higher error rates in switches than in repetitions (6.0 vs. 4.7 %) and thus switch costs of 1.3 %. The main effect of stimulus material was not significant, F \ 1, and, unlike in RT, also the interaction of transition and stimulus material was not significant, F \ 1. Even though there were more errors on incompatible trials than on compatible trials (5.7 vs. 5.1 %), which is consistent with the significant effect in the 4
Because RT were overall somewhat (but not significantly) longer with animal names, it is important to exclude that the effect of stimulus material on auditory switch costs was just due to higher costs in trials associated with longer RT. To do so, we calculated the proportional switch costs for each stimulus material by dividing switch costs by RT on repetitions. Importantly, the re-analysis with this proportional measure of switch costs confirmed the effect of stimulus material on switch costs, t(34) = 2.703, p \ 0.05.
Psychological Research Table 2 Mean error rates (%; standard errors in parentheses) as a function of transition (switch vs. repetition), S–R compatibility (incompatible vs. compatible, and stimulus material (animal names vs. number words) Transition
S–R compatibility Animal names
Number words
Incompatible
Compatible
Incompatible
Compatible
Switch
5.7 (0.014)
5.0 (0.011)
7.3 (0.014)
6.0 (0.011)
Repetition
4.8 (0.009)
4.0 (0.009)
4.9 (0.009)
5.2 (0.009)
RT data, the main effect of compatibility was not significant, F(1,34) = 2.724, MSE = 0.000, p = 0.11, 2 gp = 0.74. Like in the RT data, neither the interaction of compatibility and stimulus material, nor the interaction of transition and compatibility was significant, Fs \ 1. The three-way interaction was also not significant, F(1,34) = 1.454, MSE = 0.000, p [ 0.05, g2p = 0.04.5
Discussion We examined the performance costs of intentional attention switching and a specific auditory compatibility effect in a categorization task using an auditory task-switching variant with different types of stimulus material. To this 5
Since the mapping of relevant gender and relevant ear varied unpredictably from trial to trial, a change in the relevant gender did not necessarily go along with a change of the relevant ear. We assumed that a repetition benefit of the relevant gender was most pronounced if the relevant ear was also repeated. Therefore, we conducted a post hoc analysis to examine potential influences of changes of the relevant ear. RTs and error rates were submitted to separate ANOVAs using transition, S–R compatibility, ear transition, and stimulus material as independent variables. We only report the relevant main effects and interactions regarding ear transition. In the RTs, the main effect of ear transition was significant, F(1,34) = 26.604, MSE = 2.956, p \ 0.001, g2p = 0.44, indicating longer RTs in ear repetitions than in ear switches (1,065 vs. 1,032 ms) and thus costs if the relevant ear repeats (i.e., ‘‘ear-repetition costs’’) of 33 ms. The interaction of transition and ear transition was also significant, F(1,34) = 10.052, MSE = 28.860, p \ 0.05, g2p = 0.23, indicating larger attention switch costs when the target stimulus was presented to the other ear than to the same ear (80 vs. 40 ms). In the error rates, the main effect of ear transition was also significant, F(1,34) = 6.732, MSE = 0.001, p \ 0.05, g2p = 0.17, indicating higher error rates in ear repetitions than in ear switches (5.4 vs. 4.6 %) and thus also ear-repetition costs of 0.8 %. Additionally, the interaction of ear transition and stimulus material was significant, F(1,34) = 8.785, MSE = 0.001, p \ 0.05, g2p = 0.21, indicating that these ear-repetition costs were much larger for number words than for animal names (2.7 vs. 0.1 %). Ear transitions varied fully independently from gender transitions, so that the auditory attention switch costs are further magnified by ear switches, but they are also present in ear repetitions. However, for the present purpose, it is noteworthy that this influence refers to attention switch costs, whereas the S–R compatibility effect does not seem to be modulated by switch or repetition of ear of target presentation.
end, a size judgment of the relevant auditory stimulus relative to a reference was performed in two groups. Spoken animal names were categorized as smaller or larger than a sheep in one group, and spoken number words were categorized as smaller or larger than five in the other group. Synopsis of the results We observed that performance was worse when the gender of the relevant speaker changed from trial to trial (i.e., switch costs). These switch costs were larger for animal names than for number words. Performance was also worse when the ear to which the target number (defined by speaker gender) was presented did not correspond to the required response (defined by size judgment of the target). This auditory S–R compatibility effect did not differ across stimulus material. Auditory attention switching Since it has been shown that visual cue priming effects can contribute to auditory switch costs (Arrington et al., 2007; Mayr & Kliegl, 2003), we used two cues per gender without direct cue repetitions (Monsell & Mizon, 2006) to focus on auditory attention switching (see also Koch & Lawo, 2014). We observed ‘‘pure’’ auditory attention switch costs that replicate earlier findings (Koch et al., 2011; Koch & Lawo, 2014; Lawo & Koch, 2012). In these studies, we only used number words as stimulus material. Switch costs are typically found in trials in which the task or stimulus selection criterion switches. One interpretation of switch costs relates to the idea that switch costs reflect the time to actively change parameters of the attentional set (i.e., as part of task-set ‘‘reconfiguration’’, see, e.g., Logan & Gordon, 2001; Rogers & Monsell, 1995). Another prominent interpretation relates to the idea that switch costs reflect proactive interference on the level of attentional processing biases, which would then give rise to attentional ‘‘inertia’’ effects (Allport, Styles, & Hsieh, 1994; Koch & Lawo, 2014). However, our study does not give us the opportunity to distinguish between these main theoretical alternatives.
123
Psychological Research
We also observed switch costs for animal names, which were even larger than for number words. This finding was confirmed by controlling for latency differences across stimulus material (i.e., proportional switch costs). By using different stimulus material in a categorization task, we were able to generalize earlier findings with this auditory task-switching paradigm using only number words as stimuli. The semantic meaning of the auditory stimulus always had to be extracted first (Thioux et al., 2005), before performing the size judgment was possible. We account for the larger switch costs for animal names than for numbers primarily by assuming that judgment tasks require semantic access and that the ordinal concept referring to magnitude is arguably much more accessible in number words (e.g., mental number line) than the mental representation of the sizes of animals. For the animal names, we chose eight ‘‘representatives’’ from a potentially very large, but theoretically limited set to perform a comparable judgment task. Notably, the categories ‘‘smaller than a sheep’’ and ‘‘larger than a sheep’’ are artificial categories. Numbers are mentally represented in terms of, e.g., quantity, ordinal structure, and relation (e.g., Dehaene, 1992, 1997). For number words, the associated size is its most important characteristic and its reference system is obvious. In contrast, animals are mentally represented in a rather abstract (e.g., classification system, habitat, etc.) and structured manner (e.g., semantic networks; Collins & Loftus, 1975), and its reference system has to be created ad hoc (e.g., Barsalou, 2008). Therefore, it can be assumed that semantic classification of animals according to size is somewhat slower than for numbers. Additionally, the specific result pattern might indicate that the semantic analysis of word meaning and the physical analysis of parameters like pitch and timbre that are required to identify the target based on speaker gender occur simultaneously and influence each other, so that slowed semantic access and size categorization with animal names also impair the speed with which the gender discrimination can be made. Since the auditory switch costs refer to a gender switch and not to a semantic switch, the reference system has to be updated throughout if the gender of the relevant speaker switches (but not in a gender repetition), which might also explain the larger switch costs for animal names compared to number words. Further research appears to be needed to confirm this seemingly competitive interaction of semantic and physical analysis of auditory stimulation.
the location of the relevant auditory stimulus and the location of the response key and expected a compatibility effect (i.e., worse performance in trials in which the location of the auditory target stimulus and the location of the required response do not correspond) which is typically observed in situations of unilateral stimulus presentation (Proctor et al., 2013; see Proctor & Vu, 2006, for a review) even if stimulus location is irrelevant (i.e., Simon effect; Simon, 1990; Proctor et al., 2013). Whereas in studies on auditory compatibility (Roswarski & Proctor, 2000) only one stimulus was presented, we used two simultaneously presented stimuli in our study. We observed that participants responded more slowly in trials in which the location of the relevant stimulus and the required response were spatially incompatible (i.e., compatibility effect). This compatibility effect was similar across stimulus material (animal names vs. number words). Because the stimulus presentation was bilateral (i.e., dichotic), there was no abrupt spatial onset, which is known to activate automatically (‘‘exogenously’’) a spatial response code (e.g., Kornblum et al., 1990; Proctor & Vu, 2006, for reviews). Rather, the present auditory compatibility effect suggests that the non-intentional activation of the spatially corresponding response code is based on the ‘‘endogenous’’ process of selecting one of the left vs. right auditory stimulus words. This shift in (or allocation of) spatial attention then interacts with the response selection process based on the target categorization. That is, the spatial aspect of the target selection process (which is defined by speaker gender) influences the spatial aspect of the response selection process (which is defined by size categorization). In this context, we may note though that the target stimulus itself is also, indirectly, associated with the spatial dimension with respect to its relative position on a mental magnitude representation (i.e., as smaller vs. larger than a reference). In this sense, the present auditory compatibility effect might also have a stimulus–stimulus compatibility component. However, the magnitude representation is most likely more distinct for numbers (Dehaene, 1992, 1997) than for animal names, so that a stimulus–stimulus compatibility effect might have been expected to be larger for numbers, but this is not what we observed. Therefore, we currently do not have evidence for a stimulus-based conflict as representational basis for the present auditory compatibility effect, even though this issue seems to merit further exploration.
Auditory S–R compatibility
Relation of auditory attention switching and auditory S–R compatibility
Part of the primary aim of our study was the examination of a specific auditory compatibility effect using dichotic presentation of spoken words. We examined the relation of
Importantly, we found a dissociation between auditory attention switching and auditory compatibility. Beyond the fact that switch costs and the compatibility effect did not
123
Psychological Research
interact, switch costs differed for the type of stimulus material. In contrast, the compatibility effect was additive to the effects of attention switching and stimulus material and thus to those effects that are more likely due to stimulus-related aspects of processing. This finding further supports the notion of a response-based conflict as basis for the auditory compatibility effect. In contrast, attention switch costs can be attributed to attentional biases in the stimulus selection process based on gender. The observed switch costs seem to denote a relative inflexibility in auditory selective attention, which might be either due to the time required for active reconfiguration or the time required for overcoming residual attentional biases (i.e., proactive interference; e.g., Kiesel et al., 2010, for a discussion).
Conclusion Together, this variant of auditory task switching is a useful tool to examine underlying mechanisms of intentional switching of auditory attention with different types of stimulus material. Likewise, the novel finding of an auditory compatibility effect in a dichotic-listening situation is methodologically interesting, as it allows examining interactions of attentional processes of stimulus selection and of processes of response selection. We conclude that the underlying mechanisms of auditory attention switching and of auditory compatibility are dissociable and refer to different stages in information processing. Acknowledgments We would like to thank Frank Wefers and Bruno Masiero from the Institute of Technical Acoustics for help in producing the stimulus material and Caterina Schiffner for her assistance with data collection. We are also grateful for helpful comments by Alexandra Bendixen, Frini Karayanidis, and two anonymous reviewers. This research was supported by the Deutsche Forschungsgemeinschaft (DFG; KO 2045/11-1).
References Arrington, C. M., Logan, G. D., & Schneider, D. W. (2007). Separating cue encoding from target processing in the explicit task-cuing procedure: are there ‘‘true’’ task switch effects? Journal of Experimental Psychology: Learning, Memory, and Cognition, 33, 484–502. doi:10.1037/0278-7393.33.3.484. Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–645. doi:10.1146/annurev.psych.59. 103006.093639. Broadbent, D. E. (1958). Perception and communication. Oxford: Pergamon Press. doi:10.1037/10037-010. Cherry, E. C. (1953). Some experiments on the recognition of speech, with one and with two ears. The Journal of the Acoustical Society of America, 25, 975–979. doi:10.1121/1.1907229.
Collins, A. M., & Loftus, E. F. (1975). A spreading activation of semantic processing. Psychological Review, 82, 407–428. doi:10.1037/0033-295X.82.6.407. Dalton, P., Santangelo, V., & Spence, C. (2009). The role of working memory in auditory selective attention. The Quarterly Journal of Experimental Psychology, 62, 2126–2132. doi:10.1080/ 17470210903023646. Dehaene, S. (1992). Varieties of numerical abilities. Cognition, 44, 1–40. doi:10.1016/0010-0277(92)90049-N. Dehaene, S. (1997). The number sense. New York: Oxford University Press. Dehaene, S., Bossini, S., & Giraux, P. (1993). The Mental Representation of Parity and Number Magnitude. Journal of Experimental Psychology: General, 122, 371–396. doi:10.1037/ 0096-3445.122.3.371. Di Bono, M. G., & Zorzi, M. (2013). The spatial representation of numerical and non-numerical ordered sequences: insights from a random generation task. The Quarterly Journal of Experimental Psychology, 66, 2348–2362. doi:10.1080/17470218.2013. 779730. Fitts, P. M., & Seeger, C. M. (1953). S–R compatibility: spatial characteristics of stimulus and response codes. Journal of Experimental Psychology: General, 46, 199–210. doi:10.1037/ h0062827. Hommel, B. (2011). Attention and spatial stimulus coding in the Simon task: a rejoinder to van der Lubbe and Abrahamse (2010). Acta Psychologica, 136, 265–268. doi:10.1016/j.actpsy.2010.10. 002. Kiesel, A., Steinhauser, M., Wendt, M., Falkenstein, M., Jost, K., Philipp, A. M., & Koch, I. (2010). Control and interference in task switching—A review. Psychological Bulletin, 136, 849–874. doi:10.1037/a0019842 Koch, I., & Lawo, V. (2014). Exploring temporal dissipation of attention settings in auditory task switching. Attention, Perception, and Psychophysics, 76, 73–80. doi:10.3758/s13414-0130571-5. Koch, I., Lawo, V., Fels, J., & Vorla¨nder, M. (2011). Switching in the cocktail party—exploring intentional control of auditory selective attention. Journal of Experimental Psychology: Human Perception and Performance, 37, 1140–1147. doi:10.1037/ a0022189. Kornblum, S., Hasbroucq, T., & Osman, A. (1990). Dimensional overlap: cognitive basis for stimulus–response compatibility—a model and taxonomy. Psychological Review, 97, 253–270. doi:10.1037/0033-295X.97.2.253. Lachter, J., Forster, K. I., & Ruthruff, E. (2004). Forty-five years after Broadbent (1958): still no identification without attention. Psychological Review, 111, 880–913. doi:10.1037/0033-295X. 111.4.880. Lawo, V., & Koch, I. (2012). Examining age-related differences in auditory attention control using a task-switching procedure. The Journals of Gerontology: Series B, Psychological Sciences and Social Sciences.,. doi:10.1093/geronb/gbs107. (online first). Logan, G. D., & Gordon, R. D. (2001). Executive control of visual attention in dual-task situations. Psychological Review, 108, 393–434. doi:10.1037/0033-295X.108.2.393. Mayr, U., & Kliegl, R. (2003). Differential effects of cue changes and task changes on task-set selection costs. Journal of Experimental Psychology. Learning, Memory, and Cognition, 29, 362–372. doi:10.1037/0278-7393.29.3.362. Meiran, N. (1996). Reconfiguration of processing mode prior to task performance. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22, 1423–1442. doi:10.1037/02787393.22.6.1423. Monsell, S., & Mizon, G. A. (2006). Can the task-cuing paradigm measure an endogenous task-set reconfiguration process?
123
Psychological Research Journal of Experimental Psychology Human Perception and Performance, 32, 493–516. doi:10.1037/0096-1523.32.3.493. Moray, N. (1959). Attention in dichotic listening: affective cues and the influence of instructions. Quarterly Journal of Experimental Psychology, 11, 56–60. doi:10.1080/17470215908416289. Moyer, R. S., & Bayer, R. H. (1976). Mental comparison and the symbolic distance effect. Cognitive Psychology, 8, 228–246. doi:10.1016/0010-0285(76)90025-6. Moyer, R. S., & Landauer, T. K. (1967). Time required for judgements of numerical inequality. Nature, 215, 1519–1520. doi:10.1038/2151519a0. Pashler, H. E. (1998). The psychology of attention. Cambridge: MIT Press. Proctor, R. W., & Vu, K.-P. L. (2006). Stimulus–response compatibility principles. Data, theory, and practice. Boca-Raton: Taylor & Francis. Proctor, R. W., Yamaguchi, M., Dutt, V., & Gonzalez, C. (2013). Dissociation of S–R compatibility and Simon effects with mixed tasks and mappings. Journal of Experimental Psychology: Human Perception and Performance, 39, 593–609. doi:10. 1037/a0029923. Restle, F. (1970). Speed of adding and comparing numbers. Journal of Experimental Psychology, 83, 274–278. doi:10.1037/ h0028573.
123
Rogers, R. D., & Monsell, S. (1995). Costs of a predictable switch between simple cognitive tasks. Journal of Experimental Psychology: General, 124, 207–231. doi:10.1037/0096-3445.124.2. 207. Roswarski, T. E., & Proctor, R. W. (2000). Auditory stimulus– response compatibility: is there a contribution of stimulus–hand correspondence? Psychological Research, 63, 148–158. doi:10. 1007/PL00008173. Shinn-Cunningham, B. G. (2008). Object-based auditory and visual attention. Trends in Cognitive Sciences, 12, 182–186. doi:10. 1016/j.tics.2008.02.003. Simon, J. R. (1990). The effects of an irrelevant directional cue on human information processing. In R. W. Proctor & T. G. Reeve (Eds.), Stimulus–response compatibility (pp. 31–86). New York: North-Holland. Simon, J. R., Hinrichs, J. V., & Craft, J. L. (1970). Auditory S–R compatibility: reaction time as a function of ear–hand correspondence and ear–response-location correspondence. Journal of Experimental Psychology, 86, 97–102. doi:10.1037/h0029783. Thioux, M., Pesenti, M., Costes, N., De Volder, A., & Seron, X. (2005). Task-independent semantic activation for numbers and animals. Cognitive Brain Research, 24, 284–290. doi:10.1016/j. cogbrainres.2005.02.009.
ORIGINAL ARTICLE
Dissociable effects of auditory attention switching and stimulus–response compatibility Vera Lawo • Iring Koch
Received: 18 June 2013 / Accepted: 28 January 2014 Ó Springer-Verlag Berlin Heidelberg 2014
Abstract Using a task-switching variant of dichotic listening, we examined the ability to intentionally switch auditory attention between two speakers. We specifically focused on possible interactions with stimulus–response compatibility. In each trial, two words, one spoken by a male and another by a female, were presented dichotically via headphones. In one experimental group, two animal names were presented, and the relevant animal had to be judged as smaller or larger than a sheep by pressing a left or right response key. In another group, two number words were presented and had to be judged as smaller or larger than 5. In each trial, a visual cue indicated the gender of the relevant speaker. Performance was worse when the gender of the relevant speaker switched from trial to trial. These switch costs were larger for animal names than for number words, suggesting stronger interference with slower access to semantic categories. Responses were slower if the side of the target stimulus (as defined by the relevant gender) was spatially incompatible with the required response (as defined by the size judgment). This stimulus–response compatibility effect did not differ across stimulus material and did not interact with attentional switch costs. These results indicate that auditory switch costs and stimulus– response compatibility effects are dissociable, referring to target selection and response selection, respectively.
V. Lawo (&) I. Koch Institute of Psychology, RWTH Aachen University, Ja¨gerstr. 17-19, 52066 Aachen, Germany e-mail: [email protected]
Introduction The conversation at a cocktail party is one of the best reallife examples of auditory selective attention (Pashler, 1998), as a variety of acoustic information is simultaneously available for auditory processing, but only certain information is relevant at once. In such situations, selective attention enables us to extract and attend to this relevant information while ignoring the irrelevant information (e.g., Dalton, Santangelo, & Spence, 2009; Lachter, Forster, & Ruthruff, 2004; see Shinn-Cunningham, 2008, for a review). Early studies on selective attention used dichotic-listening procedures to examine the underlying mechanisms of this selection (Broadbent, 1958; Cherry, 1953; Moray, 1959). In dichotic listening, participants attend to information presented to one ear, while ignoring information presented simultaneously to the other ear. In these traditional studies, attention switches were not instructed, but could occur whenever the task-irrelevant auditory information was nevertheless (i.e., involuntarily) attended (Lachter et al., 2004; see Shinn-Cunningham, 2008, for a discussion). To examine intentional switching of auditory attention, Koch, Lawo, Fels, and Vorla¨nder (2011) introduced a paradigm that combined the methodologies of dichotic listening and task cuing (Meiran, 1996), which is a basic variant of the task-switching paradigm. In the task-cuing paradigm, a cue precedes the stimulus to indicate the upcoming task. Typically, performance costs, so-called switch costs, occur in task switches compared to task repetitions, (see, e.g., Kiesel, Steinhauser, Wendt, Falkenstein, Jost, Philipp, & Koch, 2010, for a review). In this initial auditory task-switching study by Koch et al. (2011), two spoken number words, spoken by a
123
Psychological Research
female and a male speaker, were presented dichotically via headphones. Participants categorized the relevant number word as smaller or larger than five. A visual cue that preceded the auditory stimuli indicated whether the number word spoken by the female or male speaker was relevant in the upcoming trial. The main finding was that a cued switch of the to-beattended speaker (as defined by gender) resulted in attention switch costs (see also Koch & Lawo, 2014; Lawo & Koch, 2012). These switch costs may be explained either by an active process that reconfigures stimulus selection criteria (‘‘attentional filter settings’’) in auditory perception or by inertia of the filter settings, as previously relevant filter settings can proactively interfere with the new filter setting (see Koch et al., 2011, for a discussion). Since it has been shown that immediate repetition of visual cues can result in priming effects at the level of perceptual cue encoding, which, in turn, can contribute to switch costs (e.g., Arrington, Logan, & Schneider, 2007; Mayr & Kliegl, 2003), Koch et al. (2011) used two cues for each gender to disentangle auditory switch costs from visual cue priming effects. Even though cuerepetition priming effects contributed to switch costs, substantial switch costs (‘‘pure’’ auditory attention switch costs; Monsell & Mizon, 2006) remained even if cue priming was excluded (see Koch & Lawo, 2014, for a discussion). In this study, we focused on these ‘‘pure’’ auditory attention switch costs by excluding any influence of visual cue repetitions. We used two cues for each gender and designed the procedure in a way that direct cue repetitions in consecutive trials could not occur. Therefore, the cue switched on all trials, but on some trials the switched cue indicated the same gender as being relevant, whereas on other trials the switched cue indicated a gender switch. The primary aim of this study was to explore the possible interaction of these auditory attention switch costs and a stimulus–response (S–R) compatibility effect. This specific correspondence effect refers to the relation of the side at which the relevant stimulus is presented and the spatial location of the required response key. The concept of S–R compatibility was introduced by Fitts and Seeger (1953; see Hommel, 2011; Proctor & Vu, 2006; Proctor, Yamaguchi, Dutt, & Gonzalez, 2013; for reviews of compatibility effects). If the stimulus location is irrelevant to the task itself, performance is usually nevertheless better in conditions, in which the location of the relevant stimulus corresponds with the response location. This specific S–R compatibility effect has been termed Simon effect (Simon, 1990). For example, in a study using tones as auditory stimuli, Simon, Hinrichs, and Craft (1970) showed auditory S–R compatibility effects, and also Roswarski and Proctor (2000) found performance to be worse in conditions in
123
which the location (i.e., ear) of the relevant stimulus did not correspond with the response location. In our paradigm, the location of the auditory target stimulus and the location of the required response (i.e., key press) could be either compatible (i.e., the relevant number was presented to the left ear and was smaller than five, or the relevant number was presented to the right ear and was larger than five) or incompatible (i.e., the relevant number was presented to the left ear, but was larger than five and therefore required a right key press, or the relevant number was presented to the right ear and was smaller than five). Note, however, that in our paradigm there are always two auditory stimuli presented simultaneously, whereas only one auditory stimulus was presented in the studies by Simon et al. (1970) and Roswarski and Proctor (2000). Hence, stimulus location was much more salient in those studies, because it was associated with an abrupt spatial onset, than in our study, where stimuli are presented simultaneously and bilaterally. Specifically, we examined whether an auditory S–R compatibility effect could also be observed in conditions of dichotic stimulus presentation. We aimed to explore whether the mechanisms underlying auditory attention switch costs are dissociable from those underlying the S–R compatibility effect. Whereas attention-related effects refer per definition more to inputrelated processes (stimulus selection), the S–R compatibility effect should rather be output or response related (response selection, Kornblum, Hasbroucq, & Osman, 1990). If the mechanisms underlying auditory attention switch costs and S–R compatibility effects are dissociable, these effects should be independent rather than interactive. To explore this issue, we examined auditory attention switching and auditory S–R compatibility using the magnitude-judgment task with number words (smaller vs. larger than a reference number) as used by Koch et al. (2011). We additionally introduced animal names as new auditory stimulus material to generalize our findings to a different kind of stimulus material. In line with the number task, participants were asked to categorize the physical size of an animal, named by the relevant speaker, according to a ‘‘reference animal’’ (smaller vs. larger than a sheep). Before performing the categorization task with respect to numbers or animals, the semantic information of the relevant word has to be retrieved. Part of the semantic content is that not only especially numbers, but also symbols are believed to be spatially represented according to a mental number line (Dehaene, Bossini, & Giraux, 1993; see also Moyer & Bayer, 1976; Moyer & Landauer, 1967; Restle, 1970) with a left-to-right and small-to-large horizontal layout (Di Bono & Zorzi, 2013). Even though number words refer to specific numerical quantities (e.g., Thioux, Pesenti, Costes, De Volder, & Seron, 2005) with a more pronounced size representation than it is the case for
Psychological Research
animal names, we did not expect a qualitatively different result pattern for animal names and number words. In summary, we examined auditory attention switch costs and auditory S–R compatibility effects in a context where two stimuli are presented simultaneously (dichotically). We expected auditory attention switch costs and S– R compatibility effects with number words and animal names. If these effects refer to dissociable mechanisms, the effects should show an additive result pattern, but if not the effects should rather show a multiplicative result pattern.
Table 1 Animal names and number words were used as stimulus material (English translation in parentheses) Stimulus material Animal names
Number words
Maus (mouse)
eins (one)
Igel (hedgehog)
zwei (two)
Ente (duck) Katze (cat)
drei (three) vier (four)
Schwein (pig)
sechs (six)
Esel (donkey)
sieben (seven)
Kuh (cow)
acht (eight)
Method
Pferd (horse)
neun (nine)
Participants
calibration was carried out for each individual number word and for all the different speakers. Furthermore, the duration for all the different speakers was adjusted across the set of animal names and number words to be the same. A time-stretching algorithm was used to shorten longer samples while the frequency was maintained (see Koch et al., 2011). The auditory stimuli were time-windowed (Butterworth), but no other filters were applied. The task was to judge the relevant auditory stimulus according to a constant reference. One experimental group judged the physical size of the animal, spoken by the relevant speaker as smaller vs. larger than a sheep, and the other group judged the magnitude of the number word, spoken by the relevant speaker, as smaller vs. larger than five. Participants were instructed to press the key left vs. right of the space key of the computer keyboard (QWERTZ) with their index fingers. The assignment to the categories (\ and [) was held constant according to the ‘‘mental number line’’ (i.e., smaller associated with left, and larger with right).
Thirty-six participants (24 women, 12 men) in the age range of 18–30 years (M = 22.00, SD = 2.66) took part and received partial course credit or 6 Euros. Participants were recruited through a participant pool at the Institute of Psychology at the RWTH Aachen University and reported no presence of hearing problems. Five participants were left handed. Stimuli and tasks The visual cues were presented in white at the center of a black 17-in. screen. The participant’s distance to the screen was about 60 cm. Two verbal cues, in Arial font, and two figurative cues (pictograms) were used. The German word ‘‘FRAU’’ (2.5 9 8.5 cm; angle of vision: 2.4° 9 8.1°) or a pictogram of a female (6.5 9 3.0 cm; 6.2° 9 2.9°) was presented to indicate the female speaker as being relevant. The German word ‘‘MANN’’ (2.5 9 9.5 cm; 2.4° 9 9.1°) or a pictogram of a male (6.5 9 2.5 cm; 6.2° 9 2.4°) was presented to indicate the male speaker as being relevant. In each trial, two auditory stimuli were presented dichotically via headphones (AKG K530 LTD).1 For one group, the auditory stimuli consisted of animal names, of which four animals were smaller and four were larger than a sheep (i.e., reference). For the other group, the auditory stimuli were number words from one to nine, without five (which served as reference). The German animal names and number words with their English translation are shown in Table 1. The auditory stimuli were spoken by three different female speakers and three male speakers. The speech was recorded in an anechoic chamber at the Institute of Technical Acoustics of RWTH Aachen University (sampling rate: 44.1 kHz, quantization: 24 bit). A subjective loudness 1
A level of 65 dB SPL on every ear measured with a calibrated artificial head (with ear coupler) by HEAD acoustics.
Procedure Before the actual experiment started, there was an online instruction. For the animal names, additionally drawings of the animals were shown in the correct order from small to large to be sure participants had the correct size representations of the animals. Participants performed a practice block with 32 trials and four experimental blocks with 128 trials each, separated by short breaks. In each trial, the visual cue was presented first. After a cue–stimulus interval (CSI) of 550 ms, the two auditory stimuli were simultaneously presented via headphones. The two auditory stimuli presented to the left and right ear were always different within each trial. The visual cue remained on the screen until participants responded to the relevant auditory stimulus. Participants responded according to the visual cue by pressing the associated key. The interval
123
Psychological Research
1200
Animal Names
Number Words Switch Repetition
1150 1100
RT (ms)
between response and next cue (response–cue interval, RCI) was also 550 ms. In case of an error, a visual feedback (‘‘Fehler!’’, German for ‘‘error’’) was displayed for 500 ms, delaying the onset of the next cue. The stimulus material varied across participants. The mapping of gender to ear (left vs. right), identity of speaker within each gender, and the identity of the relevant and irrelevant auditory stimulus varied randomly from trial to trial. Every cue occurred equally often (i.e., 25 %) and indicated in half of the trials that the male speaker was relevant and in the other half that the female speaker was relevant. In about half of the trials, the gender changed from trial to trial (i.e., equal number of gender switches and gender repetitions), but direct cue repetitions did not occur. In half of the trials (50 %), both auditory stimuli led to the same response (i.e., ‘‘congruent’’)2 as both were from the same ‘‘size category’’.
1050 1000 950 900 Incompatible
Compatible
Incompatible
Compatible
S-R Compatibility
Fig. 1 Mean reaction times (ms) as a function of transition (switch vs. repetition), S–R compatibility (incompatible vs. compatible), and stimulus material (animal names vs. number words). Error bars indicate the standard error of the mean
Design The independent variables were transition (switch vs. repetition), compatibility (incompatible vs. compatible), and stimulus material (animal names vs. number words). The dependent variables were reaction times (RTs; measured from auditory stimulus onset) and error rates.
Results The practice block (32 trials) and the first trial of each block were excluded from the analyses. In the RT analysis, we additionally excluded errors and trials after an error. As outliers, we excluded RTs below 100 ms (0.01 %) and RTs exceeding three standard deviations from the participant’s mean (1.9 %).3 In the analysis of the error rates, trials after an error and outliers were excluded. RTs and error rates were submitted to separate analyses of variance (ANOVAs) using transition, compatibility, and stimulus material (across participants) as independent variables. Mean RT as a function of transition, compatibility, and stimulus material (i.e., experimental group) are shown in Fig. 1. The 29292 (Transition [Switch, Repetition] 9 Compatibility [Incompatible, Compatible] 9 Stimulus Material [Animal Names vs. Number Words]) mixed ANOVA of the RTs yielded a significant main effect of transition, F(1,34) = 69.071, MSE = 2,703, p \ 0.001, g2p = 0.67, indicating longer RT in switches 2
Previous results indicated worse performance on response-incongruent trials (Koch et al., 2011). However, congruency was not the focus of the present study, and including congruency in the data analyses did not affect the interpretation of the present data. 3 Different RT-trimming procedures did not change the results in qualitative terms.
123
than in repetitions (1,076 vs. 1,004 ms) and thus switch costs of 72 ms. The main effect of stimulus material was not significant, F(1,34) = 1.588, p [ 0.05, g2p = 0.05, but the interaction of transition and stimulus material was significant, F(1,34) = 6.818, MSE = 2,703, p \ 0.05, g2p = 0.17, indicating that the switch costs were larger for animal names than for number words (95 vs. 50 ms).4 The main effect of compatibility was significant, F(1,34) = 5.742, MSE = 986, p \ 0.05, g2p = 0.14, indicating longer RT in incompatible than compatible trials (1,046 vs. 1,034 ms) and thus a compatibility effect of 12 ms. Importantly, the interaction of compatibility and stimulus material was not significant, just like the interaction of compatibility and transition, and the three-way interaction, Fs \ 1, suggesting that S–R compatibility is quite independent from attention switching. The same analysis of the error rates (see Table 2) yielded a significant main effect of transition, F(1,34) = 7.151, MSE = 0.001, p \ 0.05, g2p = 0.17, indicating higher error rates in switches than in repetitions (6.0 vs. 4.7 %) and thus switch costs of 1.3 %. The main effect of stimulus material was not significant, F \ 1, and, unlike in RT, also the interaction of transition and stimulus material was not significant, F \ 1. Even though there were more errors on incompatible trials than on compatible trials (5.7 vs. 5.1 %), which is consistent with the significant effect in the 4
Because RT were overall somewhat (but not significantly) longer with animal names, it is important to exclude that the effect of stimulus material on auditory switch costs was just due to higher costs in trials associated with longer RT. To do so, we calculated the proportional switch costs for each stimulus material by dividing switch costs by RT on repetitions. Importantly, the re-analysis with this proportional measure of switch costs confirmed the effect of stimulus material on switch costs, t(34) = 2.703, p \ 0.05.
Psychological Research Table 2 Mean error rates (%; standard errors in parentheses) as a function of transition (switch vs. repetition), S–R compatibility (incompatible vs. compatible, and stimulus material (animal names vs. number words) Transition
S–R compatibility Animal names
Number words
Incompatible
Compatible
Incompatible
Compatible
Switch
5.7 (0.014)
5.0 (0.011)
7.3 (0.014)
6.0 (0.011)
Repetition
4.8 (0.009)
4.0 (0.009)
4.9 (0.009)
5.2 (0.009)
RT data, the main effect of compatibility was not significant, F(1,34) = 2.724, MSE = 0.000, p = 0.11, 2 gp = 0.74. Like in the RT data, neither the interaction of compatibility and stimulus material, nor the interaction of transition and compatibility was significant, Fs \ 1. The three-way interaction was also not significant, F(1,34) = 1.454, MSE = 0.000, p [ 0.05, g2p = 0.04.5
Discussion We examined the performance costs of intentional attention switching and a specific auditory compatibility effect in a categorization task using an auditory task-switching variant with different types of stimulus material. To this 5
Since the mapping of relevant gender and relevant ear varied unpredictably from trial to trial, a change in the relevant gender did not necessarily go along with a change of the relevant ear. We assumed that a repetition benefit of the relevant gender was most pronounced if the relevant ear was also repeated. Therefore, we conducted a post hoc analysis to examine potential influences of changes of the relevant ear. RTs and error rates were submitted to separate ANOVAs using transition, S–R compatibility, ear transition, and stimulus material as independent variables. We only report the relevant main effects and interactions regarding ear transition. In the RTs, the main effect of ear transition was significant, F(1,34) = 26.604, MSE = 2.956, p \ 0.001, g2p = 0.44, indicating longer RTs in ear repetitions than in ear switches (1,065 vs. 1,032 ms) and thus costs if the relevant ear repeats (i.e., ‘‘ear-repetition costs’’) of 33 ms. The interaction of transition and ear transition was also significant, F(1,34) = 10.052, MSE = 28.860, p \ 0.05, g2p = 0.23, indicating larger attention switch costs when the target stimulus was presented to the other ear than to the same ear (80 vs. 40 ms). In the error rates, the main effect of ear transition was also significant, F(1,34) = 6.732, MSE = 0.001, p \ 0.05, g2p = 0.17, indicating higher error rates in ear repetitions than in ear switches (5.4 vs. 4.6 %) and thus also ear-repetition costs of 0.8 %. Additionally, the interaction of ear transition and stimulus material was significant, F(1,34) = 8.785, MSE = 0.001, p \ 0.05, g2p = 0.21, indicating that these ear-repetition costs were much larger for number words than for animal names (2.7 vs. 0.1 %). Ear transitions varied fully independently from gender transitions, so that the auditory attention switch costs are further magnified by ear switches, but they are also present in ear repetitions. However, for the present purpose, it is noteworthy that this influence refers to attention switch costs, whereas the S–R compatibility effect does not seem to be modulated by switch or repetition of ear of target presentation.
end, a size judgment of the relevant auditory stimulus relative to a reference was performed in two groups. Spoken animal names were categorized as smaller or larger than a sheep in one group, and spoken number words were categorized as smaller or larger than five in the other group. Synopsis of the results We observed that performance was worse when the gender of the relevant speaker changed from trial to trial (i.e., switch costs). These switch costs were larger for animal names than for number words. Performance was also worse when the ear to which the target number (defined by speaker gender) was presented did not correspond to the required response (defined by size judgment of the target). This auditory S–R compatibility effect did not differ across stimulus material. Auditory attention switching Since it has been shown that visual cue priming effects can contribute to auditory switch costs (Arrington et al., 2007; Mayr & Kliegl, 2003), we used two cues per gender without direct cue repetitions (Monsell & Mizon, 2006) to focus on auditory attention switching (see also Koch & Lawo, 2014). We observed ‘‘pure’’ auditory attention switch costs that replicate earlier findings (Koch et al., 2011; Koch & Lawo, 2014; Lawo & Koch, 2012). In these studies, we only used number words as stimulus material. Switch costs are typically found in trials in which the task or stimulus selection criterion switches. One interpretation of switch costs relates to the idea that switch costs reflect the time to actively change parameters of the attentional set (i.e., as part of task-set ‘‘reconfiguration’’, see, e.g., Logan & Gordon, 2001; Rogers & Monsell, 1995). Another prominent interpretation relates to the idea that switch costs reflect proactive interference on the level of attentional processing biases, which would then give rise to attentional ‘‘inertia’’ effects (Allport, Styles, & Hsieh, 1994; Koch & Lawo, 2014). However, our study does not give us the opportunity to distinguish between these main theoretical alternatives.
123
Psychological Research
We also observed switch costs for animal names, which were even larger than for number words. This finding was confirmed by controlling for latency differences across stimulus material (i.e., proportional switch costs). By using different stimulus material in a categorization task, we were able to generalize earlier findings with this auditory task-switching paradigm using only number words as stimuli. The semantic meaning of the auditory stimulus always had to be extracted first (Thioux et al., 2005), before performing the size judgment was possible. We account for the larger switch costs for animal names than for numbers primarily by assuming that judgment tasks require semantic access and that the ordinal concept referring to magnitude is arguably much more accessible in number words (e.g., mental number line) than the mental representation of the sizes of animals. For the animal names, we chose eight ‘‘representatives’’ from a potentially very large, but theoretically limited set to perform a comparable judgment task. Notably, the categories ‘‘smaller than a sheep’’ and ‘‘larger than a sheep’’ are artificial categories. Numbers are mentally represented in terms of, e.g., quantity, ordinal structure, and relation (e.g., Dehaene, 1992, 1997). For number words, the associated size is its most important characteristic and its reference system is obvious. In contrast, animals are mentally represented in a rather abstract (e.g., classification system, habitat, etc.) and structured manner (e.g., semantic networks; Collins & Loftus, 1975), and its reference system has to be created ad hoc (e.g., Barsalou, 2008). Therefore, it can be assumed that semantic classification of animals according to size is somewhat slower than for numbers. Additionally, the specific result pattern might indicate that the semantic analysis of word meaning and the physical analysis of parameters like pitch and timbre that are required to identify the target based on speaker gender occur simultaneously and influence each other, so that slowed semantic access and size categorization with animal names also impair the speed with which the gender discrimination can be made. Since the auditory switch costs refer to a gender switch and not to a semantic switch, the reference system has to be updated throughout if the gender of the relevant speaker switches (but not in a gender repetition), which might also explain the larger switch costs for animal names compared to number words. Further research appears to be needed to confirm this seemingly competitive interaction of semantic and physical analysis of auditory stimulation.
the location of the relevant auditory stimulus and the location of the response key and expected a compatibility effect (i.e., worse performance in trials in which the location of the auditory target stimulus and the location of the required response do not correspond) which is typically observed in situations of unilateral stimulus presentation (Proctor et al., 2013; see Proctor & Vu, 2006, for a review) even if stimulus location is irrelevant (i.e., Simon effect; Simon, 1990; Proctor et al., 2013). Whereas in studies on auditory compatibility (Roswarski & Proctor, 2000) only one stimulus was presented, we used two simultaneously presented stimuli in our study. We observed that participants responded more slowly in trials in which the location of the relevant stimulus and the required response were spatially incompatible (i.e., compatibility effect). This compatibility effect was similar across stimulus material (animal names vs. number words). Because the stimulus presentation was bilateral (i.e., dichotic), there was no abrupt spatial onset, which is known to activate automatically (‘‘exogenously’’) a spatial response code (e.g., Kornblum et al., 1990; Proctor & Vu, 2006, for reviews). Rather, the present auditory compatibility effect suggests that the non-intentional activation of the spatially corresponding response code is based on the ‘‘endogenous’’ process of selecting one of the left vs. right auditory stimulus words. This shift in (or allocation of) spatial attention then interacts with the response selection process based on the target categorization. That is, the spatial aspect of the target selection process (which is defined by speaker gender) influences the spatial aspect of the response selection process (which is defined by size categorization). In this context, we may note though that the target stimulus itself is also, indirectly, associated with the spatial dimension with respect to its relative position on a mental magnitude representation (i.e., as smaller vs. larger than a reference). In this sense, the present auditory compatibility effect might also have a stimulus–stimulus compatibility component. However, the magnitude representation is most likely more distinct for numbers (Dehaene, 1992, 1997) than for animal names, so that a stimulus–stimulus compatibility effect might have been expected to be larger for numbers, but this is not what we observed. Therefore, we currently do not have evidence for a stimulus-based conflict as representational basis for the present auditory compatibility effect, even though this issue seems to merit further exploration.
Auditory S–R compatibility
Relation of auditory attention switching and auditory S–R compatibility
Part of the primary aim of our study was the examination of a specific auditory compatibility effect using dichotic presentation of spoken words. We examined the relation of
Importantly, we found a dissociation between auditory attention switching and auditory compatibility. Beyond the fact that switch costs and the compatibility effect did not
123
Psychological Research
interact, switch costs differed for the type of stimulus material. In contrast, the compatibility effect was additive to the effects of attention switching and stimulus material and thus to those effects that are more likely due to stimulus-related aspects of processing. This finding further supports the notion of a response-based conflict as basis for the auditory compatibility effect. In contrast, attention switch costs can be attributed to attentional biases in the stimulus selection process based on gender. The observed switch costs seem to denote a relative inflexibility in auditory selective attention, which might be either due to the time required for active reconfiguration or the time required for overcoming residual attentional biases (i.e., proactive interference; e.g., Kiesel et al., 2010, for a discussion).
Conclusion Together, this variant of auditory task switching is a useful tool to examine underlying mechanisms of intentional switching of auditory attention with different types of stimulus material. Likewise, the novel finding of an auditory compatibility effect in a dichotic-listening situation is methodologically interesting, as it allows examining interactions of attentional processes of stimulus selection and of processes of response selection. We conclude that the underlying mechanisms of auditory attention switching and of auditory compatibility are dissociable and refer to different stages in information processing. Acknowledgments We would like to thank Frank Wefers and Bruno Masiero from the Institute of Technical Acoustics for help in producing the stimulus material and Caterina Schiffner for her assistance with data collection. We are also grateful for helpful comments by Alexandra Bendixen, Frini Karayanidis, and two anonymous reviewers. This research was supported by the Deutsche Forschungsgemeinschaft (DFG; KO 2045/11-1).
References Arrington, C. M., Logan, G. D., & Schneider, D. W. (2007). Separating cue encoding from target processing in the explicit task-cuing procedure: are there ‘‘true’’ task switch effects? Journal of Experimental Psychology: Learning, Memory, and Cognition, 33, 484–502. doi:10.1037/0278-7393.33.3.484. Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–645. doi:10.1146/annurev.psych.59. 103006.093639. Broadbent, D. E. (1958). Perception and communication. Oxford: Pergamon Press. doi:10.1037/10037-010. Cherry, E. C. (1953). Some experiments on the recognition of speech, with one and with two ears. The Journal of the Acoustical Society of America, 25, 975–979. doi:10.1121/1.1907229.
Collins, A. M., & Loftus, E. F. (1975). A spreading activation of semantic processing. Psychological Review, 82, 407–428. doi:10.1037/0033-295X.82.6.407. Dalton, P., Santangelo, V., & Spence, C. (2009). The role of working memory in auditory selective attention. The Quarterly Journal of Experimental Psychology, 62, 2126–2132. doi:10.1080/ 17470210903023646. Dehaene, S. (1992). Varieties of numerical abilities. Cognition, 44, 1–40. doi:10.1016/0010-0277(92)90049-N. Dehaene, S. (1997). The number sense. New York: Oxford University Press. Dehaene, S., Bossini, S., & Giraux, P. (1993). The Mental Representation of Parity and Number Magnitude. Journal of Experimental Psychology: General, 122, 371–396. doi:10.1037/ 0096-3445.122.3.371. Di Bono, M. G., & Zorzi, M. (2013). The spatial representation of numerical and non-numerical ordered sequences: insights from a random generation task. The Quarterly Journal of Experimental Psychology, 66, 2348–2362. doi:10.1080/17470218.2013. 779730. Fitts, P. M., & Seeger, C. M. (1953). S–R compatibility: spatial characteristics of stimulus and response codes. Journal of Experimental Psychology: General, 46, 199–210. doi:10.1037/ h0062827. Hommel, B. (2011). Attention and spatial stimulus coding in the Simon task: a rejoinder to van der Lubbe and Abrahamse (2010). Acta Psychologica, 136, 265–268. doi:10.1016/j.actpsy.2010.10. 002. Kiesel, A., Steinhauser, M., Wendt, M., Falkenstein, M., Jost, K., Philipp, A. M., & Koch, I. (2010). Control and interference in task switching—A review. Psychological Bulletin, 136, 849–874. doi:10.1037/a0019842 Koch, I., & Lawo, V. (2014). Exploring temporal dissipation of attention settings in auditory task switching. Attention, Perception, and Psychophysics, 76, 73–80. doi:10.3758/s13414-0130571-5. Koch, I., Lawo, V., Fels, J., & Vorla¨nder, M. (2011). Switching in the cocktail party—exploring intentional control of auditory selective attention. Journal of Experimental Psychology: Human Perception and Performance, 37, 1140–1147. doi:10.1037/ a0022189. Kornblum, S., Hasbroucq, T., & Osman, A. (1990). Dimensional overlap: cognitive basis for stimulus–response compatibility—a model and taxonomy. Psychological Review, 97, 253–270. doi:10.1037/0033-295X.97.2.253. Lachter, J., Forster, K. I., & Ruthruff, E. (2004). Forty-five years after Broadbent (1958): still no identification without attention. Psychological Review, 111, 880–913. doi:10.1037/0033-295X. 111.4.880. Lawo, V., & Koch, I. (2012). Examining age-related differences in auditory attention control using a task-switching procedure. The Journals of Gerontology: Series B, Psychological Sciences and Social Sciences.,. doi:10.1093/geronb/gbs107. (online first). Logan, G. D., & Gordon, R. D. (2001). Executive control of visual attention in dual-task situations. Psychological Review, 108, 393–434. doi:10.1037/0033-295X.108.2.393. Mayr, U., & Kliegl, R. (2003). Differential effects of cue changes and task changes on task-set selection costs. Journal of Experimental Psychology. Learning, Memory, and Cognition, 29, 362–372. doi:10.1037/0278-7393.29.3.362. Meiran, N. (1996). Reconfiguration of processing mode prior to task performance. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22, 1423–1442. doi:10.1037/02787393.22.6.1423. Monsell, S., & Mizon, G. A. (2006). Can the task-cuing paradigm measure an endogenous task-set reconfiguration process?
123
Psychological Research Journal of Experimental Psychology Human Perception and Performance, 32, 493–516. doi:10.1037/0096-1523.32.3.493. Moray, N. (1959). Attention in dichotic listening: affective cues and the influence of instructions. Quarterly Journal of Experimental Psychology, 11, 56–60. doi:10.1080/17470215908416289. Moyer, R. S., & Bayer, R. H. (1976). Mental comparison and the symbolic distance effect. Cognitive Psychology, 8, 228–246. doi:10.1016/0010-0285(76)90025-6. Moyer, R. S., & Landauer, T. K. (1967). Time required for judgements of numerical inequality. Nature, 215, 1519–1520. doi:10.1038/2151519a0. Pashler, H. E. (1998). The psychology of attention. Cambridge: MIT Press. Proctor, R. W., & Vu, K.-P. L. (2006). Stimulus–response compatibility principles. Data, theory, and practice. Boca-Raton: Taylor & Francis. Proctor, R. W., Yamaguchi, M., Dutt, V., & Gonzalez, C. (2013). Dissociation of S–R compatibility and Simon effects with mixed tasks and mappings. Journal of Experimental Psychology: Human Perception and Performance, 39, 593–609. doi:10. 1037/a0029923. Restle, F. (1970). Speed of adding and comparing numbers. Journal of Experimental Psychology, 83, 274–278. doi:10.1037/ h0028573.
123
Rogers, R. D., & Monsell, S. (1995). Costs of a predictable switch between simple cognitive tasks. Journal of Experimental Psychology: General, 124, 207–231. doi:10.1037/0096-3445.124.2. 207. Roswarski, T. E., & Proctor, R. W. (2000). Auditory stimulus– response compatibility: is there a contribution of stimulus–hand correspondence? Psychological Research, 63, 148–158. doi:10. 1007/PL00008173. Shinn-Cunningham, B. G. (2008). Object-based auditory and visual attention. Trends in Cognitive Sciences, 12, 182–186. doi:10. 1016/j.tics.2008.02.003. Simon, J. R. (1990). The effects of an irrelevant directional cue on human information processing. In R. W. Proctor & T. G. Reeve (Eds.), Stimulus–response compatibility (pp. 31–86). New York: North-Holland. Simon, J. R., Hinrichs, J. V., & Craft, J. L. (1970). Auditory S–R compatibility: reaction time as a function of ear–hand correspondence and ear–response-location correspondence. Journal of Experimental Psychology, 86, 97–102. doi:10.1037/h0029783. Thioux, M., Pesenti, M., Costes, N., De Volder, A., & Seron, X. (2005). Task-independent semantic activation for numbers and animals. Cognitive Brain Research, 24, 284–290. doi:10.1016/j. cogbrainres.2005.02.009.
