Acta Psychologica North-Holland
75 (1990) 41-54
AUDITORY ACCESSORY P.J.G. KEUSS,
October
EFFECTS ON VISUAL PROCESSING
*
F. van der ZEE and M.B.M. van den BREE
Free Unioersity, Amsterdam, Accepted
41
The Netherlands
1989
A series of experiments on intersensory facilitation demonstrates that non-informative sound of low to moderate intensity (30/80 dB) facilitates the reaction to a visual stimulus. By manipulating the preprocessing and perceptual stages of the visual signals, it appears that auditory intensity reduces choice reaction time independently from the positive influence of the intensity and duration of the visual imperative signal, but interacts with the effect of stimulus degradation. Degraded stimuli take more profit of the sound than intact stimuli. Besides a short-term activation effect, originated by accessories of the auditory modality, on the motor adjustment stage (cf. Sanders 1983) the results suggest that the accessory influences the stage of feature extraction.
Introduction An old issue in the field of sensory psychology, dating back to the pioneering work of Todd (1912) is intersensory facilitation. This term labels the phenomenon that a reaction to a stimulus shortens when that stimulus is accompanied by an accessory (see Nickerson’s (1973) review). Usually the response is given to a visual stimulus, while the accessory is non-informative implying that the subject needs not to attend to it in order to perform the reaction time (RT) task. The historically oldest explanation of intersensory facilitation is in terms of energy summation. The view is based on the fact that in either modality the intensity exerts a positive effect on RT (see Woodworth and Schlosberg (1956) for a review). Indeed, it is well established that the time to transmit information from the stimulus source to the cortex, including the delay of the sensory transducer and the afferent conduction time (see Nickerson’s (1973) study of above) is inversely related to * Requests Psychology,
for reprints should be sent to P.J.G. Keuss, Free University, De Boelelaan 1111, 1018 HV Amsterdam, The Netherlands.
OOOl-6918/90/$03.50
0 1990 - Elsevier Science Publishers
Dept.
B.V. (North-Holland)
of Cognitive
42
P.J.G. Keuss et al. / Auditory accessory effects on visual processing
stimulus intensity (Bernstein 1972; Koster and Peacock 1969). Psychophysiological support, cited by Smith (1967), is provided by the finding that the latency of the visual-evoked response in the EEG pattern also varies inversely with intensity. For auditory-evoked responses reference is made to the work of Karlin et al. (1971). One step further, the idea emerged that energy from the accessory combines in some way with the energy available for the reaction stimulus, leading to a more rapid processing of stimulus input. Currently, the energy summation account is no longer seriously considered because a visual accessory is not able to speed up a reaction to a visual stimulus (Helson 1964). Otherwise, it is well documented that facilitation of visual processing is caused by auditory stimulation (Posner et al. 1976). Hence, the alternative explanation is bound to the auditory stimuli, which are supposed to exert an alerting or activating effect on the reacting subject. The question of the present study is on what (sub)processes the activation takes place, when the subject is engaged in his visual reaction task. Sanders (1990) suggests that an auditory stimulus of a certain intensity level (75 dB) produces a short cut activation of the motor adjustment process. This assumption is in line with his earlier findings (Sanders 1980) demonstrating that muscle tension, directly related to motor adjustment, did interact with an accessory in a simple task but not in a choice task. In the latter case, the response selection stage may be bypassed on account of the accessory. Response selection is time consuming and bypassing of this stage (if possible) results in a double gain. For example, simple or very compatible responses (produced without the involvement of the response selection stage) may take full profit of the speeding up due to the activating accessory. In contrast, demands on response selection in case of incompatible stimulus/ response mappings counteract this facilitation. Indeed, Van der Molen and Keuss (1979) were able to demonstrate that, as heavier load was imposed on the response selection, the influence became more adverse in terms of slower reactions and errors. Furthermore, the intensity of the accessory did enlarge this negative effect in their study. When in the range of 80 till 110 dB performance deteriorates, the activation caused by the accessory might have obscured earlier effects on processing stages before the motor adjustment. Still open is thus the question what happens with lower intensities for which it is not clear that short cut activation determines the reaction of the subject. There-
P.J.G. Keuw et al. / Auditory accessory effects on visual processing
43
fore, the present study uses accessories of low to moderate intensities (30 till 80 dB) in order to explore a possible effect on input processing. Following the Additive Factor Logic (Stemberg 1969), experimental variables are chosen which are supposed to act on the earlier processing stages of visual input (see also Sternberg 1969). The first canditate is intensity which affects stimulus preprocessing. Hence, the study opens with an experiment, in which the visual reaction stimulus as well the auditory accessory varies in intensity. The idea behind the Additive Factor Method is that when both intensity factors independently contribute to the speed of responding each of the two factors affects another stage. Or, in psychological terms, one may infer that preprocessing of visual input lies outside the influence of the auditory accessory. A similar reasoning may be followed in the second experiment. Here the brightness aspect of the visual stimulus, which is assumingly also affecting stimulus preprocessing, is varied by presenting the stimulus with different durations. The durations used fall within the range (till 12 msec) for which the tradeoff between intensity and duration exists (Bloch’s Law), resulting in an RT/brightness gradient, similar to the RT/intensity relationship. When the intensity of the accessory does not interact with the duration of the visual stimulus, one may infer again, that the accessory does not influence the input stage being responsible for brightness processing. According to the AFM logic the reappearance of additivity in the second experiment verifies the independent contributions of the accessory and visual input variables to RT, as expected in experiment 1. The literature refers to the perceptual stage of feature extraction as the stage in which a central representation of the stimulus is built up (cf. Posner 1978). Feature extraction can be regarded as one of the (sub-)stages of perceptual processing (Sanders 1983) taking place before the identification of that stimulus is completed (Schmidt et al. 1984). The main factor that affects the quality of the stimulus or nature (in terms of Schmidt et al. 1984) is stimulus degradation (Frowein 1981). Stimulus degradation manipulation is being manipulated in the last experiment. Posner (1978) believes that in general the accessory acts to terminate input processing more quickly. In his view, the accessory is expected to lower the criterion for delivering the response before the process of information accrual is accomplished. Degraded stimuli demand a higher
44
P.J.G. Kews et al. / Auditory accessory effects on visual processing
degree of accumulating evidence than intact ones. The prediction is then that the intensity of the accessory interacts with the stimulus degradation factor. However, premature termination of the information accrual process results in a reduction of accuracy. In that case the accessory actuates a change in the speed-accuracy tradeoff, without affecting the real process of visual perception. An alternative is developed by Sanders (1983, 1990). In his cognitive-energetical linear stage model of human information processing room is left for two energetical mechanisms (arousal and activation) coupled to input and output processing stages, respectively. In line with this idea one may hypothesize that the accessory influences via the arousal mechanism the input processing, apart from its, forementioned, activation effect on output processing. As in Posner’s model, the linear stage model could account for an interaction between the intensity of the accessory and stimulus quality. But, when overarousal is circumvented by using low to moderate intensities as in the present study, a change in speed-accuracy tradeoff should be absent according to the latter model. Common to the three experiments of this study is that the visual stimulus demands a (binary) choice. The rationale is that in a no-choice task the subject could engage in a switching approach allowing a kind of incomplete processing (Gielen et al. 1983). And, the subject’s strategy could either be to preserve his primary concern to what comes first into the focus of his attention, or he could even ignore the reaction stimulus all the time and respond to the faster accessory. Such a strategy is bound to occur when we consider that the conductance time of an auditory signal is considerably shorter than that of a visual signal (Nickerson 1973). In the choice situation, however, the accessory does not tell the subject which response to make (Posner 1978). In this case, to ignore the accessory may be even helpful in order to avoid erroneous reactions. In all trials of our study the accessory is present. This deviates from the traditional method of using unimodal (catch) trials as reference to estimate the (additional) intersensory effect under the bimodal condition. A point of criticism against the catch trial method is that RT is considered to be not so much a function of the particular stimulus presented but of that stimulus in relation to other possible stimulus events (Gielen et al. 1983). Consequently, the subtraction method may overestimate the size of the intersensory effect. A theoretical reservation against interspersing unimodal among bimodal trials in the present
P.J. G. Keuss et al. / Auditory accessory effects on visual processing
45
case is that a trial with an auditory accessory is qualitatively different from one without. There is evidence that any auditory stimulus influences the subject’s way of responding, since there is a tendency to respond to the stimulus source (the Simon effect), even with sounds of low intensities (cf. Faber et al. (1986). Auditory signals also enlarge the readiness of the subject, cf. the work of Sanders (1983) on the compensatory effect of sound in case of time uncertainty. Finally, the Nickerson (1973) and Bernstein (1970) studies are worth mentioning. They showed that auditory signals were even able to affect reactions to a visual signal while arriving 100 msec later than the latter. As the present investigation focuses on real time processes in the sensory system, it makes sense to offer the visual and auditory stimuli at the same time to the subject.
Experimental procedure General
All subjects were recruited from the pool of Free University students aged 20-30 years, with males and females equally distributed over the experiments. They reported normal or corrected-to-normal vision and hearing. The subjects were paid a fixed sum for participation. The instruction was to react to the binary visual signal with a key-press response, executed by left or right index finger. Reactions were measured in msec, while the timer started at the onset of the reaction signal. Equal emphasis was laid on speed and accuracy. The subject sat in a sound-attenuating cubicle, which was pitch dark in the first and dimly lit in both other experiments. On his/her table two keys were mounted, 10 cm apart, for the right- and left-hand response, respectively. One second prior to the arrival of the visual reaction stimulus, an upper red light, fixed at eye level, came on and went out at the onset of the reaction stimulus. Concurrent with the visual signal, the accessory stimulus was delivered via a headphone. The accessory was a binaurally presented tone of 200 Hz. Its intensity was 30, 40, 50, 60, 70 or 80 dB(A) against an ambient noise background level of 10 dB(A). The subject’s reaction terminated both visual and auditory stimulus. Thereafter (s)he received a sign, by means of the lighting up of a 10 cm lower red light, at eye level, indicating that (s)he had to give a confidence judgment about the reaction. Therefore (s)he pressed a foot pedal, ranging in depth from 0 (very uncertain) to 150 (very certain). The confidence scores are omitted from analysis in the present report. Reappearance of the upper red light marked a fixed intertrial interval of 10 sec. All visual stimuli, positioned or projected at 1.5 to 2.0 meters in front of the subject, subtended a visual angle of maximal 2.0 degrees, horizontally and vertically.
46
P.J.G. Keuss et al. /Auditory accessory effects on visual processing
Left/right response (2 levels), intensity of the accessory (6 levels), and manipulation of the visual stimulus (4 levels in experiment 1, 2 and 3) were factorially combined to produce the individual type of trials. Each trial type was replicated 10 times per subject. The resultant 480 trials per experiment were randomized without replacement and presented in eight mixed blocks or series of 75 each. All subjects, assigned to an experiment, run the same test implying that the experimental factors of that experiment were tested along a within-subjects design. Sixteen subjects participated in each experiment. Directly after a practice session of 30 minutes, which was also used to accustom subject’s eyes to the darkness in experiment 1, (s)he run the eight experimental series in one session of about 2 hours, with intermissions between the series of 2 to 5 minutes. During these pauses the subjects were kept informed about their performance level in terms of error proportion and speed of reactions, both being the dependent variables of the report. An ITT Apple II-e system controlled the stimuli and intervals and assembled the data. Per experiment ANOVAs were employed on the RT data using a within-subjects design. Mean RT per (sub-)condition and per subject was used as individual value. Error frequencies were analyzed (ANOVAs) in parallel with the RT data. Experiment
1 (intensity)
The visual stimuli of this experiment are illustrated in the inset of fig. 1. On a perpendicular black panel a configuration of bulbs was fastened each with a diameter and interdistance of 5 mm. It consisted of eleven horizontal bulbs plus, at the extremes, four bulbs forming in triangular form the left and right arrow point. The middle light lit up, at an intensity of 0.09 Lux, one second before the visual stimulus and served as an (extra) warning signal. The visual stimulus comprised the lighting up of the nine bulbs to the left or the nine bulbs to the right of the middle light. All lights, with rise and decay times within 5 msec, had a red color with an intensity value of 0.01, 0.02, 0.04 or 0.16 Lux against a background illuminance of zero. Intensity was the manipulation factor of this experiment. The orientation of the arrow determined, in direct correspondence, the subject’s left versus right response. Experiment
2 (brightness)
The visual stimuli of this experiment were slides, projected on a white screen. A Kodak slide projector was used whose shutter time was effectively zero. The size of the stimuli was 24 by 36 cm. Each slide consisted of a left and right her&field of equal size, being two patches of gray, differing in brightness (see fig. 2). The luminance of the hen&fields was 3.8 and 4.4 Lux, respectively, against a background illuminance of 2.7 Lux. With either the left or the right key the subject had to indicate, in spatial correspondence, the brighter hem&field which was randomly presented to the left or right part of the slide. Presentation time of the slide was the manipulation factor of the experiment with 4 levels, corresponding to slide durations of 4, 6, 8, 10 msec. The rationale behind this
P.J.G. Keuss et al. / Auditory accessory effects on visual processing
manipulation was that the duration brightness (contrast). Experiment
of stimulus
presentation
positively
41 correlates
with
3 (degradation)
Fig. 3 provides the examples of this experiment. Again slides were presented producing stimuli of the same size and with the same apparatus as in experiment 2. Each slide contained within a dotted frame punctate arrow (points) which were formed by a number of black dots against a white background. The intact stimuli, as presented in the right side of the figure, had arrows composed of eleven dots. The lower rows show the degraded stimuli, which was the manipulation factor (with 4 levels) of this experiment. Dots are displaced in the second row, or omitted in the third row of above. The lowest row shows a stimulus with extra dots spread over the slide. Each level of degradation had several instances in order to eliminate learning effects. Pooled over all instances used the difference in luminance among the levels was negligible. Level of difficulty will be defined by the grand means as will be obtained in the present experiment. The orientation of the arrow point, which was random, determined the corresponding left versus right key press response.
Results Experiment
I - Intensity
Mean RT and proportion of errors as a function of the intensity of the visual reaction stimulus and the intensity of the accessory are depicted in the upper and lower panel of fig. 1, respectively. The inverse relationship of RT with the visual intensity is very pronounced (F(3, 15) = 22.6; p < 0.01) which is parallelled by a clear decrease of erroneous performance over intensity (F(3, 15) = 35.7; p < 0.01). The RT/intensity gradient of the accessory has also a negative slope, which is less outspoken than the RT/intensity relationship in the visual domain. Nonetheless, the accessory effect was statistically reliable (F(5, 15) = 4.84; p < 0.01). Errors appeared to be equally distributed over the intensity of the accessory (F(5, 15) = 1.6; ns). No statistically reliable interactions between both intensity manipulations were obtained in the analyses employed on the RT and error data, respectively. Experiment
2 - Brightness
Fig. 2 depicts in two panels the data of the brightness manipulation. The upper panel shows the RT/presentation time relationship. Presentation time contributed significantly to the variance (F(3, 15) = 8.8; p < 0.001). Inspection of the figure shows that the RT grows shorter for durations ranging from 4 to 8 msec followed, however,
48
P.J.G. Keuss et al. / Auditory accessory effects on visual processing
kll 550
--
$ 450-E .5 k
\:
-20
350--
_ae b l 0.01
0.02
I
I
0.W
0.16
i
LUX 500
(bl t
q;;;;;;pq 30
40
60
60
70
60
dB
Fig. 1. Mean RT and error proportions of experiment 1 (intensity) are presented. In panel (a) RT as a function of the visual intensity is displayed, collapsed over the auditory intensities. In panel (b) RT as a function of the auditory intensity (accessory) is displayed. An example of the stimuli is given in the inset.
by slight increase in RT as a function of presentation time. The lower panel shows the negative RT/intensity gradient of the accessory which was significant (F(5, 15) = 13.1; p < 0.001). The analyses on the errors revealed significance (at 0.05 level) for presentation time and intensity of the accessory (F = 2.9 and F = 2.7, respectively). No statistically reliable interaction between the brightness manipulation of the visual stimulus and the intensity of the accessory were obtained for the RT and error data, respectively.
Experiment
3 - Degradation
Stimulus degradation and intensity of the accessory reached significance (F(3, 15) = 8.1; p c 0.001 and F(5, 15) = 4.6; p < 0.01, respectively). Grand means for the degradation conditions were in ascending order: condition 1 (intact stimuli) 345 msec; condition 4 (added dots) 360 msec; condition 2 (displaced dots) 372 msec; condition 3 (deleted dots) 378 msec. The interaction of these factors also contributed to the variance (F(15, 300) = 3.1; p < 0.01). Fig. 3 illustrates the interaction of the intensity of the accessory with the most and the least degraded stimuli as a parameter. It can be
P.J.G. Keuss et al. / Auditory accessory effecis on visual processing
49
M
360 t
.E
k
340--20 ae g a
I
I
I
I
4
6
6
10
exposure time hnaecl
Eso-.c
\
k 340~-
I I I 30
40
60
III
60
70
60
d6
Fig. 2. Mean RT and error proportions as a function of exposure time, collapsed over the auditory intensity (accessory), obtained in experiment 2. In panel (b) the RT/intensity gradient for the accessory is given. An example of the stimuli is given in the inset.
seen from the graphics of the upper panel that the slope of the RT/intensity gradient is steeper for the more degraded stimuli (condition 3, dots deleted) than the less degraded stimuli (condition 4, dots added). Fig. 3 (lower panel) also shows the error proportions which follow the RT trends but the intensity of the accessory fell short of significance at 0.05 level. Stimulus degradation and its interaction with the intensity of the accessory contributed significantly to the variance (F(3, 15) = 4.0; p -C0.01, and F(15, 300) = 4.1; p < 0.01, respectively). In an additional analysis omitting the 80 dB data, the Degradation By Intensity interaction fell (at 0.05 level) short of significance (F(12, 240) = 1.9).
Discussion Intersensory facilitation was consistently manifest in the data patterns of all three experiments. Visual reactions were faster with increas-
50
P. J. G. Keuss et al. / Auditory accessory effects on visual processing
r
Condition Condition Condition
4 2 3
1
i
Condition
2
; :*::
dB
L
380
“.3
Condition
i
.*‘*
f
“‘ZJ
:.. ....... :
(bl
Condition3
i :
a’ .
*’i.
I *a . i . . . . . ..:
r20 30
40
60
60
70
BO
dB
Fig. 3. Delta RT (absolute difference between RT,,,,,, RTdegraded) as a function of auditory intensity (accessory), obtained in experiment 3, with (in descending order of difficulty) condition 3 (dots deleted), condition 2 (dots displaced) and condition 4 (dots added) as a parameter. Condition 1 (intact stimuli) is reference. Panel (b) shows the RT/auditory intensity (accessory) gradient and error proportions collapsed over ah visual degradation conditions. Examples of stimuli are given in the inset.
ing intensity of the auditory accessory. The decrement amounted to 15 msec for an intensity range of 40 to 80 dB which is in accordance with what is cited in the literature (Keuss 1972). Crucial is the non-existence of a speed-accuracy tradeoff with increasing intensities, as we clearly found in two of the three experiments. This excludes the ‘by-pass’ interpretation of the accessory effect on the subject’s decision stage which is particularly relevant to the interpretation of the third experiment. This also excludes the premature termination of the information accrual process due to the accessory, as Posner (1978) proposed. The intensities used in the present study were likely too low (30/80 dB) to produce a blocking of the choice stage. An exception may form the highest intensity in experiment 3 for which an increment of errors was obtained. This outcome is, however, not at odds with the general view
P.J.G. Keuss et al. / Auditory accessory effects on visual processing
51
that 70 to 80 dB marks the limit above which (choice-)performance deteriorates, presumably due to startle (Van der Molen and Keuss 1981). Although the present study does not give direct evidence for a general response activation originated by the accessory, the data support the view expressed by Meulenbroek and Van Galen (1988), that aspecific ‘functional’ preparation may be enhanced by this sort of manipulation. Reference is made to the work of Sanders (1980, 1983, 1990), suggesting the motor adjustment stage as the most likely candidate whereupon this sort of activation is manifest. It is further noted that the activation is more outspoken as the intensity grows. In this respect the data pattern is consistent with that of Keuss and Van der Molen (1982) who used a broader range of intensities up to 110 dB. Inferring from the activation account that is exclusively bound to the auditory modality, one may expect an independence from visual processing aspects which miss the activation function. Indeed, the intensity variation of the visual stimulus, as done in the first experiment, was manifest in processing time variation of that stimulus, whereas any relationship with the auditory accessory was absent. More intense visual stimuli yielded faster responses than less intense ones, which corresponds to the well-known inverse relationship of RT with intensity. Similarly, stimuli of longer duration and, hence, in psychological respect, greater brightness (experiment 2) yielded faster reactions than those of shorter durations. In passing it is noted that the intensity variation as used in experiment 1 was quite broad, ranging from 0.01 to 0.16 Lux, corresponding with relatively long RTs varying from 550 msec till 350 msec, respectively. Still, the frequency of errors (5%) was relatively low compared by the data pattern of experiment 2 and 3. It might suggest that the processing of the orientation of the arrow can be done in a rather compatible way, but that the low intensities used here render the subjects cautious in responding which resulted in relatively high reaction times as compared by the data of experiment 2 and 3. By taking the results of experiment 1 and 2 together, there is conclusive evidence for the view that unbalanced manipulations on the input side did not affect the influence of the accessory on the speed of reactions. We stress that the contribution of the visual intensity (experiment 1) as well as the duration (experiment 2) were additive to the contribution of the accessory. In contrast, experiment 3 revealed an interaction between the degradation factor
52
P.J.G. Keuss et al. / Auditory accessory effects on visual processing
and the intensity of the accessory. It appeared that more degraded stimuli took more profit of the intensity than less degraded stimuli. The data of experiment 3 suggest that the accessory exerts a double influence on the reacting subject. It acts on a motor stage, viz., motor adjustment, and on a perceptual stage, viz., feature extraction. In this respect the data are in accordance with Sanders’ (1983) linear stage model based on two energetical mechanisms (see also Sanders 1990). Our addition is that one and the same accessory differentially affects both energetical mechanisms in producing immediate arousal as well as activation, in terms of Sanders’ model. Note that the present results are also not at odds with the standpoint of Meulenbroek and Van Galen (1988) who stress the functional character of the accessory, comparable to foreperiod duration and instructed muscle tension. Logical grounds for the dual character in our data pattern (experiments 1 and 2 versus 3) may be the following. On the first place it can be argued that two manipulations (intensity and brightness) operate on a very peripheral level, in terms of the activity of the nervous system of the eye. On the other hand, the recognition of degraded stimuli, distorted by deleted and/or displaced information dots, might involve a processing capacity which is more centrally located in the nervous system of the visual modality. This view is suggestive for the locus where the arousal is operating on the visual system. First, there is physiological evidence that an arousal system of the brain exists, as identified by EEG patterns (cf. the classic study of Lansing et al. 1959). Event-related potentials are an index for such an attentive state produced through action of the ascending reticular system. Such a state may emerge from stimulation attributable to the auditory accessory (Karlin et al. 1971; Roth et al. 1982). The gravamen of the suggestion that the accessory acts on visual processing hinges upon the idea that the encoding of the visual stimulus takes place at a higher central level as the majority of experimental psychologists tends to believe since Sternberg (1969). To defend the thought of the present report, it is counter-intuitive to localize the feature extraction process at the same peripheral level of the nervous system where intensity and brightness are being processed by the visual modality. In this respect it is interesting to notice that Sanders’ (1983) model makes a distiction between stimulus preprocessing and the later perceptual stages. It seems acceptable to believe that feature extraction
P.J. G. Keuss et al. / Auditory accessory effects on visual processing
53
involves not only stimulus intake but also interpretation of that stimulus for which brain activity is indispensable. In conclusion, our study suggests that an auditory accessory facilitates motor reactions, due to an activation effect. A second aspect of the auditory modality, not shared by the visual system, may be the arousal effect upon perceptual processing. One may infer that feature extraction, located at that stage, benefits from this arousal effect, in particular, when (degraded) stimuli are difficult to identify. In practice, traffic signs as arrows in puntuated presentation form and barely visible in bad weather circumstances, may be better observed or identified when sound or noise of non-informative character is added to the field situation.
References Bernstein, I.H., 1970. ‘Can we see and hear at the same time? Some recent studies of intersensory facilitation of reaction time’. In: A.F. Sanders (ed.), Attention and performance, Vol. 3. Amsterdam: North Holland. pp. 21-35. Bernstein, I.H., 1972. ‘Double stimulation and intersensory information processing’. In: L.M. Herman (chm.), Information processing: Double stimulation. Symposium presented at the meeting of the American Psychological Association, Honolulu, HI, September. Faber, H.E.L., M.W. van der Molen, P.J.G. Keuss and E.J. Stoffels, 1986. An OR analysis of the tendency to react toward the stimulus source. Acta Psychologica 61, 105-115. Frowein, H.W., 1981. Selective drug effects on information processing. Enschede: Sneldruk, Boulevard. Gielen, C.C.A.M., R.A. Schmidt and P.J.M. van den Heuvel, 1983. On the nature of intersensory facilitation of reaction time. Perception & Psychophysics, 34, 161-168. Helson, H., 1964. Current trends and issues in adaptation-level theory. American Psychologist 19, 26-38. Karlin, L., M.J. Mar& S.E. Brauth and A.M. Mordkoff, 1971. Auditory evoked potentials, motor potentials and reaction time. Electroencephalography and Clinical Neurophysiology 31, 129136. Keuss, P.J.G., 1972. Reaction time to the second of two shortly spaced auditory signals both varying in intensity. Acta Psychologica 36, 226-238. Keuss, P.J.G. and M.W. van der Molen, 1982. Positive and negative effects of stimulus intensity in auditory reaction tasks: Further studies on immediate arousal. Acta Psychologica 52, 61-72. Koster, W.G. and J.B. Peacock, 1969. ‘The influence of intensity of visual stimuli on the psychological refractory phase’. In: W.G. Koster (ed.), Attention and performance, Vol. 2. Amsterdam: North-Holland. pp. 232-253. Lansing, R.W., E. Schwartz and D.B. Lindsley, 1959. Reaction time and EEG acitivation under alerted and nonalerted conditions. Journal of Experimental Psychology 58, l-7. Meulenbroek, R.G.J. and G.P. van Galen, 1988. Foreperiod duration and the analysis of motor stages in a line drawing task. Acta Pschologica 69, 19-33. Nickerson, R.S., 1973. Intersensory facilitation of reaction time: Energy summation or preparation enhancement? Psychological Review 80,489-509.
54
P.J.G. Keuss et al. / Auditory accessoty effects on visual processing
Posner, M.I., 1978. Chronometric explorations of mind. Hillsdale, NJ: Wiley. Posner, M.I., M.J. Nissen and R. Klein, 1976. Visual dominance: An information-processing account of its origins and significance. Psychological Review 83, 157-171. Roth, W.T., G.H. Blowers, C.M. Doyle and B.S. Kopell, 1982. Auditory intensity effects on components of the late positive complex. Electroencephalography and Clinical Neurophysiology 54, 132-146. Sanders, A.F., 1980. ‘Stage analysis of reaction processes’. In: G.E. Stelmach and J. Requin (eds.), Tutorials in motor behavior. Amsterdam: North-Holland. pp. 331-354. Sanders, A.F., 1983. Towards a model of stress and human performance. Acta Psychologica 53, 61-97. Sanders, A.F., 1990. Issues and trends in the debate on discrete vs. continuous processing of information. Acta Psychologica 74, 123-167. Schmidt, R.A., S.C.A.M. Gielen and P.J.M. van den Heuvel, 1984. The locus of intersensory facilitation of reaction time. Acta Psychologica 57, 145-164. Smith, M.C., 1967. Reaction time to a second stimulus as a function of intensity of the first stimulus. Quarterly Journal of Experimental Psychology 19, 125-132. Stemberg, S., 1969. ‘The discovery of processing stages: Extensions of Donders’ method’. In: W.G. Koster (ed.), Attention and performance II. Amsterdam: North Holland. Todd, J.W., 1912. Reaction to multiple stimuli. Archives of Psychology 3, l-65. Van der Molen, M.W. and P.J.G. Keuss, 1979. The relationship between reaction time and intensity in discrete auditory tasks. Quarterly Journal of Experimental Psychology 31, 95-102. Van der Molen, M.W. and P.J.G. Keuss, 1981. Response selection and the processing of auditory intensity. Quarterly Journal of Experimental Psychology 33A, 177-184. Woodworth, R.S. and H. Schlosberg, 1956. Experimental psychology (rev. ed.). New York: Holt.
75 (1990) 41-54
AUDITORY ACCESSORY P.J.G. KEUSS,
October
EFFECTS ON VISUAL PROCESSING
*
F. van der ZEE and M.B.M. van den BREE
Free Unioersity, Amsterdam, Accepted
41
The Netherlands
1989
A series of experiments on intersensory facilitation demonstrates that non-informative sound of low to moderate intensity (30/80 dB) facilitates the reaction to a visual stimulus. By manipulating the preprocessing and perceptual stages of the visual signals, it appears that auditory intensity reduces choice reaction time independently from the positive influence of the intensity and duration of the visual imperative signal, but interacts with the effect of stimulus degradation. Degraded stimuli take more profit of the sound than intact stimuli. Besides a short-term activation effect, originated by accessories of the auditory modality, on the motor adjustment stage (cf. Sanders 1983) the results suggest that the accessory influences the stage of feature extraction.
Introduction An old issue in the field of sensory psychology, dating back to the pioneering work of Todd (1912) is intersensory facilitation. This term labels the phenomenon that a reaction to a stimulus shortens when that stimulus is accompanied by an accessory (see Nickerson’s (1973) review). Usually the response is given to a visual stimulus, while the accessory is non-informative implying that the subject needs not to attend to it in order to perform the reaction time (RT) task. The historically oldest explanation of intersensory facilitation is in terms of energy summation. The view is based on the fact that in either modality the intensity exerts a positive effect on RT (see Woodworth and Schlosberg (1956) for a review). Indeed, it is well established that the time to transmit information from the stimulus source to the cortex, including the delay of the sensory transducer and the afferent conduction time (see Nickerson’s (1973) study of above) is inversely related to * Requests Psychology,
for reprints should be sent to P.J.G. Keuss, Free University, De Boelelaan 1111, 1018 HV Amsterdam, The Netherlands.
OOOl-6918/90/$03.50
0 1990 - Elsevier Science Publishers
Dept.
B.V. (North-Holland)
of Cognitive
42
P.J.G. Keuss et al. / Auditory accessory effects on visual processing
stimulus intensity (Bernstein 1972; Koster and Peacock 1969). Psychophysiological support, cited by Smith (1967), is provided by the finding that the latency of the visual-evoked response in the EEG pattern also varies inversely with intensity. For auditory-evoked responses reference is made to the work of Karlin et al. (1971). One step further, the idea emerged that energy from the accessory combines in some way with the energy available for the reaction stimulus, leading to a more rapid processing of stimulus input. Currently, the energy summation account is no longer seriously considered because a visual accessory is not able to speed up a reaction to a visual stimulus (Helson 1964). Otherwise, it is well documented that facilitation of visual processing is caused by auditory stimulation (Posner et al. 1976). Hence, the alternative explanation is bound to the auditory stimuli, which are supposed to exert an alerting or activating effect on the reacting subject. The question of the present study is on what (sub)processes the activation takes place, when the subject is engaged in his visual reaction task. Sanders (1990) suggests that an auditory stimulus of a certain intensity level (75 dB) produces a short cut activation of the motor adjustment process. This assumption is in line with his earlier findings (Sanders 1980) demonstrating that muscle tension, directly related to motor adjustment, did interact with an accessory in a simple task but not in a choice task. In the latter case, the response selection stage may be bypassed on account of the accessory. Response selection is time consuming and bypassing of this stage (if possible) results in a double gain. For example, simple or very compatible responses (produced without the involvement of the response selection stage) may take full profit of the speeding up due to the activating accessory. In contrast, demands on response selection in case of incompatible stimulus/ response mappings counteract this facilitation. Indeed, Van der Molen and Keuss (1979) were able to demonstrate that, as heavier load was imposed on the response selection, the influence became more adverse in terms of slower reactions and errors. Furthermore, the intensity of the accessory did enlarge this negative effect in their study. When in the range of 80 till 110 dB performance deteriorates, the activation caused by the accessory might have obscured earlier effects on processing stages before the motor adjustment. Still open is thus the question what happens with lower intensities for which it is not clear that short cut activation determines the reaction of the subject. There-
P.J.G. Keuw et al. / Auditory accessory effects on visual processing
43
fore, the present study uses accessories of low to moderate intensities (30 till 80 dB) in order to explore a possible effect on input processing. Following the Additive Factor Logic (Stemberg 1969), experimental variables are chosen which are supposed to act on the earlier processing stages of visual input (see also Sternberg 1969). The first canditate is intensity which affects stimulus preprocessing. Hence, the study opens with an experiment, in which the visual reaction stimulus as well the auditory accessory varies in intensity. The idea behind the Additive Factor Method is that when both intensity factors independently contribute to the speed of responding each of the two factors affects another stage. Or, in psychological terms, one may infer that preprocessing of visual input lies outside the influence of the auditory accessory. A similar reasoning may be followed in the second experiment. Here the brightness aspect of the visual stimulus, which is assumingly also affecting stimulus preprocessing, is varied by presenting the stimulus with different durations. The durations used fall within the range (till 12 msec) for which the tradeoff between intensity and duration exists (Bloch’s Law), resulting in an RT/brightness gradient, similar to the RT/intensity relationship. When the intensity of the accessory does not interact with the duration of the visual stimulus, one may infer again, that the accessory does not influence the input stage being responsible for brightness processing. According to the AFM logic the reappearance of additivity in the second experiment verifies the independent contributions of the accessory and visual input variables to RT, as expected in experiment 1. The literature refers to the perceptual stage of feature extraction as the stage in which a central representation of the stimulus is built up (cf. Posner 1978). Feature extraction can be regarded as one of the (sub-)stages of perceptual processing (Sanders 1983) taking place before the identification of that stimulus is completed (Schmidt et al. 1984). The main factor that affects the quality of the stimulus or nature (in terms of Schmidt et al. 1984) is stimulus degradation (Frowein 1981). Stimulus degradation manipulation is being manipulated in the last experiment. Posner (1978) believes that in general the accessory acts to terminate input processing more quickly. In his view, the accessory is expected to lower the criterion for delivering the response before the process of information accrual is accomplished. Degraded stimuli demand a higher
44
P.J.G. Kews et al. / Auditory accessory effects on visual processing
degree of accumulating evidence than intact ones. The prediction is then that the intensity of the accessory interacts with the stimulus degradation factor. However, premature termination of the information accrual process results in a reduction of accuracy. In that case the accessory actuates a change in the speed-accuracy tradeoff, without affecting the real process of visual perception. An alternative is developed by Sanders (1983, 1990). In his cognitive-energetical linear stage model of human information processing room is left for two energetical mechanisms (arousal and activation) coupled to input and output processing stages, respectively. In line with this idea one may hypothesize that the accessory influences via the arousal mechanism the input processing, apart from its, forementioned, activation effect on output processing. As in Posner’s model, the linear stage model could account for an interaction between the intensity of the accessory and stimulus quality. But, when overarousal is circumvented by using low to moderate intensities as in the present study, a change in speed-accuracy tradeoff should be absent according to the latter model. Common to the three experiments of this study is that the visual stimulus demands a (binary) choice. The rationale is that in a no-choice task the subject could engage in a switching approach allowing a kind of incomplete processing (Gielen et al. 1983). And, the subject’s strategy could either be to preserve his primary concern to what comes first into the focus of his attention, or he could even ignore the reaction stimulus all the time and respond to the faster accessory. Such a strategy is bound to occur when we consider that the conductance time of an auditory signal is considerably shorter than that of a visual signal (Nickerson 1973). In the choice situation, however, the accessory does not tell the subject which response to make (Posner 1978). In this case, to ignore the accessory may be even helpful in order to avoid erroneous reactions. In all trials of our study the accessory is present. This deviates from the traditional method of using unimodal (catch) trials as reference to estimate the (additional) intersensory effect under the bimodal condition. A point of criticism against the catch trial method is that RT is considered to be not so much a function of the particular stimulus presented but of that stimulus in relation to other possible stimulus events (Gielen et al. 1983). Consequently, the subtraction method may overestimate the size of the intersensory effect. A theoretical reservation against interspersing unimodal among bimodal trials in the present
P.J. G. Keuss et al. / Auditory accessory effects on visual processing
45
case is that a trial with an auditory accessory is qualitatively different from one without. There is evidence that any auditory stimulus influences the subject’s way of responding, since there is a tendency to respond to the stimulus source (the Simon effect), even with sounds of low intensities (cf. Faber et al. (1986). Auditory signals also enlarge the readiness of the subject, cf. the work of Sanders (1983) on the compensatory effect of sound in case of time uncertainty. Finally, the Nickerson (1973) and Bernstein (1970) studies are worth mentioning. They showed that auditory signals were even able to affect reactions to a visual signal while arriving 100 msec later than the latter. As the present investigation focuses on real time processes in the sensory system, it makes sense to offer the visual and auditory stimuli at the same time to the subject.
Experimental procedure General
All subjects were recruited from the pool of Free University students aged 20-30 years, with males and females equally distributed over the experiments. They reported normal or corrected-to-normal vision and hearing. The subjects were paid a fixed sum for participation. The instruction was to react to the binary visual signal with a key-press response, executed by left or right index finger. Reactions were measured in msec, while the timer started at the onset of the reaction signal. Equal emphasis was laid on speed and accuracy. The subject sat in a sound-attenuating cubicle, which was pitch dark in the first and dimly lit in both other experiments. On his/her table two keys were mounted, 10 cm apart, for the right- and left-hand response, respectively. One second prior to the arrival of the visual reaction stimulus, an upper red light, fixed at eye level, came on and went out at the onset of the reaction stimulus. Concurrent with the visual signal, the accessory stimulus was delivered via a headphone. The accessory was a binaurally presented tone of 200 Hz. Its intensity was 30, 40, 50, 60, 70 or 80 dB(A) against an ambient noise background level of 10 dB(A). The subject’s reaction terminated both visual and auditory stimulus. Thereafter (s)he received a sign, by means of the lighting up of a 10 cm lower red light, at eye level, indicating that (s)he had to give a confidence judgment about the reaction. Therefore (s)he pressed a foot pedal, ranging in depth from 0 (very uncertain) to 150 (very certain). The confidence scores are omitted from analysis in the present report. Reappearance of the upper red light marked a fixed intertrial interval of 10 sec. All visual stimuli, positioned or projected at 1.5 to 2.0 meters in front of the subject, subtended a visual angle of maximal 2.0 degrees, horizontally and vertically.
46
P.J.G. Keuss et al. /Auditory accessory effects on visual processing
Left/right response (2 levels), intensity of the accessory (6 levels), and manipulation of the visual stimulus (4 levels in experiment 1, 2 and 3) were factorially combined to produce the individual type of trials. Each trial type was replicated 10 times per subject. The resultant 480 trials per experiment were randomized without replacement and presented in eight mixed blocks or series of 75 each. All subjects, assigned to an experiment, run the same test implying that the experimental factors of that experiment were tested along a within-subjects design. Sixteen subjects participated in each experiment. Directly after a practice session of 30 minutes, which was also used to accustom subject’s eyes to the darkness in experiment 1, (s)he run the eight experimental series in one session of about 2 hours, with intermissions between the series of 2 to 5 minutes. During these pauses the subjects were kept informed about their performance level in terms of error proportion and speed of reactions, both being the dependent variables of the report. An ITT Apple II-e system controlled the stimuli and intervals and assembled the data. Per experiment ANOVAs were employed on the RT data using a within-subjects design. Mean RT per (sub-)condition and per subject was used as individual value. Error frequencies were analyzed (ANOVAs) in parallel with the RT data. Experiment
1 (intensity)
The visual stimuli of this experiment are illustrated in the inset of fig. 1. On a perpendicular black panel a configuration of bulbs was fastened each with a diameter and interdistance of 5 mm. It consisted of eleven horizontal bulbs plus, at the extremes, four bulbs forming in triangular form the left and right arrow point. The middle light lit up, at an intensity of 0.09 Lux, one second before the visual stimulus and served as an (extra) warning signal. The visual stimulus comprised the lighting up of the nine bulbs to the left or the nine bulbs to the right of the middle light. All lights, with rise and decay times within 5 msec, had a red color with an intensity value of 0.01, 0.02, 0.04 or 0.16 Lux against a background illuminance of zero. Intensity was the manipulation factor of this experiment. The orientation of the arrow determined, in direct correspondence, the subject’s left versus right response. Experiment
2 (brightness)
The visual stimuli of this experiment were slides, projected on a white screen. A Kodak slide projector was used whose shutter time was effectively zero. The size of the stimuli was 24 by 36 cm. Each slide consisted of a left and right her&field of equal size, being two patches of gray, differing in brightness (see fig. 2). The luminance of the hen&fields was 3.8 and 4.4 Lux, respectively, against a background illuminance of 2.7 Lux. With either the left or the right key the subject had to indicate, in spatial correspondence, the brighter hem&field which was randomly presented to the left or right part of the slide. Presentation time of the slide was the manipulation factor of the experiment with 4 levels, corresponding to slide durations of 4, 6, 8, 10 msec. The rationale behind this
P.J.G. Keuss et al. / Auditory accessory effects on visual processing
manipulation was that the duration brightness (contrast). Experiment
of stimulus
presentation
positively
41 correlates
with
3 (degradation)
Fig. 3 provides the examples of this experiment. Again slides were presented producing stimuli of the same size and with the same apparatus as in experiment 2. Each slide contained within a dotted frame punctate arrow (points) which were formed by a number of black dots against a white background. The intact stimuli, as presented in the right side of the figure, had arrows composed of eleven dots. The lower rows show the degraded stimuli, which was the manipulation factor (with 4 levels) of this experiment. Dots are displaced in the second row, or omitted in the third row of above. The lowest row shows a stimulus with extra dots spread over the slide. Each level of degradation had several instances in order to eliminate learning effects. Pooled over all instances used the difference in luminance among the levels was negligible. Level of difficulty will be defined by the grand means as will be obtained in the present experiment. The orientation of the arrow point, which was random, determined the corresponding left versus right key press response.
Results Experiment
I - Intensity
Mean RT and proportion of errors as a function of the intensity of the visual reaction stimulus and the intensity of the accessory are depicted in the upper and lower panel of fig. 1, respectively. The inverse relationship of RT with the visual intensity is very pronounced (F(3, 15) = 22.6; p < 0.01) which is parallelled by a clear decrease of erroneous performance over intensity (F(3, 15) = 35.7; p < 0.01). The RT/intensity gradient of the accessory has also a negative slope, which is less outspoken than the RT/intensity relationship in the visual domain. Nonetheless, the accessory effect was statistically reliable (F(5, 15) = 4.84; p < 0.01). Errors appeared to be equally distributed over the intensity of the accessory (F(5, 15) = 1.6; ns). No statistically reliable interactions between both intensity manipulations were obtained in the analyses employed on the RT and error data, respectively. Experiment
2 - Brightness
Fig. 2 depicts in two panels the data of the brightness manipulation. The upper panel shows the RT/presentation time relationship. Presentation time contributed significantly to the variance (F(3, 15) = 8.8; p < 0.001). Inspection of the figure shows that the RT grows shorter for durations ranging from 4 to 8 msec followed, however,
48
P.J.G. Keuss et al. / Auditory accessory effects on visual processing
kll 550
--
$ 450-E .5 k
\:
-20
350--
_ae b l 0.01
0.02
I
I
0.W
0.16
i
LUX 500
(bl t
q;;;;;;pq 30
40
60
60
70
60
dB
Fig. 1. Mean RT and error proportions of experiment 1 (intensity) are presented. In panel (a) RT as a function of the visual intensity is displayed, collapsed over the auditory intensities. In panel (b) RT as a function of the auditory intensity (accessory) is displayed. An example of the stimuli is given in the inset.
by slight increase in RT as a function of presentation time. The lower panel shows the negative RT/intensity gradient of the accessory which was significant (F(5, 15) = 13.1; p < 0.001). The analyses on the errors revealed significance (at 0.05 level) for presentation time and intensity of the accessory (F = 2.9 and F = 2.7, respectively). No statistically reliable interaction between the brightness manipulation of the visual stimulus and the intensity of the accessory were obtained for the RT and error data, respectively.
Experiment
3 - Degradation
Stimulus degradation and intensity of the accessory reached significance (F(3, 15) = 8.1; p c 0.001 and F(5, 15) = 4.6; p < 0.01, respectively). Grand means for the degradation conditions were in ascending order: condition 1 (intact stimuli) 345 msec; condition 4 (added dots) 360 msec; condition 2 (displaced dots) 372 msec; condition 3 (deleted dots) 378 msec. The interaction of these factors also contributed to the variance (F(15, 300) = 3.1; p < 0.01). Fig. 3 illustrates the interaction of the intensity of the accessory with the most and the least degraded stimuli as a parameter. It can be
P.J.G. Keuss et al. / Auditory accessory effecis on visual processing
49
M
360 t
.E
k
340--20 ae g a
I
I
I
I
4
6
6
10
exposure time hnaecl
Eso-.c
\
k 340~-
I I I 30
40
60
III
60
70
60
d6
Fig. 2. Mean RT and error proportions as a function of exposure time, collapsed over the auditory intensity (accessory), obtained in experiment 2. In panel (b) the RT/intensity gradient for the accessory is given. An example of the stimuli is given in the inset.
seen from the graphics of the upper panel that the slope of the RT/intensity gradient is steeper for the more degraded stimuli (condition 3, dots deleted) than the less degraded stimuli (condition 4, dots added). Fig. 3 (lower panel) also shows the error proportions which follow the RT trends but the intensity of the accessory fell short of significance at 0.05 level. Stimulus degradation and its interaction with the intensity of the accessory contributed significantly to the variance (F(3, 15) = 4.0; p -C0.01, and F(15, 300) = 4.1; p < 0.01, respectively). In an additional analysis omitting the 80 dB data, the Degradation By Intensity interaction fell (at 0.05 level) short of significance (F(12, 240) = 1.9).
Discussion Intersensory facilitation was consistently manifest in the data patterns of all three experiments. Visual reactions were faster with increas-
50
P. J. G. Keuss et al. / Auditory accessory effects on visual processing
r
Condition Condition Condition
4 2 3
1
i
Condition
2
; :*::
dB
L
380
“.3
Condition
i
.*‘*
f
“‘ZJ
:.. ....... :
(bl
Condition3
i :
a’ .
*’i.
I *a . i . . . . . ..:
r20 30
40
60
60
70
BO
dB
Fig. 3. Delta RT (absolute difference between RT,,,,,, RTdegraded) as a function of auditory intensity (accessory), obtained in experiment 3, with (in descending order of difficulty) condition 3 (dots deleted), condition 2 (dots displaced) and condition 4 (dots added) as a parameter. Condition 1 (intact stimuli) is reference. Panel (b) shows the RT/auditory intensity (accessory) gradient and error proportions collapsed over ah visual degradation conditions. Examples of stimuli are given in the inset.
ing intensity of the auditory accessory. The decrement amounted to 15 msec for an intensity range of 40 to 80 dB which is in accordance with what is cited in the literature (Keuss 1972). Crucial is the non-existence of a speed-accuracy tradeoff with increasing intensities, as we clearly found in two of the three experiments. This excludes the ‘by-pass’ interpretation of the accessory effect on the subject’s decision stage which is particularly relevant to the interpretation of the third experiment. This also excludes the premature termination of the information accrual process due to the accessory, as Posner (1978) proposed. The intensities used in the present study were likely too low (30/80 dB) to produce a blocking of the choice stage. An exception may form the highest intensity in experiment 3 for which an increment of errors was obtained. This outcome is, however, not at odds with the general view
P.J.G. Keuss et al. / Auditory accessory effects on visual processing
51
that 70 to 80 dB marks the limit above which (choice-)performance deteriorates, presumably due to startle (Van der Molen and Keuss 1981). Although the present study does not give direct evidence for a general response activation originated by the accessory, the data support the view expressed by Meulenbroek and Van Galen (1988), that aspecific ‘functional’ preparation may be enhanced by this sort of manipulation. Reference is made to the work of Sanders (1980, 1983, 1990), suggesting the motor adjustment stage as the most likely candidate whereupon this sort of activation is manifest. It is further noted that the activation is more outspoken as the intensity grows. In this respect the data pattern is consistent with that of Keuss and Van der Molen (1982) who used a broader range of intensities up to 110 dB. Inferring from the activation account that is exclusively bound to the auditory modality, one may expect an independence from visual processing aspects which miss the activation function. Indeed, the intensity variation of the visual stimulus, as done in the first experiment, was manifest in processing time variation of that stimulus, whereas any relationship with the auditory accessory was absent. More intense visual stimuli yielded faster responses than less intense ones, which corresponds to the well-known inverse relationship of RT with intensity. Similarly, stimuli of longer duration and, hence, in psychological respect, greater brightness (experiment 2) yielded faster reactions than those of shorter durations. In passing it is noted that the intensity variation as used in experiment 1 was quite broad, ranging from 0.01 to 0.16 Lux, corresponding with relatively long RTs varying from 550 msec till 350 msec, respectively. Still, the frequency of errors (5%) was relatively low compared by the data pattern of experiment 2 and 3. It might suggest that the processing of the orientation of the arrow can be done in a rather compatible way, but that the low intensities used here render the subjects cautious in responding which resulted in relatively high reaction times as compared by the data of experiment 2 and 3. By taking the results of experiment 1 and 2 together, there is conclusive evidence for the view that unbalanced manipulations on the input side did not affect the influence of the accessory on the speed of reactions. We stress that the contribution of the visual intensity (experiment 1) as well as the duration (experiment 2) were additive to the contribution of the accessory. In contrast, experiment 3 revealed an interaction between the degradation factor
52
P.J.G. Keuss et al. / Auditory accessory effects on visual processing
and the intensity of the accessory. It appeared that more degraded stimuli took more profit of the intensity than less degraded stimuli. The data of experiment 3 suggest that the accessory exerts a double influence on the reacting subject. It acts on a motor stage, viz., motor adjustment, and on a perceptual stage, viz., feature extraction. In this respect the data are in accordance with Sanders’ (1983) linear stage model based on two energetical mechanisms (see also Sanders 1990). Our addition is that one and the same accessory differentially affects both energetical mechanisms in producing immediate arousal as well as activation, in terms of Sanders’ model. Note that the present results are also not at odds with the standpoint of Meulenbroek and Van Galen (1988) who stress the functional character of the accessory, comparable to foreperiod duration and instructed muscle tension. Logical grounds for the dual character in our data pattern (experiments 1 and 2 versus 3) may be the following. On the first place it can be argued that two manipulations (intensity and brightness) operate on a very peripheral level, in terms of the activity of the nervous system of the eye. On the other hand, the recognition of degraded stimuli, distorted by deleted and/or displaced information dots, might involve a processing capacity which is more centrally located in the nervous system of the visual modality. This view is suggestive for the locus where the arousal is operating on the visual system. First, there is physiological evidence that an arousal system of the brain exists, as identified by EEG patterns (cf. the classic study of Lansing et al. 1959). Event-related potentials are an index for such an attentive state produced through action of the ascending reticular system. Such a state may emerge from stimulation attributable to the auditory accessory (Karlin et al. 1971; Roth et al. 1982). The gravamen of the suggestion that the accessory acts on visual processing hinges upon the idea that the encoding of the visual stimulus takes place at a higher central level as the majority of experimental psychologists tends to believe since Sternberg (1969). To defend the thought of the present report, it is counter-intuitive to localize the feature extraction process at the same peripheral level of the nervous system where intensity and brightness are being processed by the visual modality. In this respect it is interesting to notice that Sanders’ (1983) model makes a distiction between stimulus preprocessing and the later perceptual stages. It seems acceptable to believe that feature extraction
P.J. G. Keuss et al. / Auditory accessory effects on visual processing
53
involves not only stimulus intake but also interpretation of that stimulus for which brain activity is indispensable. In conclusion, our study suggests that an auditory accessory facilitates motor reactions, due to an activation effect. A second aspect of the auditory modality, not shared by the visual system, may be the arousal effect upon perceptual processing. One may infer that feature extraction, located at that stage, benefits from this arousal effect, in particular, when (degraded) stimuli are difficult to identify. In practice, traffic signs as arrows in puntuated presentation form and barely visible in bad weather circumstances, may be better observed or identified when sound or noise of non-informative character is added to the field situation.
References Bernstein, I.H., 1970. ‘Can we see and hear at the same time? Some recent studies of intersensory facilitation of reaction time’. In: A.F. Sanders (ed.), Attention and performance, Vol. 3. Amsterdam: North Holland. pp. 21-35. Bernstein, I.H., 1972. ‘Double stimulation and intersensory information processing’. In: L.M. Herman (chm.), Information processing: Double stimulation. Symposium presented at the meeting of the American Psychological Association, Honolulu, HI, September. Faber, H.E.L., M.W. van der Molen, P.J.G. Keuss and E.J. Stoffels, 1986. An OR analysis of the tendency to react toward the stimulus source. Acta Psychologica 61, 105-115. Frowein, H.W., 1981. Selective drug effects on information processing. Enschede: Sneldruk, Boulevard. Gielen, C.C.A.M., R.A. Schmidt and P.J.M. van den Heuvel, 1983. On the nature of intersensory facilitation of reaction time. Perception & Psychophysics, 34, 161-168. Helson, H., 1964. Current trends and issues in adaptation-level theory. American Psychologist 19, 26-38. Karlin, L., M.J. Mar& S.E. Brauth and A.M. Mordkoff, 1971. Auditory evoked potentials, motor potentials and reaction time. Electroencephalography and Clinical Neurophysiology 31, 129136. Keuss, P.J.G., 1972. Reaction time to the second of two shortly spaced auditory signals both varying in intensity. Acta Psychologica 36, 226-238. Keuss, P.J.G. and M.W. van der Molen, 1982. Positive and negative effects of stimulus intensity in auditory reaction tasks: Further studies on immediate arousal. Acta Psychologica 52, 61-72. Koster, W.G. and J.B. Peacock, 1969. ‘The influence of intensity of visual stimuli on the psychological refractory phase’. In: W.G. Koster (ed.), Attention and performance, Vol. 2. Amsterdam: North-Holland. pp. 232-253. Lansing, R.W., E. Schwartz and D.B. Lindsley, 1959. Reaction time and EEG acitivation under alerted and nonalerted conditions. Journal of Experimental Psychology 58, l-7. Meulenbroek, R.G.J. and G.P. van Galen, 1988. Foreperiod duration and the analysis of motor stages in a line drawing task. Acta Pschologica 69, 19-33. Nickerson, R.S., 1973. Intersensory facilitation of reaction time: Energy summation or preparation enhancement? Psychological Review 80,489-509.
54
P.J.G. Keuss et al. / Auditory accessoty effects on visual processing
Posner, M.I., 1978. Chronometric explorations of mind. Hillsdale, NJ: Wiley. Posner, M.I., M.J. Nissen and R. Klein, 1976. Visual dominance: An information-processing account of its origins and significance. Psychological Review 83, 157-171. Roth, W.T., G.H. Blowers, C.M. Doyle and B.S. Kopell, 1982. Auditory intensity effects on components of the late positive complex. Electroencephalography and Clinical Neurophysiology 54, 132-146. Sanders, A.F., 1980. ‘Stage analysis of reaction processes’. In: G.E. Stelmach and J. Requin (eds.), Tutorials in motor behavior. Amsterdam: North-Holland. pp. 331-354. Sanders, A.F., 1983. Towards a model of stress and human performance. Acta Psychologica 53, 61-97. Sanders, A.F., 1990. Issues and trends in the debate on discrete vs. continuous processing of information. Acta Psychologica 74, 123-167. Schmidt, R.A., S.C.A.M. Gielen and P.J.M. van den Heuvel, 1984. The locus of intersensory facilitation of reaction time. Acta Psychologica 57, 145-164. Smith, M.C., 1967. Reaction time to a second stimulus as a function of intensity of the first stimulus. Quarterly Journal of Experimental Psychology 19, 125-132. Stemberg, S., 1969. ‘The discovery of processing stages: Extensions of Donders’ method’. In: W.G. Koster (ed.), Attention and performance II. Amsterdam: North Holland. Todd, J.W., 1912. Reaction to multiple stimuli. Archives of Psychology 3, l-65. Van der Molen, M.W. and P.J.G. Keuss, 1979. The relationship between reaction time and intensity in discrete auditory tasks. Quarterly Journal of Experimental Psychology 31, 95-102. Van der Molen, M.W. and P.J.G. Keuss, 1981. Response selection and the processing of auditory intensity. Quarterly Journal of Experimental Psychology 33A, 177-184. Woodworth, R.S. and H. Schlosberg, 1956. Experimental psychology (rev. ed.). New York: Holt.
