JOURNAL
Effects
OF
EXPERIMENTAL
of Input Visual
CHILD
PSYCHOLOGY
Conditions and
19, 63-78 (1975)
on lntramodal
Kinesthetic
Matches
SUSANNA
and
Crossmodal
by Children1
MILLARD
University of Oxford, England Three experiments are reported on the relation between children’s intramodal and crossmodal visual and kinesthetic performance under conditions varying the difficulty of the input patterns. Crossmodal recognition errors exceeded intramodal errors on distance patterns with two and four constituents by ten-year-olds (Expt l), and by 5.5-year-olds on single and double distance patterns (Expt 2). Preschoolers showed no significant crossmodal deficits in the recognition of single and double patterns (Expt 2), or in recall of single lengths presented in blocked or alternating order (Expt 3). No interactions between crossmodal errors and pattern difficulty were found. Order of presenting the patterns (Expts 1 and 2), and blocked versus alternating presentations (Expt 3) had significant effects on the relation between intramodal and crossmodal errors. The results are discussed with reference to explanations of crossmodal matching.
Human adults (Krauthammer, 1968), young children (Blank & Bridger, 1964; Blank, Altman & Bridger, 1968; Millar, 1971; Rude1 & Teuber, 1971)) preverbal infants (Bryant, Jones & Claxton, 1972), and apes (Davenport & Rogers, 1970) can match information from different sense modalities. The suggestion (Ettlinger, 1967) that intersensory performance necessitates verbal mediation is thus ruled out. The processes that do underly this ability are, however, not yet well understood. A number of studies report that matching stimuli from different modalities produce more errors and/or are slower than matching inputs within the same modality (Chase & Calfee, 1969; Connolly & Jones, 1970; Harvey, 1973; Jones & Connolly, 1970; Kress & Cross, 1969; Legge, 1965; Rubinstein & Gruenberg, 1971). One explanation is that stimuli within the same modality (intramodal) can be matched directly on sensory impressions, while inputs from different modalities (crossmodal) require translation via special learned integrative or categorizing sys‘This study was supported by a Grant from the Social Science Research Council which is gratefully acknowledged. ‘Requests for reprints should be sent to Dr. S. Millar, Department of Experimental Psychology, South Parks Road, Oxford, OX1 3PS, England. 63 Copyright @ 1975 by Academic Press, Inc. All rights of reproduction in any form reserved
64
SCSANNA
hIILLAR
terns (Birch & Lefford, 1963, 1967; Connolly & Jones, 1970). However, other studies suggest that crossmodal errors reflect errors in the respective intramodal tasks (Cashdan, 1968; Garvill & Mollander, 1969; Goodnow, 1971; Rose, Blank & Bridger, 1972; Rude1 & Teuber, 1964, 1971). Gibson (1969) held that matching both within and across modalities depends upon detecting invariant, relational, amodal features in the stimuli. Crossmodal errors only arise when there are differences between modalities in detecting these. Studies manipulating such differences are worth considering in some detail. One difference is that visual inputs are simultaneous, while tactual, kinesthetic, and auditory ones are sequential. Young children certainly utilize visual information more efficiently than either tactual (Goodnow, 1971; Millar, 1971; Milner & Bryant, 1970; Rude1 & Teuber, 1964, 1971), auditory (Rude1 & Teuber, 1971), or kinesthetic (Millar, 1972a), and this difference decreases with age. The difficulty does not lie in integrating simultaneous and successive inputs as is sometimes suggested, but in processing sequential ones. Rude1 and Teuber (1964, 1971) found no specific difficulty by young children with simultaneous visual, and sequential tactual or auditory patterns. Crossmodal errors lay halfway between the least accurate intratactual or interauditory, and the most accurate intravisual matches. When visual patterns were sequentially presented, matching these not only became more difficult than matching auditory patterns, but crossmodal errors now rose to the level of the most difficult input. Milner and Bryant (1970) argued that it is permissible to attribute crossmodal errors solely to failures in the more difficult modality, only if crossmodal errors are significantly less than errors within the more difficult modality (e.g., Rude1 and Teuber’s first results). In the latter case two poorly discriminated inputs have to be compared, while in each of the crossmodal tasks only one difficult, but also one easy input is received. If crossmodal errors do not differ from errors in the most difficult modality, failure in intersensory organization cannot bc ruled out. Rude1 and Teuber explained their results by young children’s reliance on normal visual input. Millar (1971)) for instance, showed that tactual recognition by preschoolers was improved when standards were both viewed and felt. Nevertheless, in explaining the changes in error patterns, the possibility that increased input difficulty also had some specific effects on crossmodal matching cannot be ruled out. Mimer and Bryant (1970) attributed error changes to difficulties in retrieving crossmodal information under delay, on the grounds that children’s visual-tactual matches were better than intratactual matches under 0-set, but not under 5-set delays. The significant interaction between modalities and delay, which this assumption necessitates, was only
INTRAMODAL
AND
CROSSMODAL
MATCHING
65
found by Rose, Blank and Bridger (1972)) but this was contaminated by ceiling effects. None of the errors under simultaneous input were significantly different from zero. They suggested, as did Goodnow (1971)) that delay affects crossmodal errors because visual standards are coded and stored efficiently while tactual information decays rapidly with delay. Posner (1967) proposed such coding and storage differences for visual and kinesthetic judgments on the grounds that task-filled delays affect visual, but not kinesthetic judgments. The relevant comparisons between task-filled and unfilled delays were made by Millar (197213). Visual and tactual matches differed as predicted, but effects of task-filled compared to unfilled delays on crossmodal matches indicated that subjects could choose whether to rely mainly on information from visual standards or from comparison stimuli. Errors and latencies showed that crossmodal matching may indeed be limited by discrimination and coding differences between modalities, but not solely because of tactual failures. Coding differences and the fact that inputs are from different modalities add information which may require extra decisions in the choice of coding strategies. Millar (1972a) showed that crossmodal errors can change (rather than decrease) for older children who can utilize additional information. Thus, comparing inputs should probably not, in any case, be regarded as a passive process, limited only by discrimination and retention difficulties, but as requiring active choices which are affected by what strategies are available to subjects, as well as by input conditions and modality differences. Difficulties in processing sequential information, and discriminability and coding differences between modalities have thus emerged as important factors in crossmodal matching. At the same time the evidence makes it doubtful whether such differences can totally account for all crossmodal errors, especially for results showing either one, or both, crossmodal matches to be less efficient than matches within the most difficult modality (e.g., Connolly & Jones, 1970; Chase & Calfee, 1969; Harvey, 1973). The question is whether this necessitates the opposite assumption of special, learned translation mechanisms. The results on infants and the lack of evidence for differential decreases in crossmodal errors with age (Milner & Bryant, 1970) count against it, since the theory assumes crossmodal improvement with age. They are not necessarily decisive, however. Advocates could argue that infants can only so match the simplest stimuli, and that age interactions may be masked by differential rates of improvement in translation systems and in within-modality processing. A more decisive test is whether differences in stimulus complexity affect crossmodal errors differentially. Translation difficulties are
66
SUSAKNA
MILLAR
presumably greater for more complex stimuli. The assumption of special translation and integrative mechanisms should thus predict an interaction between stimulus difficulty or complexity, and crossmodal errors. On the assumption that crossmodal errors are completely accounted for by errors in the more difficult modality, this interaction is less likely. Even if increased stimulus complexity affects processing more in one modality, increases in errors within this modality would raise the average level of errors in the two within modality matches, as well as the average level of crossmodal errors. There have been few direct studies of this. Blank, Higgins and Bridger (1971) argued that children’s crossmodal errors are due to differences in processing stimuli of different complexity, in analogy with such effects on intravisual matches, but the data contain no crossmodal controls. Rubinstein and Gruenberg (1971) found that crossmodal matches of visual and auditory rhythmic patterns by adults were more difficult than intramodal ones with 4.5 stimuli per set, but not for patterns with only 3 stimuli per sec. They held that intcrsensory processing is less efficient, except in limiting cases, because inputs can only be compared in areas of joint association. However, they obtained these results in successive cxperiments in which some of the same subjects took part. The present study was, therefore, designed to test the effects of stimulus difficulty on the relation between intramodal and crossmodal visual and kinesthetic errors by children. Visual and kinesthetic inputs were made as similar as possible by presenting visual inputs by a moving light point on a joystick, and kinesthet’ic ones by the felt, movements of the joystick. Both inputs were thus sequential. Previous studies of visual and kinesthetic judgments used recall tests. Recognition rather than recall was employed here, since the latter may impose greater memory load or retrieval difficulties (Kinch, 1969), and these may be involved in crossmodal errors. EXPERIMENT
1
Two hypotheses were to be tested: (a) Crossmodal errors exceed intramodal errors in visual and kinesthetic recognition by children; (b) crossmodal errors increase differentially with an increase in the number of constituents in the input pattern. Method A Same/different recognition task was used in an Input pattern X Modality, within-S design. Subjects. Twenty-four children, mean age 10 years, 0 months (range: 9 years, 5 months to 10 years, 7 months), from a local Authority school, took part. There were equal numbers of boys and girls.
INTRAMODAL
AND
CROSSMODAL
MATCHING
67
Apparatus. The apparatus consisted of two joysticks constrained by metal bars to move in unison. The movement of S’s joystick could thus be controlled either by the subject or by the experimenter. The joysticks were housed in a unit which had a viewing window on the sloping front panel facing the S, fitted with a shutter (closing the window in kinesthetic condition), which could be operated from the experimenter’s side (opposite to the S) . Visual inputs to the subject were from a s-mm light bulb fitted to the top of S’s joystick. Movement of the lighted bulb was guided via E’s joystick and viewed by S through the viewing window, open in the appropriate conditions. In kinesthetic conditions, S held his joystick in a pencil hold just below the light bulb via a sleeved opening below the viewing window. A circular plate prevented S from grasping the stick at a lower point. For kinesthetic inputs, the S moved his joystick to prepared stops. Only extensor movements were used. A ledge, fitted inside the experimenter’s end of the unit, restricted the movements to the horizontal and bore a moveable bar to stop the S’s movements at appropriate distances before being moved on or removed. Inputs were thus from the subject’s movement of his joystick (kinesthetic) or from movements of the lightbulb on his joystick (visual) guided by the experimenter (E). Input patterns. Visual and kinesthetic analogs of Rubinstein and Gruenberg’s patterns were used. The more difficult (Q pattern) consisted of three 30” and one 10” successive movements (traced either by the light or the felt movements of the joystick) marked off from each other by 3-set stops between the distances. The smaller distance could occur either in the first, second, third or fourth position. If X stands for the 10” distance, and 0 for the 30” distance, Q patterns consisted of either X000, 0X00, 00X0 or 000X, presented in random order (an equal number of each for “same” and “different” tests, alternated in Gellermann order). The easier (D) pattern had two constituents, one 10” and one 30” SUCcessive movement, marked from each other by a 3-set stop. The D patterns (X0 or OX) were presented in random order (an equal number of each for “same” and “different” tests in Gellerman order). Half the subjects were tested first on D patterns, and after not more than 4 days on Q patterns. For the other half of the subjects the order was reversed. Modality. Presentation and Test modes were either visual or kinesthetic, yielding two intramodal (Visual-visual and Kinesthetic-Kinesthetic), and two crossmodal (Visual-kinesthetic and Kinesthetic-visual) within-S conditions, blocked in across-S counterbalanced order. Procedure. The Ss were tested singly. The S was seated in front of the apparatus at a comfortable height. The patterns were presented as car or bus journeys, with short stops where passengers could alight. The S was to notice where the stops occurred on the first journey (“remember”) and to judge (“judge”) whether stops on the second journey occurred at
68
SITSANNA
MILLAR
exactly the same distances or not. Test presentations followed after return of joystick to starting point (approx 2 set) . The Ss were familiarized with the appropriate pattern and given trial runs in every modality condition prior to tests, to ensure that instructions were followed. Trial runs were not counted. A total of eight tests (same/different, alternated in Gellermann type order) were run for each S in each of the two intramodal, and two crossmodal conditions (a total of 62 responses per subject under the two patterns). RESULTS
AND
DISCUSSION
Errors were subjected to a three-way Anova with Order of presentation (D followed by Q, Q followed by D), Input pattern (D or Q), and Modality (intramodal, crossmodal) as factors. (Mean errors are shown in Table 1). Input pattern had a significant effect (F(1,22) = 9.09, p < .Ol), showing that there were fewer errors under D than under Q patterns. The other main effect was of Modality (F(1,22) = 15.60, p < 901). Table 1 shows that errors for the two crossmodal tasks were higher than errors for either of the intramodal tasks. The hypothesis that crossmodal errors exceed intramodal errors on recognition tests was thus confirmed. The interaction between Input pattern and Modality (p < .25) was not significant. The hypothesis that crossmodal errors would increase differentially with the number of constituents in the input pattern was not supported. A significant interaction between Order of presentation and Modality (P(1,22) = 5.12, p < .05) was found. This might represent a purely fortuitous population difference. However, since subjects were allocated randomly to the two orders of presentation and this produced no main TABLE 1 MEAN RECOGNITION ERRORS BY lo-YEAR-OLDS UNDER VISUAL-VISUAL (V-V), KINESTHETIC-KINESTHETIC (K-K), AND UNDER VISUAL-KINESTHETIC (V-K) AND KINESTHETIC-VISUAL (K-V) CONDITIONS FOR DOUBLE AND QUADRUPLE INPUT PATTERNS (EXPERIMENT 1) MODALITY Crossmodal
Intramodal Input pattern Double Quadruple Means
v-v
K-K
Means
V-K
K-V
Means
.92 1.46 1.19
1.63 2.25 1.94
1.28 1.86 1.57
2.13 2.71 2.42
2.04 2.58 2.31
2.09 2.65 2.37
INTRAMODAL
AND
CROSSMODAL
MATCHING
69
AGE 10
35
0,&v
Drdn D-Q
I DFattern
Q RnHn
DPalkm
Q-D
QFaum
FIG. 1. Mean errors by lo-year-olds for intramodal (INTRA) and crossmodal (CROSS) modality conditions for Double (D) and Quadruple (Q) patterns in presentation orders D followed by Q (D-Q) and Q followed by D (Q-D), (Expt 1).
effect of difference, the possibility that crossmodal errors were affected differentially by the preceding task could not be discounted. The relevant terms are graphed in Fig. 1. This shows that when the more difficult pattern (Q) preceded the easier (D) one, crossmodal errors were relatively smaller than when the simpler D pattern preceded the more difficult Q pattern. This will be discussed further below. EXPERIMENT
2
Two arguments could be presented against accepting the results of Expt 1 as conclusive evidence against the assumption that crossmodal matching depends upon translation via special learned associations or integrative systems. Firstly, although Q patterns proved more difficult than D patterns, both involved the identification of the position of the smaller of the distances in the standard and comparison patterns as either same or different. Thus the translation problem might have been the same for the two patterns. Secondly, IO-year-olds might already have established translations systems whereby both could be accomplished. In theory this is less likely at younger ages. Two groups of younger children were, therefore, tested on the D pattern, and on single lengths of 10” and 30”. It was assumed that judging single lengths would present a
70
SUSANNA
MILLAR
different and easier ‘%ranslation” problem, since it involved only judgments of distance and not the additional task of locating the position of the smaller (or larger) of the two distances in the pattern, as in D. Three hypotheses were thus to be tested: (a) There are differential effects of input pattern difficulty on crossmodal performance; (b) these are larger for the younger of the two groups; and (c) the difference between intramodal and crossmodal errors is larger for the younger of the two groups. Method A same/different (“same, ” ‘(not the same”) recognition task was used in an Age (4.5, 5.5) X Input pattern (S,D) X Modality (intramodal, crossmodal) design with repeated measures on the last two factors. Subjects. Subjects were 24 children, an equal number of boys and girls, 12 from an Infant class in the same school as the older children in Expt 1, mean age 5 years, 8 months (5 years, 4 months to 6 years, 11 months), and 12 from a Nursery school in the same catchment area, mean age, 4 years, 6 months (3 years, 11 months to 4 years, 11 months). Input patterns. The easier pattern (S) consisted of two distances, one of 10” and one of 30”, alternated in random order (an equal number for each of same and different tests, alternated in Gellerman order). The D rather than Q pattern described in Expt 1 was used as the more difficult pattern to avoid ceiling effects on errors. Half the subjects were tested first on the S pattern, and after an interval of l-3 days on the D pattern. For the other half of the Xs, the order was reversed. Equal numbers of boys and girls were randomly allocated to the two presentation orders. Apparatus, modality conditions and procedures. These were exactly as described in Expt 1. RESULT
AND
DISCUSSION
Errors were subjected to a four-way Anova, with Age, Order of Presentation, Input pattern and Modality as factors. A significant effect of Age (F(1,44) = 5.88, p < .05) meant that the older group produced fewer errors than the preschoolers. The assumption that S patterns would be easier than D patterns was borne out by the main effect of pattern (F(1,44) = 19.92, p < .OOl). The main effect of Modality (F(1,44) = 10.28, p < .005) showed that crossmodal errors exceeded intramodal errors. (Mean errors are shown in Table 2). Order of presentation (S followed by D, D followed by S) was included as a factor because of its effect in Expt 1. It produced both a
INTRAMODAL
AND
CROSSMODAL
TABLE MEAN (V-V),
RIXOGNITION ERRORS BY 4.5 KINESTHETIC-KINESTHETIC AND KINESTHETIC-VISUAL SINGLE AND DOUBLE
71
MATCHING
2
AND
.!~~-YEAR-OLDS UNDER VISUAL-VISUAL (K-K), AND VISUAL-KINESTHETIC (V-K), (K-V) MODALITY CONDITIONS FOR PATTERNS (EXPERIMENT 2)
MODALITY Intxamodal
Crossmodal
Input pattern
v-v
K-K
Means
V-K
K-V
Means
4.5
Single Double Means
2.42 3.33 2.88
3.50 3.92 3.71
2.96 3.63 3.30
3.33 3.83 3.58
3.67 3.58 3.63
3.50 3.71 3.61
5.5
Single Double Means
1.92 3.17 2.55
2.08 3.33 2.71
2.00 3.25 2.63
2.58 3.83 3.21
2.75 3.67 3.21
2.67 3.75 3.21
Age
main effect (F(1,44) = 14.57, p < .OOl) and a significant interaction with Input pattern and Modality (F (1,44) = 8.42, p < .Ol). The interactions are shown graphically in Fig. 2. It is clear that increasing the number of constituents in patterns (D vs S) did not as such increase crossmodal errors differentially. (The Input pattern and Modality interaction was positive but not significant.) What the data for 5.5-year-olds show is that the difference between intramodal and crossmodal errors was larger for both the patterns when they were presented first than when they were presented after the other pattern. At the same time, this was not merely a warm-up or facilitating effect. When S preceded D, crossmodal errors on D were reduced compared to the first presentation of D. When D was presented first, crossmodal errors on S were not different, but intramodal errors increased compared to the first presentation of S. This suggests that there were proactive effects from the prior task, the direction of which depended on the type of input pattern, but does not support the assumption that crossmodal errors increase differentially with input pattern difficulty as such. The interaction between the main effects and age did not reach significance level. However, since such effects were predicted, and the interactions between age and pattern, age, order and modality, and age, order, pattern and modality were positive (p < .lO), analyses of simple effects for the two age levels were carried out. These showed that for 5.5-yearolds, Order of presentation (p < .Ol) , Pattern (p < .OOl), and modality (p < .005) were significant, but for 4.5-year-olds only Order of presentation (the S-D sequence was easier) was significant (p < .025). Since
72
SCSANNA
MILLAR
FIG. 2. Mean errors by 5.5- and 4.5-year-olds for intramodal (INTRA) and crossmodal (CROSS) conditions for single (S) and Double (D) patterns in presentation orders S followed by D (S-D) and D followed by S (D-S), (Expt 2).
neither Input pattern, nor modality, produced significant effects for preschoolers, the hypothesis that the difference between crossmodal and intramodal errors is larger for the younger subjects was not supported. If anything, the tendency was in the opposite direction. Inspection of Table 2 shows that while for 5.5-year-olds, errors on the crossmodal tasks exceeded errors on either of the intramodal tasks, for 45year-olds, errors on the intrakinesthetic task exceeded errors on one or both of the crossmodal tasks. EXPERIMENT
3
A third experiment was undertaken to test whether using a recall task would produce the crossmodal deficit (see above) in preschoolers performance not found in Expt 2. To minimize possible ceiling effects, only the S pattern (single length of 10” and 30”) was used, presented either as in Expt 2 (alternating) or in blocks. The latter condition was included as a, presumably, still easier task, and to ascertain whether such an apparently minor variation in presentation would affect the intramodalcrossmodal difference.
INTRAMODAL
AND
CROSSMODAL
MATCHING
73
Method Reproduction of lengths was used in Presentation type (blocked, alternating) X Modality (intramodal, crossmodal) design with repeated measures on the last factor. Subjects. The Ss were 18 local Authority Nursery school children, mean age, 4 years, 1 month (3 years, 3 months to 4 years, 8 months). An equal number of boys and girls were allocated randomly to blocked and alternating presentation conditions. Apparatus and task. The apparatus and presentation of standards were the same as for the previous experiments. For kinesthetic tests the S was to reproduce the distance by moving his joystick to the same distance as the standard. For visual tests, the S stopped the movement of the light by saying “stop” at the appropriate distance. Tests followed approximately within 2 sec. Two circular potentiometers, fitted to the E’s joystick, drove the X and Y inputs from a Hewlett Packard 7035B S-Y recorder through zeroing and scaling units. The direction and extent of movements of the joysticks were directly transcribed onto graph paper and errors measured in millimeter deviations. Presentation type and modality. Standards were 10” or 30” distances, presented either in blocks or alternated in Gellerman order (as for S patterns in Expt 2). For blocked presentation, half the runs were on the 10” length followed by the 30” length after a 5-min break (or the reverse). Equal numbers of boys and girls were allocated to the two conditions. Modality conditions were exactly as for Expts 1 and 2. Procedure. The task was presented as a game of playing bus drivers. Trial runs to ensure that instructions were being followed preceded every change in modality conditions and were not counted. The Xs received eight runs in every modality condition (32 runs). In all other respects procedures were the same as for Expts 1 and 2. RESULTS
AND
DISCUSSION
Mean absolute errors are shown in Table 3 in terms of centimeter deviations. A Presentation type (blocked, alternating) X Modality (intramodal, crossmodal) analysis of variance, showed an interaction between the two factors (F(1,14) = 5.17, p < .05). Simple effects showed that in blocked presentation, absolute errors between intramodal and crossmodal tasks did not differ, but intramodal errors were larger than crossmodal errors in alternating presentations (p < .025). This is opposite to the predicted effect. The significant interaction of the main terms demonstrated that even small differences in the type of presentation had effects on the intra-
74
SUSANNA
MILLAR
TABLE 3 MEAN ABSOLUTI: ERRORS (ON DEXIATIONS) IN REPRODUCING SINGLE LENGTHS UNDER BLOCKED AND ALTERNATING PRESENTATION IN VISUAL-VISUAL (V-V), KINESTHETIC-KINESTHETIC (K-K) AND VISUAL-KINESTHETIC (V-K) AND KINESTHETIC-VISUAL (K-V) MODALITY CONDITIONS (EXPERIMENT 3) MODALITY Intramodal
Crossmodal
Presentation type Blocked Alternating Means
v-v 1.5 1.9 1.7
K-K
Means
V-K
K-V
Means
4.5 6.5 5.5
3.0 4.2 3.6
4.5 3.4 4.0
2.0 2.8 2.4
3.3 3.1 3.2
modal-crossmodal relation. However, the effect was on intramodal performance. Table 3 shows that this was due to larger intrakinesthetic errors under alternating presentations. A presentation type X Input Modality analysis on kinesthetic test results confirmed this. Input modality (V or K), (F(1,14) = 6.33, p < .05), and the interaction of presentation type and input modality (F(lJ4) = 6.33, p < .05) were significant. This is of interest especially since it shows that presentation type increased intramodal (K-K) errors without increasing crossmodal (V-K) errors. The size of the errors showed that the patterns were not easy, nor were they so difficult as to preclude significant differences between responses. SUMMAR.Y
AND
GENERAL
DISCUSSION
There was no evidence of any differential effect on crossmodal performance of an increase from two to four constituents in distance patterns for IO-year-olds (Expt 1) or from single to double lengths for younger children (Expt 2). Crossmodal errors were significantly larger than intramodal errors for lo-year-olds (Expt l), and for 5.5-year-olds (Expt 2) in recognition tests. For preschoolers, neither recognition of single and double lengths (Expt 2), nor recall of single lengths (Expt 3) showed any difference between intramodal and crossmodal errors here. These findings add strong evidence against the theory that crossmodal performance depends upon special learned integrative systems necessary for, and specific to, crossmodal translation. It is reasonable to predict from it that more difficult patterns will be more difficult to translate. If anything, the tendency to an age interaction (Expt 2) was in the opposite direction to that predicted on the theory. A further question is how far the results support the opposite claim
INTRAMODAL
AND
CROSSMODAL
MATCHING
75
that crossmodal errors are completely explained by errors in the more difficult modality. The absence of interactions between stimulus complexity as such, and crossmodal errors, and the fact that the averages of crossmodal and intramodal errors did not differ for the two preschool groups (Expts 2 and 3) would be expected on the theory. However, the strongest assumption that can be made in its favor, in order to explain the significantly larger crossmodal than intramodal errors for older subjects (Expts 1 and 2), is that accuracy in comparing two inputs reduces to the level of the more difficult one. Even this could not allow for the fact that errors in each of the crossmodal tasks exceeded those in the more difficult of the within-modality matches (Expts 1 and 2 for the older group). More importantly, it would be difficult to justify that this assumption does not apply equally to matches by preschoolers, even more when their intrakinesthetic errors were relatively very large (Expt 3), and why significantly increased intrakinesthetic errors under alternating presentation were not reflected in crossmodal matches (Expt 3). The significant effects of order of presenting complex and simple patterns on crossmodal errors (Expts 1 and 2) also require additional assumptions. The findings, together with others reported in the literature, suggest that this theory requires modification. An alternative explanation is that differences in discriminability, coding, and input difficulty affect subjects’ decisions in the choice of coding strategies (Millar, 1972b), rather than that errors are completely determined by discrimination or storage capacities. The significant order and interaction effects throw some light on this. It is unlikely that all these could be due to fortuitous population differences in three separate experiments with random allocations of subjects, nor could they be explained by practice in crossmodal matching. Inspection of the data shows that the direction of the effect on crossmodal matches (Expts 1 and 2) depended on whether the easier or the more difficult pattern had been presented first. This suggests that the prior task affected subjects’ strategies in dealing with the inputs. The hypothesis makes the not unreasonable assumption that subjects who are asked to compare two successively presented stimuli have more than one strategy open to them. They could code either or both inputs on some abstracted feature, or amodal category, or assign labels, or picture it if the input is visual (Tversky, 1969; Millar, 1972~). Tversky (1969, 1973) showed that coding choices depend on which of these subjects judge more economical in given tasks. Such choices are available also in comparing inputs from the same modality. But in comparing stimuli from different modalities, information about discrepancies in inputs, and in the source of stimulation arc added. Extra decisions to
76
STlSANNA
MILLAR
ignore discrepancies, or to use some amodal code, or to rely more on the ‘lbetter” input may make the choice of strategy less obvious and more dependent on task variables; for instance, whether relatively easy or more complex stimuli are presented first, This could have direct effects (increasing latencies) or indirect effects (disturbing storage or retrieval accuracy) on the efficiency of crossmodal matches. More crossmodal errors and effects of variations in task requirements on these could thus be expected. The hypothesis does not imply that crossmodal matching is always, or necessarily worse. It need not be, if coding choices are not required, or obvious, or comparisons easy, or well practiced, or if subjects only have very limited choices of strategy. Apes and infants, for instance, cannot, and young children frequently do not (Conrad, 1971) use verbal codes. Moreover, they use active strategies less, are slower to shift encoding modalities when the task actually requires this (Tversky, 1973), and are less likely to take into account all the available information. Their comparisons would, therefore, be disturbed more by difficulties in discriminating or remembering inputs than by consideration of additional or discrepant information, or by alternative coding possibilities. Either reliance on one, relatively simple, amodal code, or on dominant visual information (Millar, 1971; Rude1 & Teuber, 1971) for all matches could produce the largest errors in the least discriminable modality. The apparently counterintuitive result, that older, but not the youngest groups, showed larger crossmodal than intramodal errors thus becomes reasonable, indeed predictable in these conditions. It is instructive that most of the evidence in the literature in favor of explanations in terms of discrimination and retention difficulties has come from studies on young children. Studies on adults report, rather more frequently, results not completely compatible with this. The explanation put forward here accounts for the present results, and for previous findings of longer latenties, greater variability and effects of delay and distracters on crossmodal matching, and suggests conditions in which these do not occur. The study was designed to test the hypothesis that crossmodal matching depends upon special learned integrative translation systems by varying the difficulty of input conditions. Crossmodal errors exceeded intramodal errors by older children but stimulus complexity, as such, did not have differential effects. Order of presenting complex and simpler inputs significantly affected the relation between within- and across-modality matching by older subjects. Preschoolers showed no difference between intramodal and crossmodal errors, and effects of alternating versus blocked presentation disturbed only their intrakinesthetic judgments. The results were held to counterindicate special learned translation processes,
INTRAMODAL
AND
CROSSMODAL
77
MATCHING
but could not be completely explained by difficulties in processing the less discriminable input. An alternative explanation was suggested. REFERENCES BIRCH, H. G., & LEFFORD, A. Intersensory development in the Society for Research in Child Development, 1963,28,48.
children.
Monographs
of
BIRCH, H. G., & LEFFORD, A. Visual differentiation, intersensory integration and voluntary motory control. Monographs of the Society for Research in Child Development, 1967, 32, 87. BLANK, M., ALTMAN, L. D., & BRIDGER,W. H. Crossmodal transfer of form discrimination in pre-school children. Psychonomic Science, 1968, 10, 51-52. BLANK, M., & BRIDGER, W. H. Crossmodal transfer in nursery school children. Journal of Comparative and Physiological Psychology, 1964, 58, 277-282. BLANK, M., HIGGINS, T. J., & BRIDGER,W. H. Stimulus complexity and intramodal reaction time in retarded readers. Journal of Educational Psychology, 1971, 62, 117-122. BRYANT, P. E., JONES, P., CLAXTON, V., & PERKINS, G. M. Recognition of shapes across modalities by infants. Nature (London), 1972, 240, 303-304. CASHDAN, S. Visual and haptic form discrimination under conditions of successive stimulation. Journal of Experimental Psychology, 1968,76,215218. CHASE, W. G., & CALFEE, R.. C. Modality and similarity effects in short-term recognition memory. Journal of Experimental Psychology, 1969, 81, 510-514. CONNOLLY, K., & JONES, B. A. A developmental study of afferent-reafferent integration. British Journal of Psychology, 1970, 61, 259-268. CONRAD, R. The chronology of the development of covert speech in children. Developmental Psychology, 1971, 5, 398405. DAVENPORT, R. K., & ROGERS, C. M. Intermodal equivalence of stimuli in apes. Science, 1970, 168, 279-280. ETTLINGER, G. Analysis of cross-modal effects and their relationship to language. In J. Millikan & F. Darley (Eds.), Mechanism underlying speech and language. New York: Grune & Stratton, 1967. GARVILL, J., & MOLLANDER, B. Intra-modal and cross-modal accuracy in a form discrimination task. Umeci Psychological Report, 1969, No. 11, 8. GIBSON, E. J. Principles of perceptual learning and development. Appleton-CenturyCrofts : New York, 1969. G~~DNOW, J. J. Eye and hand: Differential memory and its effect on matching. Neuropsychologica, 1971, 9, 89-95. HARVEY, N. Does intermodal equivalence exist between heteromodal stimulus dimensions or between stimulus values on those dimensions? Quarterly Journal of Experimental Psychology, 1973, 25, 476491. JONES, B., & CONNOLLY, K. Memory effects in crossmodal matching. British Journal of Psychology, 1970, 61, 267-270. KINTON, W. Learning, memory and conceptual processes. Wiley: New York, 1969. KRAUTHAMER, G. Form perception across sensory modalities. Neuropsychologica, 1968, 6, 105-113. KRESS, G., & CROSS, J. Visual and tactual interaction in judgments of the vertical. Psychonomic Science, 1969, 14, 166166. LEGGE, D. Analysis of visual and proprioceptive components of motor skill by means of a drug. British Journal of Psychology, 1965, 56,243254.
78
SCSANNA
MILLAR
S. Visual and haptic cue utilization by preschool children: The recognition of visual and haptic stimuli presented separately or together. Journnl of Experimental Child Psychology, 1971, 12, 88-94. MILLAR, S. The development of visual and kinesthetic judgments of distance. British Journal of Psychology, 1972a, 63, 271-282. MILLAR. S. The effects of interpolated tasks on latency and accuracy of intmmodal and crossmodal shape recognition by children. Journal of Experimental Psychology, 197213, 96, 170-175. MILLAR, S. Effects of instructions to visualize stimuli during delay on visual recognition by preschool children. Child Development, 1972c, 43, 1073-1075. MILNER, A. D., B: BRYANT. P. E. Cross-modal matching by young children. Journal MILLAR,
of Comparative
and
Physiological
Psychology,
1970, 71, 453458.
M. I. Short-term memory systems in human information processing. Actn Psychologica, 1967, 27, 267-284. RUBINSTEIN, L., & GRUENBERG, E. M. Intramodal and crossmodal sensory transfer of visual and auditory sensory patterns. Perception and Psychophysics, 1971, 9, 385-390. RUDEL, R. G., & TEUBER, H. L. Crossmodal transfer of shape discrimination by children. Neuropsychologica, 1964, 2, 1-18. RUDEL, R. G., & TEUBER, H. L. Pattern recognition within and across modalities in normal and brain-injured children. T~ERSKY. B. Pictorial and verbal encoding in a short-term memory task. Perception POSNER,
and
Psychophysics,
1969, 6, 225-233.
B. Pictorial and verbal Psychology, 1973, 8, 149-153.
TVERSKY,
encoding
in preschool
children.
Developmental
Effects
OF
EXPERIMENTAL
of Input Visual
CHILD
PSYCHOLOGY
Conditions and
19, 63-78 (1975)
on lntramodal
Kinesthetic
Matches
SUSANNA
and
Crossmodal
by Children1
MILLARD
University of Oxford, England Three experiments are reported on the relation between children’s intramodal and crossmodal visual and kinesthetic performance under conditions varying the difficulty of the input patterns. Crossmodal recognition errors exceeded intramodal errors on distance patterns with two and four constituents by ten-year-olds (Expt l), and by 5.5-year-olds on single and double distance patterns (Expt 2). Preschoolers showed no significant crossmodal deficits in the recognition of single and double patterns (Expt 2), or in recall of single lengths presented in blocked or alternating order (Expt 3). No interactions between crossmodal errors and pattern difficulty were found. Order of presenting the patterns (Expts 1 and 2), and blocked versus alternating presentations (Expt 3) had significant effects on the relation between intramodal and crossmodal errors. The results are discussed with reference to explanations of crossmodal matching.
Human adults (Krauthammer, 1968), young children (Blank & Bridger, 1964; Blank, Altman & Bridger, 1968; Millar, 1971; Rude1 & Teuber, 1971)) preverbal infants (Bryant, Jones & Claxton, 1972), and apes (Davenport & Rogers, 1970) can match information from different sense modalities. The suggestion (Ettlinger, 1967) that intersensory performance necessitates verbal mediation is thus ruled out. The processes that do underly this ability are, however, not yet well understood. A number of studies report that matching stimuli from different modalities produce more errors and/or are slower than matching inputs within the same modality (Chase & Calfee, 1969; Connolly & Jones, 1970; Harvey, 1973; Jones & Connolly, 1970; Kress & Cross, 1969; Legge, 1965; Rubinstein & Gruenberg, 1971). One explanation is that stimuli within the same modality (intramodal) can be matched directly on sensory impressions, while inputs from different modalities (crossmodal) require translation via special learned integrative or categorizing sys‘This study was supported by a Grant from the Social Science Research Council which is gratefully acknowledged. ‘Requests for reprints should be sent to Dr. S. Millar, Department of Experimental Psychology, South Parks Road, Oxford, OX1 3PS, England. 63 Copyright @ 1975 by Academic Press, Inc. All rights of reproduction in any form reserved
64
SCSANNA
hIILLAR
terns (Birch & Lefford, 1963, 1967; Connolly & Jones, 1970). However, other studies suggest that crossmodal errors reflect errors in the respective intramodal tasks (Cashdan, 1968; Garvill & Mollander, 1969; Goodnow, 1971; Rose, Blank & Bridger, 1972; Rude1 & Teuber, 1964, 1971). Gibson (1969) held that matching both within and across modalities depends upon detecting invariant, relational, amodal features in the stimuli. Crossmodal errors only arise when there are differences between modalities in detecting these. Studies manipulating such differences are worth considering in some detail. One difference is that visual inputs are simultaneous, while tactual, kinesthetic, and auditory ones are sequential. Young children certainly utilize visual information more efficiently than either tactual (Goodnow, 1971; Millar, 1971; Milner & Bryant, 1970; Rude1 & Teuber, 1964, 1971), auditory (Rude1 & Teuber, 1971), or kinesthetic (Millar, 1972a), and this difference decreases with age. The difficulty does not lie in integrating simultaneous and successive inputs as is sometimes suggested, but in processing sequential ones. Rude1 and Teuber (1964, 1971) found no specific difficulty by young children with simultaneous visual, and sequential tactual or auditory patterns. Crossmodal errors lay halfway between the least accurate intratactual or interauditory, and the most accurate intravisual matches. When visual patterns were sequentially presented, matching these not only became more difficult than matching auditory patterns, but crossmodal errors now rose to the level of the most difficult input. Milner and Bryant (1970) argued that it is permissible to attribute crossmodal errors solely to failures in the more difficult modality, only if crossmodal errors are significantly less than errors within the more difficult modality (e.g., Rude1 and Teuber’s first results). In the latter case two poorly discriminated inputs have to be compared, while in each of the crossmodal tasks only one difficult, but also one easy input is received. If crossmodal errors do not differ from errors in the most difficult modality, failure in intersensory organization cannot bc ruled out. Rude1 and Teuber explained their results by young children’s reliance on normal visual input. Millar (1971)) for instance, showed that tactual recognition by preschoolers was improved when standards were both viewed and felt. Nevertheless, in explaining the changes in error patterns, the possibility that increased input difficulty also had some specific effects on crossmodal matching cannot be ruled out. Mimer and Bryant (1970) attributed error changes to difficulties in retrieving crossmodal information under delay, on the grounds that children’s visual-tactual matches were better than intratactual matches under 0-set, but not under 5-set delays. The significant interaction between modalities and delay, which this assumption necessitates, was only
INTRAMODAL
AND
CROSSMODAL
MATCHING
65
found by Rose, Blank and Bridger (1972)) but this was contaminated by ceiling effects. None of the errors under simultaneous input were significantly different from zero. They suggested, as did Goodnow (1971)) that delay affects crossmodal errors because visual standards are coded and stored efficiently while tactual information decays rapidly with delay. Posner (1967) proposed such coding and storage differences for visual and kinesthetic judgments on the grounds that task-filled delays affect visual, but not kinesthetic judgments. The relevant comparisons between task-filled and unfilled delays were made by Millar (197213). Visual and tactual matches differed as predicted, but effects of task-filled compared to unfilled delays on crossmodal matches indicated that subjects could choose whether to rely mainly on information from visual standards or from comparison stimuli. Errors and latencies showed that crossmodal matching may indeed be limited by discrimination and coding differences between modalities, but not solely because of tactual failures. Coding differences and the fact that inputs are from different modalities add information which may require extra decisions in the choice of coding strategies. Millar (1972a) showed that crossmodal errors can change (rather than decrease) for older children who can utilize additional information. Thus, comparing inputs should probably not, in any case, be regarded as a passive process, limited only by discrimination and retention difficulties, but as requiring active choices which are affected by what strategies are available to subjects, as well as by input conditions and modality differences. Difficulties in processing sequential information, and discriminability and coding differences between modalities have thus emerged as important factors in crossmodal matching. At the same time the evidence makes it doubtful whether such differences can totally account for all crossmodal errors, especially for results showing either one, or both, crossmodal matches to be less efficient than matches within the most difficult modality (e.g., Connolly & Jones, 1970; Chase & Calfee, 1969; Harvey, 1973). The question is whether this necessitates the opposite assumption of special, learned translation mechanisms. The results on infants and the lack of evidence for differential decreases in crossmodal errors with age (Milner & Bryant, 1970) count against it, since the theory assumes crossmodal improvement with age. They are not necessarily decisive, however. Advocates could argue that infants can only so match the simplest stimuli, and that age interactions may be masked by differential rates of improvement in translation systems and in within-modality processing. A more decisive test is whether differences in stimulus complexity affect crossmodal errors differentially. Translation difficulties are
66
SUSAKNA
MILLAR
presumably greater for more complex stimuli. The assumption of special translation and integrative mechanisms should thus predict an interaction between stimulus difficulty or complexity, and crossmodal errors. On the assumption that crossmodal errors are completely accounted for by errors in the more difficult modality, this interaction is less likely. Even if increased stimulus complexity affects processing more in one modality, increases in errors within this modality would raise the average level of errors in the two within modality matches, as well as the average level of crossmodal errors. There have been few direct studies of this. Blank, Higgins and Bridger (1971) argued that children’s crossmodal errors are due to differences in processing stimuli of different complexity, in analogy with such effects on intravisual matches, but the data contain no crossmodal controls. Rubinstein and Gruenberg (1971) found that crossmodal matches of visual and auditory rhythmic patterns by adults were more difficult than intramodal ones with 4.5 stimuli per set, but not for patterns with only 3 stimuli per sec. They held that intcrsensory processing is less efficient, except in limiting cases, because inputs can only be compared in areas of joint association. However, they obtained these results in successive cxperiments in which some of the same subjects took part. The present study was, therefore, designed to test the effects of stimulus difficulty on the relation between intramodal and crossmodal visual and kinesthetic errors by children. Visual and kinesthetic inputs were made as similar as possible by presenting visual inputs by a moving light point on a joystick, and kinesthet’ic ones by the felt, movements of the joystick. Both inputs were thus sequential. Previous studies of visual and kinesthetic judgments used recall tests. Recognition rather than recall was employed here, since the latter may impose greater memory load or retrieval difficulties (Kinch, 1969), and these may be involved in crossmodal errors. EXPERIMENT
1
Two hypotheses were to be tested: (a) Crossmodal errors exceed intramodal errors in visual and kinesthetic recognition by children; (b) crossmodal errors increase differentially with an increase in the number of constituents in the input pattern. Method A Same/different recognition task was used in an Input pattern X Modality, within-S design. Subjects. Twenty-four children, mean age 10 years, 0 months (range: 9 years, 5 months to 10 years, 7 months), from a local Authority school, took part. There were equal numbers of boys and girls.
INTRAMODAL
AND
CROSSMODAL
MATCHING
67
Apparatus. The apparatus consisted of two joysticks constrained by metal bars to move in unison. The movement of S’s joystick could thus be controlled either by the subject or by the experimenter. The joysticks were housed in a unit which had a viewing window on the sloping front panel facing the S, fitted with a shutter (closing the window in kinesthetic condition), which could be operated from the experimenter’s side (opposite to the S) . Visual inputs to the subject were from a s-mm light bulb fitted to the top of S’s joystick. Movement of the lighted bulb was guided via E’s joystick and viewed by S through the viewing window, open in the appropriate conditions. In kinesthetic conditions, S held his joystick in a pencil hold just below the light bulb via a sleeved opening below the viewing window. A circular plate prevented S from grasping the stick at a lower point. For kinesthetic inputs, the S moved his joystick to prepared stops. Only extensor movements were used. A ledge, fitted inside the experimenter’s end of the unit, restricted the movements to the horizontal and bore a moveable bar to stop the S’s movements at appropriate distances before being moved on or removed. Inputs were thus from the subject’s movement of his joystick (kinesthetic) or from movements of the lightbulb on his joystick (visual) guided by the experimenter (E). Input patterns. Visual and kinesthetic analogs of Rubinstein and Gruenberg’s patterns were used. The more difficult (Q pattern) consisted of three 30” and one 10” successive movements (traced either by the light or the felt movements of the joystick) marked off from each other by 3-set stops between the distances. The smaller distance could occur either in the first, second, third or fourth position. If X stands for the 10” distance, and 0 for the 30” distance, Q patterns consisted of either X000, 0X00, 00X0 or 000X, presented in random order (an equal number of each for “same” and “different” tests, alternated in Gellermann order). The easier (D) pattern had two constituents, one 10” and one 30” SUCcessive movement, marked from each other by a 3-set stop. The D patterns (X0 or OX) were presented in random order (an equal number of each for “same” and “different” tests in Gellerman order). Half the subjects were tested first on D patterns, and after not more than 4 days on Q patterns. For the other half of the subjects the order was reversed. Modality. Presentation and Test modes were either visual or kinesthetic, yielding two intramodal (Visual-visual and Kinesthetic-Kinesthetic), and two crossmodal (Visual-kinesthetic and Kinesthetic-visual) within-S conditions, blocked in across-S counterbalanced order. Procedure. The Ss were tested singly. The S was seated in front of the apparatus at a comfortable height. The patterns were presented as car or bus journeys, with short stops where passengers could alight. The S was to notice where the stops occurred on the first journey (“remember”) and to judge (“judge”) whether stops on the second journey occurred at
68
SITSANNA
MILLAR
exactly the same distances or not. Test presentations followed after return of joystick to starting point (approx 2 set) . The Ss were familiarized with the appropriate pattern and given trial runs in every modality condition prior to tests, to ensure that instructions were followed. Trial runs were not counted. A total of eight tests (same/different, alternated in Gellermann type order) were run for each S in each of the two intramodal, and two crossmodal conditions (a total of 62 responses per subject under the two patterns). RESULTS
AND
DISCUSSION
Errors were subjected to a three-way Anova with Order of presentation (D followed by Q, Q followed by D), Input pattern (D or Q), and Modality (intramodal, crossmodal) as factors. (Mean errors are shown in Table 1). Input pattern had a significant effect (F(1,22) = 9.09, p < .Ol), showing that there were fewer errors under D than under Q patterns. The other main effect was of Modality (F(1,22) = 15.60, p < 901). Table 1 shows that errors for the two crossmodal tasks were higher than errors for either of the intramodal tasks. The hypothesis that crossmodal errors exceed intramodal errors on recognition tests was thus confirmed. The interaction between Input pattern and Modality (p < .25) was not significant. The hypothesis that crossmodal errors would increase differentially with the number of constituents in the input pattern was not supported. A significant interaction between Order of presentation and Modality (P(1,22) = 5.12, p < .05) was found. This might represent a purely fortuitous population difference. However, since subjects were allocated randomly to the two orders of presentation and this produced no main TABLE 1 MEAN RECOGNITION ERRORS BY lo-YEAR-OLDS UNDER VISUAL-VISUAL (V-V), KINESTHETIC-KINESTHETIC (K-K), AND UNDER VISUAL-KINESTHETIC (V-K) AND KINESTHETIC-VISUAL (K-V) CONDITIONS FOR DOUBLE AND QUADRUPLE INPUT PATTERNS (EXPERIMENT 1) MODALITY Crossmodal
Intramodal Input pattern Double Quadruple Means
v-v
K-K
Means
V-K
K-V
Means
.92 1.46 1.19
1.63 2.25 1.94
1.28 1.86 1.57
2.13 2.71 2.42
2.04 2.58 2.31
2.09 2.65 2.37
INTRAMODAL
AND
CROSSMODAL
MATCHING
69
AGE 10
35
0,&v
Drdn D-Q
I DFattern
Q RnHn
DPalkm
Q-D
QFaum
FIG. 1. Mean errors by lo-year-olds for intramodal (INTRA) and crossmodal (CROSS) modality conditions for Double (D) and Quadruple (Q) patterns in presentation orders D followed by Q (D-Q) and Q followed by D (Q-D), (Expt 1).
effect of difference, the possibility that crossmodal errors were affected differentially by the preceding task could not be discounted. The relevant terms are graphed in Fig. 1. This shows that when the more difficult pattern (Q) preceded the easier (D) one, crossmodal errors were relatively smaller than when the simpler D pattern preceded the more difficult Q pattern. This will be discussed further below. EXPERIMENT
2
Two arguments could be presented against accepting the results of Expt 1 as conclusive evidence against the assumption that crossmodal matching depends upon translation via special learned associations or integrative systems. Firstly, although Q patterns proved more difficult than D patterns, both involved the identification of the position of the smaller of the distances in the standard and comparison patterns as either same or different. Thus the translation problem might have been the same for the two patterns. Secondly, IO-year-olds might already have established translations systems whereby both could be accomplished. In theory this is less likely at younger ages. Two groups of younger children were, therefore, tested on the D pattern, and on single lengths of 10” and 30”. It was assumed that judging single lengths would present a
70
SUSANNA
MILLAR
different and easier ‘%ranslation” problem, since it involved only judgments of distance and not the additional task of locating the position of the smaller (or larger) of the two distances in the pattern, as in D. Three hypotheses were thus to be tested: (a) There are differential effects of input pattern difficulty on crossmodal performance; (b) these are larger for the younger of the two groups; and (c) the difference between intramodal and crossmodal errors is larger for the younger of the two groups. Method A same/different (“same, ” ‘(not the same”) recognition task was used in an Age (4.5, 5.5) X Input pattern (S,D) X Modality (intramodal, crossmodal) design with repeated measures on the last two factors. Subjects. Subjects were 24 children, an equal number of boys and girls, 12 from an Infant class in the same school as the older children in Expt 1, mean age 5 years, 8 months (5 years, 4 months to 6 years, 11 months), and 12 from a Nursery school in the same catchment area, mean age, 4 years, 6 months (3 years, 11 months to 4 years, 11 months). Input patterns. The easier pattern (S) consisted of two distances, one of 10” and one of 30”, alternated in random order (an equal number for each of same and different tests, alternated in Gellerman order). The D rather than Q pattern described in Expt 1 was used as the more difficult pattern to avoid ceiling effects on errors. Half the subjects were tested first on the S pattern, and after an interval of l-3 days on the D pattern. For the other half of the Xs, the order was reversed. Equal numbers of boys and girls were randomly allocated to the two presentation orders. Apparatus, modality conditions and procedures. These were exactly as described in Expt 1. RESULT
AND
DISCUSSION
Errors were subjected to a four-way Anova, with Age, Order of Presentation, Input pattern and Modality as factors. A significant effect of Age (F(1,44) = 5.88, p < .05) meant that the older group produced fewer errors than the preschoolers. The assumption that S patterns would be easier than D patterns was borne out by the main effect of pattern (F(1,44) = 19.92, p < .OOl). The main effect of Modality (F(1,44) = 10.28, p < .005) showed that crossmodal errors exceeded intramodal errors. (Mean errors are shown in Table 2). Order of presentation (S followed by D, D followed by S) was included as a factor because of its effect in Expt 1. It produced both a
INTRAMODAL
AND
CROSSMODAL
TABLE MEAN (V-V),
RIXOGNITION ERRORS BY 4.5 KINESTHETIC-KINESTHETIC AND KINESTHETIC-VISUAL SINGLE AND DOUBLE
71
MATCHING
2
AND
.!~~-YEAR-OLDS UNDER VISUAL-VISUAL (K-K), AND VISUAL-KINESTHETIC (V-K), (K-V) MODALITY CONDITIONS FOR PATTERNS (EXPERIMENT 2)
MODALITY Intxamodal
Crossmodal
Input pattern
v-v
K-K
Means
V-K
K-V
Means
4.5
Single Double Means
2.42 3.33 2.88
3.50 3.92 3.71
2.96 3.63 3.30
3.33 3.83 3.58
3.67 3.58 3.63
3.50 3.71 3.61
5.5
Single Double Means
1.92 3.17 2.55
2.08 3.33 2.71
2.00 3.25 2.63
2.58 3.83 3.21
2.75 3.67 3.21
2.67 3.75 3.21
Age
main effect (F(1,44) = 14.57, p < .OOl) and a significant interaction with Input pattern and Modality (F (1,44) = 8.42, p < .Ol). The interactions are shown graphically in Fig. 2. It is clear that increasing the number of constituents in patterns (D vs S) did not as such increase crossmodal errors differentially. (The Input pattern and Modality interaction was positive but not significant.) What the data for 5.5-year-olds show is that the difference between intramodal and crossmodal errors was larger for both the patterns when they were presented first than when they were presented after the other pattern. At the same time, this was not merely a warm-up or facilitating effect. When S preceded D, crossmodal errors on D were reduced compared to the first presentation of D. When D was presented first, crossmodal errors on S were not different, but intramodal errors increased compared to the first presentation of S. This suggests that there were proactive effects from the prior task, the direction of which depended on the type of input pattern, but does not support the assumption that crossmodal errors increase differentially with input pattern difficulty as such. The interaction between the main effects and age did not reach significance level. However, since such effects were predicted, and the interactions between age and pattern, age, order and modality, and age, order, pattern and modality were positive (p < .lO), analyses of simple effects for the two age levels were carried out. These showed that for 5.5-yearolds, Order of presentation (p < .Ol) , Pattern (p < .OOl), and modality (p < .005) were significant, but for 4.5-year-olds only Order of presentation (the S-D sequence was easier) was significant (p < .025). Since
72
SCSANNA
MILLAR
FIG. 2. Mean errors by 5.5- and 4.5-year-olds for intramodal (INTRA) and crossmodal (CROSS) conditions for single (S) and Double (D) patterns in presentation orders S followed by D (S-D) and D followed by S (D-S), (Expt 2).
neither Input pattern, nor modality, produced significant effects for preschoolers, the hypothesis that the difference between crossmodal and intramodal errors is larger for the younger subjects was not supported. If anything, the tendency was in the opposite direction. Inspection of Table 2 shows that while for 5.5-year-olds, errors on the crossmodal tasks exceeded errors on either of the intramodal tasks, for 45year-olds, errors on the intrakinesthetic task exceeded errors on one or both of the crossmodal tasks. EXPERIMENT
3
A third experiment was undertaken to test whether using a recall task would produce the crossmodal deficit (see above) in preschoolers performance not found in Expt 2. To minimize possible ceiling effects, only the S pattern (single length of 10” and 30”) was used, presented either as in Expt 2 (alternating) or in blocks. The latter condition was included as a, presumably, still easier task, and to ascertain whether such an apparently minor variation in presentation would affect the intramodalcrossmodal difference.
INTRAMODAL
AND
CROSSMODAL
MATCHING
73
Method Reproduction of lengths was used in Presentation type (blocked, alternating) X Modality (intramodal, crossmodal) design with repeated measures on the last factor. Subjects. The Ss were 18 local Authority Nursery school children, mean age, 4 years, 1 month (3 years, 3 months to 4 years, 8 months). An equal number of boys and girls were allocated randomly to blocked and alternating presentation conditions. Apparatus and task. The apparatus and presentation of standards were the same as for the previous experiments. For kinesthetic tests the S was to reproduce the distance by moving his joystick to the same distance as the standard. For visual tests, the S stopped the movement of the light by saying “stop” at the appropriate distance. Tests followed approximately within 2 sec. Two circular potentiometers, fitted to the E’s joystick, drove the X and Y inputs from a Hewlett Packard 7035B S-Y recorder through zeroing and scaling units. The direction and extent of movements of the joysticks were directly transcribed onto graph paper and errors measured in millimeter deviations. Presentation type and modality. Standards were 10” or 30” distances, presented either in blocks or alternated in Gellerman order (as for S patterns in Expt 2). For blocked presentation, half the runs were on the 10” length followed by the 30” length after a 5-min break (or the reverse). Equal numbers of boys and girls were allocated to the two conditions. Modality conditions were exactly as for Expts 1 and 2. Procedure. The task was presented as a game of playing bus drivers. Trial runs to ensure that instructions were being followed preceded every change in modality conditions and were not counted. The Xs received eight runs in every modality condition (32 runs). In all other respects procedures were the same as for Expts 1 and 2. RESULTS
AND
DISCUSSION
Mean absolute errors are shown in Table 3 in terms of centimeter deviations. A Presentation type (blocked, alternating) X Modality (intramodal, crossmodal) analysis of variance, showed an interaction between the two factors (F(1,14) = 5.17, p < .05). Simple effects showed that in blocked presentation, absolute errors between intramodal and crossmodal tasks did not differ, but intramodal errors were larger than crossmodal errors in alternating presentations (p < .025). This is opposite to the predicted effect. The significant interaction of the main terms demonstrated that even small differences in the type of presentation had effects on the intra-
74
SUSANNA
MILLAR
TABLE 3 MEAN ABSOLUTI: ERRORS (ON DEXIATIONS) IN REPRODUCING SINGLE LENGTHS UNDER BLOCKED AND ALTERNATING PRESENTATION IN VISUAL-VISUAL (V-V), KINESTHETIC-KINESTHETIC (K-K) AND VISUAL-KINESTHETIC (V-K) AND KINESTHETIC-VISUAL (K-V) MODALITY CONDITIONS (EXPERIMENT 3) MODALITY Intramodal
Crossmodal
Presentation type Blocked Alternating Means
v-v 1.5 1.9 1.7
K-K
Means
V-K
K-V
Means
4.5 6.5 5.5
3.0 4.2 3.6
4.5 3.4 4.0
2.0 2.8 2.4
3.3 3.1 3.2
modal-crossmodal relation. However, the effect was on intramodal performance. Table 3 shows that this was due to larger intrakinesthetic errors under alternating presentations. A presentation type X Input Modality analysis on kinesthetic test results confirmed this. Input modality (V or K), (F(1,14) = 6.33, p < .05), and the interaction of presentation type and input modality (F(lJ4) = 6.33, p < .05) were significant. This is of interest especially since it shows that presentation type increased intramodal (K-K) errors without increasing crossmodal (V-K) errors. The size of the errors showed that the patterns were not easy, nor were they so difficult as to preclude significant differences between responses. SUMMAR.Y
AND
GENERAL
DISCUSSION
There was no evidence of any differential effect on crossmodal performance of an increase from two to four constituents in distance patterns for IO-year-olds (Expt 1) or from single to double lengths for younger children (Expt 2). Crossmodal errors were significantly larger than intramodal errors for lo-year-olds (Expt l), and for 5.5-year-olds (Expt 2) in recognition tests. For preschoolers, neither recognition of single and double lengths (Expt 2), nor recall of single lengths (Expt 3) showed any difference between intramodal and crossmodal errors here. These findings add strong evidence against the theory that crossmodal performance depends upon special learned integrative systems necessary for, and specific to, crossmodal translation. It is reasonable to predict from it that more difficult patterns will be more difficult to translate. If anything, the tendency to an age interaction (Expt 2) was in the opposite direction to that predicted on the theory. A further question is how far the results support the opposite claim
INTRAMODAL
AND
CROSSMODAL
MATCHING
75
that crossmodal errors are completely explained by errors in the more difficult modality. The absence of interactions between stimulus complexity as such, and crossmodal errors, and the fact that the averages of crossmodal and intramodal errors did not differ for the two preschool groups (Expts 2 and 3) would be expected on the theory. However, the strongest assumption that can be made in its favor, in order to explain the significantly larger crossmodal than intramodal errors for older subjects (Expts 1 and 2), is that accuracy in comparing two inputs reduces to the level of the more difficult one. Even this could not allow for the fact that errors in each of the crossmodal tasks exceeded those in the more difficult of the within-modality matches (Expts 1 and 2 for the older group). More importantly, it would be difficult to justify that this assumption does not apply equally to matches by preschoolers, even more when their intrakinesthetic errors were relatively very large (Expt 3), and why significantly increased intrakinesthetic errors under alternating presentation were not reflected in crossmodal matches (Expt 3). The significant effects of order of presenting complex and simple patterns on crossmodal errors (Expts 1 and 2) also require additional assumptions. The findings, together with others reported in the literature, suggest that this theory requires modification. An alternative explanation is that differences in discriminability, coding, and input difficulty affect subjects’ decisions in the choice of coding strategies (Millar, 1972b), rather than that errors are completely determined by discrimination or storage capacities. The significant order and interaction effects throw some light on this. It is unlikely that all these could be due to fortuitous population differences in three separate experiments with random allocations of subjects, nor could they be explained by practice in crossmodal matching. Inspection of the data shows that the direction of the effect on crossmodal matches (Expts 1 and 2) depended on whether the easier or the more difficult pattern had been presented first. This suggests that the prior task affected subjects’ strategies in dealing with the inputs. The hypothesis makes the not unreasonable assumption that subjects who are asked to compare two successively presented stimuli have more than one strategy open to them. They could code either or both inputs on some abstracted feature, or amodal category, or assign labels, or picture it if the input is visual (Tversky, 1969; Millar, 1972~). Tversky (1969, 1973) showed that coding choices depend on which of these subjects judge more economical in given tasks. Such choices are available also in comparing inputs from the same modality. But in comparing stimuli from different modalities, information about discrepancies in inputs, and in the source of stimulation arc added. Extra decisions to
76
STlSANNA
MILLAR
ignore discrepancies, or to use some amodal code, or to rely more on the ‘lbetter” input may make the choice of strategy less obvious and more dependent on task variables; for instance, whether relatively easy or more complex stimuli are presented first, This could have direct effects (increasing latencies) or indirect effects (disturbing storage or retrieval accuracy) on the efficiency of crossmodal matches. More crossmodal errors and effects of variations in task requirements on these could thus be expected. The hypothesis does not imply that crossmodal matching is always, or necessarily worse. It need not be, if coding choices are not required, or obvious, or comparisons easy, or well practiced, or if subjects only have very limited choices of strategy. Apes and infants, for instance, cannot, and young children frequently do not (Conrad, 1971) use verbal codes. Moreover, they use active strategies less, are slower to shift encoding modalities when the task actually requires this (Tversky, 1973), and are less likely to take into account all the available information. Their comparisons would, therefore, be disturbed more by difficulties in discriminating or remembering inputs than by consideration of additional or discrepant information, or by alternative coding possibilities. Either reliance on one, relatively simple, amodal code, or on dominant visual information (Millar, 1971; Rude1 & Teuber, 1971) for all matches could produce the largest errors in the least discriminable modality. The apparently counterintuitive result, that older, but not the youngest groups, showed larger crossmodal than intramodal errors thus becomes reasonable, indeed predictable in these conditions. It is instructive that most of the evidence in the literature in favor of explanations in terms of discrimination and retention difficulties has come from studies on young children. Studies on adults report, rather more frequently, results not completely compatible with this. The explanation put forward here accounts for the present results, and for previous findings of longer latenties, greater variability and effects of delay and distracters on crossmodal matching, and suggests conditions in which these do not occur. The study was designed to test the hypothesis that crossmodal matching depends upon special learned integrative translation systems by varying the difficulty of input conditions. Crossmodal errors exceeded intramodal errors by older children but stimulus complexity, as such, did not have differential effects. Order of presenting complex and simpler inputs significantly affected the relation between within- and across-modality matching by older subjects. Preschoolers showed no difference between intramodal and crossmodal errors, and effects of alternating versus blocked presentation disturbed only their intrakinesthetic judgments. The results were held to counterindicate special learned translation processes,
INTRAMODAL
AND
CROSSMODAL
77
MATCHING
but could not be completely explained by difficulties in processing the less discriminable input. An alternative explanation was suggested. REFERENCES BIRCH, H. G., & LEFFORD, A. Intersensory development in the Society for Research in Child Development, 1963,28,48.
children.
Monographs
of
BIRCH, H. G., & LEFFORD, A. Visual differentiation, intersensory integration and voluntary motory control. Monographs of the Society for Research in Child Development, 1967, 32, 87. BLANK, M., ALTMAN, L. D., & BRIDGER,W. H. Crossmodal transfer of form discrimination in pre-school children. Psychonomic Science, 1968, 10, 51-52. BLANK, M., & BRIDGER, W. H. Crossmodal transfer in nursery school children. Journal of Comparative and Physiological Psychology, 1964, 58, 277-282. BLANK, M., HIGGINS, T. J., & BRIDGER,W. H. Stimulus complexity and intramodal reaction time in retarded readers. Journal of Educational Psychology, 1971, 62, 117-122. BRYANT, P. E., JONES, P., CLAXTON, V., & PERKINS, G. M. Recognition of shapes across modalities by infants. Nature (London), 1972, 240, 303-304. CASHDAN, S. Visual and haptic form discrimination under conditions of successive stimulation. Journal of Experimental Psychology, 1968,76,215218. CHASE, W. G., & CALFEE, R.. C. Modality and similarity effects in short-term recognition memory. Journal of Experimental Psychology, 1969, 81, 510-514. CONNOLLY, K., & JONES, B. A. A developmental study of afferent-reafferent integration. British Journal of Psychology, 1970, 61, 259-268. CONRAD, R. The chronology of the development of covert speech in children. Developmental Psychology, 1971, 5, 398405. DAVENPORT, R. K., & ROGERS, C. M. Intermodal equivalence of stimuli in apes. Science, 1970, 168, 279-280. ETTLINGER, G. Analysis of cross-modal effects and their relationship to language. In J. Millikan & F. Darley (Eds.), Mechanism underlying speech and language. New York: Grune & Stratton, 1967. GARVILL, J., & MOLLANDER, B. Intra-modal and cross-modal accuracy in a form discrimination task. Umeci Psychological Report, 1969, No. 11, 8. GIBSON, E. J. Principles of perceptual learning and development. Appleton-CenturyCrofts : New York, 1969. G~~DNOW, J. J. Eye and hand: Differential memory and its effect on matching. Neuropsychologica, 1971, 9, 89-95. HARVEY, N. Does intermodal equivalence exist between heteromodal stimulus dimensions or between stimulus values on those dimensions? Quarterly Journal of Experimental Psychology, 1973, 25, 476491. JONES, B., & CONNOLLY, K. Memory effects in crossmodal matching. British Journal of Psychology, 1970, 61, 267-270. KINTON, W. Learning, memory and conceptual processes. Wiley: New York, 1969. KRAUTHAMER, G. Form perception across sensory modalities. Neuropsychologica, 1968, 6, 105-113. KRESS, G., & CROSS, J. Visual and tactual interaction in judgments of the vertical. Psychonomic Science, 1969, 14, 166166. LEGGE, D. Analysis of visual and proprioceptive components of motor skill by means of a drug. British Journal of Psychology, 1965, 56,243254.
78
SCSANNA
MILLAR
S. Visual and haptic cue utilization by preschool children: The recognition of visual and haptic stimuli presented separately or together. Journnl of Experimental Child Psychology, 1971, 12, 88-94. MILLAR, S. The development of visual and kinesthetic judgments of distance. British Journal of Psychology, 1972a, 63, 271-282. MILLAR. S. The effects of interpolated tasks on latency and accuracy of intmmodal and crossmodal shape recognition by children. Journal of Experimental Psychology, 197213, 96, 170-175. MILLAR, S. Effects of instructions to visualize stimuli during delay on visual recognition by preschool children. Child Development, 1972c, 43, 1073-1075. MILNER, A. D., B: BRYANT. P. E. Cross-modal matching by young children. Journal MILLAR,
of Comparative
and
Physiological
Psychology,
1970, 71, 453458.
M. I. Short-term memory systems in human information processing. Actn Psychologica, 1967, 27, 267-284. RUBINSTEIN, L., & GRUENBERG, E. M. Intramodal and crossmodal sensory transfer of visual and auditory sensory patterns. Perception and Psychophysics, 1971, 9, 385-390. RUDEL, R. G., & TEUBER, H. L. Crossmodal transfer of shape discrimination by children. Neuropsychologica, 1964, 2, 1-18. RUDEL, R. G., & TEUBER, H. L. Pattern recognition within and across modalities in normal and brain-injured children. T~ERSKY. B. Pictorial and verbal encoding in a short-term memory task. Perception POSNER,
and
Psychophysics,
1969, 6, 225-233.
B. Pictorial and verbal Psychology, 1973, 8, 149-153.
TVERSKY,
encoding
in preschool
children.
Developmental
