Scand. J. Psychol. 20, 77-80, 1979
Attempts to predict intelligent behavior 1. The Categorizing Test KJELL RAAHEIM GEIR KAUFMANN ASTRID KAUFMANN
University of Bergen, Norway
Raaheim, K., Kaufmann, G. & Kaufmann, A,: Attempts to predict intelligent behavior. 1. The Categorizing Test. Scand. J. Psychol. 20, 77-80, 1979. The possibility of predicting future intelligent behavior by means of a test calling for flexible categorizing of a present situation is discussed. The test, called The Categorizing Test, consisting initially of 22 items, was tried out on 94 Ss. The results showed high, positive correlations between the items, suggesting that they to a strong degree tap the same cognitive function, which is interpreted as an ability to categorize flexibly. A shorter 10-item version of the test was compared with a test of verbal intelligence and a flexibility test of a traditional kind. On the whole, high, positive correlations were obtained, with the Categorizing Test apparently occupying a central position. These results constitute validating evidence for The Categorizing Test. K. Raaheim, Psykologisk Insiiiuti, University of Bergen, Norway.
It has been argued that the problem-solving paradigm constitutes a fruitful basis for the study of human intelligence, provided a problem situation is defined in a careful and rigorous way (Raaheim, 1961, 1974). To achieve this it is necessary to relate problem-solving behavior to the individual’s past experience. A problem situation might be viewed as located between two thresholds: a lower threshold which marks the transition from a mere routine task into a situation where a deviation from what has previously been experienced has to be dealt with, and an upper threshold, marking the transition to a situation where a familiar pattern is no longer recognizable. When this upper threshold is transcended we should no longer speak of a problem situation, but rather of a difficult task. The thresholds are determined by individually based criteria, since past experience relevant to the present situation obviously differs between individuals, a reminder of the difficulties we face if, on a given occasion, we want to predict, or just explain, the behavior of a particular person. On this basis intelligence might be conceived as the ability to transform what at first glance are new or unknown situations into familiar, more or less
easily mastered situations. This does not imply that intelligent adjustment is merely a question of incorporating the situation confronting the individual into a ready-made category since also the individual’s past experience must be thought of as undergoing certain changes when mobilized in the problem-solving process. When the aim is to predictfuture intelligent behavior, it follows from the above considerations that an assessment of the individual’s ability to look upon his experience in a flexible manner is needed. A possible basis for predicting future intelligent modes of adjustment, may thus consist of recording the individual’s ability to categorize the present situation in a flexible way. The present report describes an attempt to measure this ability. I. THE CATEGORIZING TEST Rationale for the test A person who is given the task of discovering some kind of reward in one of three boxes in a variety of Harlow’s “oddity problem” situation (Harlow, 1958) will in most cases find himself able to describe the characteristics of the “correct” box in more Scand. J . Psychol. 20
78
K . Raaheim et al.
than one way. If for example the arrangement is a box of triangular shape in the middle with a cylindrical box on either side, the subject, when finding the reward in box 2 starting from the left, may categorize the situation in various ways. The “second from the left” may be replaced by the “second from the right” or by the “box in the middle”. Turning to the shape of the boxes rather than their location the subject may tell that the reward was found in “the triangular box”, or “the box with sharp edges”, and finally the subject may simply tell that the reward was in the “odd one”. It is the future arrangement of boxes that the subject will be confronted with which gives the clue as to which formulation is the most useful one. But if the person who is able to give several different formulations, or interpretations, of his present experience, is also one who will be able to choose the proper one when later faced with a similar task, a test of intelligent behavior may be constructed in an indirect way. Providing the subjects with a number of situations equivalent to the one just sketched, and asking for as many ways of formulating their experience as possible, one might get a reliable assessment of an ability of central importance for human adjustment. It was decided to construct a group test of the paper and pencil type. Preliminary studies indicated sufficient variability between individuals in mastering such a task to permit the construction of a test with a reasonable number of items. Procedure From a large number of possible test items a total of 22 were chosen. Each item presents the subject with three named objects which are all alike in certain respects, but differ in others. The subject was asked to describe in as many ways as possible, what characterized one of the objects as different from the two others. The time limit for each item was 3 min. Example: Aeroplanexar-bicycle. Here the task is to characterize the bicycle in as many ways as possible as different from aeroplane and car. Examples of answers “The bicycle is the most inexpensive one”, “one does not have to have a license to drive”. “the slowest one” etc. Scoring A subject’s score on the test was the number of acceptable answers on the 22 items. Different answers referring to the same categorizing were scored as one answer. Example: “The bicycle does not use gas”, “the bicycle has no motor”. Scand. J . Psychol. 20
Answers categorizing the object in relation to only one of the two other objects were not accepted. Example: “The bicycle drives on land.” Subjects were 48 boys and 46 girls, on the average 15 years of age. They were tested in a classroom setting. The purpose of the first administration of the test was to investigate if a general ability could be identified.
Results and discussion Since on each item the subject had to place the object in question within as many classes, or categories, as possible, the test will be referred to as The Categorizing Test. The number of categories listed by the subjects ranged from 0 to 15 for the individual item. The total score ranged from 37 to 162 for the 22 items of the present edition of the test. A correlation analysis showed high, positive correlations between the items, and when the itemtotal score correlations were worked out, a homogenous pattern of high, positive correlations emerged. The correlations varied from 0.55 to 0.75. This indicates that the 22 items tap the same cognitive function as further confirmed by a factor analysis showing one factor accounting for 46% of the variance. Two additional, small factors added only 10% to the variance. It thus appears reasonable to talk about an ability to categorize flexibly when a human being is faced with the task of interpreting a particular experience. The high intercorrelations allow the construction of a shorter edition of the test, desirable if the test scores are to be compared with other measures of intellectual abilities obtained during the same session. Such comparisons are important t o determine its validity. Since the item-total score correlations are homogenous it is possible to make the choice of the final items on the basis of content. A shorter, final edition of the test was constructed of 10 items. The split-half reliability coefficient for this edition was 0.93.
A VALIDATION STUDY
Here the first steps in the process of validating the Categorizing Test are reported. It was decided to compare performance on the test with performance on a verbal intelligence test of the traditional type and with performance on a flexibility test.
Attempts to predict intelligent behavior
Table 1. Means and standard deviations of the different tests Test
Mean
S.D.
Msnnesland Total Score Sentence Completion Arithmetic Problems Verbal Grouping Numeral Senes Verbal Analogies Comprehension Categorizing Test Situational Flexibility
104.69 9.41 7.78 13.45 10.91 9.80 9.35 40.48 59.59
10.98 2.63 2.80 2.17 3.98 4.04 2.30 12.07 16.42
Procedure The subjects were 48 boys and 46 girls, on the average 15 years of age. They were tested in a classroom setting. The following tests were administered: i. The Categorizing Test. The time limit for each of the 10 items was 3 min. With the general introductory remarks and instructions the total time needed to go through the test was 45 min. ii. The M0nnesland Group Test of Verbal Intelligence. This is an intelligence test of the traditional type, composed of the following sub-tests: (1) Sentence Completion, (2) Arithmetical Problems, (3) Verbal Grouping or Classification, (4) Numeral Series, (5) Verbal Analogies, (6) Comprehension. The total time needed for the administration of the test is 45 min. For details of administration and scoring procedure, see Msnnesland (1948). iii. The Situational Flexibility Test. This test is intended to measure a person’s capacity to interpret social situations in different ways. The test consists of 21 items, all of the same type: A particular event in a social context is described, and the subject asked to provide as many different interpretations as possible. Example: “A man suddenly woke up in the night.” Question: “What is happening? Write down as many proposals as possible.” Possible answers: “He has had a nightmare”, “noises in the house”, etc.
79
The time limit for each item is 1 min, and the total time needed to go through the test is 35 min. The subject’s score on the test is the number of acceptable answers as evaluated by the scorer.
Results and discussion Since the three tests all seem anchored in the Ss’ world of experience, a pattern of positive correlations between the tests was expected. Absence of correlation between the Categorizing Test and the other tests would be invalidating. At the same time, however, very high correlations would also be invalidating, because the different test tasks vary in their informational content and their degree of familiarity, some tasks being strongly reproductive, while others are more loosely related to past experience. Table 1 presents the Means and Standard Deviations of the different tests. Table 2 shows that there is a homogenous pattern of positive, statistically significant correlations between The Categorizing Test and the other tests. This is in accordance with our initial hypotheses. The magnitudes of the correlations are also as expected. The clear relationship between The Categorizing Test and The Msnnesland Test of Verbal Intelligence is worth noticing. In several investigations (see e.g. Getzels & Jackson, 1962; Wallach & Kogan, 1965; Wallach & Wing, 1969) non-significant correlations have been reported between socalled “creativity” tests resembling the present Categorizing Test and traditional intelligence tests. Kaufmann (1971) has, however, argued that these findings are due to methodological weaknesses in the design of the investigations in question such as restriction in range, confounding of trait and
Table 2. Correlation coefficients between scores on the different tests ( N =94) A correlation of 21 equals a p of 5% and one of 0.27 a p of 1 %, two-tailed test Variable no. Test 1
2 3 4 5 6 7 8 9
Msnnesland Total Score Sentence Completion Arithmetic Problems Verbal Grouping Numeral Series Verbal Analogies Comprehension Categorizing Test Situation Flexibility
1
.66 .77 .74 .80 .75 .71 .5 1
.27
2
.39 .45 .37 .36 .56 .37 .I7
3
4
5
6
7
8
.31 .42 .I4
.45 .25
.56
.46 59 .47 .55 .39 .23
.50 .52 .53 .39 .24
.49 .45 .3 1
.21
Scand. 1.Psycho!. 20
80
K . Raaheim et a /
method factors and use of inappropriate statistics. From the present theoretical view our data seem the more plausible ones. The results are also in accordance with previous findings (see e.g. Raaheim, 1969). The Situational Flexibility Test also correlates significantly with the Msnnesland Test, but this correlation is rather low. The more interesting finding is that, while correlating highly with the Msnnesland Test, the Categorizing Test correlates equally high with the Situational Flexibility Test. The Categorizing Test thus seems to stand in the centre of the tasks. It has previously been argued (Raaheim, 1974) that intelligence may be conceived as having its optimal function in tasks of medium familiarity and to be less essential towards the two thresholds. One might thus speculate that the three tests contain tasks which are different as regards the deviation from past experience. Traditional intelligence tests of the Msnnesland type are commonly criticized for being based on reproductive kind of content, and would thus consist of tasks close to the lower threshold of familiarity. The Situational Flexibility Test obviously is more loosely related to past experience. The Categorizing Test may be thought of as occupying some intermediate position on the familiarity continuum since it poses certain restrictions on the individual’s search field in relation to his experience. (The object is to be categorized in relation to two other objects.) If so, the Categorizing Test is the more representative of intelligence as defined within the present context. However, further evidence is needed for a more conclusive validation of this hypothesis.
REFERENCES Getzels, J. W. & Jackson, P. W. (1962). Creativity and intelligence: Explorations with gifted students. New York: Wiley. Harlow, H. F. (1958). The evolution of learning. In A. Roe & G. G. Simpson (Eds.), Behavior and evolution. New Haven: Yale University Press. Kaufmann, G. (1971). Kreativitet og skolen. In A. Folkvord (Ed.), Normalplanen: Et opplegg till misfostring eller gagns menneske? Oslo: Universitetsforlaget. Msnnesland, K. (1948). Intelligensprmer f o r voksne. Oslo: Olaf Nordlie. Raaheim, K. (1961). Problem solving: A new approach. A c t a Univ. Bergensis, Ser. H u m . L i f t . , N o . 5. Raaheim, K. (1969). Problem solving and intelligence. A Si~md.J . Psychol. 20
comparison of test scores. [ R e p . Inst. Psychol., Univ. Bergen, I . ] Raaheim, K. (1974). Problem solving and intelligence. Bergen/Oslo/Tromse: Universitetsforlaget. Wallach, M. A. & Kogan, N . (1965). Modes of fhinking in young children: A study of the creativity-intelligence distinction. New York: Holt, Rinehart &Winston. Wallach, M. A. & Wing, C. W., Jr (1969). The talented student: A validation of the creativity-intelligence distinction. New York: Holt, Rinehart &Winston.
Attempts to predict intelligent behavior 1. The Categorizing Test KJELL RAAHEIM GEIR KAUFMANN ASTRID KAUFMANN
University of Bergen, Norway
Raaheim, K., Kaufmann, G. & Kaufmann, A,: Attempts to predict intelligent behavior. 1. The Categorizing Test. Scand. J. Psychol. 20, 77-80, 1979. The possibility of predicting future intelligent behavior by means of a test calling for flexible categorizing of a present situation is discussed. The test, called The Categorizing Test, consisting initially of 22 items, was tried out on 94 Ss. The results showed high, positive correlations between the items, suggesting that they to a strong degree tap the same cognitive function, which is interpreted as an ability to categorize flexibly. A shorter 10-item version of the test was compared with a test of verbal intelligence and a flexibility test of a traditional kind. On the whole, high, positive correlations were obtained, with the Categorizing Test apparently occupying a central position. These results constitute validating evidence for The Categorizing Test. K. Raaheim, Psykologisk Insiiiuti, University of Bergen, Norway.
It has been argued that the problem-solving paradigm constitutes a fruitful basis for the study of human intelligence, provided a problem situation is defined in a careful and rigorous way (Raaheim, 1961, 1974). To achieve this it is necessary to relate problem-solving behavior to the individual’s past experience. A problem situation might be viewed as located between two thresholds: a lower threshold which marks the transition from a mere routine task into a situation where a deviation from what has previously been experienced has to be dealt with, and an upper threshold, marking the transition to a situation where a familiar pattern is no longer recognizable. When this upper threshold is transcended we should no longer speak of a problem situation, but rather of a difficult task. The thresholds are determined by individually based criteria, since past experience relevant to the present situation obviously differs between individuals, a reminder of the difficulties we face if, on a given occasion, we want to predict, or just explain, the behavior of a particular person. On this basis intelligence might be conceived as the ability to transform what at first glance are new or unknown situations into familiar, more or less
easily mastered situations. This does not imply that intelligent adjustment is merely a question of incorporating the situation confronting the individual into a ready-made category since also the individual’s past experience must be thought of as undergoing certain changes when mobilized in the problem-solving process. When the aim is to predictfuture intelligent behavior, it follows from the above considerations that an assessment of the individual’s ability to look upon his experience in a flexible manner is needed. A possible basis for predicting future intelligent modes of adjustment, may thus consist of recording the individual’s ability to categorize the present situation in a flexible way. The present report describes an attempt to measure this ability. I. THE CATEGORIZING TEST Rationale for the test A person who is given the task of discovering some kind of reward in one of three boxes in a variety of Harlow’s “oddity problem” situation (Harlow, 1958) will in most cases find himself able to describe the characteristics of the “correct” box in more Scand. J . Psychol. 20
78
K . Raaheim et al.
than one way. If for example the arrangement is a box of triangular shape in the middle with a cylindrical box on either side, the subject, when finding the reward in box 2 starting from the left, may categorize the situation in various ways. The “second from the left” may be replaced by the “second from the right” or by the “box in the middle”. Turning to the shape of the boxes rather than their location the subject may tell that the reward was found in “the triangular box”, or “the box with sharp edges”, and finally the subject may simply tell that the reward was in the “odd one”. It is the future arrangement of boxes that the subject will be confronted with which gives the clue as to which formulation is the most useful one. But if the person who is able to give several different formulations, or interpretations, of his present experience, is also one who will be able to choose the proper one when later faced with a similar task, a test of intelligent behavior may be constructed in an indirect way. Providing the subjects with a number of situations equivalent to the one just sketched, and asking for as many ways of formulating their experience as possible, one might get a reliable assessment of an ability of central importance for human adjustment. It was decided to construct a group test of the paper and pencil type. Preliminary studies indicated sufficient variability between individuals in mastering such a task to permit the construction of a test with a reasonable number of items. Procedure From a large number of possible test items a total of 22 were chosen. Each item presents the subject with three named objects which are all alike in certain respects, but differ in others. The subject was asked to describe in as many ways as possible, what characterized one of the objects as different from the two others. The time limit for each item was 3 min. Example: Aeroplanexar-bicycle. Here the task is to characterize the bicycle in as many ways as possible as different from aeroplane and car. Examples of answers “The bicycle is the most inexpensive one”, “one does not have to have a license to drive”. “the slowest one” etc. Scoring A subject’s score on the test was the number of acceptable answers on the 22 items. Different answers referring to the same categorizing were scored as one answer. Example: “The bicycle does not use gas”, “the bicycle has no motor”. Scand. J . Psychol. 20
Answers categorizing the object in relation to only one of the two other objects were not accepted. Example: “The bicycle drives on land.” Subjects were 48 boys and 46 girls, on the average 15 years of age. They were tested in a classroom setting. The purpose of the first administration of the test was to investigate if a general ability could be identified.
Results and discussion Since on each item the subject had to place the object in question within as many classes, or categories, as possible, the test will be referred to as The Categorizing Test. The number of categories listed by the subjects ranged from 0 to 15 for the individual item. The total score ranged from 37 to 162 for the 22 items of the present edition of the test. A correlation analysis showed high, positive correlations between the items, and when the itemtotal score correlations were worked out, a homogenous pattern of high, positive correlations emerged. The correlations varied from 0.55 to 0.75. This indicates that the 22 items tap the same cognitive function as further confirmed by a factor analysis showing one factor accounting for 46% of the variance. Two additional, small factors added only 10% to the variance. It thus appears reasonable to talk about an ability to categorize flexibly when a human being is faced with the task of interpreting a particular experience. The high intercorrelations allow the construction of a shorter edition of the test, desirable if the test scores are to be compared with other measures of intellectual abilities obtained during the same session. Such comparisons are important t o determine its validity. Since the item-total score correlations are homogenous it is possible to make the choice of the final items on the basis of content. A shorter, final edition of the test was constructed of 10 items. The split-half reliability coefficient for this edition was 0.93.
A VALIDATION STUDY
Here the first steps in the process of validating the Categorizing Test are reported. It was decided to compare performance on the test with performance on a verbal intelligence test of the traditional type and with performance on a flexibility test.
Attempts to predict intelligent behavior
Table 1. Means and standard deviations of the different tests Test
Mean
S.D.
Msnnesland Total Score Sentence Completion Arithmetic Problems Verbal Grouping Numeral Senes Verbal Analogies Comprehension Categorizing Test Situational Flexibility
104.69 9.41 7.78 13.45 10.91 9.80 9.35 40.48 59.59
10.98 2.63 2.80 2.17 3.98 4.04 2.30 12.07 16.42
Procedure The subjects were 48 boys and 46 girls, on the average 15 years of age. They were tested in a classroom setting. The following tests were administered: i. The Categorizing Test. The time limit for each of the 10 items was 3 min. With the general introductory remarks and instructions the total time needed to go through the test was 45 min. ii. The M0nnesland Group Test of Verbal Intelligence. This is an intelligence test of the traditional type, composed of the following sub-tests: (1) Sentence Completion, (2) Arithmetical Problems, (3) Verbal Grouping or Classification, (4) Numeral Series, (5) Verbal Analogies, (6) Comprehension. The total time needed for the administration of the test is 45 min. For details of administration and scoring procedure, see Msnnesland (1948). iii. The Situational Flexibility Test. This test is intended to measure a person’s capacity to interpret social situations in different ways. The test consists of 21 items, all of the same type: A particular event in a social context is described, and the subject asked to provide as many different interpretations as possible. Example: “A man suddenly woke up in the night.” Question: “What is happening? Write down as many proposals as possible.” Possible answers: “He has had a nightmare”, “noises in the house”, etc.
79
The time limit for each item is 1 min, and the total time needed to go through the test is 35 min. The subject’s score on the test is the number of acceptable answers as evaluated by the scorer.
Results and discussion Since the three tests all seem anchored in the Ss’ world of experience, a pattern of positive correlations between the tests was expected. Absence of correlation between the Categorizing Test and the other tests would be invalidating. At the same time, however, very high correlations would also be invalidating, because the different test tasks vary in their informational content and their degree of familiarity, some tasks being strongly reproductive, while others are more loosely related to past experience. Table 1 presents the Means and Standard Deviations of the different tests. Table 2 shows that there is a homogenous pattern of positive, statistically significant correlations between The Categorizing Test and the other tests. This is in accordance with our initial hypotheses. The magnitudes of the correlations are also as expected. The clear relationship between The Categorizing Test and The Msnnesland Test of Verbal Intelligence is worth noticing. In several investigations (see e.g. Getzels & Jackson, 1962; Wallach & Kogan, 1965; Wallach & Wing, 1969) non-significant correlations have been reported between socalled “creativity” tests resembling the present Categorizing Test and traditional intelligence tests. Kaufmann (1971) has, however, argued that these findings are due to methodological weaknesses in the design of the investigations in question such as restriction in range, confounding of trait and
Table 2. Correlation coefficients between scores on the different tests ( N =94) A correlation of 21 equals a p of 5% and one of 0.27 a p of 1 %, two-tailed test Variable no. Test 1
2 3 4 5 6 7 8 9
Msnnesland Total Score Sentence Completion Arithmetic Problems Verbal Grouping Numeral Series Verbal Analogies Comprehension Categorizing Test Situation Flexibility
1
.66 .77 .74 .80 .75 .71 .5 1
.27
2
.39 .45 .37 .36 .56 .37 .I7
3
4
5
6
7
8
.31 .42 .I4
.45 .25
.56
.46 59 .47 .55 .39 .23
.50 .52 .53 .39 .24
.49 .45 .3 1
.21
Scand. 1.Psycho!. 20
80
K . Raaheim et a /
method factors and use of inappropriate statistics. From the present theoretical view our data seem the more plausible ones. The results are also in accordance with previous findings (see e.g. Raaheim, 1969). The Situational Flexibility Test also correlates significantly with the Msnnesland Test, but this correlation is rather low. The more interesting finding is that, while correlating highly with the Msnnesland Test, the Categorizing Test correlates equally high with the Situational Flexibility Test. The Categorizing Test thus seems to stand in the centre of the tasks. It has previously been argued (Raaheim, 1974) that intelligence may be conceived as having its optimal function in tasks of medium familiarity and to be less essential towards the two thresholds. One might thus speculate that the three tests contain tasks which are different as regards the deviation from past experience. Traditional intelligence tests of the Msnnesland type are commonly criticized for being based on reproductive kind of content, and would thus consist of tasks close to the lower threshold of familiarity. The Situational Flexibility Test obviously is more loosely related to past experience. The Categorizing Test may be thought of as occupying some intermediate position on the familiarity continuum since it poses certain restrictions on the individual’s search field in relation to his experience. (The object is to be categorized in relation to two other objects.) If so, the Categorizing Test is the more representative of intelligence as defined within the present context. However, further evidence is needed for a more conclusive validation of this hypothesis.
REFERENCES Getzels, J. W. & Jackson, P. W. (1962). Creativity and intelligence: Explorations with gifted students. New York: Wiley. Harlow, H. F. (1958). The evolution of learning. In A. Roe & G. G. Simpson (Eds.), Behavior and evolution. New Haven: Yale University Press. Kaufmann, G. (1971). Kreativitet og skolen. In A. Folkvord (Ed.), Normalplanen: Et opplegg till misfostring eller gagns menneske? Oslo: Universitetsforlaget. Msnnesland, K. (1948). Intelligensprmer f o r voksne. Oslo: Olaf Nordlie. Raaheim, K. (1961). Problem solving: A new approach. A c t a Univ. Bergensis, Ser. H u m . L i f t . , N o . 5. Raaheim, K. (1969). Problem solving and intelligence. A Si~md.J . Psychol. 20
comparison of test scores. [ R e p . Inst. Psychol., Univ. Bergen, I . ] Raaheim, K. (1974). Problem solving and intelligence. Bergen/Oslo/Tromse: Universitetsforlaget. Wallach, M. A. & Kogan, N . (1965). Modes of fhinking in young children: A study of the creativity-intelligence distinction. New York: Holt, Rinehart &Winston. Wallach, M. A. & Wing, C. W., Jr (1969). The talented student: A validation of the creativity-intelligence distinction. New York: Holt, Rinehart &Winston.
