B r . J . educ. Psychol., 60,161-170,1990
RESPONSE FEEDBACK, CERTITUDE AND LEARNING FROM TEXT BYRAYMOND W. KULHAW, WILLIAM A. STOCK, NANCYE. THORNTON, KRISTENS . WINSTON, AND JOHN T. BEHRENS (Arizona State University, U.S.A.) SUMMARY.Based upon the premise that persons modify their study behaviom in order to maintain their certainty of responding conectly, a study was conducted with 40 undergraduate volunteers examining the effects of differentpresentationrates and feedback context onresponse confidence andcorrectresponding. Significanteffects in the anticipateddirectionwere found forpresentation rate but not for the context of feedback.
INTRODUCTION INFORMATION processing frameworks have freed instructionalfeedback from its early association with operant psychology, and, as a consequence, such feedback is infrequently assumed to have reinforcing properties. Research on the effects of instructional feedback shows that one of its important functions is the correction of errors (e.g. Kulhavy, 1977). However, there are also data which suggest that simple error correction is an inadequate explanation for important events which occur as a result of being exposed to a post-response knowledge standard (Kulhavy et al., 1976, 1979). It is our view that a key to understanding how feedback influences instructional behaviour lies in a learner’s perceptions of tasks, feedback, and tests, rather than in the objective content of feedback. In a previous paper we have defined a learner’s perceptions of the correctness of their response as their degree of certitude in that response (Kulhavy and Stock, 1989). In this case we accepted the O.E.D. (1933) definition of certitude as, “Subjective certainty; the state of being certain or sure of anything”. Although certitudejudgments about instructionalresponses are not perfect, they do provide useful information about the cognitive status of the response in question. When certitudeis operationalisedas a rating of correct response confidence, Kulhavy et al. (1976, 1979) found a consistent relation between response ratings and learning when questions dealt with a previously studied text segment. Results from these studies indicate post-response study time is directly related to response confidence for persons making an incorrect response and inversely related to confidence for those making a correct response. Individualschoosing incorrectresponseswith high confidencearemore likely to correct such errorsthan thosechoosing incorrectresponses with low confidence.Such results suggest that response certitude has an important role in the acquisition of knowledge from text. This article presents a rationale for the inclusion of certitude in models of text processing, and reports a study designed to evaluate this rationale. We begin with the basic assumption that an instructionaltask involves theachievement of some performance criterion, simply because this production is a way of showing that a planned increment of learning has occurred. Hence, a critical variable is the learner’s perception that the criterion demand can be met after studying.Our contention is that it is the learner’s perception that he will be able to perform correctly which shapes subsequent behaviour in the instructional milieu. The student’s estimate of certitude is determined by the degree of match between perceived comprehension of the criterion and the episodic/semantic prior knowledge available to the learner. We propose that learners compare their perception of the criterion 161
162
Feedback and Certitude
demand against a set of cognitive referents consisting of both content (“Do I know enough about this material to understand it?”) and experience (“Do I generally perform well on this type of task?”) information. The degree of match between the perceived comprehensionand the cognitivereferentscreates an expectation that the criterionresponse produced will be the correct one (e.g., the one desired by the creator of the instruction). We further argue that it is the magnitude of this expectation which determines the behaviours of the learner when feedback is presented after the instructionalresponse. Our model of how response expectancy influences future instructional behaviours employs concepts derived from control theory (e.g., Powers, 1978). Using the metaphor of control theory, “perceived comprehension” is a controlled quantity, and the cognitive referents to which it is compared form the basis for a reference standard. In order to simplify our presentation, we characterise perceived comprehension as a representation of the perceptions of the instructional materials and the response demands (e.g., the questions testing these materials). This representation is matched to materials known by the learner, including information in the instructional message which have become incorporated in the knowledge structure of the individual (Kulhavy and Stock, 1989). In control theory, the operation of a comparator (here the matching of perceived comprehensionwith the knowledge base standard) is a continuousprocess, with control of processing managed by negative feedback loops. In a negative feedback loop, behaviour is directedatmaintainingthevalueofacontrol variableataspecifiedreferencevalue. A similar position may be held here, and one can view such a comparison as controlling behaviours involved in the task of acquiring the instructionalmaterial. Theoretically, we view the feedback message which follows a response as being composed of the two separable components, verification and elaboration (Kulhavy and Stock, 1989).Theverification componentrefersto the simple dichotomousjudgment that the initial instructionalresponse was right or wrong. Hence, when a student is simply told “yes” or “no”, the only function served by feedback is verification. The elaboration component consistsofall substantiveinformationcontainedin the feedbackstatement,such as representing the entire item, or providing an explanation regarding why the response was right or wrong. For ease of understanding we label the verification componentas (f,) and any added elaborations as (fi). We can now construct a notational shorthand for the two effects that feedback has on a learner: (1) Feedback = f, + fi In the present context, we prefer to focus on two points in the instructional sequence, the first shortly after a learner’s response, and the second after the presentation of instructional feedback. Therefore, we suggest that followinginstruction and a response the learner is sensitive to the likelihood of a response being correct (a consequenceof the knowledge monitoring comparison). The magnitude of this likelihood will range from low to high, with higher values reflecting a higher likelihood that the response is correct, according to the standards and perceptions of the learner. With presentation of instructionalfeedback, the learner is informed with respect to an external standard for the response. As a consequence of a comparison between the two perceptions derived from feedback and the prior response (as well as the expectation of responding correctly), the original estimate of performance certitude is transformed into a measure of discrepancy. According to Kulhavy and Stock (1989), discrepancy (d) is a measure of error between input and reference standard which takes into account the degree of match between the initial response (r;) and the feedback message to produce a verification estimate (f ). The verification estimate is then combined with the initial response certitude (c) to yield a discrepancy value for the response in question. Hence, using our notational shorthand, discrepancy is defined as: (2) f, x c = d
RAYMONDW. KULHAWET AL.
163
The scale used to distribute (d) for initially correct and error responses is developed more completely in the Appendix which follows this paper. It is the magnitude of the discrepancy value that we believe controls subsequent studying behaviours. Here, we suggest that a simple behavioural mechanism is engaged, where the learnerundertakesstudyingbehavioursin order to learn responseswhich eliminate mismatchesbetween the initial response and feedback. These subsequent study behaviours will emergeimmediately.For example, we expect learners to study feedback longer when it is associated with incorrect responses. Also, if learners are given a choice to restudy instruction,they will choose to restudy more often those text framesassociatedwith incorrect responses, and to spend more time studying those particular frames. There will also be a greaterlikelihoodthat high confidenceerrors will be replaced by correctresponsescompared to low confidence errors. In designing the present study, we reasoned that limiting the time available to study initial instruction would have two effects. First, shorter study times would directly affect the level of certitude learners expressed in being able to answer questions over the material studied, with shorter presentation intervals associated with lower expressions of response certitude. Second, we expected that subsequent feedback study time would be affected similarly, with shorter text presentation rates being associated with longer feedback study times. Thus, in this study abetween-subjectsfactor included three levels of text presentation, with fast, average, and slow rates determined by times available from a norming study. Next, we were interested in whetherstructuralchangesin the feedback messages would affect feedback study time. While Kulhavy et al. (1985) found that feedback study times increased as a functionof increasingcomplexity of feedback (e.g., increasesinthe magnitude off.) our intent was to examine the effects of changes which did not alter the length of the feedback messages. Therefore, we altered the context of the feedback message, as a withinsubjects factor, by using either the original test item as the basis of feedback (in context),or the original test item with all originaldistractorsreplacedby new plausiblealternativeswhich were structurally equivalent to the old. In this way we hoped to tease out the influence of minor changes in the context of (fJ on subsequent responding. METHOD Design and participants Threelevelsof framereadingtimevariedbetween-subjectsandcrossedwith two levels of feedback context within-subjects.Three reading times had values of slow, average, and fast, while feedback was formed by an in vs. out of context comparison.Participants were 54 undergraduatevolunteers who were randomly assigned to between-subjectscells in order of their appearance for the experiment.Data from 14 individualswere omitted when the microcomputercontrol program failed to transfer to diskettea single time measure on each of these persons, leaving a final sample size of 40. Materials Text materials consisted of 12 paragraph length frames describing the United States Navy. Frames averaged 147 words in length, and were written so that they could be read in any order without breaking the generalcontinuity of the passage. A five-alternative,multiplechoicequestion was based on each frame.Both framesand questionswere modified versions of materials developed and normed earlier (Kulhavy et a f . 1985). The text was easily comprehensibleby an undergraduatepopulation,and all test items could be answered correctly in a straightforwardfashion from the information contained in the paragraph. An example paragraph and its correspondingtest item are displayed below.
164
Feedback and Certitude
The fleet that is able to carry on its own commerce and stop that of an opponent is said to be in “commandof the sea” in the region where it enjoys that markedadvantage.The term “command”in this case does not mean completecontrol, but only that sea-borne communicationsover certain areas are restricted. A fleet can secure command of the sea either by destroying the enemy or by denying him access to the sea area. The first of these alternatives is best because it settles the matter completely and permanently. A fleet that refuses battle and stays in port does not cease to be a threat. The act of avoiding battle while still maintaining the maximum threat value is known as the strategy of the “fleet in being” and forces the fleet in command of the sea area to be constantly on guard. Concerning command of the sea, which of the following statements is true? When a fleet commands a sea area, it is not actually interested in movement of a. the enemy ships themselves. The phrase “fleet in being” refers to a mobile fleet that is eager to come to battle b. as soon as possible. Only the ships of a major fleet are able to exercise command of the sea in a given c. area. Command of the sea area simply means that a fleet controls communications d. over that area. e. The best possible way to command a sea area is to deny an enemy access to that area. In-contextfeedback items were constructed by capitalisingand marking with an asterisk the correct alternative in the corresponding question, otherwise leaving the original question intact. Out-of-context feedback items were created by substituting new distractors in the respective in-context feedback item, and marking the correct alternative in the Same way as in-context items. The four new distractors were constructed from information contained in the target paragraph, were of the same length as the originals, but differed entirely in their content. Each learner saw six in-context and six out-of-context feedback items, with the specific selection of feedback items separately randomised for each participant. During the course of the study learners responded three separate times to a five-point scale which asked them to rate their confidence in their current task. Following the reading of a paragraph they were asked to rate their confidence in being able to answer a multiplechoice question over the materialjust read. Response options ranged from not at all sure (l), through somewhat sure (3), to absolutely sure (5). with points (2) and (4) not labelled. Following their initial response to a test item, and again after their response on the post-test, they wereaskedtorate how confidenttheywerethattheanswerjustgiven tothequestion was the correct one. The scale used in the later two cases had point labels identical to those described above. A post-test consisted of the 12 items originally responded to during the text presentation. Items were printed on 8%x 11 inch (21.25 x 27.50 cm.) white paper and stapled into test booklets for each participant,with the order of items separately randomised for each learner. Frame presentationrates were derived from a norming study using 20 undergraduates whoread the paragraphs at their own rate. Our averagepresentation time was theoverallmean per frame time (M = 52 sec.). Our slow and fast presentation times were one pooled standard deviation (SD = 16sec.), above (68 sec.) and below (36 sec.) the average time,respectively. Procedure An IBM BASIC program controlled presentation of materials in the study, with individual randomisations of text frames and context items for each learner. As learners entered theroom they were randomlyassigned to one of the between-subjects conditions and
RAYMOND W. KULHAVY ET
165
AL.
seated in front of a microcomputer.A monitor told them that they would be reading a series of paragraphs about the Navy, answering a question over each paragraph, and rating their confidence in their ability to answer questionsfrom the text, and in how correct they thought their response was to each test item. A monitor also provided an introduction to use of the terminal, answered procedural questions, and started the program. All participants completeda practice sequence identical in form to the presentationof experimental materials. During the practice session learners read a paragraph on the screen, rated confidence, responded to a test item, rated confidence,and practised the reread option describedbelow.Subsequently,they worked through the stimulusportionof theexperiment. For each of the 12 paragraphs, learners studied the text for their allotted time, rated their confidence in their ability to answer questionsover the material,responded to test item, rated their response confidence, studied the feedback (in or out of context), and were finally given the option of rereading the paragraph prior to moving to a new frame. Each learner saw six in-context and six out-of-context feedback items. Each element of the steps above was presented as a separate screen, with the material for a screen presented simultaneously,rather than line by line. Response choices, confidence ratings, and reread selections were recorded on-line by the computer. Latencies were also recorded for text reading, item responding, feedback study, and reread selections. All responses were automatically written to diskette. When the stimulus portion of the experiment was Completed,learners were given a copy of the post-test booklet and given unlimited time to answer the original 12 items. The last page of the booklet contained a general questionnaireasking participants about what strategies they used in completing the experimental tasks. RESULTS Responses to multiple choice questions were scored for correctness;number of reread options were summed for correct and incorrect responses items within context classifications; and question, feedback and text restudy times were converted to log times according to procedures recommended by Winer (1971, p. 401). Generally, two planned comparisons were used throughout to decompose effects associated with presentation rate; one contrast compared the fast presentation group (N = 14) to the average presentation group (N = 12), while the second contrast compared the combined fast and average presentation groups against the slow presentation group (N = 14). Examined first was correct responding as a function of text presentation rate, context of feedback, and occasion (items immediately following instruction vs. on the post-test). Correct responses Table 1 provides summary statisticsfor correct responses. TABLE 1 MEANCORRECTINSTRUCIIONAL AND POST-TEST RFSFONSESFOR FAST,AVERAGE, AND SLOW PFSSE~ATION GROUPS Presentation Group Fast M€SUR
Instluctional corrects
M a
SD
5.93
2.64
11.07
1.38
Average Mean
Slow
SD
Mean
SD
8.17
2.95
9.36
2.41
9.83
3.04
11.21
1.53
Post-test
Corrects N=
B
14
12
14
Feedback and Certitude
166
Analysis of variance confirmed that overall recall for the fast and average rates of presentation group did not differ (F(, ,$ = 1.62), while these groups combined differed significantly from the slow presentation group (F(, ,) = 4.87, P
RESPONSE FEEDBACK, CERTITUDE AND LEARNING FROM TEXT BYRAYMOND W. KULHAW, WILLIAM A. STOCK, NANCYE. THORNTON, KRISTENS . WINSTON, AND JOHN T. BEHRENS (Arizona State University, U.S.A.) SUMMARY.Based upon the premise that persons modify their study behaviom in order to maintain their certainty of responding conectly, a study was conducted with 40 undergraduate volunteers examining the effects of differentpresentationrates and feedback context onresponse confidence andcorrectresponding. Significanteffects in the anticipateddirectionwere found forpresentation rate but not for the context of feedback.
INTRODUCTION INFORMATION processing frameworks have freed instructionalfeedback from its early association with operant psychology, and, as a consequence, such feedback is infrequently assumed to have reinforcing properties. Research on the effects of instructional feedback shows that one of its important functions is the correction of errors (e.g. Kulhavy, 1977). However, there are also data which suggest that simple error correction is an inadequate explanation for important events which occur as a result of being exposed to a post-response knowledge standard (Kulhavy et al., 1976, 1979). It is our view that a key to understanding how feedback influences instructional behaviour lies in a learner’s perceptions of tasks, feedback, and tests, rather than in the objective content of feedback. In a previous paper we have defined a learner’s perceptions of the correctness of their response as their degree of certitude in that response (Kulhavy and Stock, 1989). In this case we accepted the O.E.D. (1933) definition of certitude as, “Subjective certainty; the state of being certain or sure of anything”. Although certitudejudgments about instructionalresponses are not perfect, they do provide useful information about the cognitive status of the response in question. When certitudeis operationalisedas a rating of correct response confidence, Kulhavy et al. (1976, 1979) found a consistent relation between response ratings and learning when questions dealt with a previously studied text segment. Results from these studies indicate post-response study time is directly related to response confidence for persons making an incorrect response and inversely related to confidence for those making a correct response. Individualschoosing incorrectresponseswith high confidencearemore likely to correct such errorsthan thosechoosing incorrectresponses with low confidence.Such results suggest that response certitude has an important role in the acquisition of knowledge from text. This article presents a rationale for the inclusion of certitude in models of text processing, and reports a study designed to evaluate this rationale. We begin with the basic assumption that an instructionaltask involves theachievement of some performance criterion, simply because this production is a way of showing that a planned increment of learning has occurred. Hence, a critical variable is the learner’s perception that the criterion demand can be met after studying.Our contention is that it is the learner’s perception that he will be able to perform correctly which shapes subsequent behaviour in the instructional milieu. The student’s estimate of certitude is determined by the degree of match between perceived comprehension of the criterion and the episodic/semantic prior knowledge available to the learner. We propose that learners compare their perception of the criterion 161
162
Feedback and Certitude
demand against a set of cognitive referents consisting of both content (“Do I know enough about this material to understand it?”) and experience (“Do I generally perform well on this type of task?”) information. The degree of match between the perceived comprehensionand the cognitivereferentscreates an expectation that the criterionresponse produced will be the correct one (e.g., the one desired by the creator of the instruction). We further argue that it is the magnitude of this expectation which determines the behaviours of the learner when feedback is presented after the instructionalresponse. Our model of how response expectancy influences future instructional behaviours employs concepts derived from control theory (e.g., Powers, 1978). Using the metaphor of control theory, “perceived comprehension” is a controlled quantity, and the cognitive referents to which it is compared form the basis for a reference standard. In order to simplify our presentation, we characterise perceived comprehension as a representation of the perceptions of the instructional materials and the response demands (e.g., the questions testing these materials). This representation is matched to materials known by the learner, including information in the instructional message which have become incorporated in the knowledge structure of the individual (Kulhavy and Stock, 1989). In control theory, the operation of a comparator (here the matching of perceived comprehensionwith the knowledge base standard) is a continuousprocess, with control of processing managed by negative feedback loops. In a negative feedback loop, behaviour is directedatmaintainingthevalueofacontrol variableataspecifiedreferencevalue. A similar position may be held here, and one can view such a comparison as controlling behaviours involved in the task of acquiring the instructionalmaterial. Theoretically, we view the feedback message which follows a response as being composed of the two separable components, verification and elaboration (Kulhavy and Stock, 1989).Theverification componentrefersto the simple dichotomousjudgment that the initial instructionalresponse was right or wrong. Hence, when a student is simply told “yes” or “no”, the only function served by feedback is verification. The elaboration component consistsofall substantiveinformationcontainedin the feedbackstatement,such as representing the entire item, or providing an explanation regarding why the response was right or wrong. For ease of understanding we label the verification componentas (f,) and any added elaborations as (fi). We can now construct a notational shorthand for the two effects that feedback has on a learner: (1) Feedback = f, + fi In the present context, we prefer to focus on two points in the instructional sequence, the first shortly after a learner’s response, and the second after the presentation of instructional feedback. Therefore, we suggest that followinginstruction and a response the learner is sensitive to the likelihood of a response being correct (a consequenceof the knowledge monitoring comparison). The magnitude of this likelihood will range from low to high, with higher values reflecting a higher likelihood that the response is correct, according to the standards and perceptions of the learner. With presentation of instructionalfeedback, the learner is informed with respect to an external standard for the response. As a consequence of a comparison between the two perceptions derived from feedback and the prior response (as well as the expectation of responding correctly), the original estimate of performance certitude is transformed into a measure of discrepancy. According to Kulhavy and Stock (1989), discrepancy (d) is a measure of error between input and reference standard which takes into account the degree of match between the initial response (r;) and the feedback message to produce a verification estimate (f ). The verification estimate is then combined with the initial response certitude (c) to yield a discrepancy value for the response in question. Hence, using our notational shorthand, discrepancy is defined as: (2) f, x c = d
RAYMONDW. KULHAWET AL.
163
The scale used to distribute (d) for initially correct and error responses is developed more completely in the Appendix which follows this paper. It is the magnitude of the discrepancy value that we believe controls subsequent studying behaviours. Here, we suggest that a simple behavioural mechanism is engaged, where the learnerundertakesstudyingbehavioursin order to learn responseswhich eliminate mismatchesbetween the initial response and feedback. These subsequent study behaviours will emergeimmediately.For example, we expect learners to study feedback longer when it is associated with incorrect responses. Also, if learners are given a choice to restudy instruction,they will choose to restudy more often those text framesassociatedwith incorrect responses, and to spend more time studying those particular frames. There will also be a greaterlikelihoodthat high confidenceerrors will be replaced by correctresponsescompared to low confidence errors. In designing the present study, we reasoned that limiting the time available to study initial instruction would have two effects. First, shorter study times would directly affect the level of certitude learners expressed in being able to answer questions over the material studied, with shorter presentation intervals associated with lower expressions of response certitude. Second, we expected that subsequent feedback study time would be affected similarly, with shorter text presentation rates being associated with longer feedback study times. Thus, in this study abetween-subjectsfactor included three levels of text presentation, with fast, average, and slow rates determined by times available from a norming study. Next, we were interested in whetherstructuralchangesin the feedback messages would affect feedback study time. While Kulhavy et al. (1985) found that feedback study times increased as a functionof increasingcomplexity of feedback (e.g., increasesinthe magnitude off.) our intent was to examine the effects of changes which did not alter the length of the feedback messages. Therefore, we altered the context of the feedback message, as a withinsubjects factor, by using either the original test item as the basis of feedback (in context),or the original test item with all originaldistractorsreplacedby new plausiblealternativeswhich were structurally equivalent to the old. In this way we hoped to tease out the influence of minor changes in the context of (fJ on subsequent responding. METHOD Design and participants Threelevelsof framereadingtimevariedbetween-subjectsandcrossedwith two levels of feedback context within-subjects.Three reading times had values of slow, average, and fast, while feedback was formed by an in vs. out of context comparison.Participants were 54 undergraduatevolunteers who were randomly assigned to between-subjectscells in order of their appearance for the experiment.Data from 14 individualswere omitted when the microcomputercontrol program failed to transfer to diskettea single time measure on each of these persons, leaving a final sample size of 40. Materials Text materials consisted of 12 paragraph length frames describing the United States Navy. Frames averaged 147 words in length, and were written so that they could be read in any order without breaking the generalcontinuity of the passage. A five-alternative,multiplechoicequestion was based on each frame.Both framesand questionswere modified versions of materials developed and normed earlier (Kulhavy et a f . 1985). The text was easily comprehensibleby an undergraduatepopulation,and all test items could be answered correctly in a straightforwardfashion from the information contained in the paragraph. An example paragraph and its correspondingtest item are displayed below.
164
Feedback and Certitude
The fleet that is able to carry on its own commerce and stop that of an opponent is said to be in “commandof the sea” in the region where it enjoys that markedadvantage.The term “command”in this case does not mean completecontrol, but only that sea-borne communicationsover certain areas are restricted. A fleet can secure command of the sea either by destroying the enemy or by denying him access to the sea area. The first of these alternatives is best because it settles the matter completely and permanently. A fleet that refuses battle and stays in port does not cease to be a threat. The act of avoiding battle while still maintaining the maximum threat value is known as the strategy of the “fleet in being” and forces the fleet in command of the sea area to be constantly on guard. Concerning command of the sea, which of the following statements is true? When a fleet commands a sea area, it is not actually interested in movement of a. the enemy ships themselves. The phrase “fleet in being” refers to a mobile fleet that is eager to come to battle b. as soon as possible. Only the ships of a major fleet are able to exercise command of the sea in a given c. area. Command of the sea area simply means that a fleet controls communications d. over that area. e. The best possible way to command a sea area is to deny an enemy access to that area. In-contextfeedback items were constructed by capitalisingand marking with an asterisk the correct alternative in the corresponding question, otherwise leaving the original question intact. Out-of-context feedback items were created by substituting new distractors in the respective in-context feedback item, and marking the correct alternative in the Same way as in-context items. The four new distractors were constructed from information contained in the target paragraph, were of the same length as the originals, but differed entirely in their content. Each learner saw six in-context and six out-of-context feedback items, with the specific selection of feedback items separately randomised for each participant. During the course of the study learners responded three separate times to a five-point scale which asked them to rate their confidence in their current task. Following the reading of a paragraph they were asked to rate their confidence in being able to answer a multiplechoice question over the materialjust read. Response options ranged from not at all sure (l), through somewhat sure (3), to absolutely sure (5). with points (2) and (4) not labelled. Following their initial response to a test item, and again after their response on the post-test, they wereaskedtorate how confidenttheywerethattheanswerjustgiven tothequestion was the correct one. The scale used in the later two cases had point labels identical to those described above. A post-test consisted of the 12 items originally responded to during the text presentation. Items were printed on 8%x 11 inch (21.25 x 27.50 cm.) white paper and stapled into test booklets for each participant,with the order of items separately randomised for each learner. Frame presentationrates were derived from a norming study using 20 undergraduates whoread the paragraphs at their own rate. Our averagepresentation time was theoverallmean per frame time (M = 52 sec.). Our slow and fast presentation times were one pooled standard deviation (SD = 16sec.), above (68 sec.) and below (36 sec.) the average time,respectively. Procedure An IBM BASIC program controlled presentation of materials in the study, with individual randomisations of text frames and context items for each learner. As learners entered theroom they were randomlyassigned to one of the between-subjects conditions and
RAYMOND W. KULHAVY ET
165
AL.
seated in front of a microcomputer.A monitor told them that they would be reading a series of paragraphs about the Navy, answering a question over each paragraph, and rating their confidence in their ability to answer questionsfrom the text, and in how correct they thought their response was to each test item. A monitor also provided an introduction to use of the terminal, answered procedural questions, and started the program. All participants completeda practice sequence identical in form to the presentationof experimental materials. During the practice session learners read a paragraph on the screen, rated confidence, responded to a test item, rated confidence,and practised the reread option describedbelow.Subsequently,they worked through the stimulusportionof theexperiment. For each of the 12 paragraphs, learners studied the text for their allotted time, rated their confidence in their ability to answer questionsover the material,responded to test item, rated their response confidence, studied the feedback (in or out of context), and were finally given the option of rereading the paragraph prior to moving to a new frame. Each learner saw six in-context and six out-of-context feedback items. Each element of the steps above was presented as a separate screen, with the material for a screen presented simultaneously,rather than line by line. Response choices, confidence ratings, and reread selections were recorded on-line by the computer. Latencies were also recorded for text reading, item responding, feedback study, and reread selections. All responses were automatically written to diskette. When the stimulus portion of the experiment was Completed,learners were given a copy of the post-test booklet and given unlimited time to answer the original 12 items. The last page of the booklet contained a general questionnaireasking participants about what strategies they used in completing the experimental tasks. RESULTS Responses to multiple choice questions were scored for correctness;number of reread options were summed for correct and incorrect responses items within context classifications; and question, feedback and text restudy times were converted to log times according to procedures recommended by Winer (1971, p. 401). Generally, two planned comparisons were used throughout to decompose effects associated with presentation rate; one contrast compared the fast presentation group (N = 14) to the average presentation group (N = 12), while the second contrast compared the combined fast and average presentation groups against the slow presentation group (N = 14). Examined first was correct responding as a function of text presentation rate, context of feedback, and occasion (items immediately following instruction vs. on the post-test). Correct responses Table 1 provides summary statisticsfor correct responses. TABLE 1 MEANCORRECTINSTRUCIIONAL AND POST-TEST RFSFONSESFOR FAST,AVERAGE, AND SLOW PFSSE~ATION GROUPS Presentation Group Fast M€SUR
Instluctional corrects
M a
SD
5.93
2.64
11.07
1.38
Average Mean
Slow
SD
Mean
SD
8.17
2.95
9.36
2.41
9.83
3.04
11.21
1.53
Post-test
Corrects N=
B
14
12
14
Feedback and Certitude
166
Analysis of variance confirmed that overall recall for the fast and average rates of presentation group did not differ (F(, ,$ = 1.62), while these groups combined differed significantly from the slow presentation group (F(, ,) = 4.87, P
