LSHSS

Research Article

Influence of Prior Knowledge and Interest on Fourth- and Fifth-Grade Passage Comprehension on the Qualitative Reading Inventory—4 Mary Kristen Clarka and Alan G. Kamhia

Purpose: In 2 experiments, we examined the influence of prior knowledge and interest on 4th- and 5th-grade students’ passage comprehension scores on the Qualitative Reading Inventory—4 (QRI–4) and 2 experimenter-constructed passages. Method: In Experiment 1, 4th- and 5th-grade students were administered 4 Level 4 passages or 4 Level 5 passages from the QRI–4. Prior knowledge was assessed by key concept questions from the QRI–4. Interest was rated on a 5-point scale. In Experiment 2, 4th- and 5th-grade students were administered 2 passages from the QRI–4 and 2 experimenter-constructed passages. One general question was used to assess prior knowledge, and 3 measures were used to assess interest. Results: Prior knowledge as measured by key concept questions on the QRI–4 or a general question had minimal

impact on passage comprehension. More important, only one third of the prior knowledge questions on the QRI–4 were related to a comprehension question. When these question pairs were analyzed, having prior knowledge was still not predictive of comprehension performance. Interest level had minimal impact on comprehension performance. Conclusions: This study’s findings raise concerns about the usefulness of prior knowledge assessments on the QRI–4. Interest had little impact on comprehension performance. Educational implications are discussed.

M

consistency in the literature or in educational practice in how prior knowledge is defined, assessed, and measured (Dochy et al., 1999). The conceptualization of prior knowledge is often vague and too general, as exemplified by Jonassen and Gabrowski’s (1993) definition of prior knowledge as “the knowledge, skills, or ability that students bring to the learning process” (p. 417). Contributing to the lack of definitional clarity is that other terms such as background knowledge, world knowledge, domain knowledge, and topic knowledge are also used to refer to prior knowledge (Dochy et al., 1999). Given this lack of definitional clarity, it is impressive that 91.5% of the 183 articles and research reports reviewed by Dochy et al. (1999) showed positive effects of prior knowledge on some measure of comprehension performance. Studies that did not find positive effects for prior knowledge were viewed as methodologically flawed (Dochy et al., 1999). In this study, we questioned whether responses to prior knowledge questions (PKQs) predicted comprehension performance of fourth- and fifth-grade students on the Qualitative Reading Inventory—4 (QRI–4; Leslie & Caldwell, 2006), one of the most frequently used informal

ore than 30 years of research has indicated that it is difficult to overestimate the influence of prior knowledge on reading comprehension and future learning (Dochy, Segers, & Buehl, 1999). Models of comprehension and learning (e.g., Glaser & DeCorte, 1992; Kintsch, 1998) have shown that a well-organized and coherent knowledge base plays a significant role in inference generation, conceptualization, and principled understanding. At the same time, inaccuracies in prior knowledge or misconceptions can negatively affect comprehension and future learning if they are not identified and corrected (Alvermann, Smith, & Readence, 1985; Kendeou & van den Broek, 2007). Despite the obvious importance of prior knowledge for understanding and learning, there has been little

a

University of North Carolina at Greensboro

Correspondence to Mary Kristen Clark: [email protected] Editor: Marilyn Nippold Associate Editor: Donna Boudreau Received September 24, 2013 Revision received April 24, 2014 Accepted July 26, 2014 DOI: 10.1044/2014_LSHSS-13-0074

Key Words: comprehension, prior knowledge, interest, assessment

Disclosure: The authors have declared that no competing interests existed at the time of publication.

Language, Speech, and Hearing Services in Schools • Vol. 45 • 291–301 • October 2014 • © American Speech-Language-Hearing Association

291

reading inventories. The QRI–4 measures prior knowledge by asking children to answer questions that measure their understanding of key concepts. The key concept notion was taken from a study by Langer (1984) that demonstrated the benefits of a prereading activity that required some reflection and discussion of key concepts in the passages to be read. The benefits of prereading discussionbased activities for text comprehension have been confirmed in many subsequent studies (e.g., Carr & Thompson, 1996; McKeown, Beck, Sinatra, & Loxterman, 1992; Pressley, Johnson, Symons, McGoldrick, & Kurita, 1989; Pressley et al., 1992). The key concept questions on the QRI–4 (Leslie & Caldwell, 2006) were “judged to be important to comprehension of the passage” (p. 48). Some questions were written to reflect the topic of the passage (e.g., railroads) or the person the passage was about (e.g., Amelia Earhart). Other questions were included because the authors “believed that if students understood them, they would be more likely to answer the implicit comprehension questions correctly (e.g., questions involving animal defenses in the fifth grade passage ‘The Octopus’)” (p. 48). A 4-point scale is used to score responses: 0 points = unconnected or no responses; 1 point = general association (e.g., “What are seeds?”/“You can buy them in a store”); 2 points = an example of the concept (e.g., “What is wool used for?”/“You can make coats and pants”); 3 points = a precise definition or an answer specifically related to passage content (e.g., “Why do people work?”/“To get money for their families”). Leslie and Caldwell (2006, 2011) reported some pilot data showing that at the first-grade level and higher, conceptual knowledge was significantly correlated with some form of comprehension, either retelling or answers to questions. On the basis of these data, they suggested that students who score at least 55% of the possible points on the knowledge questions score higher than 70% on the comprehension questions (CQs). Other than these pilot data, no other analysis of the relationship between prior knowledge and comprehension performance was conducted. In this study, we examined the actual associations between the PKQs on four Level 4 passages and four Level 5 passages. We predicted that the relationship between prior knowledge and CQs would be higher when PKQs tapped knowledge assessed by a CQ. In a second experiment, we questioned whether student responses to a general question about prior knowledge (i.e., “Do you know much about [topic]?”) were related to comprehension performance. Although Dochy et al. (1999) raised some concerns about the use of self-evaluation to assess prior knowledge, we thought it would be worthwhile to determine whether a straightforward evaluation of topic knowledge was related to comprehension performance. The impact that prior knowledge has on learning is also influenced by topic interest, but untangling the relationship between prior knowledge and interest has not proven to be easy. As with prior knowledge, interest is defined and measured in different ways. There are two general approaches to the study of interest (Hidi, 2001). The

292

first approach focuses on how personal preferences and an individual’s already-formed interests affect performance. The second approach focuses on situational interest, such as how the interestingness of a specific text influences performance (Hidi, 1990, 2001). There was initially some debate (Hidi & McLaren, 1990; Schiefele, 1996; Schiefele & Krapp, 1996) about whether topic interest is best viewed as a type of situational interest or as a form of individual interest, but subsequent research by Ainley, Hidi, and Berndorff (2002) found that both situational and individual factors make significant contributions to topic interest. Investigations into the role of interest in reading have shown that all types of interest tend to facilitate comprehension and recall (Hidi, 2001). Variables that have been found to influence situational interest include text characteristics (e.g., novelty, intensity, and ease of comprehension), aspects of the learning environment, and readers’ self-regulatory activities (Hidi & Harackiewicz, 2000). Sex has also been shown to affect interest ratings and performance (Ainley, Andrews, & Hoey, 2002; Asher & Markell, 1974; Bray & Barron, 2003; Oakhill & Petrides, 2007), but the direction of effects has varied among studies. Bray and Barron (2003), for example, found a small but significant relationship between interest and performance on the Iowa Test of Basic Abilities, whereas Oakhill and Petrides (2007) found that boys’ reading performance was more influenced by interest levels than girls’. Boys performed similarly to girls on a high-interest passage about spiders (60% vs. 65% correct) but significantly worse than girls on a low-interest passage about the evacuation of children during World War II (38% vs. 61% correct). In the first experiment, interest level was assessed using a 5-point scale (ranging from low to high), a common way to measure interest (Bray & Barron, 2003; Hidi, 2001). In the second experiment, we used three measures to assess interest. In addition to the 5-point scale, students ranked the topics from least interesting (1) to most interesting (4) and completed a forced-choice selection of topics. In sum, the two experiments in this study examined the influence of prior knowledge and interest on fourth- and fifth-grade students’ comprehension scores on the QRI–4.1 The findings should provide important information about how prior knowledge and interest level affect comprehension scores on recent versions of the QRI.

Experiment 1 Participants Eighteen students (10 boys, eight girls) enrolled in fourth- and fifth-grade general education classrooms and ranging in age from 9;3 (years;months) to 11;1 (M = 10;2) participated in the first experiment. After written explanation of the nature of the study, parents provided informed 1 The QRI–5 was published during the collection of the data for the second experiment. It has the same Level 4 and 5 passages, prior knowledge, and comprehension questions as the QRI–4.

Language, Speech, and Hearing Services in Schools • Vol. 45 • 291–301 • October 2014

consent, and students completed assent forms. All students performed within normal limits on the Word Identification, Word Attack, and Passage Comprehension subtests of the Woodcock Reading Mastery Test—Revised (Woodcock, 1997) and received Oral Reading Fluency scores on the Dynamic Indicators of Basic Early Literacy Skills (Good & Kaminski, 2002) associated with low risk (118 for fourthgrade students and 124 for fifth-grade students). Median Oral Reading Fluency scores were used to determine placement at Level 4 (n = 9; four boys, five girls) or Level 5 (n = 9; six boys, three girls) passages in the QRI–4 (Leslie & Caldwell, 2006). All students enrolled in the study read passages at their grade level. Table 1 provides means and standard deviations for these measures. In addition, to be included in the study, word reading accuracy on a passage had to be at least 90%.

Procedure The test battery consisted of the Word Identification, Word Attack, and Passage Comprehension subtests from the Woodcock Reading Mastery Test—Revised (Woodcock, 1997); the Oral Reading Fluency subtest from the Dynamic Indicators of Basic Early Literacy Skills (Good & Kaminski, 2002); an interest rating scale; and four passages from the QRI–4 (Leslie & Caldwell, 2006). Two of the passages were narratives, and two were expository (see Table 2). With the exception of the interest rating scale, which was completed in small groups, the test battery was administered to individual students in three 20-min sessions spread out over multiple days within a 2-week period. Testing for all participants was completed in the fall semester. Interest level was evaluated using a 5-point scale (ranging from low to high). Students were asked to rate their interest in passage topics on the basis of the passage’s title or a short description of the passage when titles might be unfamiliar to most readers. For example, “Living in a Foreign Country” was used instead of “Margaret Mead.” A copy of the interest form appears in the Appendix. The presentation of each passage began with students orally responding to the four PKQs associated with each passage. After students answered these questions, they were asked to read the passage aloud and were told that they would be asked some questions about the passage once they finished. Four explicit and four implicit CQs were asked for each passage. Answers to explicit questions were in the text, and answers to implicit questions were not. For example, an explicit question for the beaver

Table 2. Qualitative Reading Inventory—4 passage titles. Level 4 (n = 9) 5 (n = 9)

Narrative

Expository

Amelia Earhart Tomie dePaola Margaret Mead Patricia McKissack

Early Railroads Plant Structures for Survival Farming on the Great Plains How Does Your Body Take in Oxygen?

passage was “What does the beaver eat in the winter?” An example of an implicit question was “Why does the beaver build a dam?” Students were allowed to look at the passages to answer the questions. As Leslie and Caldwell (2006, 2011) noted, determining comprehension level without allowing students to look back at the passages increases the memory demands of the task and may thus underestimate their comprehension abilities. Associations between the PKQs and CQs were independently evaluated by the first author and a graduate student. Agreement was 80%. Disagreements were resolved through discussion between the two authors. Examples of PKQs that had no relationship to CQs were “What is an adventurer?” (“Amelia Earhart”), “What does it mean to be gifted?” (“Tomie dePaola”), “What are the Great Plains?” (“Farming on the Great Plains”), and “What does it mean to inspire someone?” (“Patricia McKissack”). PKQs that were not related to CQs often assessed vocabulary knowledge that was not needed for a correct comprehension response. Although these questions were not related to a CQ, they were related to the content of the passage. PKQs that were associated with a CQ typically assessed general knowledge about a topic. Examples of related PKQ/CQ pairs were “Who was Amelia Earhart?”/“What was Amelia Earhart’s main goal?” and “What are problems caused by beavers?”/“Why might some people dislike beavers?” Two PKQ/CQ pairs were almost identical questions: “Who was Tomie dePaola?”/“Who is Tomie dePaola?” and “What does ‘adapt’ mean?”/“What is adaptation?” A 3-point scoring system was used to score the answers to the PKQs: 0 = incorrect, 1 = partially correct (i.e., general associations), and 2 = correct (i.e., precise definitions or answers). We used a 3-point system rather than Leslie and Caldwell’s (2006) 4-point system because we had difficulty differentiating between general associations (scored as 1 point) and examples of the concept (scored as 2 points). Answers to the CQs were scored as either correct

Table 1. Woodcock Reading Mastery Test—Revised and Dynamic Indicators of Basic Early Literacy Skills scores: Experiment 1.

Participants

Word ID M(SD)

Word Attack M(SD)

Passage Comp M(SD)

ORF M(SD)

Total (N = 18) Boys (n = 10) Girls (n = 8)

109.22 (10.83) 113.50 (9.77) 103.88 (10.16)

107.78 (6.26) 110.20 (4.85) 104.75 (6.78)

107.06 (9.74) 108.80 (8.60) 104.88 (11.19)

132.44 (34.72) 141.40 (35.73) 121.25 (32.07)

Note. ID = identification; Comp = comprehension; ORF = Oral Reading Fluency.

Clark & Kamhi: Influence of Prior Knowledge and Interest

293

or incorrect because they were based on information presented in the texts. No credit was given for partially correct answers. All responses were independently scored by the first author and a graduate student. Interrater reliability was calculated for 35% of the PKQs and all of the CQs. Agreement was 88.9% and 74.2% for PKQs and CQs, respectively. Disagreements were resolved through discussion between the authors.

Results Prior knowledge. Table 3 presents the number of PKQs that corresponded to a CQ for each passage. Approximately one third (11/32; 34.4%) of the PKQs were related to a CQ. One passage did not have any related PKQ–CQ pairs. Five passages only had one related pair; one passage had two pairs. The passage about oxygen had four related PKQ–CQ pairs, but one of the PKQs corresponded to two CQs. This means that only one quarter (25%) of the PKQs for seven of the eight passages queried for information required to answer a CQ. Not surprisingly, performance on the PKQs was not significantly related to comprehension performance, r(71) = –.04, p > .05. Table 4 presents these data. The next analysis considered the variability in prior knowledge scores across the four Level 4 passages and the four Level 5 passages. Prior knowledge scores were comparable for the Level 4 passages but varied for the Level 5 passages, F(3, 35) = 3.46, p < .05. Post hoc tests with the Bonferroni correction revealed a significant difference in the mean prior knowledge scores for the passages about Patricia McKissack (M = 4.78) and Margaret Mead (M = 2.67; p < .05). The effect size for this analysis (d = 1.74) exceeded Cohen’s convention for a large effect (d = 0.80). The limited variability in prior knowledge scores for six of the eight passages made it unlikely that prior knowledge would be predictive of comprehension performance. The next series of analyses examined student performance on the 11 related PKQ–CQ pairs. There were a total of 98 responses to these 11 PKQ–CQ pairs with four possible response patterns: 1.

+PKQ/+CQ (prior knowledge/correct comprehension response),

Table 4. Prior knowledge (maximum = 8) and comprehension scores (maximum = 8).

Level and Passage Level 4 (n = 9) Amelia Earhart Tomie dePaola Early railroadsa Plant structures Level 5 (n = 9) Margaret Mead Patricia McKissack Farming Oxygen Combined (N = 18)

Level 4

Amelia Earhart Tomie dePaola Early railroads Plant structures

1 1 1 1

Passage Margaret Mead Patricia McKissack Farming Oxygen

5.33 (1.58) 5.67 (1.00) 4.75 (1.04) 3.89 (1.90)

2.67 (1.00) 4.78 (1.39) 3.89 (2.03) 3.33 (1.12) 4.68 (1.73)

5.78 (1.20) 5.44 (1.33) 6.11 (1.76) 4.78 (1.64) 5.23 (1.55)

2.

+PKQ/–CQ (prior knowledge/incorrect comprehension response),

3.

–PKQ/+CQ (no prior knowledge/correct comprehension response), and

4.

–PKQ/–CQ (no prior knowledge/incorrect comprehension response).

Figure 1 visually displays the proportion of each response type. The two most frequent patterns were +PKQ/+CQ (35/98; 35.7%) and –PKQ/+CQ (33/98, 33.7%). This means that students were just as likely to answer a CQ correctly when they had prior knowledge of a key concept as when they did not have prior knowledge. Having prior knowledge also did not ensure that students produced the correct comprehension response. The +PKQ/–CQ response pattern occurred 25% (24/98) of the time. This occurred most often when the knowledge activated by the PKQ was counterfactual to explicit text material. For example, most students responded to the PKQ “Why do people run races?” by referring to the quality of speed (i.e., to see who is the fastest Figure 1. Influence of prior knowledge (PK) on related comprehension question (CQ) performance.

No. 2 1 0 4a

Note. Maximum = 4. a

One prior knowledge question corresponded to two comprehension questions.

294

5.44 (1.33) 5.78 (1.39) 5.50 (1.20) 6.11 (1.27)

n = 8.

Level 5 No.

Comprehension M(SD)

a

Table 3. Number of prior knowledge questions that corresponded to a comprehension question, by passage.

Passage

Prior Knowledge M(SD)

Language, Speech, and Hearing Services in Schools • Vol. 45 • 291–301 • October 2014

or to win). The correct response to the corresponding CQ “Why did Cooper set up the race between Tom Thumb and the horse?” does not involve this key concept. The text explicitly indicates that Cooper set up the race to advertise his new invention, the steam engine. The adverse effect of prior knowledge is most evident in a direct comparison of comprehension responses with and without prior knowledge. The +PKQ/–CQ pattern occurred 41% (24/59) of the time, whereas the –PKQ/–CQ pattern occurred only 15% (6/39) of the time. In addition, not having prior knowledge was more likely to lead to a correct response than having prior knowledge. The –PKQ/+CQ pattern occurred an overwhelming 85% (33/39) of the time, compared with the –PKQ/–CQ pattern, which occurred 59% (35/59) of the time. Prior knowledge most aided comprehension for some implicit questions about unfamiliar topics. Four implicit CQs from four passages accounted for 45% of these responses. For example, the ability to answer the PKQ “What are problems learning a new language?” helped students answer the related CQ “What made Margaret Mead decide she would be able to stay in Samoa?” Prior knowledge tended to have little influence on comprehension when questions were explicit and answers were easily found in the text. Four explicit CQs from four passages accounted for 75% of these responses. For example, the inability to answer the PKQ “Who was Tomie dePaola?” had no influence on students’ ability to answer the identical CQ because the entire passage was about this person and the first line of the text explicitly told readers who he is. Interest. Table 5 presents the mean topic interest and comprehension scores by passage for boys, girls, and the total sample. Mean scores for all variables, collapsed across passages, were similar for boys and girls. A significant relationship was not found between interest and comprehension

for the total sample, r(69) = .03, p > .05; for boys, r(40) = .08, p >.05; or for girls, r(31) = –.07, p > .05. Interest scores were not significantly different for Level 4 or Level 5 students ( p > .05). Summary Prior knowledge had no influence on comprehension performance for the eight Level 4 and Level 5 passages. The lack of influence of prior knowledge can be explained, however, by examining the association between PKQs and CQs. Two thirds of PKQs (66%; 21/32) were not related to a CQ. Only one passage (“Oxygen”) had more than two PKQs related to a CQ. Three fourths of the PKQs for the other seven passages queried information that had no relationship to a CQ. In Experiment 2, we questioned whether the simple, general question, “Do you know much about [topic]?” would provide a more accurate assessment of prior knowledge than the QRI–4’s PKQs. The interest and sex findings were inconsistent with those of previous studies (e.g., Hidi, 2001; Oakhill & Petrides, 2007). None of the passages on the QRI–4 (Leslie & Caldwell, 2006) showed a clear sex preference. This may be because none of the passages were particularly interesting to boys. The way interest was measured may also have contributed to the lack of significant interest effects. Experiment 2 was designed to address these shortcomings. To ensure that topic interest truly varied from high to low, two passages were created on topics for which boys expressed high (Komodo dragons) and low (camels) interest. Students were also administered two passages from the QRI–4. Three measures were used to measure interest. In addition to a 5-point scale, students ranked the topics from least interesting (1) to most interesting (4) and were asked to choose which of two topics was most interesting (forced choice). Requiring children to rank order their interest levels for

Table 5. Topic interest (maximum = 5) and comprehension (maximum = 8) scores, by sex. Boys Level and Passage Level 4 n Amelia Earhart Tomie dePaola Early railroads Plant structures Level 5 n Margaret Mead Patricia McKissack Farming Oxygen Combined n Collapsed

Interest M(SD)

Girls Comp M(SD)

Interest M(SD)

5.75 (1.26) 5.50 (0.58) 5.50 (1.00) 5.00 (1.41)

2.80 (1.79) 3.60 (1.14) 1.25a (0.50) 3.00 (1.23)

6.00 (1.55) 6.00 (1.27) 6.33 (2.25) 5.17 (1.94)

3.00 (1.73) 3.33 (1.53) 2.33 (0.58) 3.00 (1.00)

5.43 (1.52)

2.81 (1.33)

4 3.50 (1.29) 3.75 (0.96) 3.00 (1.63) 3.00 (1.41)

Interest M(SD)

6.00 (1.41) 6.00 (1.00) 5.00a (1.41) 4.40 (1.67)

3.11 (1.54) 3.67 (1.00) 2.13 (1.46) 3.00 (1.23)

5.67 (1.53) 6.33 (1.53) 6.33 (0.58) 5.00 (1.73)

3.33 (1.66) 3.00 (1.23) 1.78 (1.09) 3.44 (1.42)

4.97 (1.58)

2.94 (1.41)

Comp M(SD) 9

3

10 3.05 (1.48)

Comp M(SD) 5

6 3.50 (1.76) 2.83 (1.17) 1.50 (1.23) 3.67 (1.63)

Total

5.89 (1.27) 5.78 (0.83) 5.25 (1.17) 4.67 (1.50) 9

8

5.89 (1.45) 6.11 (1.27) 6.33 (1.80) 5.11 (1.76) 18 5.23 (1.55)

Note. Comp = comprehension. n = 4.

a

Clark & Kamhi: Influence of Prior Knowledge and Interest

295

passages avoids the clustering that occurred with the freechoice 5-point ranking system used in the first experiment.

Experiment 2 The purpose of Experiment 2 was twofold: (a) to determine whether a straightforward self-evaluation of prior knowledge would be a better predictor of comprehension than the key concept questions on the QRI–4 and (b) to determine whether high-interest passages and more sensitive measures of interest would lead to positive interest effects for passage, gender, or both.

Participants Twenty-five students enrolled in fourth-grade (eight boys, 10 girls) and fifth-grade (two boys, five girls) general education classrooms and ranging in age from 9;2 to 10;11 (M = 10;1) participated in the second experiment. None of the students participated in the first study. Parent consent and student assent was obtained as in Experiment 1. To be included in the study, students had to score within normal limits on the Word Identification, Word Attack, and Passage Comprehension subtests of the Woodcock Reading Mastery Test—Revised (Woodcock, 1997) and demonstrate age-appropriate median Oral Reading Fluency scores on the Dynamic Indicators of Basic Early Literacy Skills (Good & Kaminski, 2002). See Table 6 for group means and standard deviations.

Procedure Testing was conducted individually during three 25-min sessions over approximately 3 weeks in the fall semester. During the first session, students completed the Oral Reading Fluency subtest from the Dynamic Indicators of Basic Early Literacy Skills (Good & Kaminski, 2002); the Word Identification, Word Attack, and Passage Comprehension subtests from the Woodcock Reading Mastery Test—Revised (Woodcock, 1997); and topic interest rating scales. Over the next two sessions, students read four expository passages, two from the QRI–4 (Leslie & Caldwell, 2006) and two experimenter-constructed passages. Passages were presented in random order, based on results from a random-number generator (http://www.randomizer.org/form.htm). At the end of the study, interest levels were reassessed. The topics for the experimenter-constructed passages were selected on the basis of additional data collected in

Experiment 1 about boys’ interest in animals. The boys were most interested in Komodo dragons and least interested in camels. Lexile estimates for the QRI–4 and experimental passages were calculated to evaluate readability. The Lexile Framework for Reading (http://www.lexile. com/analyzer/) takes into consideration word count, word frequency, and sentence length. Lexile measures for the six passages ranged from 650L to 1050L (see Table 7). The higher Lexile scores for the two experimental passages were due to the greater number of words and conjoined sentences. The Lexile range for fourth and fifth grade is 740L to 1010L according to the Common Core State Standards (Council of Chief State School Officers, 2013). The QRI–4 passages were at the low end of this range, whereas the experimental passages were at the high end or slightly above. The correlation between passage comprehension and Lexile scores was not significant, r(4) = –.45, p > .05, however, and the findings reported in the next section indicate that differences in Lexile scores had no significant impact on comprehension. Readability was also evaluated by asking the four teachers of the participants whether the two passages were appropriate for fourth- and fifth-grade students. All the teachers indicated that the passages were grade appropriate. Prior knowledge was assessed immediately before students read each passage with the question “Do you know much about [topic]?” Students were then asked to read the passage aloud to determine accuracy. Students then orally answered eight comprehension (four explicit, four implicit) questions. After reading all four passages, students were reminded of their original topic interests and told that it was okay if their interest in a passage changed (increased or decreased) after reading it. They were then asked to rank their interest in the four passages on a scale ranging from least interesting (1) to most interesting (4). As seen below, topic interest ratings and rankings were consistent. Students’ responses to each PKQ were recorded as either yes or no. CQs were scored as correct (1) or incorrect (0). All responses were independently scored by the first or second author and a graduate student. Interrater reliability for scores for all CQs was 94%. Disagreements were resolved through discussion between the authors. Three separate interest rating scales were designed to evaluate student interest levels for the four topics. As in Experiment 1, students first completed a 5-point interest scale for each topic. Students then ranked the topics from least interesting (1) to most interesting (4) and completed a

Table 6. Woodcock Reading Mastery Test—Revised and Dynamic Indicators of Basic Early Literacy Skills scores: Experiment 2.

Participants

Word ID M(SD)

Word Attack M(SD)

Passage Comp M(SD)

ORF M(SD)

Total (N = 25) Boys (n = 10) Girls (n = 15)

90.40 (3.16) 88.70 (3.30) 91.53 (2.59)

102.17 (5.92) 102.30 (6.48) 102.07 (5.75)

90.08 (2.40) 89.30 (2.31) 90.60 (2.38)

136.40 (32.41) 127.20 (41.48) 142.53 (24.36)

Note. ID = identification; Comp = comprehension; ORF = Oral Reading Fluency.

296

Language, Speech, and Hearing Services in Schools • Vol. 45 • 291–301 • October 2014

Table 7. Lexical values for QRI–4 and experimental passages. Passage and Title

Lexile

QRI–4 Level 4 The Busy Beaver Early Railroads QRI–4 Level 5 The Octopus Farming on the Great Plains Experimental Camels Komodo Dragons

670L 810L 650L 810L 1010L 1050L

Note. QRI–4 = Qualitative Reading Inventory—4.

forced-choice selection, indicating which of the two QRI–4 passage topics was more interesting and which of the two experimental topics was more interesting.

Results Table 8 presents the responses to the PKQs according to sex. The boys did not know much about beavers and camels, but half said they knew a lot about railroads and Komodo dragons. Girls reported knowledge of farming and octopi, but few had knowledge of the other four topics. Table 9 presents the mean passage comprehension scores as a function of prior knowledge. As can be seen in this table, prior knowledge about a topic had no impact on comprehension performance. The total mean scores summed across the passages were essentially identical: 5.1/ 5.2 (+PK/–PK). The girls trended in the predicted direction, having slightly higher comprehension scores when they had prior knowledge than when they did not. The boys trended in the opposite direction; their comprehension scores were slightly higher when they did not have prior knowledge. Significant differences were found in scaled interest scores between some of the passages (see Table 10 for values). Girls were more interested in beavers than railroads, t(9) = 2.94, p < .05, d = 1.96. Boys had similar interest levels for beavers and railroads. As expected, boys preferred the passage about Komodo dragons over the passage about Table 8. Prior knowledge responses: Experiment 2. Boys Topic QRI–4 Level 4 Beavers Railroads QRI–4 Level 5 Octopus Farming Experimental Camels Komodo Dragons

Girls

Yes

No

Yes

No

1 4

6 4

2 2

8 8

1 1

1 1

4 5

1 0

2 5

8 5

5 4

10 10

Note. QRI–4 = Qualitative Reading Inventory—4.

camels, t(9) = 4.36, p < .01, d = 2.91. Findings from the other topic interest measures corroborated the scaled interest rankings. Boys showed equivalent interest levels for the QRI–4 passages, whereas girls showed clear preferences— beavers over railroads and octopi over farming. Nine of the 10 girls at Level 4 chose beavers over railroads, and all of the Level 5 girls chose octopi over farming. As expected, boys ranked Komodo dragons first (3.5) and camels last (1.3). When forced to choose between the two, all of them chose dragons. Girls, in contrast, showed similar interest rankings for the Komodo dragons (2.5) and camels (2.7). Forced to choose, nine of 15 selected Komodo dragons. These findings appear to confirm our belief that the QRI–4 passages were not of high interest to boys. Despite boys’ high level of interest in Komodo dragons and low level of interest in camels, there was no difference in their comprehension performance (Table 10). The mean scores (5.0) were actually identical. Surprisingly, it was the girls’ comprehension, not boys’, that was affected by interest. The girls answered significantly more questions correctly for the passage about beavers than for the passage about railroads, t(17) = 2.30, p < .05, d = 1.12. Interestingly, the boys also showed significantly better comprehension of the beaver passage than the railroad passage, t(7) = 2.87, p < .05, d = 2.17, despite their similar levels of interest in beavers and railroads. It may be that the questions for the railroad passage were more difficult than those for the other passages. The mean number correct (3.83) was 1.29 lower than for the next most difficult passage (camels).

Discussion Two experiments were conducted that examined the influence of prior knowledge and interest on comprehension of Level 4 and Level 5 passages on the QRI–4 (Leslie & Caldwell, 2006) and two experimenter-created passages. In Experiment 1, we found no relationship between prior knowledge and comprehension and little variability in prior knowledge performance across eight QRI–4 passages. This finding was not surprising, however, because the majority of PKQs (65%) were not related to a CQ on the QRI–4. Only one passage (“Oxygen”) had more than two PKQs related to a CQ. Three fourths of the PKQs for the other seven passages queried information that had no relationship to a CQ. These findings raise questions about the value of the QRI–4’s PKQs for Levels 4 and 5. Interestingly, Leslie and Caldwell (2006) also found that at Level 4, prior knowledge did not predict narrative or expository comprehension. Only retelling was predicted by prior knowledge scores. No data were available for Level 5. At Level 6, prior knowledge was correlated with retelling and comprehension of narrative texts but not expository tests. Leslie and Caldwell did not attempt to explain why prior knowledge was not related to comprehension at Level 4 or why prior knowledge did not predict performance on expository texts at Level 6.

Clark & Kamhi: Influence of Prior Knowledge and Interest

297

Table 9. Comprehension scores (maximum = 8) by prior knowledge. Boys Passage QRI–4 Level 4 Beavers Railroads QRI–4 Level 5 Octopus Farming Experimental Camels Komodo Dragons Collapsed

Girls

Total

Yes M(SD)

No M(SD)

Yes M(SD)

No M(SD)

Yes M(SD)

No M(SD)

7.00 (—) 3.50 (1.73)

5.50 (1.05) 4.00 (1.63)

6.00 (0.00) 3.50 (2.12)

6.00 (1.20) 4.00 (1.07)

6.33 (0.58) 3.50 (1.64)

5.79 (1.12) 4.00 (1.21)

7.00 (—) 4.00 (—)

6.00 (—) 7.00 (—)

6.25 (1.50) 6.40 (1.14)

6.00 (—) —

6.40 (1.34) 6.00 (1.41)

6.00 (0.00) 7.0 (—)

3.50 (0.71) 5.00 (1.00) 4.55 (0.98)

5.38 (1.30) 5.00 (1.23) 5.28 (0.75)

5.40 (1.14) 5.50 (1.30) 5.41 (1.43)

5.10 (1.20) 4.90 (0.88) 5.10 (0.71)

4.86 (1.35) 5.22 (1.09) 5.07 (1.32)

5.22 (1.22) 4.93 (0.96) 5.17 (0.71)

Note. QRI–4 = Qualitative Reading Inventory—4. Dashes indicate no values could be obtained based on the sample size once it was broken down by sex/passage.

A different picture of the relationship between prior knowledge and comprehension emerged from the analysis of the PKQs that were related to a CQ. Having prior knowledge was not an advantage for comprehension. In fact, the data suggest that it was actually preferable not to have prior knowledge. Having prior knowledge did not improve students’ abilities to provide the correct answer to a CQ. The correct response percentage was similar for having and not having prior knowledge. Students answered CQs correctly 36% of the time when they had prior knowledge compared with 34% when they did not. There was a difference, however, in the incorrect response rate based on prior knowledge. Having prior knowledge led to an incorrect comprehension response 25% of the time, whereas no prior knowledge led to comprehension errors only 6% of the time. The adverse effects of prior knowledge were more evident when the effects of having and not having prior knowledge were directly compared. When students had prior knowledge, they produced an incorrect response 41% of

the time compared with only 15% when they did not have prior knowledge. It seems that students mistakenly assumed that the knowledge they had about a topic was correct and complete. Belief in the accuracy of their knowledge led them to ignore or discount the inconsistent or novel information in the passage. Not having prior knowledge was also more likely to lead to a correct response than having prior knowledge. Without prior knowledge, students produced correct comprehension answers 85% of the time, compared with 60% of the time when they did have prior knowledge. Taken together, these data indicate that even when inaccurate or incomplete prior knowledge was contradicted or modified by new information, students used their prior knowledge tapped by a PKQ to answer CQs. Prior knowledge appeared to help students answer implicit CQs about an unfamiliar topic. Almost half of the +PK/+CQ responses (45%) occurred for four implicit CQs from four passages. The importance of prior knowledge for answering implicit questions was not unexpected. Leslie

Table 10. Scaled topic interest (maximum = 5) and comprehension (maximum = 8) scores, by sex. Boys Passage QRI–4 Level 4 n Beavers Railroads QRI–4 Level 5 n Octopus Farming Experimental n Camels Komodo Dragons

Interest M(SD)

2.88a (1.55) 2.63 (0.74)

8

Girls Comp M(SD)

Interest M(SD)

5.71a (1.11) 3.75 (1.58)

3.90 (1.20) 2.50 (1.43)

6.50 (0.71) 5.50 (2.12)

3.80 (1.30) 2.80 (1.64)

5.00 (1.41) 5.00 (1.05)

3.20 (1.15) 2.87a (1.51)

Interest M(SD)

6.00 (1.05) 3.90 (1.20)

3.44a (1.42) 2.56 (1.15)

6.20 (1.30) 6.40 (1.14)

3.71 (1.11) 2.86 (1.35)

5.20 (1.15) 5.07a (1.00)

2.60 (1.38) 3.24a (1.48)

5

10 1.70 (1.25) 3.80 (1.32)

Comp M(SD) 10

2 3.50 (0.71) 3.00 (0.00)

Total

18

5.88a (1.05) 3.83 (1.34)

7

15

6.29 (1.11) 6.14 (1.35) 25

Note. Comp = comprehension; QRI–4 = Qualitative Reading Inventory—4. n is 1 less than indicated because of the omission of the frustration-level passage.

a

298

Comp M(SD)

Language, Speech, and Hearing Services in Schools • Vol. 45 • 291–301 • October 2014

5.12 (1.24) 5.04a (1.00)

and Caldwell (2006) selected the key concepts for the PKQs ion the basis of their relationship to responses to implicit CQs. Leslie and Caldwell cited an unpublished 1991 dissertation by Chrystal showing that key concepts related to implicit questions were more highly correlated with comprehension than the original QRI concepts. This finding provided the rationale for the creation of the PKQs first used in the QRI–3. The QRI–4 and QRI–5 use the same PKQs. In Experiment 2, prior knowledge was assessed by asking students to evaluate whether or not they knew about a topic. We had hoped that a straightforward selfevaluation of prior knowledge would be a better predictor of comprehension than the key concept questions on the QRI–4, but this was not the case. When students said they knew about a topic, they answered an average of 5.1 CQs correctly. This was almost identical to the average of 5.2 correct responses when they did not know much about a topic. These findings offer support for the finding in Dochy et al. (1999) that self-estimation measures are not valid measures of prior knowledge. It is possible, however, that students’ evaluation of their topic knowledge was accurate, but this knowledge provided no advantage in correctly answering passage CQs. The use of flawed assessment procedures may lead practitioners and educators to underestimate the strength of the relationship between prior knowledge and comprehension performance. The systematic review conducted by Dochy et al. (1999) found that prior knowledge explained between 30% and 60% of the variance in comprehension performance. The importance of prior knowledge is best seen in studies in which low-ability readers with prior knowledge outperform high-ability readers without prior knowledge (Recht & Leslie, 1988; Schneider, Körkel, & Weinert, 1989). These studies use passages about rulebased games such as baseball or soccer in which familiarity with the games provides a clear advantage in understanding the text. In Experiment 1, interest had no effect on comprehension performance for either boys or girls. We suggested that this might be because none of the passages on the QRI–4 (Leslie & Caldwell, 2006) were particularly interesting to boys. The findings from Experiment 2 indicated that we were right. Boys showed no clear interest preferences for the four QRI–4 passages but had clear preferences for the two passages we created about Komodo dragons and camels. In the forced-choice task, every boy chose the passage about Komodo dragons over the passage about camels. In contrast, the girls showed clear preferences for the QRI–4 passages (beavers over railroads and octopi over farming) but no clear preference for the Komodo dragon or camel passages. Despite the fact that boys were more interested in Komodo dragons than camels, their mean comprehension performance for the two passages was identical. Surprisingly, it was girls’ comprehension that appeared to be influenced by interest, not boys’. The girls had their poorest mean comprehension score on the passage (“Railroads”)

that had the lowest mean interest level. They also answered significantly more questions correctly in the passage about beavers, for which they showed high interest, than the passage about railroads. If interest truly influenced girls’ comprehension, its effects should be seen for other passages as well. Girls’ interest in farming was almost as low as their interest in railroads (2.8/2.5), yet despite their low interest, they obtained their highest comprehension scores on this passage (6.4). Boys also showed significantly better comprehension of the beaver passage than the railroad passage, but unlike the girls, they had similar levels of interest in beavers and railroads. Although the beaver and railroad passages are both Level 4 passages, the mean comprehension score of 3.8 for the railroad passage was 1.3 points lower than that for the next most difficult passage (camels). In light of these findings, what appeared to be an interest effect for girls is actually a passage effect. The implicit questions for the railroad passage were particularly difficult for all of the students. Only one student answered three of the four questions correctly, whereas two thirds (12/18) of the students answered one or none of the implicit questions correctly. The absence of a significant sex difference for interest raises questions about whether the sex differences found in the Oakhill and Petrides (2007) study are the exception rather than the rule. We know of no other study showing such dramatic differences. Perhaps sex differences only occur at the extremes—when topics are viewed very favorably by one sex and highly unfavorably by the other. Although students in Experiment 2 preferred Komodo dragons over camels, maybe they did not dislike camels enough to affect comprehension. The important finding here, though, was that differences in interest did not affect comprehension performance. Perhaps the small-group and individual testing sessions motivated students to perform at their maximum levels. More important, lack of interest in reading material could have adverse effects on student motivation and performance. Reading programs that are based on choice such as concept-oriented reading instruction have been found to be very effective in improving reading performance in less skilled readers (e.g., Guthrie et al., 2007; Guthrie, Klauda, & Ho, 2013).

Limitations and Future Directions As with many studies, this one would have benefited from more participants at each reading level. Selecting students who read at instructional levels (90%) may have been another limitation of the study. Interest may have had a greater influence on comprehension performance for students who were less proficient readers (e.g., those reading at frustration level). Prior knowledge effects may also have been influenced by reading level. It is also possible that interest and prior knowledge are more noticeable at higher levels of knowledge demonstration, such as analysis, synthesis, and interpretation in Bloom’s taxonomy (Krathwohl, 2002). The findings of this study raise concerns about the usefulness of prior knowledge assessments on the QRI–4.

Clark & Kamhi: Influence of Prior Knowledge and Interest

299

Comprehension performance was actually better when students did not have prior knowledge than when they did. This means that comprehension performance on the QRI–4 may actually be better when the PKQs are not asked before the passage is read. A future study could easily test this possibility. Given the potential problems with prior knowledge assessments, educators should consider using prereading activities that include active discussion of key concepts rather than static assessments of students’ prior knowledge of these concepts. If prior knowledge is assessed, educators should consider including specific instruction to modify knowledge based on new text information that is incongruent with prior knowledge.

Acknowledgments We express our appreciation to the teachers, parents, and students who participated. Thanks also go to Patricia Wasco, Ashley Jones, Brandi Carter, Michelle Middleton, and Kelly Shelton for their assistance with the project.

References Ainley, M., Andrews, M., & Hoey, L. (2002, April). Topic interest, individual interest, and reading comprehension scores. Paper presented at the Interactive Symposium on Interest and Reading Comprehension, Annual Meeting of the American Educational Research Association, New Orleans, LA. Ainley, M. D., Hidi, S., & Berndorff, D. (2002). Interest, learning, and the psychological processes that mediate their relationship. Journal of Educational Psychology, 94, 545–561. Alvermann, D. E., Smith, L. C., & Readence, J. E. (1985). Prior knowledge activation and the comprehension of compatible and incompatible text. Reading Research Quarterly, 20, 420–436. Asher, S. M., & Markell, R. A. (1974). Sex differences in comprehension of high- and low-interest reading material. Journal of Educational Psychology, 66, 680–687. Bray, G. B., & Barron, S. (2003). Assessing reading comprehension: The effects of text-based interest, gender, and ability. Educational Assessment, 9, 107–128. Carr, S. C., & Thompson, B. (1996). The effects of prior knowledge and schema activation strategies on the inferential reading comprehension of children with and without learning disabilities. Learning Disability Quarterly, 19, 48–61. Council of Chief State School Officers. (2013). Supplemental information for Appendix A of the Common Core State Standards for English Language Arts and Literacy: New research on text complexity. Retrieved from http://achievethecore.org/page/60/ supplement-to-appendix-a-detail-pg Dochy, F., Segers, M., & Buehl, M. M. (1999). The relation between assessment practices and outcomes of studies: The case of research on prior knowledge. Review of Educational Research, 69, 145–186. Glaser, R., & DeCorte, E. (1992). Preface. In F. Dochy (Ed.), Assessment of prior knowledge as a determinant for future learning (pp. 1–2). London, England: Jessica Kingsley. Good, R. H., & Kaminski, R. A. (Eds.). (2002). Dynamic Indicators of Basic Early Literacy Skills (6th ed.). Eugene, OR: Institute for the Development of Educational Achievement. Retrieved from http://dibels.uoregon.edu/ Guthrie, J. T., Hoa, L. W., Wigfield, A., Tonks, S. M., Humenick, N. M., & Littles, E. (2007). Reading motivation and reading

300

comprehension growth in the later elementary years. Contemporary Educational Psychology, 32, 282–313. Guthrie, J. T., Klauda, S. L., & Ho, A. N. (2013). Modeling the relationships among reading instruction, motivation, engagement, and achievement for adolescents. Reading Research Quarterly, 48, 9–26. Hidi, S. (1990). Interest and its contribution as a mental resource for learning. Review of Educational Research, 60, 549–571. Hidi, S. (2001). Interest, reading, and learning: Theoretical and practical considerations. Educational Psychology Review, 13, 191–209. Hidi, S., & Harackiewicz, J. (2000). Motivating the academically unmotivated: A critical issue for the 21st century. Review of Educational Research, 70, 151–179. Hidi, S., & McLaren, J. (1990). The effect of topic and theme interestingness on the production of school expositions. In H. Mandl, E. De Corte, N. Bennet, & H. F. Friedrick (Eds.), Learning and instruction: European research in an international context: Vol. 2.2 (pp. 295–308). Oxford, England: Pergamon Press. Jonassen, D. H., & Gabrowski, B. L. (1993). Handbook on individual differences, learning, and instruction: Part VII, prior knowledge. Hillsdale, NJ: Erlbaum. Kendeou, P., & van den Broek, P. (2007). The effects of prior knowledge and text structure on comprehension processes during reading of scientific texts. Memory & Cognition, 35, 1567–1577. Kintsch, W. (1998). Comprehension: A paradigm for cognition. New York, NY: Cambridge University Press. Krathwohl, D. (2002). A revision of Bloom’s taxonomy: An overview. Theory Into Practice, 41, 212–218. Langer, J. (1984). Examining background knowledge and text comprehension. Reading Research Quarterly, 14, 468–481. Leslie, L., & Caldwell, J. S. (2006). Qualitative Reading Inventory (4th ed.). Boston, MA: Pearson. Leslie, L., & Caldwell, J. S. (2011). Qualitative Reading Inventory (5th ed.). Boston, MA: Pearson. McKeown, M. G., Beck, I. L., Sinatra, G. M., & Loxterman, J. A. (1992). The relative contribution of prior knowledge and coherent text to comprehension. Reading Research Quarterly, 27, 78–93. Oakhill, J. V., & Petrides, A. (2007). Sex differences in the effects of interest on boys’ and girls’ reading comprehension. British Journal of Psychology, 98, 223–235. Pressley, M., Johnson, C. J., Symons, S., McGoldrick, J. A., & Kurita, J. A. (1989). Strategies that improve children’s memory and comprehension of text. Elementary School Journal, 90, 3–32. Pressley, M., Wood, E., Woloshyn, V. E., Martin, V., King, A., & Menke, D. (1992). Encouraging mindful use of prior knowledge: Attempting to construct explanatory answers facilitates learning. Educational Psychologist, 27, 91–109. Recht, D. R., & Leslie, L. (1988). Effects of prior knowledge on good and poor readers’ memory of text. Journal of Educational Psychology, 80, 16–20. Schiefele, U. (1996). Topic interest, text representation, and quality of experience. Contemporary Educational Psychology, 12, 3–18. Schiefele, U., & Krapp, A. (1996). Topic interest and free recall of expository text. Learning and Individual Differences, 8, 141–160. Schneider, W., Körkel, J., & Weinert, F. E. (1989). Domainspecific knowledge and memory performance: A comparison of high- and low-aptitude children. Journal of Educational Psychology, 81, 306–312. Woodcock, R. N. (1997). Woodcock Reading Mastery Tests— Revised/normative update. Circle Pines, MN: American Guidance Service.

Language, Speech, and Hearing Services in Schools • Vol. 45 • 291–301 • October 2014

Appendix Topic Interest Form Please tell us how interested you are in the following areas using a scale of 1 to 5, with 1 being not interested at all, 3 being kind of interested, and 5 being very interested. Circle only one number for each topic. Level 4 Flying a plane 1 Not Interested at All

2

3 Kind of Interested

4

5 Very Interested

Becoming an author and illustrator 1 Not Interested at All

2

3 Kind of Interested

4

5 Very Interested

Railroads 1 Not Interested at All

2

3 Kind of Interested

4

5 Very Interested

How plants grow 1 Not Interested at All

2

3 Kind of Interested

4

5 Very Interested

Living in a foreign country 1 Not Interested at All

2

3 Kind of Interested

4

5 Very Interested

Becoming an author 1 Not Interested at All

2

3 Kind of Interested

4

5 Very Interested

Farming new land 1 Not Interested at All

2

3 Kind of Interested

4

5 Very Interested

Breathing oxygen 1 Not Interested at All

2

3 Kind of Interested

4

5 Very Interested

Level 5

Clark & Kamhi: Influence of Prior Knowledge and Interest

301

Copyright of Language, Speech & Hearing Services in Schools is the property of American Speech-Language-Hearing Association and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.