Nurse Educator Vol. 39, No. 2, pp. 72-76 Copyright * 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins
Nurse Educator
A Concept Analysis of ‘‘High Stakes Testing’’ Debra Sullivan, PhD, RN, CNE, COI The nursing community is troubled by the growing use of standardized exit examinations as a graduation requirement. After years of preparation, a single test score could prevent a student from graduating or taking a licensing examination. The tremendous importance placed on exit examinations qualifies them as ‘‘high-stakes testing,’’ a concept not well studied in nursing education. This concept analysis provides a greater understanding of the term to help operationalize its use in the nursing discipline.
A
primary nursing school objective has always been to produce competent, professional nurses. The NCLEX-RN examination is written to test the entrylevel competence of RN licensure candidates who have graduated from nursing schools.1 Nursing schools monitor the NCLEX-RN pass rates of 1st-time test takers as an indicator of program effectiveness.2,3 Nursing schools are challenged to lower the test’s 10% failure rate by improving knowledge facilitation.3 The pressure on nursing schools to maintain high NCLEX-RN pass rates has fueled the recent trend of using standardized exit examinations to identify students at risk for NCLEX-RN failure.4,5 Many nursing schools use computerized comprehensive standardized exit examinations (referred to here as exit exams) designed to simulate the NCLEX-RN testing environment, both in format and content.6(chapter 27) Research supports exit exams’ reliability and validity in predicting NCLEX-RN success.7 This has led to some nursing schools requiring students to score the minimum proficiency benchmark prior to graduating from the program.8 The practice of using an exit exam to block graduation or deny eligibility to sit for the NCLEX-RN has led the National League for Nursing to create formal documents to guide nursing programs in making more balanced decisions regarding nursing student competence.2,9,10
Student Perspective Students who complete required course work but do not score above the benchmark on a standardized exit exam can find themselves in a holding pattern. Officially, their education is complete, yet they are not allowed to graduate. Author Affiliation: Assistant Professor, School of Nursing, Middle Tennessee State University, Murfreesboro. The author declares no conflicts of interest. Correspondence: Dr Sullivan, School of Nursing, Middle Tennessee State University, Cason-Kennedy, Nursing Building, Box 81, Murfreesboro, TN 37132 ([email protected]). Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www .nurseeducatoronline.com). DOI: 10.1097/NNE.0000000000000021
72
Volume 39 & Number 2 & March/April 2014
This situation could cause a great deal of emotional distress to an individual who devoted 4 or more years and invested personal and/or family funds for tuition. Because nursing schools and faculty’s use of exit exams has serious ramifications for students, determining their ability to graduate or progress in their careers, the term ‘‘high-stakes testing’’ can be applied in these situations.2
Organizing Framework The organizing framework used in this study was adapted from that developed by Wilson11 and modified by Walker and Avant12 to include 8 iterative procedural steps. This study’s analysis included 7 stages that are described in Table 1.
Concept The concept of high-stakes testing in nursing education is not well studied. A search of scholarly databases was performed using the following terms: ‘‘high-stakes examination’’ + nursing; ‘‘high-stakes examinations’’+ nursing’’; ‘‘high-stakes testing’’ + nursing; ‘‘high-stakes examination’’; ‘‘high-stakes examinations’’; and ‘‘high-stakes testing.’’ The CINAHL, PubMed, Ebsco Academic Search Primer, Google Scholar, and ERIC databases were searched. Tabulations of ‘‘hits’’ are presented in Tables, Supplemental Digital Content 1, http://links.lww .com/NE/A119 and Supplemental Digital Content 2, http:// links.lww.com/NE/A120. While thousands of articles addressed high-stakes testing in education and public schools, only 1 specifically examined high-stakes testing in nursing education.13
Purpose The purpose of this concept analysis was to distinguish the use of the concept of high-stakes testing in nursing education from its use in other fields, such as public education. This process included examining the concept, how it is being used, and its similarities and differences to other related words or concepts.12 Such an examination can clarify the ambiguities and consequences surrounding high-stakes testing for both nursing students and faculty and help initiate a dialogue about the concept.14 Nurse Educator
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Table 1. Seven Stages of Analysis Used to Examine High-Stakes Testing in Nursing Education Stage 1 2 3 4 5 6 7
Action Select a concept for study. Determine the purpose of the analysis. Identify the concept’s uses. Determine the concept’s defining attributes. Identify concept cases, including model, contrary, related, and borderline cases. Identify concept antecedents and consequences. Define empirical referents.
Adapted from the organizing framework developed by Wilson11 and modified by Walker and Avant.12(p65)
Uses of the Concept High stakes is a gambling phrase describing a game with uncertainty and a potential for significant losses for the players. Applying this concept to testing in the nursing discipline means that the test takers are the players or gamblers and that, instead of reaching their goals through studying, they must pass an examination to ‘‘win.’’15
Origins The term stakes was searched in dictionaries to differentiate how it is used as colloquial language. Stakes was defined as something wagered in a game, race, or contest; invested interest with personal, monetary, or commercial gains; a piece of wood or metal pointed at one end for driving into the ground as a marker, fence pole, or tent peg; a vertical post to which an offender is bound for execution by burning; to keep under surveillance; and used as a verb, to risk something, that is, staked his reputation, or mark something, that is, stake your claim.16 The etymology of stake dates to before the 12th century and is Middle English, originally from the Old English term, staca. This term is akin to the Middle Low German term, stake pole, and perhaps the Latin term, tignum beam.17 In Public Education In public education, the concept of high-stakes testing has taken on new meanings since the 1990s. Previously, examinations generally were used to gauge a student’s knowledge level. However, examinations are now used not only to hold students accountable for their performance, but also to hold teachers, principals, administrators, and entire school systems accountable for student performance.18 In public education, this type of testing has been associated with several problems. Some policymakers award or deny school funding based on students achieving testing benchmarks. Some schools base their curricula on examination results. Teachers have been accused of teaching to the test, thereby forsaking a richer curriculum. Another cited problem is discrimination against poor and minority students who do not have the same learning opportunities as students from affluent districts.19 A distinction can be made between ‘‘high stress examination’’ and ‘‘high-stakes tests.’’ In public education, a course’s comprehensive final examination can be stressful, whereas a test required by the state for promotion to the next grade is a high-stakes test.20 The comprehensive examination is Nurse Educator
stressful because it covers a lot of material and makes up a substantial part of the course grade, but a student could still pass the course after failing the examination. If a student fails a course, he/she may still be promoted to the next school grade. However, when a student fails a state standardized test required for promotion, he/she would be required to repeat the entire year of course work.
In Nursing In parallel to the public school scenario, standardized exit exams and the NCLEX-RN can be considered ‘‘high-stakes examinations’’ because they hold students accountable for their performance, and they hold nursing schools accountable for meeting accreditation standards. In nursing education, an admission assessment examination is an example of a high-stress examination that may cause anxiety but would not necessarily exclude a student from nursing school admission. Nursing school admission usually is based on several criteria including grade point average, interviews, and so on. In nursing, a high-stakes test would be one that prevented students from graduating if they did not achieve a certain score. The test taker’s perspective is another important issue in high-stress versus high-stakes testing in public and nursing education. Whereas 1 person may consider failure life changing, another may not be as deeply affected. Spurlock13 directly addressed high-stakes testing in nursing education when he said, ‘‘When used as a basis for progression or graduation decisions, the HESI Exit Examination becomes a high-stakes test for students.’’13(p298) He added that when the first-time NCLEX-RN pass rate is tied to a school’s accreditation to operate, the school is justified in its concern about the implications of low pass rates.13 In Other Professions High-stakes testing is used in other occupations in which licensing boards use professional certifications to measure an individual’s competency. For such tests, there is a clear distinction between the consequences for passing (students receive a certification or license) and failing (students do not receive a certification or license). Examples include medical, drivers, and realtor licenses. Most standardized examinations are produced by private companies that use psychometric standards to ensure they are fair, reliable, and valid. Medical or nursing license examination results also can have consequences for the schools that prepared the examinees. If a failure trend is found, a governing board could revoke a school’s accreditation.21
Defining Attributes For students attending nursing schools, 2 types of examinations qualify as high-stakes examinations, standardized exit exams and NCLEX-RN examinations. High-stakes tests in nursing education have 4 defining attributes (Table 2). The first 2 defining attributes of high-stakes testing in nursing education are similar to those used in public education. The last 2 attributes distinguish high-stakes testing in nursing versus public education.
Cases To enhance understanding of a concept, Walker and Avant12 (2005) recommend using a model case of a real-life scenario Volume 39 & Number 2 & March/April 2014
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
73
Table 2. The Four Defining Attributes of High Stakes Tests in Nursing Education Number 1 2 3 4
Attribute They hold students accountable because they carry serious consequences for failure. They hold schools accountable because schools with a low 1st-time NCLEX-RN pass rate can lose accreditation. There is a clear distinction between the outcomes for passing or failing. Those who take these tests have vested (personal, monetary, or emotional) interests in the examinations.
that uses each of the concept’s defining attributes. Additional cases—including contrary, related, and borderline cases—also can help clarify the concept. Chinn and Kramer22 report that such cases should ‘‘stimulate thinking about nuances of meaning.’’22(p200)
Model Case: NCLEX-RN, 1st Attempt A 33-year-old nursing student was excited about completing the nursing program at a state university. She had sacrificed time with her family to complete her studies, often missing activities and special events that were important to her. During the previous 4 years, her husband worked 2 jobs to support the family financially. She felt prepared to take the NCLEX-RN. However, she knew that if she failed she would not be able to work as an RN, and her husband and family would continue to sacrifice until she could pass. The nursing faculty had worked hard to prepare students for the examination. The faculty emphasized the importance of passing the 1st time so they did not lower the school’s 1st-time pass rate, thus possibly jeopardizing accreditation. Passing the 1st time was important to her because of her loyalty to the school. When her appointed test time arrived, she sat down, took a deep breath, cleared her mind, and started taking the examination. The defining attributes of this case were that 1. the student was accountable for passing the examination because of the serious consequences of failure; 2. she acknowledged the importance of passing the 1st time to keep the nursing school’s 1st-time pass rate intact; 3. she would not get her nursing license if she failed; and 4. she had made substantial personal and financial sacrifices to prepare for the examination.
Contrary Case: Nursing School Admission Exam An 18-year-old state university student could not decide if she wanted to be a nurse, physical therapist, or chemist. Her parents did not care which path she chose, as long as she completed a degree. She decided to apply to the nursing school, which required her to sit for an admission examination. Because she had a 3.0 grade point average and high scholastic achievement test scores, she was not worried about the examination. When the test day arrived, she dressed quickly and ran to the testing center. As she sat down to take the examination, she thought about her weekend party plans. 74
Volume 39 & Number 2 & March/April 2014
The defining attributes of this case were that 1. this student was not accountable because passing did not matter to her; 2. the nursing school was not accountable for her performance; 3. this was not a pass/fail examination; instead, the score would be combined with other admission criteria; and 4. she did not have a vested interest in passing the examination and would be happy studying to be a physical therapist or a chemist.
Related Case: Nursing School Exit Exam, 2nd Attempt A 20-year-old state university student had completed all the nursing school course requirements for graduation. She had wanted to be a nurse since her mother had been ill when she was very young. While she had worked hard to complete the nursing program, her 1st attempt taking the exit exam was unsuccessful. She was scheduled to take the examination a 2nd time. To graduate, she had to pass the examination in 3 attempts. After her 1st attempt, the nursing faculty had spent several hours reviewing the material with her. She was upset that, after spending a great deal of her and her family’s money, she was at risk of not graduating, which would delay her ability to sit for the NCLEX-RN. In her despair, she asked her advisor why the school had this policy. She was told that the school wanted to identify and help students at risk of failing the NCLEX-RN because it could lose accreditation if 1st-time NCLEX-RN pass rates fell below a designated percentage. She had taken off work that weekend and spent 3 days preparing for the examination. On the day of the examination, she had risen early to review her notes and reported a half hour early. When she started the examination, she realized that she was shaking. This case is related because this student was taking the examination a 2nd time and had only 1 more opportunity to pass and graduate. The defining attributes of this case were that 1. the student was accountable, perceived the examination as high pressure, and placed a high value on passing the examination; 2. the school was worried about their accountability because the nursing faculty had spent time to remediate the student, and her exit exam success could indicate her NCLEX-RN success; 3. this was not a pass/fail examination as she would have another opportunity to pass; and 4. the student put a great deal of value on passing this examination, as she would not need to take off work again to study for a 3rd attempt, and she could look forward to graduation.
Borderline Case: Standardized Nursing School Exit Exam A 22-year-old student in her last semester of nursing school was excited about graduating and starting her new life as a nurse. While she had worked in the summer to earn spending money, her parents had paid for her tuition and Nurse Educator
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
school expenses. The nursing school required that she sit for a standardized exit exam. While she was not required to pass or achieve a minimum score, the faculty had explained that success on this examination predicted her ability to pass the NCLEX-RN. Because of her other nursing course assignments, she was able to study for only 8 hours. As she sat down for the examination, she hoped that she would pass, but thought it would be okay if she failed because she would have plenty of time to study after graduation and before taking the NCLEX-RN. This is a borderline case because, unlike the model case, the student was not required to pass the exit exam. The defining attributes of this case were that 1. she was not accountable because she was not required to pass the examination (while passing the examination could help her to feel more confident about passing the NCLEX-RN, she would not suffer any negative consequences if she failed); 2. the nursing school would ultimately be accountable for the student’s 1st-time NCLEX-RN, pass rate; 3. this was not a pass/fail examination; and 4. she was minimally invested (personally or monetarily) in the examination because passing it was not a graduation requirement.
Antecedents and Consequences Antecedents and consequences related to the concept of high-stakes testing can shed additional light on the social contexts in which the concept is used.12
Antecedents Events, behaviors, or incidents that must occur prior to highstakes testing are referred to as antecedents.12 Some highstakes testing antecedents can have a negative impact on test results, including weak nursing curriculum, poor student performance in nursing courses and clinical settings, and high faculty turnover. When test results contribute to decisions about graduation or promotion, there must be evidence that the test addresses the specific or generalized content and skills that students have had the opportunity to master.19 Other possible negative antecedents include students’ minority, low-income, English as a 2nd language, and learning disability status. For these cases, the American Psychological Association (APA)19 emphasized that the authority mandating a test is responsible for identifying the potential negative consequences of such testing and monitoring the examination’s impact on racial and ethnic-minority students, as well as students who are disabled, are from a lower socioeconomic status, or have limited English proficiency. Consequences Events or incidents that result from high-stakes testing are considered consequences of the concept.12 Poor performance on high-stakes examinations could result in unintended consequences for students and nursing schools. For students, these can include test anxiety, delayed graduation, delayed workforce entry, emotional distress, family difficulties, and financial stress.2,13 The American Educational Research Association, the APA, and the National Council on Measurement Nurse Educator
in Education23 indicated that test results might not reflect an accurate measurement for qualified students with language or cultural difficulties, causing them to fail. Nursing schools with substandard 1st-time NCLEX-RN pass rates face several problems: & increasingly difficult recruitment efforts when prospective students consider pass rates as a criterion for nursing school selection; & lower application rates when high-potential, academically successful students decline to apply; & loss of operating revenue when at-risk students are unsuccessful, resulting in increased attrition; & scrutiny of the program from local boards of nursing and professional accrediting organizations such as National League of Nursing Accrediting Commission; & decreased satisfaction of graduates, parents, and community members; and & an increased risk of regulatory intervention for program accreditation.3 Heubert and Hauser20 stated that, if used inappropriately, high-stakes tests could undermine the quality of education and the equality of opportunity.
Empiric Referents The final step in concept analysis is to define the empiric referents. These are actual phenomena that give evidence for the concept’s existence. The obvious evidence that the NCLEX-RN is a ‘‘high stakes’’ test is what the test determines: who will receive and who will be denied a license to practice nursing in a particular state. When schools require a successful exit exam score as a progression or graduation requirement,7 such examinations become ‘‘high-stakes tests’’ as defined by Spurlock.13 The serious consequences for students who do not achieve the designated minimum score include denial of graduation eligibility, an incomplete or failing score in a capstone course, or permission withheld to take the NCLEX-RN.4
Conclusion This concept analysis provides further understanding of the concept of high-stakes testing by defining its uses, attributes, cases, antecedents, consequences, and empiric referents. The concept of high-stakes testing in nursing education is aligned with its use in public education by the 2 attributes that result in serious consequences: student accountability and nursing school accountability. The concept is differentiated from its use public education because, unlike highstakes testing in public education, there is a clear distinction between the outcomes for those who pass and those who fail. In other words, after having invested personally, monetarily, and/or emotionally in the test’s outcome, a student may be unable to graduate or get a license to practice. The cases provided illustrate the presence or absence of the concept’s 4 basic attributes. It is hoped that nursing educators and administrators can gain new awareness from this concept analysis to help guide nursing education policy changes that benefit the discipline of nursing and, ultimately, patient care. Future research is needed to uncover more insights about high-stakes testing for students, faculty, nursing Volume 39 & Number 2 & March/April 2014
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
75
schools, the NCSBN, and state nursing certification boards. Responsible administration and careful monitoring are important to avoid unintended consequences from high-stakes testing for students, faculty, and nursing schools.
References 1. National Council of State Boards of Nursing. 2013 Number of candidates taking NCLEX examination and percent passing, by type of candidate. National Council of State Boards of Nursing Web site. Available at https://www.ncsbn.org/Table_of_Pass_Rates_ 2013.pdf. Published August 26, 2013. Accessed September 4, 2013. 2. National League for Nursing. The fair testing imperative in nursing education. National League for Nursing Web site. Available at http://www.nln.org/aboutnln/livingdocuments/pdf/ nlnvision_4.pdf. Published February 2012. Accessed September 4, 2013. 3. Norton C, Relf M, Cox C, Farley J, Lachat M, Tucker M, Murray J. Ensuring NCLEX-RN success for first-time test-takers. J Prof Nurs. 2006;22(5):322-326. 4. Nibert AT, Young A, Britt R. The HESI Exit Exam: progression benchmark and remediation guide. Nurse Educ. 2003;28(3): 141-145. 5. National League for Nursing. High stakes testing addressed by National League for Nursing. National League for Nursing Web site. Available at http://www.nln.org/newsreleases/hst_121510 .htm. Published December 15, 2010. Accessed September 4, 2013. 6. Lowenstein A, Bradshaw M. Fuszard’s Innovative Teaching Strategies in Nursing. 3rd ed. Sudbury, MA: Jones and Bartlett; 2001. 7. Morrison S, Free KW, Newman M. Do progression and remediation policies improve NCLEX-RN pass rates? Comput Inform Nurs. 2005;23(3S):46S-48S. 8. Spurlock DR Jr, Hunt LA. A study of the usefulness of the HESI Exit Exam in predicting NCLEX-RN failure. J Nurs Educ. 2008; 47(4):157-166. 9. National League for Nursing. High-stakes testing addressed by National League for Nursing with publication of ‘‘Fair Testing Imperative in Nursing Education’’ and ‘‘NLN Fair Testing Guidelines for Nursing Education.’’ National League for Nursing Web site. Available at http://www.nln.org/facultyprograms/ facultyresources/fairtestingguidelines.pdf. Accessed September 4, 2013. 10. National League for Nursing. NLN fair testing guidelines for nursing education. National League for Nursing Web site. Available
76
Volume 39 & Number 2 & March/April 2014
at http://www.nln.org/facultyprograms/facultyresources/ fairtestingguidelines.pdf. Accessed September 4, 2013. 11. Wilson J. Thinking With Concepts. New York, NY: Cambridge University Press; 1963. 12. Walker LO, Avant KC. Strategies for Theory Construction in Nursing. 4th ed. Upper Saddle River, NJ: Pearson Prentice Hall; 2005. 13. Spurlock D Jr. Do no harm: ‘‘high stakes testing’’ in nursing education. J Nurs Educ. 2006;45(8):297-302. 14. Weld KK, Garman Bibb SC. Concept analysis: malpractice and modern-day nursing practice. Nursing Forum. 2009;44(1):2-10. 15. Sillman E, Wilkinson L, eds. Language and Literacy Learning in Schools. New York, NY: The Guilford Press; 2004. 16. Dictionary.com. Stakes. Dictionary.com Web site. Available at http://dictionary.reference.com/browse/stakes. Accessed September 4, 2013. 17. Merriam Webster. Stake. Merriam Webster Web site. Available at http://www.merriam-webster.com/dictionary/stake. Accessed September 4, 2013. 18. Loschert K, Gleason B, Carter G. Raising the ante for students, teachers, and schools. Info Brief. 2000;23. ASCD Web site. Available at http://www.ascd.org/publications/newsletters/infobrief/nov00/ num23/toc.aspx. Accessed September 4, 2013. 19. American Psychological Association. Appropriate Use of HighStakes Testing in Our Nation’s Schools. American Psychological Association Web site. Available at http://www.apa.org/pubs/info/ brochures/testing.aspx. Accessed September 4, 2013. 20. Heubert JP, Hauser RM, eds. High Stakes: Testing for Tracking, Promotion, and Graduation. Washington, DC: National Academy Press; 1999. National Academy Press Web site. Available at http:// books.nap.edu/openbook.php?record_id=6336&page=13http:// bob.nap.edu/html/highstakes/Hubert. Accessed September 4, 2013. 21. Mitchell R. A guide to standardized testing: the nature of assessment. Center for Public Education Web site. Available at http://www.centerforpubliceducation.org/Main-Menu/Evaluatingperformance/A-guide-to-standardiz
Nurse Educator
A Concept Analysis of ‘‘High Stakes Testing’’ Debra Sullivan, PhD, RN, CNE, COI The nursing community is troubled by the growing use of standardized exit examinations as a graduation requirement. After years of preparation, a single test score could prevent a student from graduating or taking a licensing examination. The tremendous importance placed on exit examinations qualifies them as ‘‘high-stakes testing,’’ a concept not well studied in nursing education. This concept analysis provides a greater understanding of the term to help operationalize its use in the nursing discipline.
A
primary nursing school objective has always been to produce competent, professional nurses. The NCLEX-RN examination is written to test the entrylevel competence of RN licensure candidates who have graduated from nursing schools.1 Nursing schools monitor the NCLEX-RN pass rates of 1st-time test takers as an indicator of program effectiveness.2,3 Nursing schools are challenged to lower the test’s 10% failure rate by improving knowledge facilitation.3 The pressure on nursing schools to maintain high NCLEX-RN pass rates has fueled the recent trend of using standardized exit examinations to identify students at risk for NCLEX-RN failure.4,5 Many nursing schools use computerized comprehensive standardized exit examinations (referred to here as exit exams) designed to simulate the NCLEX-RN testing environment, both in format and content.6(chapter 27) Research supports exit exams’ reliability and validity in predicting NCLEX-RN success.7 This has led to some nursing schools requiring students to score the minimum proficiency benchmark prior to graduating from the program.8 The practice of using an exit exam to block graduation or deny eligibility to sit for the NCLEX-RN has led the National League for Nursing to create formal documents to guide nursing programs in making more balanced decisions regarding nursing student competence.2,9,10
Student Perspective Students who complete required course work but do not score above the benchmark on a standardized exit exam can find themselves in a holding pattern. Officially, their education is complete, yet they are not allowed to graduate. Author Affiliation: Assistant Professor, School of Nursing, Middle Tennessee State University, Murfreesboro. The author declares no conflicts of interest. Correspondence: Dr Sullivan, School of Nursing, Middle Tennessee State University, Cason-Kennedy, Nursing Building, Box 81, Murfreesboro, TN 37132 ([email protected]). Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www .nurseeducatoronline.com). DOI: 10.1097/NNE.0000000000000021
72
Volume 39 & Number 2 & March/April 2014
This situation could cause a great deal of emotional distress to an individual who devoted 4 or more years and invested personal and/or family funds for tuition. Because nursing schools and faculty’s use of exit exams has serious ramifications for students, determining their ability to graduate or progress in their careers, the term ‘‘high-stakes testing’’ can be applied in these situations.2
Organizing Framework The organizing framework used in this study was adapted from that developed by Wilson11 and modified by Walker and Avant12 to include 8 iterative procedural steps. This study’s analysis included 7 stages that are described in Table 1.
Concept The concept of high-stakes testing in nursing education is not well studied. A search of scholarly databases was performed using the following terms: ‘‘high-stakes examination’’ + nursing; ‘‘high-stakes examinations’’+ nursing’’; ‘‘high-stakes testing’’ + nursing; ‘‘high-stakes examination’’; ‘‘high-stakes examinations’’; and ‘‘high-stakes testing.’’ The CINAHL, PubMed, Ebsco Academic Search Primer, Google Scholar, and ERIC databases were searched. Tabulations of ‘‘hits’’ are presented in Tables, Supplemental Digital Content 1, http://links.lww .com/NE/A119 and Supplemental Digital Content 2, http:// links.lww.com/NE/A120. While thousands of articles addressed high-stakes testing in education and public schools, only 1 specifically examined high-stakes testing in nursing education.13
Purpose The purpose of this concept analysis was to distinguish the use of the concept of high-stakes testing in nursing education from its use in other fields, such as public education. This process included examining the concept, how it is being used, and its similarities and differences to other related words or concepts.12 Such an examination can clarify the ambiguities and consequences surrounding high-stakes testing for both nursing students and faculty and help initiate a dialogue about the concept.14 Nurse Educator
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Table 1. Seven Stages of Analysis Used to Examine High-Stakes Testing in Nursing Education Stage 1 2 3 4 5 6 7
Action Select a concept for study. Determine the purpose of the analysis. Identify the concept’s uses. Determine the concept’s defining attributes. Identify concept cases, including model, contrary, related, and borderline cases. Identify concept antecedents and consequences. Define empirical referents.
Adapted from the organizing framework developed by Wilson11 and modified by Walker and Avant.12(p65)
Uses of the Concept High stakes is a gambling phrase describing a game with uncertainty and a potential for significant losses for the players. Applying this concept to testing in the nursing discipline means that the test takers are the players or gamblers and that, instead of reaching their goals through studying, they must pass an examination to ‘‘win.’’15
Origins The term stakes was searched in dictionaries to differentiate how it is used as colloquial language. Stakes was defined as something wagered in a game, race, or contest; invested interest with personal, monetary, or commercial gains; a piece of wood or metal pointed at one end for driving into the ground as a marker, fence pole, or tent peg; a vertical post to which an offender is bound for execution by burning; to keep under surveillance; and used as a verb, to risk something, that is, staked his reputation, or mark something, that is, stake your claim.16 The etymology of stake dates to before the 12th century and is Middle English, originally from the Old English term, staca. This term is akin to the Middle Low German term, stake pole, and perhaps the Latin term, tignum beam.17 In Public Education In public education, the concept of high-stakes testing has taken on new meanings since the 1990s. Previously, examinations generally were used to gauge a student’s knowledge level. However, examinations are now used not only to hold students accountable for their performance, but also to hold teachers, principals, administrators, and entire school systems accountable for student performance.18 In public education, this type of testing has been associated with several problems. Some policymakers award or deny school funding based on students achieving testing benchmarks. Some schools base their curricula on examination results. Teachers have been accused of teaching to the test, thereby forsaking a richer curriculum. Another cited problem is discrimination against poor and minority students who do not have the same learning opportunities as students from affluent districts.19 A distinction can be made between ‘‘high stress examination’’ and ‘‘high-stakes tests.’’ In public education, a course’s comprehensive final examination can be stressful, whereas a test required by the state for promotion to the next grade is a high-stakes test.20 The comprehensive examination is Nurse Educator
stressful because it covers a lot of material and makes up a substantial part of the course grade, but a student could still pass the course after failing the examination. If a student fails a course, he/she may still be promoted to the next school grade. However, when a student fails a state standardized test required for promotion, he/she would be required to repeat the entire year of course work.
In Nursing In parallel to the public school scenario, standardized exit exams and the NCLEX-RN can be considered ‘‘high-stakes examinations’’ because they hold students accountable for their performance, and they hold nursing schools accountable for meeting accreditation standards. In nursing education, an admission assessment examination is an example of a high-stress examination that may cause anxiety but would not necessarily exclude a student from nursing school admission. Nursing school admission usually is based on several criteria including grade point average, interviews, and so on. In nursing, a high-stakes test would be one that prevented students from graduating if they did not achieve a certain score. The test taker’s perspective is another important issue in high-stress versus high-stakes testing in public and nursing education. Whereas 1 person may consider failure life changing, another may not be as deeply affected. Spurlock13 directly addressed high-stakes testing in nursing education when he said, ‘‘When used as a basis for progression or graduation decisions, the HESI Exit Examination becomes a high-stakes test for students.’’13(p298) He added that when the first-time NCLEX-RN pass rate is tied to a school’s accreditation to operate, the school is justified in its concern about the implications of low pass rates.13 In Other Professions High-stakes testing is used in other occupations in which licensing boards use professional certifications to measure an individual’s competency. For such tests, there is a clear distinction between the consequences for passing (students receive a certification or license) and failing (students do not receive a certification or license). Examples include medical, drivers, and realtor licenses. Most standardized examinations are produced by private companies that use psychometric standards to ensure they are fair, reliable, and valid. Medical or nursing license examination results also can have consequences for the schools that prepared the examinees. If a failure trend is found, a governing board could revoke a school’s accreditation.21
Defining Attributes For students attending nursing schools, 2 types of examinations qualify as high-stakes examinations, standardized exit exams and NCLEX-RN examinations. High-stakes tests in nursing education have 4 defining attributes (Table 2). The first 2 defining attributes of high-stakes testing in nursing education are similar to those used in public education. The last 2 attributes distinguish high-stakes testing in nursing versus public education.
Cases To enhance understanding of a concept, Walker and Avant12 (2005) recommend using a model case of a real-life scenario Volume 39 & Number 2 & March/April 2014
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
73
Table 2. The Four Defining Attributes of High Stakes Tests in Nursing Education Number 1 2 3 4
Attribute They hold students accountable because they carry serious consequences for failure. They hold schools accountable because schools with a low 1st-time NCLEX-RN pass rate can lose accreditation. There is a clear distinction between the outcomes for passing or failing. Those who take these tests have vested (personal, monetary, or emotional) interests in the examinations.
that uses each of the concept’s defining attributes. Additional cases—including contrary, related, and borderline cases—also can help clarify the concept. Chinn and Kramer22 report that such cases should ‘‘stimulate thinking about nuances of meaning.’’22(p200)
Model Case: NCLEX-RN, 1st Attempt A 33-year-old nursing student was excited about completing the nursing program at a state university. She had sacrificed time with her family to complete her studies, often missing activities and special events that were important to her. During the previous 4 years, her husband worked 2 jobs to support the family financially. She felt prepared to take the NCLEX-RN. However, she knew that if she failed she would not be able to work as an RN, and her husband and family would continue to sacrifice until she could pass. The nursing faculty had worked hard to prepare students for the examination. The faculty emphasized the importance of passing the 1st time so they did not lower the school’s 1st-time pass rate, thus possibly jeopardizing accreditation. Passing the 1st time was important to her because of her loyalty to the school. When her appointed test time arrived, she sat down, took a deep breath, cleared her mind, and started taking the examination. The defining attributes of this case were that 1. the student was accountable for passing the examination because of the serious consequences of failure; 2. she acknowledged the importance of passing the 1st time to keep the nursing school’s 1st-time pass rate intact; 3. she would not get her nursing license if she failed; and 4. she had made substantial personal and financial sacrifices to prepare for the examination.
Contrary Case: Nursing School Admission Exam An 18-year-old state university student could not decide if she wanted to be a nurse, physical therapist, or chemist. Her parents did not care which path she chose, as long as she completed a degree. She decided to apply to the nursing school, which required her to sit for an admission examination. Because she had a 3.0 grade point average and high scholastic achievement test scores, she was not worried about the examination. When the test day arrived, she dressed quickly and ran to the testing center. As she sat down to take the examination, she thought about her weekend party plans. 74
Volume 39 & Number 2 & March/April 2014
The defining attributes of this case were that 1. this student was not accountable because passing did not matter to her; 2. the nursing school was not accountable for her performance; 3. this was not a pass/fail examination; instead, the score would be combined with other admission criteria; and 4. she did not have a vested interest in passing the examination and would be happy studying to be a physical therapist or a chemist.
Related Case: Nursing School Exit Exam, 2nd Attempt A 20-year-old state university student had completed all the nursing school course requirements for graduation. She had wanted to be a nurse since her mother had been ill when she was very young. While she had worked hard to complete the nursing program, her 1st attempt taking the exit exam was unsuccessful. She was scheduled to take the examination a 2nd time. To graduate, she had to pass the examination in 3 attempts. After her 1st attempt, the nursing faculty had spent several hours reviewing the material with her. She was upset that, after spending a great deal of her and her family’s money, she was at risk of not graduating, which would delay her ability to sit for the NCLEX-RN. In her despair, she asked her advisor why the school had this policy. She was told that the school wanted to identify and help students at risk of failing the NCLEX-RN because it could lose accreditation if 1st-time NCLEX-RN pass rates fell below a designated percentage. She had taken off work that weekend and spent 3 days preparing for the examination. On the day of the examination, she had risen early to review her notes and reported a half hour early. When she started the examination, she realized that she was shaking. This case is related because this student was taking the examination a 2nd time and had only 1 more opportunity to pass and graduate. The defining attributes of this case were that 1. the student was accountable, perceived the examination as high pressure, and placed a high value on passing the examination; 2. the school was worried about their accountability because the nursing faculty had spent time to remediate the student, and her exit exam success could indicate her NCLEX-RN success; 3. this was not a pass/fail examination as she would have another opportunity to pass; and 4. the student put a great deal of value on passing this examination, as she would not need to take off work again to study for a 3rd attempt, and she could look forward to graduation.
Borderline Case: Standardized Nursing School Exit Exam A 22-year-old student in her last semester of nursing school was excited about graduating and starting her new life as a nurse. While she had worked in the summer to earn spending money, her parents had paid for her tuition and Nurse Educator
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
school expenses. The nursing school required that she sit for a standardized exit exam. While she was not required to pass or achieve a minimum score, the faculty had explained that success on this examination predicted her ability to pass the NCLEX-RN. Because of her other nursing course assignments, she was able to study for only 8 hours. As she sat down for the examination, she hoped that she would pass, but thought it would be okay if she failed because she would have plenty of time to study after graduation and before taking the NCLEX-RN. This is a borderline case because, unlike the model case, the student was not required to pass the exit exam. The defining attributes of this case were that 1. she was not accountable because she was not required to pass the examination (while passing the examination could help her to feel more confident about passing the NCLEX-RN, she would not suffer any negative consequences if she failed); 2. the nursing school would ultimately be accountable for the student’s 1st-time NCLEX-RN, pass rate; 3. this was not a pass/fail examination; and 4. she was minimally invested (personally or monetarily) in the examination because passing it was not a graduation requirement.
Antecedents and Consequences Antecedents and consequences related to the concept of high-stakes testing can shed additional light on the social contexts in which the concept is used.12
Antecedents Events, behaviors, or incidents that must occur prior to highstakes testing are referred to as antecedents.12 Some highstakes testing antecedents can have a negative impact on test results, including weak nursing curriculum, poor student performance in nursing courses and clinical settings, and high faculty turnover. When test results contribute to decisions about graduation or promotion, there must be evidence that the test addresses the specific or generalized content and skills that students have had the opportunity to master.19 Other possible negative antecedents include students’ minority, low-income, English as a 2nd language, and learning disability status. For these cases, the American Psychological Association (APA)19 emphasized that the authority mandating a test is responsible for identifying the potential negative consequences of such testing and monitoring the examination’s impact on racial and ethnic-minority students, as well as students who are disabled, are from a lower socioeconomic status, or have limited English proficiency. Consequences Events or incidents that result from high-stakes testing are considered consequences of the concept.12 Poor performance on high-stakes examinations could result in unintended consequences for students and nursing schools. For students, these can include test anxiety, delayed graduation, delayed workforce entry, emotional distress, family difficulties, and financial stress.2,13 The American Educational Research Association, the APA, and the National Council on Measurement Nurse Educator
in Education23 indicated that test results might not reflect an accurate measurement for qualified students with language or cultural difficulties, causing them to fail. Nursing schools with substandard 1st-time NCLEX-RN pass rates face several problems: & increasingly difficult recruitment efforts when prospective students consider pass rates as a criterion for nursing school selection; & lower application rates when high-potential, academically successful students decline to apply; & loss of operating revenue when at-risk students are unsuccessful, resulting in increased attrition; & scrutiny of the program from local boards of nursing and professional accrediting organizations such as National League of Nursing Accrediting Commission; & decreased satisfaction of graduates, parents, and community members; and & an increased risk of regulatory intervention for program accreditation.3 Heubert and Hauser20 stated that, if used inappropriately, high-stakes tests could undermine the quality of education and the equality of opportunity.
Empiric Referents The final step in concept analysis is to define the empiric referents. These are actual phenomena that give evidence for the concept’s existence. The obvious evidence that the NCLEX-RN is a ‘‘high stakes’’ test is what the test determines: who will receive and who will be denied a license to practice nursing in a particular state. When schools require a successful exit exam score as a progression or graduation requirement,7 such examinations become ‘‘high-stakes tests’’ as defined by Spurlock.13 The serious consequences for students who do not achieve the designated minimum score include denial of graduation eligibility, an incomplete or failing score in a capstone course, or permission withheld to take the NCLEX-RN.4
Conclusion This concept analysis provides further understanding of the concept of high-stakes testing by defining its uses, attributes, cases, antecedents, consequences, and empiric referents. The concept of high-stakes testing in nursing education is aligned with its use in public education by the 2 attributes that result in serious consequences: student accountability and nursing school accountability. The concept is differentiated from its use public education because, unlike highstakes testing in public education, there is a clear distinction between the outcomes for those who pass and those who fail. In other words, after having invested personally, monetarily, and/or emotionally in the test’s outcome, a student may be unable to graduate or get a license to practice. The cases provided illustrate the presence or absence of the concept’s 4 basic attributes. It is hoped that nursing educators and administrators can gain new awareness from this concept analysis to help guide nursing education policy changes that benefit the discipline of nursing and, ultimately, patient care. Future research is needed to uncover more insights about high-stakes testing for students, faculty, nursing Volume 39 & Number 2 & March/April 2014
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
75
schools, the NCSBN, and state nursing certification boards. Responsible administration and careful monitoring are important to avoid unintended consequences from high-stakes testing for students, faculty, and nursing schools.
References 1. National Council of State Boards of Nursing. 2013 Number of candidates taking NCLEX examination and percent passing, by type of candidate. National Council of State Boards of Nursing Web site. Available at https://www.ncsbn.org/Table_of_Pass_Rates_ 2013.pdf. Published August 26, 2013. Accessed September 4, 2013. 2. National League for Nursing. The fair testing imperative in nursing education. National League for Nursing Web site. Available at http://www.nln.org/aboutnln/livingdocuments/pdf/ nlnvision_4.pdf. Published February 2012. Accessed September 4, 2013. 3. Norton C, Relf M, Cox C, Farley J, Lachat M, Tucker M, Murray J. Ensuring NCLEX-RN success for first-time test-takers. J Prof Nurs. 2006;22(5):322-326. 4. Nibert AT, Young A, Britt R. The HESI Exit Exam: progression benchmark and remediation guide. Nurse Educ. 2003;28(3): 141-145. 5. National League for Nursing. High stakes testing addressed by National League for Nursing. National League for Nursing Web site. Available at http://www.nln.org/newsreleases/hst_121510 .htm. Published December 15, 2010. Accessed September 4, 2013. 6. Lowenstein A, Bradshaw M. Fuszard’s Innovative Teaching Strategies in Nursing. 3rd ed. Sudbury, MA: Jones and Bartlett; 2001. 7. Morrison S, Free KW, Newman M. Do progression and remediation policies improve NCLEX-RN pass rates? Comput Inform Nurs. 2005;23(3S):46S-48S. 8. Spurlock DR Jr, Hunt LA. A study of the usefulness of the HESI Exit Exam in predicting NCLEX-RN failure. J Nurs Educ. 2008; 47(4):157-166. 9. National League for Nursing. High-stakes testing addressed by National League for Nursing with publication of ‘‘Fair Testing Imperative in Nursing Education’’ and ‘‘NLN Fair Testing Guidelines for Nursing Education.’’ National League for Nursing Web site. Available at http://www.nln.org/facultyprograms/ facultyresources/fairtestingguidelines.pdf. Accessed September 4, 2013. 10. National League for Nursing. NLN fair testing guidelines for nursing education. National League for Nursing Web site. Available
76
Volume 39 & Number 2 & March/April 2014
at http://www.nln.org/facultyprograms/facultyresources/ fairtestingguidelines.pdf. Accessed September 4, 2013. 11. Wilson J. Thinking With Concepts. New York, NY: Cambridge University Press; 1963. 12. Walker LO, Avant KC. Strategies for Theory Construction in Nursing. 4th ed. Upper Saddle River, NJ: Pearson Prentice Hall; 2005. 13. Spurlock D Jr. Do no harm: ‘‘high stakes testing’’ in nursing education. J Nurs Educ. 2006;45(8):297-302. 14. Weld KK, Garman Bibb SC. Concept analysis: malpractice and modern-day nursing practice. Nursing Forum. 2009;44(1):2-10. 15. Sillman E, Wilkinson L, eds. Language and Literacy Learning in Schools. New York, NY: The Guilford Press; 2004. 16. Dictionary.com. Stakes. Dictionary.com Web site. Available at http://dictionary.reference.com/browse/stakes. Accessed September 4, 2013. 17. Merriam Webster. Stake. Merriam Webster Web site. Available at http://www.merriam-webster.com/dictionary/stake. Accessed September 4, 2013. 18. Loschert K, Gleason B, Carter G. Raising the ante for students, teachers, and schools. Info Brief. 2000;23. ASCD Web site. Available at http://www.ascd.org/publications/newsletters/infobrief/nov00/ num23/toc.aspx. Accessed September 4, 2013. 19. American Psychological Association. Appropriate Use of HighStakes Testing in Our Nation’s Schools. American Psychological Association Web site. Available at http://www.apa.org/pubs/info/ brochures/testing.aspx. Accessed September 4, 2013. 20. Heubert JP, Hauser RM, eds. High Stakes: Testing for Tracking, Promotion, and Graduation. Washington, DC: National Academy Press; 1999. National Academy Press Web site. Available at http:// books.nap.edu/openbook.php?record_id=6336&page=13http:// bob.nap.edu/html/highstakes/Hubert. Accessed September 4, 2013. 21. Mitchell R. A guide to standardized testing: the nature of assessment. Center for Public Education Web site. Available at http://www.centerforpubliceducation.org/Main-Menu/Evaluatingperformance/A-guide-to-standardiz
