Original Article
Faculty Perception of the Effectiveness of EBP Courses for Graduate Nursing Students Renata ´ Zelen´ıkova, ´ RN, PhD • Michael Beach, DNP, ACNP-BC, PNP • Dianxu Ren, MD, PhD • Emily Wolff • Paula Sherwood, RN, PhD, CNRN, FAAN
ABSTRACT Keywords evidence-based practice, faculty, teaching, curricula
Background: Effective teaching is key in preparing students to become successful evidencebased healthcare professionals. The effectiveness of graduate evidence-based practice (EBP) pedagogy is not often a subject of research studies. Purpose: The purpose of this study was to determine how faculty from the 50 top graduate nursing schools in the United States perceived the effectiveness of EBP courses for graduate nursing students. Methods: A descriptive cross-sectional design was used to explore faculty perception of the effectiveness of EBP courses. A web-based survey was used for data collection. A total of 45 questionnaires were subjected to statistical analysis. Results: The mean perception of the effectiveness of EBP courses for the whole sample, on a scale from 1 to 7, was 5.58 (min. 4.29; max. 6.73), a higher score signifying higher perceived effectiveness. The highest rated item concerned a school’s access to different databases. The strongest correlations were found between the total score and the scores for items describing students’ opportunities to strengthen and apply their EBP skills (rs = .66). The internal consistency of the Perception of Effectiveness of EBP Courses scale, based on standardized Cronbach’s alpha, was .84, which signifies strong internal consistency. Faculty perceived themselves as most competent at the following EBP skills: (a) “Asking questions regarding patients’ care” (6.56), (b) “Considering patient preferences when implementing EBP” (6.40), and (c) “Critically appraising the relevant body of evidence to address clinical questions” (6.40). Discussion: To strengthen the effectiveness of EBP courses, students should have more opportunities to implement their EBP knowledge and skills after completing EBP courses. Linking Evidence to Action: Evaluation of faculty perceptions of the effectiveness of EBP courses can help to guide the development of nursing school curricula that better integrate EBP. Further evaluation of the psychometric properties of the instrument used to measure perception of the effectiveness of EBP courses is required along with objective measures of faculty knowledge and skills in teaching EBP.
BACKGROUND AND SIGNIFICANCE Worldwide, evidence-based practice (EBP) has emerged as a major healthcare initiative (Thiel & Ghosh, 2008). One of the most consistent findings in health service research is the gap between best practice (as determined by scientific evidence) and actual clinical care (Flores-Mateo & Argimon, 2007). To accelerate the translation of research findings into clinical practice, two major outcomes must be achieved: (a) Advanced practice and direct care nurses must acquire sufficient EBP knowledge and skills as well as strong beliefs about the value of EBP in clinical settings, and (b) educators must teach their students the EBP process to instill in them lifelong skills and the motivation to deliver the highest quality of care (Melnyk, Fineout-Overholt, Feinstein, Sadler, & Green-Hernandez, 2008).
Worldviews on Evidence-Based Nursing, 2014; 11:6, 401–413. C 2014 Sigma Theta Tau International
Findings from a recent national survey by Melnyk, FineoutOverholt, Gallagher-Ford, and Kaplan (2012) indicated that nurses surveyed across the country are ready for and do value EBP. The majority of participants who responded to the survey reported wanting to gain more knowledge and skills in order to deliver evidence-based care in their institutions. Nurses cited a top requirement for helping them implement EBP in daily practice as education. For students to become evidence-based healthcare professionals, the teaching of EBP has to be effective (Spek, Wolf, Dijk, & Lucas, 2012). Little has been published about teaching EBP to nursing students (Stiffler & Cullen, 2010). Though there are systematic reviews and meta-analyses of teaching EBP in schools of medicine, there is a dearth of research in nursing,
401
Perception of Effectiveness of EBP Courses
especially in regard to graduate-level EBP pedagogy. According to Fineout-Overholt and Johnston (2005), further research is needed to assess effective teaching and evaluation strategies for EBP. Teaching EBP to nursing students is usually based on the basic steps of EBP. Melnyk and Fineout-Overholt (2011) added two more steps to the five basic steps of EBP, which include: (a) cultivate a spirit of inquiry; (b) ask the burning clinical question in PICOT (P = patient population; I = intervention or area of interest; C = comparison intervention or group; O = outcomes; and T = time) format; (c) search for and collect the most relevant best evidence; (d) critically appraise the evidence; (e) integrate the best evidence with one’s clinical expertise and patient preferences and values in making a practice decision or change; (f) evaluate outcomes of the practice decision or change based on evidence; and (g) disseminate the outcomes of the EBP decision or change. The seven steps of EBP can serve as a structure around which to build EBP curriculum for graduate nursing students. The effectiveness of graduate EBP pedagogy is not often a subject of research studies. One of the problems is the complexity of the EBP process and the difficulty of assessing all aspects of its effectiveness. Shaneyefelt and colleagues (2006) performed a systematic review of EBP instruments. Their results showed that the majority of instruments targeted students and postgraduate trainees, whereas nonphysicians were rarely evaluated. The available instruments most commonly evaluated EBP skills (predominantly focusing on the critical appraisal of evidence), knowledge, attitudes, and behaviors. Most instruments are designed for specific purposes, such as the evaluation of theoretical EBP courses (instruments to assess cognitive skills), or the evaluation of EBP in clinical practice (instruments to assess performance-based skills and application; Ilic, 2009). The instrument used in this study for measuring the perception of the effectiveness of EBP courses was inspired by the ARCC-E (advancing research and clinical practice through close collaboration and education) conceptual model for teaching EBP (Melnyk & Fineout-Overholt, 2011). This model shows that organizational support, the identification of curricular strengths and barriers to teaching EBP and use of faculty EBP mentors play an important role in the effectiveness of EBP pedagogy. Based on this model, the parameters of effective EBP courses were set. The necessary conditions for successful EBP courses are: curricular strengths, organizational support, teachers’ mastery of EBP, and teachers’ involvement in EBP learning. In our instrument, we focused on the evaluation of the effectiveness of EBP courses from a faculty perspective. Therefore, items reflected faculty perception of the integration of EBP in the curriculum, faculty involvement in learning and expectations they had of their students, mastery of subject matter, and organizational support. The instrument for measuring faculty perception of their own and students’ competence in EBP skills was designed to
402
address all seven steps of the EBP process based on a modified version of the list of EBP skills in Melnyk et al. (2008). EBP skills can be divided into three groups:
r Skills related to the implementation and dissemination of EBP (e.g., applying synthesized evidence to initiate change, mentoring/teaching EBP to others).
r Skills in searching for and appraising evidence (e.g., searching efficiently for evidence, critically appraising the relevant body of evidence to address clinical questions).
r Skills relating to cooperation with the clinical environment (e.g., assessing the clinical environment for readiness for EBP, asking questions regarding patients’ care). Teacher competency is one of the most important factors in the learning process. To be able to deliver EBP knowledge and skills properly, faculty have to be experts in these skills. However, there is a lack of studies focusing on the competence of teachers of EBP courses and on how they perceive their own EBP skills. It is not well known how competent graduate nursing faculty members are in EBP knowledge and skills. This study focuses particularly on faculty perception of the effectiveness of EBP courses and faculty perception of their own competence in EBP skills. Inclusion of a core EBP course in graduate nursing curricula is essential to establishing a foundation from which EBP knowledge and skills can be further developed in the remainder of the students’ didactic and clinical coursework (Melnyk et al., 2008). Evaluation of the effectiveness of EBP courses can inform the improvement of future programs. Three main research questions guided this study: (1) What are faculty perceptions of the effectiveness of EBP courses for graduate nursing students? (2) What are faculty perceptions of their own competence in EBP skills? (3) As judged by their teachers, how competent in EBP skills are students upon completion of EBP courses?
PURPOSE The purpose of this descriptive cross-sectional study was to determine how faculty from the 50 top graduate nursing schools in the United States perceived the effectiveness of EBP courses for graduate nursing students.
METHODS Study Design and Methods A descriptive cross-sectional design was used to explore perception of the effectiveness of EBP courses and perception of competence in EBP skills. A web-based survey was used Worldviews on Evidence-Based Nursing, 2014; 11:6, 401–413. C 2014 Sigma Theta Tau International
Original Article for data collection. Data collection was conducted by sending potential respondents an e-mail with a hyperlink to the webbased survey. Faculty members agreeable to participating were required to click on the link in the e-mail directing them to a study-specific, secured Survey Monkey website. The website consisted of a questionnaire that was developed by the investigators to assess perceptions of the effectiveness of EBP courses and perceptions of EBP skills. The questionnaire took approximately 5–10 minutes to complete.
Sample and Data Collection The sample population consisted of faculty from the top 50 graduate nursing schools across the United States according to the Top Universities in U.S. website (http://www.university-list.net/us/rank/univ-20131118.htm), who taught EBP courses for graduate nursing students. Sixty-two nursing schools were contacted through their deans’ offices. E-mails were sent to the deans. Each e-mail contained a cover letter addressed to the dean and a cover letter for faculty members involved in teaching EBP courses. Deans who were agreeable to their faculty participating in the study were requested to forward an e-mail including the latter cover letter and a study-specific, secure, and de-identified link to the web-based survey to faculty members involved in teaching EBP courses. Deans of five nursing schools did not agree to their faculty participating in the study. E-mails were sent only once. Data were collected from the middle of January 2013 through the middle of March 2013. The study was conducted in three steps.
Step 1: Development of a questionnaire for evaluating the effectiveness of teaching EBP in nursing. The survey items were developed by the authors. Survey items were derived from the literature and from the authors’ teaching experiences. Items measuring perception of the effectiveness of EBP courses were devised in consultation with an expert from the Center for Instructional Development and Distance Education at the University of Pittsburgh. The survey was comprised of the following sets of questions: (1) Fifteen questions on the survey employed a scale of 1 (strongly disagree) to 7 (strongly agree) to assess perception of the effectiveness of EBP courses. These questions covered four areas: perception of the integration of EBP in the curriculum, faculty involvement in EBP learning and the expectations they had of students, mastery of subject matter, and organizational support. (2) Fourteen questions included in the survey employed a scale of 1 (not at all competent) to 7 (extremely competent) to assess faculty perceptions of their own competence in EBP skills. A list of EBP skills was adapted from Melnyk and colleagues (2008) with the consent of the first author. The list of skills did not include those with an immediate bearing on patient care. Worldviews on Evidence-Based Nursing, 2014; 11:6, 401–413. C 2014 Sigma Theta Tau International
These skills were added following an analysis of the definition of EBP by the researchers, and reflected a consensus reached among them. Besides this modification, some items on the original list were omitted and others were paraphrased to make the questionnaire more accessible to participants. (3) Fourteen questions employed a scale of 0 (not well at all) to 7 (extremely well) to measure faculty perceptions of students’ competence in EBP skills after completing EBP courses. Faculty members were asked how well students who had completed EBP courses had mastered a specific set of EBP skills. The list of EBP skills was the same as the list of EBP skills for faculty. (4) Sample characteristics (e.g., gender, age, years of experience teaching EPB courses, number of years as a faculty member, and training in teaching EBP). (5) Questions regarding teaching strategies (e.g., barriers to teaching EBP, the most important EBP skills to focus on). Once the survey items were developed, they were reviewed for content validity and clarity by an interdisciplinary group of EBP experts.
Step 2: Conversion of paper survey to web-based survey and pilot testing. The paper survey was converted to a web version and piloted by the authors and several colleagues. Issues and limitations with the design of the web-based survey were discussed within the research team and modifications were made to increase ease of use.
Step 3: Web-based survey. Questionnaires were completed online via SurveyMonkey, which keeps data private, safe, and secure.
ETHICAL CONSIDERATIONS Approval was obtained from the Institutional Review Board of the University of Pittsburgh prior to participant recruitment. A cover letter explaining the aim of the study was attached to all surveys. The cover letter also addressed the voluntary nature of the survey and guaranteed confidentiality to the fullest possible extent. Completing the questionnaire served as a participant’s consent to take part in the research study. The total duration of subject participation was the time taken to complete a one-time questionnaire.
STATISTICAL ANALYSIS Descriptive statistics were used to describe the sample characteristics and perceptions of effectiveness of EBP courses as well as perceptions of competence in EBP skills. The Spearman Correlation coefficient was used to examine the correlation between the total score for the instrument and the score for each item. Reliability of the scale was measured by Cronbach’s
403
Perception of Effectiveness of EBP Courses
Alpha. Interitem correlations were calculated and an itemtotal analysis was performed. The Kaiser-Meyer-Olkin Measure of Sampling Adequacy and Bartlett’s Test of Sphericity were conducted before factor analysis. Factor analysis (extraction method: principal component analysis) with Varimax rotation was performed. Statistical analyses were conducted using the Statistical Package for the Social Sciences 16.0 for Windows (SPSS, Chicago, IL, USA).
The Most Important EBP Skills to Teach According to our sample, the most important EBP skills to focus on when teaching EBP courses are:
r Critically appraising the relevant body of evidence to address clinical questions (n = 37; 82.2%);
r Searching efficiently for evidence that answers the clinical questions (n = 29; 64.4%);
r Synthesizing evidence to make decisions about paRESULTS Fifty-seven nursing schools deemed likely to agree to participate were contacted. In response, 74 questionnaires from faculty were received, though not all of them were complete. A total of 45 completed questionnaires (60%) had less than 5% of the data missing and were used in the statistical analysis. The majority of participants were involved in teaching EBP courses. Approximately two-thirds of the participants were teaching EBP courses for graduate students (62.3%), others were teaching EBP courses for both graduate and undergraduate students (15.5%), or were teaching EBP courses for undergraduate students (13.3%), or else included EBP content in other courses (8.9%). Of the entire sample, 42 (93.3%) participants were female and 3 were male (6.7%). The largest segment of the sample, 24 (53.4%), were 56 years old or older, whereas 15 (33.3%) were 46–55 years old. More than 80% of those surveyed had been faculty members for more than 6 years. Approximately two-thirds of participants had been teaching EBP courses for 4 or more years. Of all the participants, 30 (66.7%) cited selfstudy as their primary training in the teaching of EBP, whereas 12 (26.7%) received several days of training, and 3 (6.6%) of the participants used online training (see Table 1).
Teaching Strategies Faculty from the top 50 graduate nursing schools across the United States cited lectures (n = 32; 71.1%), projects (n = 32; 71.1%), group work (n = 31; 68.9%), computer-based teaching strategies (n = 26; 57.8%), training students to critically appraise a published article (n = 26; 57.8%), and training students to develop clinical protocols (n = 8; 17.8%) as the most frequently used teaching strategies in EBP courses. The number of students enrolled in EBP courses was 5–20 (n = 21; 46.6%), 21–40 students (n = 12; 26.7%), and 41 or more (n = 12; 26.7%).
Barriers to the Teaching of EBP The main barriers to the teaching of EBP identified by faculty were (a) time (n = 8) “limited class time and competition for that time with so many other skills and theory,” (b) insufficient statistical and research background of students (n = 5), (c) lack of faculty experts in the area of EBP (n = 4), and (d) large class size (n = 3).
404
tient care (n = 27; 60%);
r Formulating searchable, answerable clinical questions in PICOT format (n = 24; 53.3%);
r Selecting the best evidence from what is found in the search (n = 21; 46.7%);
r Analyzing outcomes of evidence-based interventions, practice changes, and clinical guidelines (n = 20; 44.4%).
Perception of the Effectiveness of EBP Courses The mean perception of the effectiveness of EBP courses measured on a scale from 1 to 7 was 5.58 (s2 0.45; min. 4.29; max. 6.73) for the whole sample, and the mean perception of faculty competence in EBP skills was 6.16 (s2 0.06; min. 5.73; max. 6.56). In both cases, a higher score signifies higher perceived effectiveness or competency (see Table 1).
Reliability. The interitem correlations of the 15 items ranged from −.16 to .98, with the majority being around .40. The internal consistency of the scale, based on standardized Cronbach’s alpha, was .84, which signifies strong internal consistency. The item-total analysis showed a range for Cronbach’s alpha if item deleted from .81 to .85, showing that every item contributes to the overall reliability. Factor analysis of perception of the effectiveness of EBP courses. A factor analysis (extraction method: principal component analysis) with Varimax rotation in SPSS was performed on the 15 items of the Perception of the Effectiveness of EBP Courses scale. A ratio of 3:1 (subject-to-variables) was used in the sample. The minimum sample size according to Mundfrom, Shaw, and Ke (2005), with good level for variables-tofactors ratio set at seven and the number of factors set at three, is 40–65. The Kaiser-Meyer-Olkin Measure of Sampling Adequacy was .647, above the .6 suggested minimum. Bartlett’s Test of Sphericity was significant (
Faculty Perception of the Effectiveness of EBP Courses for Graduate Nursing Students Renata ´ Zelen´ıkova, ´ RN, PhD • Michael Beach, DNP, ACNP-BC, PNP • Dianxu Ren, MD, PhD • Emily Wolff • Paula Sherwood, RN, PhD, CNRN, FAAN
ABSTRACT Keywords evidence-based practice, faculty, teaching, curricula
Background: Effective teaching is key in preparing students to become successful evidencebased healthcare professionals. The effectiveness of graduate evidence-based practice (EBP) pedagogy is not often a subject of research studies. Purpose: The purpose of this study was to determine how faculty from the 50 top graduate nursing schools in the United States perceived the effectiveness of EBP courses for graduate nursing students. Methods: A descriptive cross-sectional design was used to explore faculty perception of the effectiveness of EBP courses. A web-based survey was used for data collection. A total of 45 questionnaires were subjected to statistical analysis. Results: The mean perception of the effectiveness of EBP courses for the whole sample, on a scale from 1 to 7, was 5.58 (min. 4.29; max. 6.73), a higher score signifying higher perceived effectiveness. The highest rated item concerned a school’s access to different databases. The strongest correlations were found between the total score and the scores for items describing students’ opportunities to strengthen and apply their EBP skills (rs = .66). The internal consistency of the Perception of Effectiveness of EBP Courses scale, based on standardized Cronbach’s alpha, was .84, which signifies strong internal consistency. Faculty perceived themselves as most competent at the following EBP skills: (a) “Asking questions regarding patients’ care” (6.56), (b) “Considering patient preferences when implementing EBP” (6.40), and (c) “Critically appraising the relevant body of evidence to address clinical questions” (6.40). Discussion: To strengthen the effectiveness of EBP courses, students should have more opportunities to implement their EBP knowledge and skills after completing EBP courses. Linking Evidence to Action: Evaluation of faculty perceptions of the effectiveness of EBP courses can help to guide the development of nursing school curricula that better integrate EBP. Further evaluation of the psychometric properties of the instrument used to measure perception of the effectiveness of EBP courses is required along with objective measures of faculty knowledge and skills in teaching EBP.
BACKGROUND AND SIGNIFICANCE Worldwide, evidence-based practice (EBP) has emerged as a major healthcare initiative (Thiel & Ghosh, 2008). One of the most consistent findings in health service research is the gap between best practice (as determined by scientific evidence) and actual clinical care (Flores-Mateo & Argimon, 2007). To accelerate the translation of research findings into clinical practice, two major outcomes must be achieved: (a) Advanced practice and direct care nurses must acquire sufficient EBP knowledge and skills as well as strong beliefs about the value of EBP in clinical settings, and (b) educators must teach their students the EBP process to instill in them lifelong skills and the motivation to deliver the highest quality of care (Melnyk, Fineout-Overholt, Feinstein, Sadler, & Green-Hernandez, 2008).
Worldviews on Evidence-Based Nursing, 2014; 11:6, 401–413. C 2014 Sigma Theta Tau International
Findings from a recent national survey by Melnyk, FineoutOverholt, Gallagher-Ford, and Kaplan (2012) indicated that nurses surveyed across the country are ready for and do value EBP. The majority of participants who responded to the survey reported wanting to gain more knowledge and skills in order to deliver evidence-based care in their institutions. Nurses cited a top requirement for helping them implement EBP in daily practice as education. For students to become evidence-based healthcare professionals, the teaching of EBP has to be effective (Spek, Wolf, Dijk, & Lucas, 2012). Little has been published about teaching EBP to nursing students (Stiffler & Cullen, 2010). Though there are systematic reviews and meta-analyses of teaching EBP in schools of medicine, there is a dearth of research in nursing,
401
Perception of Effectiveness of EBP Courses
especially in regard to graduate-level EBP pedagogy. According to Fineout-Overholt and Johnston (2005), further research is needed to assess effective teaching and evaluation strategies for EBP. Teaching EBP to nursing students is usually based on the basic steps of EBP. Melnyk and Fineout-Overholt (2011) added two more steps to the five basic steps of EBP, which include: (a) cultivate a spirit of inquiry; (b) ask the burning clinical question in PICOT (P = patient population; I = intervention or area of interest; C = comparison intervention or group; O = outcomes; and T = time) format; (c) search for and collect the most relevant best evidence; (d) critically appraise the evidence; (e) integrate the best evidence with one’s clinical expertise and patient preferences and values in making a practice decision or change; (f) evaluate outcomes of the practice decision or change based on evidence; and (g) disseminate the outcomes of the EBP decision or change. The seven steps of EBP can serve as a structure around which to build EBP curriculum for graduate nursing students. The effectiveness of graduate EBP pedagogy is not often a subject of research studies. One of the problems is the complexity of the EBP process and the difficulty of assessing all aspects of its effectiveness. Shaneyefelt and colleagues (2006) performed a systematic review of EBP instruments. Their results showed that the majority of instruments targeted students and postgraduate trainees, whereas nonphysicians were rarely evaluated. The available instruments most commonly evaluated EBP skills (predominantly focusing on the critical appraisal of evidence), knowledge, attitudes, and behaviors. Most instruments are designed for specific purposes, such as the evaluation of theoretical EBP courses (instruments to assess cognitive skills), or the evaluation of EBP in clinical practice (instruments to assess performance-based skills and application; Ilic, 2009). The instrument used in this study for measuring the perception of the effectiveness of EBP courses was inspired by the ARCC-E (advancing research and clinical practice through close collaboration and education) conceptual model for teaching EBP (Melnyk & Fineout-Overholt, 2011). This model shows that organizational support, the identification of curricular strengths and barriers to teaching EBP and use of faculty EBP mentors play an important role in the effectiveness of EBP pedagogy. Based on this model, the parameters of effective EBP courses were set. The necessary conditions for successful EBP courses are: curricular strengths, organizational support, teachers’ mastery of EBP, and teachers’ involvement in EBP learning. In our instrument, we focused on the evaluation of the effectiveness of EBP courses from a faculty perspective. Therefore, items reflected faculty perception of the integration of EBP in the curriculum, faculty involvement in learning and expectations they had of their students, mastery of subject matter, and organizational support. The instrument for measuring faculty perception of their own and students’ competence in EBP skills was designed to
402
address all seven steps of the EBP process based on a modified version of the list of EBP skills in Melnyk et al. (2008). EBP skills can be divided into three groups:
r Skills related to the implementation and dissemination of EBP (e.g., applying synthesized evidence to initiate change, mentoring/teaching EBP to others).
r Skills in searching for and appraising evidence (e.g., searching efficiently for evidence, critically appraising the relevant body of evidence to address clinical questions).
r Skills relating to cooperation with the clinical environment (e.g., assessing the clinical environment for readiness for EBP, asking questions regarding patients’ care). Teacher competency is one of the most important factors in the learning process. To be able to deliver EBP knowledge and skills properly, faculty have to be experts in these skills. However, there is a lack of studies focusing on the competence of teachers of EBP courses and on how they perceive their own EBP skills. It is not well known how competent graduate nursing faculty members are in EBP knowledge and skills. This study focuses particularly on faculty perception of the effectiveness of EBP courses and faculty perception of their own competence in EBP skills. Inclusion of a core EBP course in graduate nursing curricula is essential to establishing a foundation from which EBP knowledge and skills can be further developed in the remainder of the students’ didactic and clinical coursework (Melnyk et al., 2008). Evaluation of the effectiveness of EBP courses can inform the improvement of future programs. Three main research questions guided this study: (1) What are faculty perceptions of the effectiveness of EBP courses for graduate nursing students? (2) What are faculty perceptions of their own competence in EBP skills? (3) As judged by their teachers, how competent in EBP skills are students upon completion of EBP courses?
PURPOSE The purpose of this descriptive cross-sectional study was to determine how faculty from the 50 top graduate nursing schools in the United States perceived the effectiveness of EBP courses for graduate nursing students.
METHODS Study Design and Methods A descriptive cross-sectional design was used to explore perception of the effectiveness of EBP courses and perception of competence in EBP skills. A web-based survey was used Worldviews on Evidence-Based Nursing, 2014; 11:6, 401–413. C 2014 Sigma Theta Tau International
Original Article for data collection. Data collection was conducted by sending potential respondents an e-mail with a hyperlink to the webbased survey. Faculty members agreeable to participating were required to click on the link in the e-mail directing them to a study-specific, secured Survey Monkey website. The website consisted of a questionnaire that was developed by the investigators to assess perceptions of the effectiveness of EBP courses and perceptions of EBP skills. The questionnaire took approximately 5–10 minutes to complete.
Sample and Data Collection The sample population consisted of faculty from the top 50 graduate nursing schools across the United States according to the Top Universities in U.S. website (http://www.university-list.net/us/rank/univ-20131118.htm), who taught EBP courses for graduate nursing students. Sixty-two nursing schools were contacted through their deans’ offices. E-mails were sent to the deans. Each e-mail contained a cover letter addressed to the dean and a cover letter for faculty members involved in teaching EBP courses. Deans who were agreeable to their faculty participating in the study were requested to forward an e-mail including the latter cover letter and a study-specific, secure, and de-identified link to the web-based survey to faculty members involved in teaching EBP courses. Deans of five nursing schools did not agree to their faculty participating in the study. E-mails were sent only once. Data were collected from the middle of January 2013 through the middle of March 2013. The study was conducted in three steps.
Step 1: Development of a questionnaire for evaluating the effectiveness of teaching EBP in nursing. The survey items were developed by the authors. Survey items were derived from the literature and from the authors’ teaching experiences. Items measuring perception of the effectiveness of EBP courses were devised in consultation with an expert from the Center for Instructional Development and Distance Education at the University of Pittsburgh. The survey was comprised of the following sets of questions: (1) Fifteen questions on the survey employed a scale of 1 (strongly disagree) to 7 (strongly agree) to assess perception of the effectiveness of EBP courses. These questions covered four areas: perception of the integration of EBP in the curriculum, faculty involvement in EBP learning and the expectations they had of students, mastery of subject matter, and organizational support. (2) Fourteen questions included in the survey employed a scale of 1 (not at all competent) to 7 (extremely competent) to assess faculty perceptions of their own competence in EBP skills. A list of EBP skills was adapted from Melnyk and colleagues (2008) with the consent of the first author. The list of skills did not include those with an immediate bearing on patient care. Worldviews on Evidence-Based Nursing, 2014; 11:6, 401–413. C 2014 Sigma Theta Tau International
These skills were added following an analysis of the definition of EBP by the researchers, and reflected a consensus reached among them. Besides this modification, some items on the original list were omitted and others were paraphrased to make the questionnaire more accessible to participants. (3) Fourteen questions employed a scale of 0 (not well at all) to 7 (extremely well) to measure faculty perceptions of students’ competence in EBP skills after completing EBP courses. Faculty members were asked how well students who had completed EBP courses had mastered a specific set of EBP skills. The list of EBP skills was the same as the list of EBP skills for faculty. (4) Sample characteristics (e.g., gender, age, years of experience teaching EPB courses, number of years as a faculty member, and training in teaching EBP). (5) Questions regarding teaching strategies (e.g., barriers to teaching EBP, the most important EBP skills to focus on). Once the survey items were developed, they were reviewed for content validity and clarity by an interdisciplinary group of EBP experts.
Step 2: Conversion of paper survey to web-based survey and pilot testing. The paper survey was converted to a web version and piloted by the authors and several colleagues. Issues and limitations with the design of the web-based survey were discussed within the research team and modifications were made to increase ease of use.
Step 3: Web-based survey. Questionnaires were completed online via SurveyMonkey, which keeps data private, safe, and secure.
ETHICAL CONSIDERATIONS Approval was obtained from the Institutional Review Board of the University of Pittsburgh prior to participant recruitment. A cover letter explaining the aim of the study was attached to all surveys. The cover letter also addressed the voluntary nature of the survey and guaranteed confidentiality to the fullest possible extent. Completing the questionnaire served as a participant’s consent to take part in the research study. The total duration of subject participation was the time taken to complete a one-time questionnaire.
STATISTICAL ANALYSIS Descriptive statistics were used to describe the sample characteristics and perceptions of effectiveness of EBP courses as well as perceptions of competence in EBP skills. The Spearman Correlation coefficient was used to examine the correlation between the total score for the instrument and the score for each item. Reliability of the scale was measured by Cronbach’s
403
Perception of Effectiveness of EBP Courses
Alpha. Interitem correlations were calculated and an itemtotal analysis was performed. The Kaiser-Meyer-Olkin Measure of Sampling Adequacy and Bartlett’s Test of Sphericity were conducted before factor analysis. Factor analysis (extraction method: principal component analysis) with Varimax rotation was performed. Statistical analyses were conducted using the Statistical Package for the Social Sciences 16.0 for Windows (SPSS, Chicago, IL, USA).
The Most Important EBP Skills to Teach According to our sample, the most important EBP skills to focus on when teaching EBP courses are:
r Critically appraising the relevant body of evidence to address clinical questions (n = 37; 82.2%);
r Searching efficiently for evidence that answers the clinical questions (n = 29; 64.4%);
r Synthesizing evidence to make decisions about paRESULTS Fifty-seven nursing schools deemed likely to agree to participate were contacted. In response, 74 questionnaires from faculty were received, though not all of them were complete. A total of 45 completed questionnaires (60%) had less than 5% of the data missing and were used in the statistical analysis. The majority of participants were involved in teaching EBP courses. Approximately two-thirds of the participants were teaching EBP courses for graduate students (62.3%), others were teaching EBP courses for both graduate and undergraduate students (15.5%), or were teaching EBP courses for undergraduate students (13.3%), or else included EBP content in other courses (8.9%). Of the entire sample, 42 (93.3%) participants were female and 3 were male (6.7%). The largest segment of the sample, 24 (53.4%), were 56 years old or older, whereas 15 (33.3%) were 46–55 years old. More than 80% of those surveyed had been faculty members for more than 6 years. Approximately two-thirds of participants had been teaching EBP courses for 4 or more years. Of all the participants, 30 (66.7%) cited selfstudy as their primary training in the teaching of EBP, whereas 12 (26.7%) received several days of training, and 3 (6.6%) of the participants used online training (see Table 1).
Teaching Strategies Faculty from the top 50 graduate nursing schools across the United States cited lectures (n = 32; 71.1%), projects (n = 32; 71.1%), group work (n = 31; 68.9%), computer-based teaching strategies (n = 26; 57.8%), training students to critically appraise a published article (n = 26; 57.8%), and training students to develop clinical protocols (n = 8; 17.8%) as the most frequently used teaching strategies in EBP courses. The number of students enrolled in EBP courses was 5–20 (n = 21; 46.6%), 21–40 students (n = 12; 26.7%), and 41 or more (n = 12; 26.7%).
Barriers to the Teaching of EBP The main barriers to the teaching of EBP identified by faculty were (a) time (n = 8) “limited class time and competition for that time with so many other skills and theory,” (b) insufficient statistical and research background of students (n = 5), (c) lack of faculty experts in the area of EBP (n = 4), and (d) large class size (n = 3).
404
tient care (n = 27; 60%);
r Formulating searchable, answerable clinical questions in PICOT format (n = 24; 53.3%);
r Selecting the best evidence from what is found in the search (n = 21; 46.7%);
r Analyzing outcomes of evidence-based interventions, practice changes, and clinical guidelines (n = 20; 44.4%).
Perception of the Effectiveness of EBP Courses The mean perception of the effectiveness of EBP courses measured on a scale from 1 to 7 was 5.58 (s2 0.45; min. 4.29; max. 6.73) for the whole sample, and the mean perception of faculty competence in EBP skills was 6.16 (s2 0.06; min. 5.73; max. 6.56). In both cases, a higher score signifies higher perceived effectiveness or competency (see Table 1).
Reliability. The interitem correlations of the 15 items ranged from −.16 to .98, with the majority being around .40. The internal consistency of the scale, based on standardized Cronbach’s alpha, was .84, which signifies strong internal consistency. The item-total analysis showed a range for Cronbach’s alpha if item deleted from .81 to .85, showing that every item contributes to the overall reliability. Factor analysis of perception of the effectiveness of EBP courses. A factor analysis (extraction method: principal component analysis) with Varimax rotation in SPSS was performed on the 15 items of the Perception of the Effectiveness of EBP Courses scale. A ratio of 3:1 (subject-to-variables) was used in the sample. The minimum sample size according to Mundfrom, Shaw, and Ke (2005), with good level for variables-tofactors ratio set at seven and the number of factors set at three, is 40–65. The Kaiser-Meyer-Olkin Measure of Sampling Adequacy was .647, above the .6 suggested minimum. Bartlett’s Test of Sphericity was significant (
