Student-teacher education programme (STEP) by step: transforming medical students into competent, confident teachers.

2014, 36: 322–332

Student–teacher education programme (STEP) by step: Transforming medical students into competent, confident teachers DEBORAH R. ERLICH & ALLEN F. SHAUGHNESSY Tufts University, USA

Background: While most medical schools have students teach other students, few offer formal education in teaching skills, and fewer provide teaching theory together with experiential teaching practice. Furthermore, curriculum evaluation of teaching education is lacking. Aim: This study aimed to examine effects of a novel didactic teaching curriculum for students embedded in a practical teaching experience. Methods: A longitudinal 12-week curriculum with complementary didactic and practical components for final-year students learning how to teach was developed, implemented and evaluated using a multi-level evaluation based on the Kirkpatrick approach with qualitative and quantitative methods. Results: Thirteen student–teachers acquired measureable knowledge, skills and attitudes necessary for teaching excellence. Confidence in teaching increased (p50.001), particularly in four key areas: oral feedback, written feedback, mentoring, and the difficult learner. Student–teachers demonstrated teaching competence as determined by self-assessment, student feedback, and faculty observation. Top teachers impacted their first-year students’ performance in patient interviewing as measured by Objective Structured Clinical Examination (OSCE). Conclusions: Reinforcing educational theory with practical teaching experience under direct faculty supervision promotes teaching competency for graduating medical students. The intertwined didactic plus practical model can be applied to various teaching contexts to fulfil the mandate that medical schools train graduates in core teaching knowledge, skills and attitudes in preparation for their future roles as clinical teachers.

Practice points Ever since the dawn of modern medicine, the overwhelming majority of practising clinicians teach and instruct their junior colleagues. (Bannard-Smith et al. 2012).

Introduction Historically, physicians have always been teachers. The etymology of the word doctor from the Latin verb docere ‘‘to teach’’ is a reminder that the act of teaching, both of patients and of apprentices, is a defining quality of the physician’s role. Junior doctors are expected to teach students (Dandavino et al. 2007), but untrained or even poor teachers abound. Students say that resident [junior doctor] teaching varies widely (Kerfoot et al. 2004) and is often ineffective (Morrison & Hafler 2000). Most residents report feeling uncomfortable teaching (Bing-You & Greenberg 1990). To achieve competency in educating students, future physicians must learn how to teach even while still in medical school.

Embedding a longitudinal didactic series on education in a practical teaching experience reinforces knowledge, skills and attitudes of teaching. After a teaching curriculum, final-year students realise increased confidence in teaching. After a teaching curriculum, students demonstrate competence in specific teaching abilities. It is still inconclusive whether lasting outcomes emerge in first-year students taught by trained teachers.

20 14

Med Teach Downloaded from informahealthcare.com by Memorial University of Newfoundland on 01/26/15 For personal use only.

Abstract

A medical school’s obligation to provide teaching training for students extends beyond merely providing the opportunity to teach (Soriano et al. 2010). In recognition of the long-held de facto teaching role of physicians, medical schools are today explicitly challenged to prepare graduates to teach in their next career phase. Internationally, LCME and ACGME (US),

Correspondence: Deborah R. Erlich, MD, MMedEd, Department of Family Medicine, Tufts University, 136 Harrison Avenue, Stearns 107, Boston, MA 02111, USA. Tel: 617 636 2455; Fax: 617.474.3868; E-mail: [email protected]

322

ISSN 0142-159X print/ISSN 1466-187X online/14/40322–332 ß 2014 Informa UK Ltd. DOI: 10.3109/0142159X.2014.887835


Student–teacher education programme

CanMEDS (Canada), GMC’s Tomorrow’s Doctors (UK) all now mandate that residents be competent in teaching. Extensive prior literature discusses giving senior students opportunities to teach (Pasquinelli & Greenberg 2008), often as faculty extenders. However, trainees without explicit instruction in teaching feel uncomfortable in their teaching skills, while students with formal training are stronger teachers, learners and communicators; teaching, while a universal task of medical trainees, is not intuitive and requires on-going reinforcement and practice (Dandavino et al. 2007; Soriano et al. 2010). Most medical schools lack formal teaching training for students, and most medical students receive no teaching training (Greenberg & Jewett 1987; Ten Cate 2007). Only one-quarter of US medical schools offer such training (Henry et al. 2006), and in the UK, only one of 29 medical schools has a compulsory curriculum for teaching students how to become teachers (Rodrigues et al. 2010).

Peer teaching The model of students teaching other students falls under the label peer teaching, or ‘‘people from similar social groupings who are not professional teachers helping each other to learn and learning themselves by teaching’’ (Topping 1996). In medical education, most ‘‘peer’’ teaching connotes a more senior student teaching a junior student, not same-year peer teaching (Topping 1996). Students indeed prefer to learn from peers who are slightly more advanced: neither at the same level, nor too far ahead (Field et al. 2004, 2007). Not only does peer teaching give senior students an opportunity to practise teaching, but it also confers distinct advantages for learners because of two key concepts: cognitive congruence and social congruence. Because of their cognitive congruence, or thinking similarly, peer teachers can augment learning, even when they have no particular expertise in the material taught (Nnodim 1997; Rao & DiCarlo 2000; Giuliodori et al. 2006; Mazur 2009; Smith et al. 2009). Peers remember what it was like not to know (Baillie et al. 2009) and can explain concepts accessibly to learners (Hill et al. 2010). Peers can teach learners something new, but are not at such an expert level that learners get lost. Social congruence, or shared social context, experience, age and other generational qualities (Ten Cate & Durning 2007; Lockspeiser et al. 2008), implies that peer teachers can relate to learners better than can seasoned faculty (Bulte et al. 2007). Students feel more comfortable learning from a peer than from an expert teacher (Ten Cate & Durning 2007). Peer teachers are less intimidating, more relaxed and less formal than are qualified doctors (Hudson & Tonkin 2008; Baillie et al. 2009) and create less pressure and embarrassment in learners (Weidner & Popp 2007).

Good teachers matter Students of better teachers perform better. In three separate studies, excellent teachers, as rated by a resident consensus panel with high inter-rater reliability, improved their students’

end-of clerkship marks and national United States Medical Licensing Examination (USMLE) scores, whereas poor teachers actually lowered student marks (Griffith et al. 1997, 2000; Stern et al. 2000). Another study corroborated that peer-assisted learning in preparation for the USMLE Step 1 examination improved tutees’ scores (Alcamo et al. 2010). Conversely, poor teachers can hinder learning (Young et al. 2009).

Student-as-teacher programmes Medical schools have developed various models for teaching education. Some schools offer a didactic session or series about core teaching principles. Most are brief. In Germany, for instance, one school offered a three-day ‘‘Mentoring-Tutoring Programme’’ in presentation skills, communication and group leadership (Juenger et al. 2009). In Massachusetts (US), a one-week teaching elective for senior students taught concepts such as needs assessment of learners, establishing goals and objectives, teaching methods, and evaluation and feedback (Pasquale & Pugnaire 2002). In Ireland, a three-hour workshop on teaching for senior students preceded a six-month commitment to teaching physical diagnosis to first-year students (Salerno-Kennedy et al. 2010). Other medical schools give senior students experience teaching as part of the formal curriculum for junior students, often in small groups. Students have taught peers in patient interviewing (Barnes et al. 1978; Blatt & Greenberg 2007; Leeper et al. 2007), problem-based learning (Johansen et al. 1992; Kassab et al. 2005; Jafri et al. 2007), clinical problemsolving (Pepe et al. 1980), anatomy (Nnodim 1997; Houwink et al. 2004; Bentley & Hill 2009; Correa et al. 2009; Shiozawa et al. 2010), physical diagnosis (Haist et al. 1998; Field et al. 2004; Kim et al. 2010; Dickson et al. 2011), musculoskeletal examination (Burke et al. 2007; Graham et al. 2008; Perry et al. 2010), physiology (Kibble et al. 2006), other basic science (Resnick & MacDougall 1976; Pasquinelli & Greenberg 2008), and ultrasound use (Knobe et al. 2010). Students also help junior students with professional development, addressing organisational skills and career exploration (Juenger et al. 2009), and recruitment (Drouin et al. 2006). Extensive prior literature, then, describes opportunities for senior students to teach, and some discusses didactic programmes, but scant research exists on offering didactic principles of teaching in concert with practical application of these tenets. Two European examples are notable. In the Netherlands, didactic sessions on teaching sandwiched a practical teaching experience for final-year students, who were very satisfied. The experience comprised an introductory discussion, a preparatory meeting, the teaching session itself, a reflection exercise and a microteaching experience (Zijdenbos et al. 2011). A German study described a novel peer tutor curriculum addressing both teaching skills and anatomy content. It included a three-week preparatory curriculum, microteaching, didactic exercises, and dissection experience. Peer tutors appreciated both elements (Shiozawa et al. 2010). A further research gap is that most teaching training interventions are brief, lasting an average of one day. Little is known about the effect of more substantial teaching

323

D. R. Erlich & A. F. Shaughnessy


programmes, such as a month-long block or a longitudinal programme over several months. There are also insufficient data on assessment of students as teachers, whether via direct observation, anonymous survey or other assessment tools. At our medical school, final-year students provide formal teaching and assessment of first-year medical students in a medical interviewing course. In the present study, student– teachers received a novel didactic teaching programme embedded in this practical teaching experience. The aim of this study was to implement the didactic teaching programme and then evaluate its effectiveness using four levels of Kirkpatrick’s (1959) evaluation model.

Methods Context The present study, conducted at Tufts University School of Medicine (TUSM) in Boston, Massachusetts, received approval by the TUSM Institutional Review Board, the Dean of Education and the Dean for Students. Each year, final-year students (in their fourth year) volunteer to lead a small group of first-year students in the compulsory Medical Interviewing and the Doctor–Patient Relationship course. The course, presented weekly for 12 sessions over the course of a semester, consists of a morning lecture given by faculty members on medical interview and history-taking techniques, followed by an afternoon of practising in small groups with actual patients. The first-year students are randomly assigned to stable groups of four; each small group has a group leader, who is either faculty or a student–teacher. This leader facilitates a discussion on main points of the lecture, directly observes interviews and gives feedback on interviewing skills. In addition to receiving this formative feedback, first-year students receive formal midpoint and final summative assessments from their group leaders. Teachers rate students with ‘‘meets expectations’’ or ‘‘does not meet expectations’’ for each of 11 competencies in patient interviewing and history-taking and may write qualitative comments for each. Students also participate in an end-of-course Objective Structured Clinical Examination (OSCE), graded by standardised patient educators. At the course end, student–teachers give a final mark of pass or fail. Students must pass both the OSCE and the small group to pass the course. Students with a failing grade may appeal and remediate with faculty course leadership. Student–teachers are volunteers who use this experience to satisfy the teaching requirement of the curriculum. The opportunity is well-known among final-year students. They receive no other incentive outside of inclusion of this experience in their CV as they apply for post-graduate training positions. They have no prior training in performing needs assessments, facilitating group discussions, providing feedback, or developing summative evaluations and final course marks, all of which is expected of them. All student–teachers have previously taken the course in their own first year of study. 324

Participants For this study, the complete class of 189 first-year students enrolled in the Medical Interviewing and the Doctor–Patient Relationship course and were randomly assigned to groups of four distributed among six patient care sites. A total of 13 student–teachers who oversaw 12 small groups of 48 firstyear students participated in the intervention. The other 35 student–teachers (control groups) led 35 small groups totalling 140 students at five other sites. These student– teachers received no teaching training. Assignment of student– teachers to groups was based on availability and logistical factors such as transportation to the various sites and pre-existing schedule of clinical rotations; student–teachers were not randomised to control or intervention. Within each site, however, student–teachers were randomly assigned to groups of four first-year students (Figure 1).

Curriculum The primary aim of the Student–Teacher Education Programme (STEP) was to develop the knowledge, skills and confidence student–teachers need to competently serve as tutors and assessors of first-year students. The learning objectives for student–teachers included knowledge-based objectives from Bloom’s Taxonomy of Cognitive Domains (Bloom 1956): By the end of STEP, student–teachers will be able to: (1) Formulate guiding principles on which effective teaching can be built; (2) Apply a variety of methods to teach during a patient interview; (3) Distinguish between, and apply, formative feedback and summative assessment; (4) Deliver feedback using various methods; (5) Facilitate an effective student-centred group discussion; (6) Observe and assess students conducting a patient interview;

189 first-year students

24 groups of 3 or 4

48 student-teachers

13 student-teachers at intervention site

Figure 1.

35 student-teachers at control site

Study participants.



(7) Solicit feedback on their own teaching; (8) Assess student written assignments; (9) Demonstrate qualities of mentorship for first-year students; (10) Teach with confidence; (11) Show reflective ability regarding their own teaching; (12) Show professionalism by fulfilling expectations of section leader via attendance, dress, timely submission of marks to course director. Curricular content, based on both a review of the literature on existing student-as-teacher programmes and a needs assessment, included topics such as small group facilitation skills, reflection, mentorship, frameworks for giving feedback, and basic education theory. The programme spanned 12 weeks including orientation and final exam, meeting each Thursday afternoon. The daily schedule was as follows: 1330–1330: Student–teacher seminar part I 1330–1415: Student–teacher meets with first-year students, reviews morning lecture, leads discussion 1415–1445: Patient interview #1 (student interviews patient while 3 peers and group leader observe) 1445–1515: Feedback and debrief #1 1515–1545: Patient interview #2 (second student) 1545–1615: Feedback and debrief #2, dismiss first-year students 1615–1700: Student–teacher seminar part II A physician faculty tutor conducted teaching seminars for student–teachers before and after each of their teaching sessions. Each session covered a portion of the planned curriculum, sandwiching an experiential opportunity to practise their new skills. Teaching methods for these seminars, drawn from commonly accepted teaching practices and intending to meet learners at their Vygotskian Zone of Proximal Development (Vygotsky, 1986), included free discussion,

structured discussion, snowball discussion, fish bowl discussion, mini-lecture, role-plays, written reflection, learning plans, microteaching and peer observation of teaching.

Assessment of student–teachers Assessments of the teachers included self-assessment, assessment from the first-year students, and assessment from the STEP tutor. STEP participants completed surveys, participated in focus groups, were directly observed in simulated role-plays and in ‘‘live’’ teaching, and wrote reflection pieces. They also, of course, underwent practical task practice.

Curriculum evaluation Outcome evaluation of STEP followed a modified Kirkpatrick’s hierarchy of curriculum evaluation (1959): Reaction, Learning, Behaviour, and Results (Figure 2).

Data collection and analysis We collected data before, during, and after the course (Table 1). Student–teachers completed pre- and post-course paper questionnaires (design informed by previously-published standards) (McAleer 1994; Boynton 2004) and submitted them in person on the same day. We established content and face validity using a pilot survey. We collected field notes during STEP seminar and during ‘‘live’’ student teaching following a standard two-column ethnographic technique containing objective observations and observer reactions (Morse 1995). Midway through the course and at its end, first-year students anonymously assessed their teachers on nine written questions (numeric 1–5 Likert scale rating and descriptive comments). We analysed changes in scores at the individual teacher level using the student’s t-test as well as in the aggregate using the Wilcoxon ranked sum test.

Results In what ways did the teaching curriculum impact the outcomes of the first-year students? Behaviour Were student-teachers more competent at the curriculum’s end? Learning In what ways did participants demonstrate learning of the basic principles of teaching, and in what ways were they confident in these objectives?

Reaction How did student-teachers react to or enjoy the experience?

Figure 2.

Modified Kirkpatrick pyramid of curriculum evaluation.

325


Table 1. Data collection methods sorted by research objective.

Research objective

Level

(1) To evaluate the curriculum by assessing the teaching confidence of final-year student small group teachers

Reaction

Student–teacher perception (feedback) on the curriculum’s content and conduct, obtained via post-curriculum questionnaire

(2) To evaluate the curriculum by assessing the teaching competence of final-year small group teachers

Learning

Focus group querying confidence Student–teacher self-assessments in teaching skills, before and after the curriculum Field notes on STEP tutor’s observations of student–teachers’ demonstrated learning in seminars Assessments of student–teachers, performed by first-year students; before and after STEP completion STEP tutor’s assessment of STEP participants’ teaching skills, via recorded observations of student–teacher teaching ‘‘live’’ in their sections Peer assessments between student–teachers

Behaviour


Data collection method

(3) To evaluate the curriculum by assessing the competence and confidence in medical interviewing skills of first-year students taught by STEP completers, in comparison with students taught by control group small group teachers

Results

We performed content analysis on descriptive comments from pre- and post-course surveys of the student–teachers. We coded qualitative questionnaire items and merged codes into concepts per grounded theory methodology (Kennedy 2006). We conducted, audio-recorded and transcribed an endof-course semi-structured focus group of student–teachers’ reflections. We also generated reports of first-year student OSCE scores and of mandatory first-year student electronic course evaluations. To assess the Kirkpatrick Reaction level, we analysed student–teachers’ anonymous questionnaire items, both quantitative Likert ratings and qualitative comments, pertaining to reaction to the programme. We also examined comments from the focus group. We evaluated student–teachers’ Learning via both confidence and competence in teaching as measured by self-report (surveys) and demonstration of learning observed by the researcher (field notes from seminar). To evaluate the STEP curriculum at the Behaviour level, we used two data sources: the assessments done by STEP participants’ first-year students, and observations made by the STEP tutor during actual teaching practice. We measured Kirkpatrick’s Results level, pertaining to downstream outcomes, by first-year student performance on a standardised examination of interviewing skills (OSCE), and student–teachers’ observations of their learners’ performance. Using a mixed methods design (Maudsley 2011), we analysed both numerical and qualitative data and triangulated the data. We used an intention-to-treat (or intentionto-educate!) analysis, including all data even from student– teachers who had absences from sessions.

Results Kirkpatrick level I: Reaction Student–teachers reacted positively to STEP. Half (55%) of the respondents agreed or strongly agreed with the statement ‘‘the formal teaching skills programme was valuable to me’’ and 90% agreed or strongly agreed with ‘‘I would choose to teach in [Medical Interviewing] again.’’ Table 2 lists the themes 326

OSCE of interviewing skills, administered to all first-year students at the end of the Medical Interviewing course Student–teacher assessment of first-years’ interviewing skills

Table 2. Themes of STEP participant written feedback (reaction).

Theme Benefits Gained skills

Gained knowledge Attitude: Enjoyment

Attitude: Comfort Areas for improvement Time commitment

Insufficient mentor role Insufficient teaching practice

Example ‘‘I gained more practice and experience teaching’’ ‘‘I got feedback on my teaching’’ ‘‘I appreciated the ‘teaching theory’ lessons’’ ‘‘I really enjoyed working with the students’’ ‘‘The gratitude from my students was surprising and touching’’ ‘‘I became more comfortable in the teacher role’’ ‘‘The hours of 1300-1700 are a lot to be expected . . . especially since that doesn’t include travel time’’ ‘‘I hoped to interact more as a mentor’’ ‘‘More practice teaching’’

developed from analysis of the descriptive comments as well as from the focus group; these represent at least a large minority of participants. One STEP participant, for instance, appreciated working with two different groups of first-year students: I realised that it’s actually nice in terms of us learning to teach to have a second group, because we had to repeat what we had just done, and we had to do it in a little bit different context, because we had new students, so we could refine things we had learned before . . . and for me, when I’m learning something, I need repetition. Overall, students appreciated the experience: I have definitely learned that this is not as easy as I thought it was . . . Definitely after this, I definitely want . . . teaching to be a big part of my career and life.


An indirect measure of final-year students’ reaction to the value of the experience was their attendance. We observed excellent attendance (94% overall) in the STEP group, with more sporadic attendance (close to 50%) in the group without teaching training.


Kirkpatrick level II: Learning In pre- and post-course survey questions on self-reported confidence in nine different teaching activities using a numerical Likert scale (1 ¼ ‘‘extremely uncomfortable’’; 5 ¼ ‘‘extremely comfortable’’), 8 of 11 student–teacher aggregate scores for all 9 items were significantly higher after experiencing the curriculum. For individual confidence items, scores for giving oral feedback, giving written feedback, working with the difficult learner, and mentoring were significantly higher following the curriculum (Tables 3 and 4). We deliberately included survey items on teaching topics not offered in STEP (such as giving large-group lectures) as control questions. Most students qualitatively reported increased confidence. For instance, one student–teacher in the focus group noted, ‘‘I think my teaching has definitely improved and I feel more confident.’’ Another was more specific:

I taught in college; maybe I could do it again’ whereas now I feel like I have some tools, that I sort of know what I’m doing. By the end of the course, over 90% of student–teachers were comfortable or extremely comfortable with the topics covered in the curriculum (Table 5). The topic with the greatest increase in confidence was giving oral feedback; three quarters of STEP participants increased their confidence, and no one decreased. The numerical patterns correlated with qualitative data: I think I, like a lot of people, always knew how to give positive feedback, but in terms of giving people constructive criticism, and helping people make positive changes . . . it was really nice to see that I can do that. Attitudes about teaching also changed: . . . I learned a lot. I was sort of surprised by how much science is behind it. Really I just thought you just show up, teach people how to do something. It’s pretty simple, right? But I didn’t realise . . . all of the nuances, and the science behind it . . . , I was pretty happy with the way it went.

[teaching in this course] got me sort of excited about residency where I know I’ll be teaching a lot. I feel more, definitely more confident, where at the beginning of the course I was like, ‘well maybe,

Table 5. Average group confidence scores. Table 3. Scores on Likert scales on pre- and post-course questionnaires.

Survey item (teaching skill)

Wilcoxon matched pair rank sum test

One-to-one teaching Small group teaching Large group lecturing Giving oral feedback Giving written feedback Marking written assignments Designing lesson plans Working with the difficult learner Mentoring

p40.05 p40.05 p40.05 p ¼ 0.005 p ¼ 0.01 p40.05 p40.1 p ¼ 0.01 p ¼ 0.01

Average group Average group Change from confidence confidence before score before score after to after course course p-value One-to-one teaching Small group Large group Oral feedback Written feedback Written assignment Lesson plan Difficult learner Mentor

4.45

4.73

0.192

4.18 3.00 3.55 3.73 3.82 3.00 2.91 3.55

4.55 3.27 4.91 4.73 4.36 3.27 4.00 4.64

0.103 0.391 0.0013 0.0041 0.0519 0.341 0.0061 0.0061

Student’s two-tailed t-test. Bold values are statistically significant.

Bold values are statistically significant.

Table 4. Change in teaching confidence before and after STEP.

One-to-one teaching Small group Large groupa Oral feedback Written feedback Written assignment Lesson plana Difficult Learner Mentor a

Number of teachers with no change

Number of teachers with increase

Number of teachers with decrease

% Increase or no change

% Increase

6 5 6 3 4 4 5 4 4

4 5 3 8 7 6 4 7 7

1 1 2 0 0 1 2 0 0

91 91 82 100 100 91 82 100 100

36 45 27 73 64 55 36 64 64

Not covered in STEP; control question. Bold values are statistically significant.

327


For example, a group of first-year students described their top teacher as

Similarly:


I definitely think it’s been a good experience. I have definitely learned that this is not as easy as I thought it was. You know sometimes, you think that if you’re, like, a personable person it will be easy to talk to people and come up with good advice, but it definitely . . . necessitates a lot more thoughtfulness.

. . . flexible and understanding, and provided a lot of support . . . very approachable . . . nonthreatening . . . very sensitive to everyone’s needs and different styles . . . I would feel comfortable to talking to [her] about almost anything and feel very comfortable asking her questions . . . [She] facilitates discussion well and lets everyone contribute . . .

We categorised qualitative data of self-report into themes, which represent at least a large minority of data, as demonstrated in Table 6. Many faculty observations of student–teachers’ abilities corresponded to student–teachers’ self-assessments.

Kirkpatrick level III: Behaviour

As for educational content, students commented that the best teacher expertly explained the topic each week . . . did a great job focusing us on the task at hand . . . sometimes she covers the material even better than in lecture.

First-year student assessments of their teachers At the midpoint and at the end of the course, first-year students rated their teachers on a 1–5 Likert scale and also provided descriptive comments. Table 7 shows two representative teachers from the top, middle, and bottom third of teachers as sorted by numeric rating. Qualitative comments about teachers reflected the numeric ratings given by first-year students to their teachers. Comments tended to cluster around certain themes, such as a teacher’s ability to foster an effective and safe learning environment, or a teacher’s skill at giving feedback.

Table 6. Confidence themes from post-course questionnaires: ‘‘In what ways are you confident in teaching?’’.

Theme

Example

Attitude: Confidence

Skill: Small group facilitation

Skill: Needs assessment Skill: Difficult situations Skill: Specific teaching tools

‘‘[I gained] confidence in my ability’’ ‘‘I think my teaching has definitely improved and I feel more confident’’ ‘‘I think I am now better at managing a small group and encouraging group participation’’ ‘‘I became more proficient at . . . assessing needs of students/peers’’ ‘‘problem-solving solutions in difficult situations’’ ‘‘[I] learned teaching techniques . . . I got immediate practice with techniques which helped reinforce them’’

Table 7.

Likert scale rating (in thirds) Top Top Mid Mid Bottom Bottom

328

A near-peer teacher as a role model was also the theme of first-year student comments: [Our teacher was] smart and confident but also humble and understanding . . . [she] was a wonderful role model and I hope she continues to teach throughout her training . . . I am hope I am like her as a 4th year! Conversely, teachers earning lower ratings received comments highlighting a less effective learning environment: [He] listens to us when we have concerns but I’m not sure he completely empathises or takes our opinions into consideration . . . The group feels safe but very forced. [He] rarely gives us constructive criticism and always just asks how we thought we did. The group discussions could have been more . . . interactive . . . He didn’t care too much about what we had to say.

Faculty assessment of student–teachers Faculty field notes corroborated assessments by students, especially for high and low performers. Faculty observed that the two lowest-rated student–teachers on the first-year student assessments had difficulties leading their small groups due to problems with giving feedback and managing a difficult learner.

Ratings of top, middle and bottom teachers.

Percentage of questions on which teachers earned 5 of 5 on Likert scale (%)

Percentage of questions on which teachers earned 4 of 5 (%)

Percentage of questions on which teachers earned 3 of 5 or below (%)

100 100 99 97 90 75

0 0 1 3 5 15

0 0 0 0 5 10


Conversely, the field notes contained examples of the teachers most highly rated by students demonstrating achievement of STEP learning objectives. For instance, one teacher deliberately used a specific feedback model; another explicitly laid out that day’s goals.


Kirkpatrick level IV: Results Examining first-year students’ OSCE scores as an overall group revealed no statistically significant differences among the various sites. However, the top individual student–teachers as rated by their students’ Likert ratings were also the teachers whose students performed at the highest level on the OSCE. Correspondingly, the students of the lowest-rated teachers achieved the lowest OSCE scores. Teachers whose students achieved the highest scores on the OSCE also had uniformly positive comments from their pupils; the teachers of the bottom-performing students on the OSCE had mixed or negative comments. Faculty observed that these same highly-rated STEP participants demonstrated competence both in STEP tutorial and in actual teaching practice. Specifically, they were noted to effectively give feedback, use the interview teaching techniques taught in seminar, set expectations, perform needs assessment and submit detailed written summative assessments. In contrast, faculty noted that the lowest-ranked teachers—again, whose students were the bottom performers on the OSCE—gave suboptimal feedback and had more difficulty facilitating their small groups.

Discussion We found that a curriculum to develop teaching knowledge, skills and attitudes for student–teachers corresponded with improvements in their teaching confidence, observable teaching behaviours and outcomes of their students. There was good correlation among data from written survey, descriptive comments, focus group and direct observation. The majority of STEP participants found the didactic programme valuable, and nearly all indicated that they would teach in this course again. The four skills showing greatest increase in teacher self-confidence were giving oral feedback, giving written feedback, working with a difficult learner, and mentoring. These four areas had been emphasised heavily in the teachers’ curriculum via a variety of teaching methods. STEP participant teaching competence improved in these areas: facilitating small groups, using specific teaching tools, creating a safe learning environment, optimising social congruence, giving useful feedback, performing needs assessment and demonstrating enthusiasm for teaching. Student– teachers attributed specific gains in competence to STEP. Measurements of competence using the ‘‘objects’’ of teachers’ teaching, namely, the first-year students, revealed striking correlations. An excellent teacher should have a positive impact on his or her learners (Griffith et al. 2000); in this course, an indirect measure of how well students were taught may be their high-stakes OSCE scores. At the extremes, first-year students’ numerical and descriptive ratings of their teachers correlated with the students’ OSCE

performance. These top teachers were also observed by faculty to demonstrate teaching competence. It appears, then, that first-year students were able to identify teachers whose groups were going to perform lower. This phenomenon corroborates previous findings that students can identify teachers who have been trained (Edwards et al. 1988). These results suggest that peer teachers who demonstrate competence at the Learning (in seminar) and Behaviour (actual teaching practice) levels according to faculty assessment not only are identified by their students as being excellent teachers, but also deliver on the important outcome of preparing their students for high performance. Excellent teachers who are rated well by their students also result in clinical excellence (OSCE success) of their learners; therefore, Learning and Behaviour evaluations may be predictive of the more difficult to measure Results-level evaluations. These findings support previous evidence that students who are trained to teach enjoy it more, are more effective teachers (Blatt & Greenberg 2007; Pasquinelli & Greenberg 2008), and are more prepared for future teaching roles as residents (Resnick & MacDougall 1976; Topping 1996; Josephson & Whelan 2002; Sobral 2002; Tang et al. 2004). Data from the current study also bear out prior theories of cognitive and social congruence; students learn well from peers, because they use accessible language to describe concepts and relate to the feeling of being a novice. Given the obvious progression from student to junior doctor, trainees must be actively prepared for their future roles as teachers while still in medical school. After all, the reality of the wards is that junior doctors will carry great responsibility for the learning of their students, and peer congruence dictates that they will have great influence on their students. Indeed, students cite residents as their most impactful educators (Mann et al. 2007). This study furthered existing knowledge in three important ways. First, models of student-as-teacher programmes are still not fully described in existing literature. The present study enhanced the current understanding of the effectiveness of such programmes via a multi-level curriculum evaluation. Most prior research on student-as-teacher programmes describes the Reaction level of Kirkpatrick (1959)’s pyramid of curricular evaluation, discussing whether student–teachers enjoyed or appreciated the experience. There are few data on higher-level outcomes such as Learning (whether student–teachers demonstrate principles of teaching), Behaviour (whether trained student–teachers implement newly-gained teaching skills in the clinical setting), or Results (whether learners of trained student–teachers perform better). These gaps in the student-asteacher literature called for further investigation to evaluate and study the effects of a didactic plus experiential programme preparing students how to teach. Although Best Evidence Medical Education (BEME) recommends designing and evaluating educational programmes according to the best available evidence, just as clinical practice guidelines should be based on best available medical evidence (Harden et al. 2000), only 12% of educational literature actually evaluates curricula at the highest two Kirkpatrick levels, typically due to difficulty of analysis (Yardley & Dornan 2012). The present study sought to evaluate higher levels on the Kirkpatrick pyramid: achievement

329



of STEP objectives by participants (Learning), demonstration of teaching skills in actual teaching practice (Behaviour), and even the first-year students’ outcomes (Results). Second, few institutions have studied a longitudinal programme such as the present model. A 2009 review of 29 US resident-as-teacher studies found that although residents of all levels have undergone some teaching training, most training is brief, not longitudinal, and optional (Hill et al. 2009); moreover, similar teaching training programmes for students lag far behind (Henry et al. 2006). The current study designed and examined a longitudinal curriculum spanning three months, rather than a concentrated block or a one-day course; students learning to become teachers need multiple exposures over time to gain competence in teaching. Finally, the present programme uniquely enmeshed a didactic teaching component within a real teaching experience for final-year medical students such that didactic learning about teaching was reinforced with immediate teaching practice, under faculty supervision. This feature is critical given our existing knowledge that graduating medical students typically feel uncomfortable with their teaching skills, that teaching skills can be taught and that formal teaching training can impact both teachers and their learners.

Limitations Because the primary study author served many roles, during STEP seminars, she had to simultaneously instruct and record notes. During live student teaching, all 12 groups operated concurrently, so the investigator could observe only student– teacher at a time. Details of the control groups’ experiences also remain unknown, again, because the investigator was observing the intervention group. Additionally, to preserve anonymity, we were not able to triangulate individual student– teacher survey data with field observations. At the intervention site, attendance of teachers was higher than at control sites, so STEP participants took part in a more stable group process than the control groups, which may have positively contributed to their gains as teachers. Better attendance at the intervention site may have itself been due to the STEP programme; the orientation to the student–teacher experience emphasised the importance of a teacher’s role in the education of students. At the control sites, final-year students may not have viewed their group leader role with the same level of significance and thus missed more sessions. Another limitation of the study is that the control groups may not have been pure controls, in that they also likely discussed some teaching concepts even if not intentionally or formally. There may also have been cross-contamination if STEP participants discussed their curriculum content with classmates who were control group student–teachers. Finally, the range of numerical data was limited for certain items. OSCE scores were uniformly high, regardless of intervention versus control groups; the limited range of marks limited the ability to analyse the data. Two STEP participants’ final post-course surveys were lost to follow-up, reducing the data. 330

Conclusion Because of the many benefits demonstrated in the current project and in prior publications, and because medical students will shortly need to supervise, teach, and assess the next generation of students, there is a recommendation that all medical schools using medical students as teachers in any capacity offer those students formal training in teaching (Soriano et al. 2010). The present findings may help other institutions create or improve teaching-to-teach programmes for medical students. As for how to design a programme for training student– teachers how to teach, our embedded programme is one model. We believe that the two key critical elements of success were, first, interleaving sessions devoted to didactic educational principles—such as small group facilitation theory and techniques—into a practical teaching experience with actual students in a real course, and, second, direct faculty guidance in developing knowledge, skills, and attitudes of teaching. The faculty guidance component confers validity to the final-year students as teachers and evaluators. If these concepts were to be transferred to peer teaching in an anatomy course, for instance, the content would need modification, but the structure, with didactic components sandwiching the practical session, could be applied easily. Much as attending lectures on the rules of the game would not turn a novice into a tennis player, and neither would merely attempting to hit the ball without knowledge of how to hold the racquet or learning the rules, so too does learning how to teach require both theory and practice. Regardless of the technique, tomorrow’s physician–educators need a deliberate approach to learning how to teach. Declaration of interest: The authors have no declarations of interest.

Notes on Contributors DEBORAH R. ERLICH, MD, MMedEd, is residency programme director of Carney Family Medicine Residency in Boston, Massachusetts, assistant professor at Tufts University School of Medicine and clinical instructor at Harvard Medical School, both in Boston. She completed a family medicine residency, faculty development fellowship and master’s degree in medical education. ALLEN F. SHAUGHNESSY, PharmD, MMedEd, is a professor at Tufts University School of Medicine and director of the Master Teacher Fellowship at Tufts Family Medicine Residency at Cambridge Health Alliance in Malden, Massachusetts. He completed a faculty development fellowship and the Department of Health and Human Services primary care health policy fellowship.

Glossary Peer teacher: Person from similar social groupings who is not a professional teachers helping others to learn and learning themselves by teaching Reference: Topping KJ. 1996. The effectiveness of peer tutoring in further and higher education: A typology and review of the literature. Higher Educ 32(3):321–345.



References Alcamo AM, Davids AR, Way DP, Lynn DJ, Vandre DD. 2010. ‘The impact of a peer-designed and -led USMLE Step 1 review course: Improvement in preparation and scores’. Acad Med: J Assoc Am Med Coll 85(10 Suppl):S45–S48. Baillie S, Shore H, Gill D, May SA. 2009. ‘Introducing peer-assisted learning into a veterinary curriculum: A trial with a simulator. J Vet Med Educ 36(2):174–179. Bannard-Smith J, Bishop S, Gawne S, Halder N. 2012. Twelve tips for junior doctors interested in a career in medical education. Med Teach 34(12):1012–1016. Barnes HV, Albanese M, Schroeder J, Reiter S. 1978. Senior medical students teaching the basic skills of history and physical examination. J Med Educ 53(5):432–434. Bentley BS, Hill RV. 2009. Objective and subjective assessment of reciprocal peer teaching in medical gross anatomy laboratory. Anatom Sci Educ 2(4):143–149. Bing-You RG, Greenberg LW. 1990. Training residents in clinical teaching skills: A resident-managed program. Med Teacher 12(3–4):305–309. Blatt B, Greenberg L. 2007. A multi-level assessment of a program to teach medical students to teach. Adv Health Sci Educ 12(1):7–18. Bloom BS. 1956. Taxonomy of educational objectives; the classification of educational goals, 1st ed. New York: Longmans, Green. Boynton PM. 2004. Administering, analysing and reporting your questionnaire. Br Med J 328:1372–1375. Bulte C, Betts A, Garner K, Durning S. 2007. Student teaching: Views of student near-peer teachers and learners. Med Teach 29(6):583–590. Burke J, Fayaz S, Graham K, Matthew R, Field M. 2007. Peer-assisted learning in the acquisition of clinical skills: A supplementary approach to musculoskeletal system training. Med Teach 29(6):577–582. Correa R, Brugal Y, Valentin L, Perez J, Perez-Guma J. 2009. ‘Peer-teaching: An effective learning experience?’ Puerto Rico Health Sci J 28(1):83–83. Dandavino M, Snell L, Wiseman J. 2007. Why medical students should learn how to teach. Med Teacher 29(6):558–565. Dickson JM, Harrington R, Carter MJ. 2011. Teaching clinical examination using peer-assisted learning amongst graduate-entry students. Clin Teach 8(1):8–12. Drouin J, Denis M, Nadeau L, Chenier Y. 2006. Medical students as teachers and role models for their future colleagues. Med Teach 28(7):618–624. Edwards JC, Kissling GE, Plauche WC, Marier RL. 1988. Evaluation of a teaching skills improvement programme for residents. Med Educ 22(6):514–517. Field M, Burke J, Lloyd D, McAllister D. 2004. Peer-assisted learning in clinical examination. Lancet 363(9407):490–491. Field M, Burke JM, McAllister D, Lloyd DM. 2007. Peer-assisted learning: A novel approach to clinical skills learning for medical students. Med Educ 41(4):411–418. Giuliodori MJ, Lujan HL, DiCarlo SE. 2006. Peer instruction enhanced student performance on qualitative problem-solving questions. Adv Physiol Educ 30(4):168–173. Graham K, Burke JM, Field M. 2008. Undergraduate rheumatology: Can peer-assisted learning by medical students deliver equivalent training to that provided by specialist staff? Rheumatol 47(5):652–655. Greenberg LW, Jewett LS. 1987. Preparing medical students to teach: An educational program using three approaches. Med Teach 9(4):409–414. Griffith III CH, Wilson JF, Haist SA, Ramsbottom-Lucier M. 1997. Relationships of how well attending physicians teach to their students’ performances and residency choices. Acad Med: J Assoc Am Med Coll 72(10 Suppl 1):S118–S120. Griffith CH, Georgesen JC, Wilson JF. 2000. Six-year documentation of the association between excellent clinical teaching and improved students’ examination performances. Acad Med 75(10):S62–S64. Haist SA, Wilson JF, Brigham NL, Fosson SE, Blue AV. 1998. Comparing fourth-year medical students with faculty in the teaching of physical examination skills to first-year students’. Acad Med: J Assoc Am Med Coll 73(2):198–200. Harden RM, Grant J, Buckley G, Hart IR. 2000. ‘Best evidence medical education’. Adv Health Sci Educ: Theory Pract 5(1):71–90.

Henry BW, Haworth JG, Hering P. 2006. ‘Perceptions of medical school graduates and students regarding their academic preparation to teach. Postgrad Med J 82(971):607–612. Hill AG, Yu TC, Barrow M, Hattie J. 2009. A systematic review of residentas-teacher programmes. Med Educ 43(12):1129–1140. Hill E, Liuzzi F, Giles J. 2010. Peer-assisted learning from three perspectives: Student, tutor and co-ordinator. Clin Teach 7(4):244–246. Houwink AP, Kurup AN, Kollars JP, Kollars CAK, Carmichael SW, Pawlina W. 2004. Help of third-year medical students decreases first-year medical students’ negative psychological reactions on the first day of gross anatomy dissection. Clin Anat 17(4):328–333. Hudson JN, Tonkin AL. 2008. Clinical skills education: Outcomes of relationships between junior medical students, senior peers and simulated patients. Med Educ 42(9):901–908. Jafri W, Mumtaz K, Burdick WP, Morahan PS, Freeman R, Zehra T. 2007. Improving the teaching skills of residents as tutors/facilitators and addressing the shortage of faculty facilitators for PBL modules. BMC Med Educ 7:34. Johansen ML, Martenson DF, Bircher J. 1992. Students as tutors in problembased learning: Does it work? Med Educ 26(2):163–165. Josephson SA, Whelan AJ. 2002. A new first-year course designed and taught by a senior medical student. Acad Med 77(12):1207–1211. Juenger J, Schultz JH, Schoenemann J, Wagener S, Drude N, Duelli R, Resch F. 2009. AMEE Guide Supplements: Peer-assisted learning: A planning and implementation framework. Guide supplement 30.6–practical application. Med Teach 31(1):55–56. Kassab S, Abu-Hijleh MF, Al-Shboul Q, Hamdy H. 2005. Student-led tutorials in problem-based learning: Educational outcomes and students’ perceptions. Med Teach 27(6):521–526. Kennedy TTJTAL. 2006. Making sense of grounded theory in medical education. Med Educ 40:101–108. Kerfoot BP, Nabha KS, Hafler JP. 2004. Designing a surgical ‘resident-asteacher’ programme. Med Educ 38(11):1190. Kibble J, Hansen PA, Nelson L. 2006. Use of modified SOAP notes and peer-led small-group discussion in a Medical Physiology course: Addressing the hidden curriculum. Adv Physiol Educ 30(4):230–236. Kim JE, Schickedanz AD, Chou CL. 2010. Near-peer workshops for preclerkship physical examination skills. Med Educ 44(5):499–500. Kirkpatrick DL. 1959. Techniques for evaluating training programs. J Am Soc Training Directors (Training and Development Journal) 13(11):3–9. Knobe M, Munker R, Sellei RM, Holschen M, Mooij SC, Schmidt-Rohlfing B, Niethard FU, Pape HC. 2010. Peer teaching: A randomised controlled trial using student-teachers to teach musculoskeletal ultrasound. Med Educ 44(2):148–155. Leeper H, Chang E, Cotter G, MacIntosh P, Scott F, Apantaku L, Broutman L, Lazarus C. 2007. A student-designed and student-led sexual-historytaking module for second-year medical students. Teach Learn Med 19(3):293–301. Lockspeiser TM, O’Sullivan P, Teherani A, Muller J. 2008. Understanding the experience of being taught by peers: The value of social and cognitive congruence. Adv Health Sci Educ 13(3):361–372. Mann KV, Sutton E, Frank B. 2007. Twelve tips for preparing residents as teachers. Med Teach 29(4):301–206. Maudsley G. 2011. Mixing it but not mixed-up: Mixed methods research in medical education (a critical narrative review). Med Teach 33(2):e92–e104. Mazur E. 2009. Education. Farewell, lecture? Science 323(5910):50–51. McAleer S. 1994. Designing a questionnaire. Educ Gen Pract 5:38–43. Morrison EH, Hafler JP. 2000. Yesterday a learner, today a teacher too: Residents as teachers in 2000. Pediatrics 105(1):238–241. Morse JMFPA. 1995. Qualitative research methods for health professionals, 2nd ed. Thousand Oaks: SAGE. Nnodim JO. 1997. A controlled trial of peer-teaching in practical gross anatomy. Clin Anat 10(2):112–117. Pasquale SJ, Pugnaire MP. 2002. Preparing medical students to teach. Acad Med J Assoc Am Med Coll 77(11):1175–1176. Pasquinelli LM, Greenberg LW. 2008. A review of medical school programs that train medical students as teachers (MED-SATS). Teach Learn Med 20(1):73–81.

331



Pepe EA, Hodel CG, Bosshart DA. 1980. Use of peers to teach interviewing and clinical problem-solving. J Med Educ 55(9):800. Perry ME, Burke JM, Friel L, Field M. 2010. Can training in musculoskeletal examination skills be effectively delivered by undergraduate students as part of the standard curriculum? Rheumatology 49(9):1756–1761. Rao SP, DiCarlo SE. 2000. Peer instruction improves performance on quizzes. Adv Physiol Educ 24(1):51–55. Resnick PJ, MacDougall E. 1976. The use of senior medical students as preceptors in freshman clinical science. J Med Educ 51(09):763–765. Rodrigues J, Sengupta A, Ramoutar D, Maxwell S. 2010. Are UK junior doctors being taught to teach? Med Educ 44(3):324. Salerno-Kennedy R, Henn P, O’Flynn S. 2010. Implementing peer tutoring in a graduate medical education programme. Clin Teach 7(2):83–89. Shiozawa T, Hirt B, Celebi N, Baur F, Weyrich P, Lammerding-Koppel M. 2010. ‘Development and implementation of a technical and didactical training program for student tutors in the dissection course’. Ann AnatAnatomischer Anzeiger 192(6):355–360. Smith MK, Wood WB, Adams WK, Wieman C, Knight JK, Guild N, Su TT. 2009. Why peer discussion improves student performance on in-class concept questions. Science 323(5910):122–124. Sobral DT. 2002. Cross-year peer tutoring experience in a medical school: Conditions and outcomes for student tutors. Med Educ 36(11):1064–1070. Soriano RP, Blatt B, Coplit L, CichoskiKelly E, Kosowicz L, Newman L, Pasquale SJ, Pretorius R, Rosen JM, Saks NS, Greenberg L. 2010. Teaching medical students how to teach: A national survey of students-

332

as-teachers programs in U.S. medical schools. Acad Med 85(11):1725–1731. Stern DT, Williams BC, Gill A, Gruppen LD, Woolliscroft JO, Grum CM. 2000. Is there a relationship between attending physicians’ and residents’ teaching skills and students’ examination scores? Acad Med: J Assoc Am Med Coll 75(11):1144–1146. Tang TS, Hernandez EJ, Adams BS. 2004. ‘‘Learning by teaching’’: A peer-teaching model for diversity training in medical school. Teach Learn Med 16(1):60–63. Ten Cate O. 2007. A teaching rotation and a student teaching qualification for senior medical students. Med Teach 29(6):566–571. Ten Cate O, Durning S. 2007. Dimensions and psychology of peer teaching in medical education. Med Teach 29(6):546–552. Topping KJ. 1996. The effectiveness of peer tutoring in further and higher education: A typology and review of the literature. Higher Educ 32(3):321–345. Vygotsky L. 1986. Thought and language. Cambridge: The MIT Press. Weidner TG, Popp JK. 2007. ‘Peer-assisted learning and orthopaedic evaluation psychomotor skills. J Athletic Train 42(1):113–119. Yardley S, Dornan T. 2012. Kirkpatrick’s levels and education ‘evidence’. Med Educ 46(1):97–106. Young L, Orlandi A, Galichet B, Heussler H. 2009. Effective teaching and learning on the wards: Easier said than done? Med Educ 43(8):808–817. Zijdenbos I, Fick T, ten Cate O. 2011. How we offer all medical students training in basic teaching skills. Med Teach 33(1):24–26.

Step by step.

Step-by-step Laparoscopic Vesiculectomy for Hemospermia.

Insights into HER2 signaling from step-by-step optimization of anti-HER2 antibodies.

Residents-as-teachers across graduate medical education - expanding into the undergraduate medical curriculum.

Step-by-Step to Achieve our Goals.

Golgi-Cox Staining Step by Step.

Recognising and developing students as teachers: Introduction of a novel Undergraduate Certificate in Veterinary Medical Education.

Is there a place for medical students as teachers in the education of junior residents?

Changing attitudes of medical teachers towards medical education.

Budget holding: a step into the unknown.

Confident assignment of site-specific glycosylation in complex glycoproteins in a single step.

Gene transfer into humans: a first step.

Gene set analysis: A step-by-step guide.

The meiotic checkpoint network: step-by-step through meiotic prophase.

Step-by-step fracture of two-layer stacked graphene membranes.

Surface Aesthetics in Tip Rhinoplasty: A Step-by-Step Guide.

Editorial Comment: Step-by-step Laparoscopic Vesiculectomy for Hemospermia.

[Operation methods: safe, logical and step by step].

Robotic surgery - a step into the future.

Preparing to take the USMLE Step 1: a survey on medical students' self-reported study habits.

Intraosseous access can be taught to medical students using the four-step approach.

A skills programme for preclinical medical students.

Step-by-step quantitative analysis of focal adhesions.

Accessing critical care big data: a step by step approach.