DOI 10.1515/jpm-2013-0247 J. Perinat. Med. 2014; 42(4): 479–486
Sumit Saraf*, Jyothshna Bayya, Jeremy Weedon, Howard Minkoff and Nelli Fisher
The relationship of praise/criticism to learning during obstetrical simulation: a randomized clinical trial Abstract Aims: The effect of positive vs. negative comments (praise vs. criticism) on trainees’ subsequent cognitive and technical performance is unknown, but of potential importance. We performed a randomized trial of giving either praise or criticism during simulated normal vaginal deliveries (using a high-fidelity birthing simulator) to assess the differential effect of these types of comments on students’ cognitive and technical performance, and perceived confidence after their learning experience. Methods: Medical and nursing students underwent stratified randomization to praise or criticism. Students (n = 59) initially participated in a teaching demonstration and practiced normal spontaneous vaginal delivery using a birthing simulator. A baseline assessment of cognitive and technical skills, and of self-confidence, was followed by a second simulation during which positive or negative comments were given using standardized scripts. Cognitive performance, technical performance and confidence measures were then scored again. Results: Cognitive and technical performance scores in the “praise” group improved significantly by 2.5 (P = 0.007) and 1.8 (P = 0.032), respectively, while those in the “criticism” group remained unchanged. The self-reported confidence scores did not show any significant change from baseline in either group. Conclusions: Praise strengthens students’ cognitive and technical performances, while criticism does not. Keywords: Criticism; feedback; learning; obstetrics and gynecology; praise; resident education; simulation; teaching. *Corresponding author: Sumit Saraf, MD, Maimonides Medical Center, 967 48 St., Brooklyn, NY 11219, USA, Tel.: +1 718 283 7048, Fax: +1 718 283 8468, E-mail: [email protected] Sumit Saraf, Jyothshna Bayya and Nelli Fisher: Department of Obstetrics and Gynecology at Maimonides Medical Center, New York, NY, USA Jeremy Weedon: Department of Epidemiology/Biostatistics, State University of New York Downstate Medical Center, Brooklyn, NY, USA
Howard Minkoff: Department of Obstetrics and Gynecology at Maimonides Medical Center, Brooklyn, NY, USA; and Professor of Obstetrics and Gynecology at SUNY Downstate Medical Center, Brooklyn, NY, USA
Introduction Behavioral research has revealed a great deal about the effect of the environment on learning, how new behaviors are learned, and what motivates people to change or remain the same. That environment includes the behavior and attitude of colleagues, as well as the decorum and the milieu of the clinical workplace (e.g., the operating or delivery room). What is less clear is which environment provides the best opportunity for learning: one that is stressful and hostile, such as military school teaching, or one that is facilitative and encouraging. For many years, some physicians have subscribed to an unwritten rule of medical pedagogy, i.e., learning under stress is the optimal means of clinical training. Zakay and Wooler [26] have suggested that training under nonstress conditions does not improve task performance when the task itself has to be performed under stress conditions and recommended stress-inclusive training or simulations. The stress of criticism induced by clinical teachers (e.g., use of punishment, pressure and criticism) was justified by the theory that the trainee’s actual clinical environment would be similarly stressful. However, stress may be unfavorable for learning. Severe life stress hampers spatial learning in animals [21], and childhood stress can lead to behavioral disorders later in life [20]. The question of whether punitive and critical comments (criticism) or encouraging and nurturing comments (praise) best strengthens the clinicianlearner’s targeted behavior in clinical settings remains unanswered. While a great deal has been published about feedback, what generally happens in a clinical setting does not always meet that definition. Comments from senior physicians to trainees might be more appropriately categorized as “praise” or “criticism.” In medical education,
Brought to you by | University of Manitoba Authenticated Download Date | 6/6/15 1:49 AM
480 Saraf et al., Praise/criticism during obstetric simulation feedback is one of the crucial elements of learning [10]. Studies have concluded that effective feedback is necessary, valuable and critical for clinical performance improvement [10] and for the acquisition of clinical skills [11] by students and residents. It is associated with students’ perception of high-quality learning [25]. However, much less is known about praise or criticism, which is ubiquitous in clinical settings, and the differential impact of praise and criticism on teaching in obstetrics has not, to our knowledge, been previously assessed. We chose simulation to study the effect of praise/ criticism on learning in different types of learning environments. Simulation is increasingly used as an effective learning tool [24]. In obstetrics, simulation has been shown to improve knowledge-based scores [9], safety attitudes [12], technical skills [7], communication skills [4] and perceived competence [22, 23]. However, studies to date have not assessed the effect of different pedagogic approaches (praise or criticism) on trainee learning in a simulated environment. Our objectives were to compare the cognitive and technical performance of students randomized to receive either praise or criticism during a simulated normal vaginal delivery and to assess students’ perceived confidence in their abilities after their learning experience. We hypothesized that there would be an overall increase in cognitive and technical scores for the “praise” group and a decrease in all scores for the “criticism” group.
NSVD lecture+ Demonstration (N=59)
Methods Design We conducted a randomized clinical trial approved by the Maimonides Medical Center’s Institutional Review Board. We enrolled medical students (n = 40) and nursing students (n = 19) present at the hospital between June 2010 and May 2011. Only students who had not witnessed and had not performed a normal spontaneous vaginal delivery (NSVD) were included. Teaching and evaluation were performed with the use of Noelle S575 Gaumard Scientific, FL, USA, advanced birthing simulator.
Pilot and randomization We conducted a pilot study (n = 10), to assess the appropriateness of the scripts. The pilot study included students randomized to praise (n = 5) and criticism (n = 5), and assessed their performance using a previously published checklist [6]. The pilot confirmed the feasibility of the study and the participants’ understanding of all instruments. The study protocol is shown in Figure 1. Block randomization was performed with stratification by profession, i.e., medical or nursing student. A computer-generated table of random numbers and sequentially numbered opaque sealed envelopes containing allocation cards were used to achieve randomization, designating group assignment to praise or criticism groups. The students were blinded to the purpose of the study and to their group assignments.
Consent A day before the simulation session, we distributed handouts to all participants, which described the steps used in the performance of
Participant practice (Simulation)
Simulation 1 Baseline assessment
Medical students N=40
Praise group (N=19)
Nursing students N=19
Criticism group (N=21)
Praise group (N=9)
Criticism group (N=10)
Simulation 2 Intervention/Second assessment Debriefing
Figure 1 Materials and methods – Flowchart utilized for randomization and recruitment of participants.
Brought to you by | University of Manitoba Authenticated Download Date | 6/6/15 1:49 AM
Saraf et al., Praise/criticism during obstetric simulation 481 a normal vaginal delivery [5]. On the day of simulation, we obtained informed consent, demographic data and distributed the sealed opaque envelopes. The consent informed the participants both about simulation of normal vaginal deliveries and that they might be video taped (a subset was videotaped to allow validation of the scores rendered by proctors in the room).
The instrument Next, we showed the steps to perform an NSVD. Each student practiced in a simulated NSVD and had an opportunity to ask questions. During both the baseline and the intervention sessions (see below), two board-certified obstetricians were in the room with the students. One served as the proctor and the second scored the students. After the teaching demonstration and practice session, a baseline assessment was undertaken with each student performing an NSVD simulation. No comments were provided during the baseline simulation. The trainees were scored using a checklist that was a modification of a standardized published checklist [6]. This checklist (Figure 2) assessed cognitive (knowledge) and technical parameters (applied performance). At the completion of the baseline simulation, the students filled out a confidence survey based on a previously published Likert scale survey [6].
Sr. No. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Interventions/Checklist Introduces self to patient Asks for parity Asks for gestational age Asks for medical problems Assesses fetal tracing Assesses clinical EFW Calls for supervisor help Pelvic exam: Cervical dilatation Pelvic exam: Cervical effacement Pelvic exam: Cervical station Instructs patient how to push Controls delivery of the head Supports the perineum Notes the time of delivery of head Asks to stop pushing (after delivery of head) Checks for nuchal cord Suctions the nose and mouth Checks and delivers the anterior shoulder Delivers posterior shoulder Delivers rest of the body Clamps and cuts the cord Waits for signs of placental separation Delivers placenta using suprapubic pressure Asks for pitocin postpartum Provides fundal massage Inspects perineum for lacerations Inspects placenta Assesses and announces blood loss Asks for fetal well being and APGAR scores Counsels and talks to the patient Total score
These checklists and scoring systems were constructed based on three of Kirkpatrick’s [14] four levels of learning evaluations, an established method for judging learning processes. The levels of learning assessed were as follows: (A) Reaction (how the participants felt) – confidence score (Likert self-report scale); (B) Learning (increase in knowledge) – cognitive score; and (C) Behavior (applied learning, i.e., performance) – technical score. We did not assess the fourth of Kirkpatrick’s levels – Results, i.e., long-term outcome. During the intervention simulation performed later on the same day, each participant again performed a simulated NSVD. However, this time, based on their random group assignment, the proctor provided one of two interventions; either scripted praise or criticism (Figure 3). The scripts included comments at initial proctor greetings and then at five predetermined points during the NSVD. Comments varied depending on whether the trainee had performed the task correctly. For example, after a correct placental delivery using the traction/counter-traction method, a participant in the praise arm would be told, “Excellent, that’s the classic way to deliver the placenta.” A participant in the criticism arm would be told, “OK, that’s fine, but hurry up and keep moving.” If a participant in the praise arm performed an incorrect placental delivery she/he would be told, “Good job, but make sure you apply suprapubic pressure for counter traction,” while a participant in the criticism arm would be told, “That
Cognitive assessment
Technical assessment
/28
/22
Figure 2 Checklist 1 – Baseline simulation checklist. Assessment of students’ technical and cognitive baseline performance.
Brought to you by | University of Manitoba Authenticated Download Date | 6/6/15 1:49 AM
482 Saraf et al., Praise/criticism during obstetric simulation Sr. No
1 2 3 4
5 6 7 8 9 10
11 12 13
14 15 16 17 18 19 20 21 22 23
24 25 26 27 28 29 30
Interventions/Checklist
Cognitive assessment
Technical assessment
(Outside the room) What’s your name? Which medical school do you attend? You did/did not perform well the last time. I was satisfied/disappointed I hope you realize this is/is not going to affect your grades. Introduces self to patient Asks for parity Asks for gestational age Asks for medical problems a) Wonderful, you requested all the appropriate information b) Great introduction, it is also important to ask about… (mention what they missed) c) Keep going, you did fine, but we need to get to the point d) This is not how you do it. You have to ask specific questions, for example if she has a heart failure, are you going to let her push? Assesses fetal tracing Assesses clinical EFW Calls for supervisor help Pelvic exam: Cervical dilatation Pelvic exam: Cervical effacement Pelvic exam: Cervical station a) Excellent, that’s exactly how you do a complete pelvic exam b) Excellent start just let me show you how you could perfect your exam. c) Ok, your exam was fine, that does not mean that you can deliver the baby easily, let’s see. d) Watch out how you do it. You can cause lacerations and make her uncomfortable. ( shows) Instructs patient how to push Controls delivery of the head Supports the perineum a) Great job- That’s exactly how you support the perineum b) Overall you are doing great, but let me show you how to do this better c) Yeah OK, finally you did something right d) What do you think you are doing? you are not supporting the perineum properly and this could results in more injuries Notes the time of delivery of head Asks to stop pushing (after delivery of head) Checks for nuchal cord Suctions the nose and mouth Checks and delivers the anterior shoulder Delivers posterior shoulder Delivers rest of the body Clamps and cuts the cord Waits for signs of placental separation Delivers placenta using suprapubic pressure(traction/counter traction) a) Excellent, That’s the classic way of delivering the placenta b) Good job, but make sure you apply the suprapubic pressure for counter traction c) OK, that’s fine. But hurry up and keep moving ahead . d) This is totally wrong, without suprapubic pressure, the uterus is going to flip inside out and the mother is going to bleed to death Asks for pitocin postpartum Provides fundal massage Inspects perineum for lacerations Inspects placenta Assesses and announces blood loss Asks for fetal well being and APGAR scores Counsels and talks to the patient Total score
/28
/22
Figure 3 Checklist 2 – Intervention simulation checklist. Assessment of students’ technical and cognitive post-intervention performance. was totally wrong, without suprapubic pressure, the uterus is going to flip inside out and the mother is going to bleed to death.” Whenever technical errors were observed by trainees in either group, the proctor immediately demonstrated the correct way of performing the task. The scoring provider did not have any interaction with students during either the baseline or the intervention simulation. The same scoring checklist that was used for baseline assessment of cognition and technical skills was again utilized for the intervention assessment, and that assessment was followed by a second self-scored
Likert confidence survey. The participants were debriefed about the study details and its purpose at the end of participation. The “change in score” from baseline to intervention, obtained from the praise group, was compared with that obtained from the criticism group for cognition, technical skills and confidence. We used a video-recording device to mitigate possible grading bias by the scoring provider who heard the proctor’s comments during the second simulated NSVD and in order to determine the reliability of the scoring assessments. A separate reviewer, blinded to the
Brought to you by | University of Manitoba Authenticated Download Date | 6/6/15 1:49 AM
Saraf et al., Praise/criticism during obstetric simulation 483
participants’ group assignment, evaluated video recordings of the subset (20%) of participants. This blinded reviewer scored students’ performance by watching the simulation videos, which had been edited so that the comments were not heard, and the provider giving the comments was not seen.
Statistical analysis Demographic differences between participants in the two arms were evaluated with two-way frequency tables and Fisher exact tests. Then, for each of the three outcome measures (cognitive, technical and self-reported confidence) as dependent variable, a mixed linear model was constructed with profession (medical, nursing), assessment time (baseline, intervention), study arm (praise, criticism) and subject gender. An unstructured within-subject covariance matrix was modeled. Satterthwaite corrections were applied to denominator degrees of freedom. Non-significant fourth-order terms were excluded from models. Model residuals were inspected for outliers and skew. Reported means for outcome measures were model estimated and, therefore, properly adjusted for differences between genders and professions. Since there were no relevant studies from which to project effect size, we conducted an interim analysis of the first 25 subjects. That data suggested that 25 subjects in each group would provide 80% power to detect a significant difference for our principal outcomes (cognitive and technical scores) using two-way analysis of variance with significance level set at 0.05. One outlier with extremely low,
Sumit Saraf*, Jyothshna Bayya, Jeremy Weedon, Howard Minkoff and Nelli Fisher
The relationship of praise/criticism to learning during obstetrical simulation: a randomized clinical trial Abstract Aims: The effect of positive vs. negative comments (praise vs. criticism) on trainees’ subsequent cognitive and technical performance is unknown, but of potential importance. We performed a randomized trial of giving either praise or criticism during simulated normal vaginal deliveries (using a high-fidelity birthing simulator) to assess the differential effect of these types of comments on students’ cognitive and technical performance, and perceived confidence after their learning experience. Methods: Medical and nursing students underwent stratified randomization to praise or criticism. Students (n = 59) initially participated in a teaching demonstration and practiced normal spontaneous vaginal delivery using a birthing simulator. A baseline assessment of cognitive and technical skills, and of self-confidence, was followed by a second simulation during which positive or negative comments were given using standardized scripts. Cognitive performance, technical performance and confidence measures were then scored again. Results: Cognitive and technical performance scores in the “praise” group improved significantly by 2.5 (P = 0.007) and 1.8 (P = 0.032), respectively, while those in the “criticism” group remained unchanged. The self-reported confidence scores did not show any significant change from baseline in either group. Conclusions: Praise strengthens students’ cognitive and technical performances, while criticism does not. Keywords: Criticism; feedback; learning; obstetrics and gynecology; praise; resident education; simulation; teaching. *Corresponding author: Sumit Saraf, MD, Maimonides Medical Center, 967 48 St., Brooklyn, NY 11219, USA, Tel.: +1 718 283 7048, Fax: +1 718 283 8468, E-mail: [email protected] Sumit Saraf, Jyothshna Bayya and Nelli Fisher: Department of Obstetrics and Gynecology at Maimonides Medical Center, New York, NY, USA Jeremy Weedon: Department of Epidemiology/Biostatistics, State University of New York Downstate Medical Center, Brooklyn, NY, USA
Howard Minkoff: Department of Obstetrics and Gynecology at Maimonides Medical Center, Brooklyn, NY, USA; and Professor of Obstetrics and Gynecology at SUNY Downstate Medical Center, Brooklyn, NY, USA
Introduction Behavioral research has revealed a great deal about the effect of the environment on learning, how new behaviors are learned, and what motivates people to change or remain the same. That environment includes the behavior and attitude of colleagues, as well as the decorum and the milieu of the clinical workplace (e.g., the operating or delivery room). What is less clear is which environment provides the best opportunity for learning: one that is stressful and hostile, such as military school teaching, or one that is facilitative and encouraging. For many years, some physicians have subscribed to an unwritten rule of medical pedagogy, i.e., learning under stress is the optimal means of clinical training. Zakay and Wooler [26] have suggested that training under nonstress conditions does not improve task performance when the task itself has to be performed under stress conditions and recommended stress-inclusive training or simulations. The stress of criticism induced by clinical teachers (e.g., use of punishment, pressure and criticism) was justified by the theory that the trainee’s actual clinical environment would be similarly stressful. However, stress may be unfavorable for learning. Severe life stress hampers spatial learning in animals [21], and childhood stress can lead to behavioral disorders later in life [20]. The question of whether punitive and critical comments (criticism) or encouraging and nurturing comments (praise) best strengthens the clinicianlearner’s targeted behavior in clinical settings remains unanswered. While a great deal has been published about feedback, what generally happens in a clinical setting does not always meet that definition. Comments from senior physicians to trainees might be more appropriately categorized as “praise” or “criticism.” In medical education,
Brought to you by | University of Manitoba Authenticated Download Date | 6/6/15 1:49 AM
480 Saraf et al., Praise/criticism during obstetric simulation feedback is one of the crucial elements of learning [10]. Studies have concluded that effective feedback is necessary, valuable and critical for clinical performance improvement [10] and for the acquisition of clinical skills [11] by students and residents. It is associated with students’ perception of high-quality learning [25]. However, much less is known about praise or criticism, which is ubiquitous in clinical settings, and the differential impact of praise and criticism on teaching in obstetrics has not, to our knowledge, been previously assessed. We chose simulation to study the effect of praise/ criticism on learning in different types of learning environments. Simulation is increasingly used as an effective learning tool [24]. In obstetrics, simulation has been shown to improve knowledge-based scores [9], safety attitudes [12], technical skills [7], communication skills [4] and perceived competence [22, 23]. However, studies to date have not assessed the effect of different pedagogic approaches (praise or criticism) on trainee learning in a simulated environment. Our objectives were to compare the cognitive and technical performance of students randomized to receive either praise or criticism during a simulated normal vaginal delivery and to assess students’ perceived confidence in their abilities after their learning experience. We hypothesized that there would be an overall increase in cognitive and technical scores for the “praise” group and a decrease in all scores for the “criticism” group.
NSVD lecture+ Demonstration (N=59)
Methods Design We conducted a randomized clinical trial approved by the Maimonides Medical Center’s Institutional Review Board. We enrolled medical students (n = 40) and nursing students (n = 19) present at the hospital between June 2010 and May 2011. Only students who had not witnessed and had not performed a normal spontaneous vaginal delivery (NSVD) were included. Teaching and evaluation were performed with the use of Noelle S575 Gaumard Scientific, FL, USA, advanced birthing simulator.
Pilot and randomization We conducted a pilot study (n = 10), to assess the appropriateness of the scripts. The pilot study included students randomized to praise (n = 5) and criticism (n = 5), and assessed their performance using a previously published checklist [6]. The pilot confirmed the feasibility of the study and the participants’ understanding of all instruments. The study protocol is shown in Figure 1. Block randomization was performed with stratification by profession, i.e., medical or nursing student. A computer-generated table of random numbers and sequentially numbered opaque sealed envelopes containing allocation cards were used to achieve randomization, designating group assignment to praise or criticism groups. The students were blinded to the purpose of the study and to their group assignments.
Consent A day before the simulation session, we distributed handouts to all participants, which described the steps used in the performance of
Participant practice (Simulation)
Simulation 1 Baseline assessment
Medical students N=40
Praise group (N=19)
Nursing students N=19
Criticism group (N=21)
Praise group (N=9)
Criticism group (N=10)
Simulation 2 Intervention/Second assessment Debriefing
Figure 1 Materials and methods – Flowchart utilized for randomization and recruitment of participants.
Brought to you by | University of Manitoba Authenticated Download Date | 6/6/15 1:49 AM
Saraf et al., Praise/criticism during obstetric simulation 481 a normal vaginal delivery [5]. On the day of simulation, we obtained informed consent, demographic data and distributed the sealed opaque envelopes. The consent informed the participants both about simulation of normal vaginal deliveries and that they might be video taped (a subset was videotaped to allow validation of the scores rendered by proctors in the room).
The instrument Next, we showed the steps to perform an NSVD. Each student practiced in a simulated NSVD and had an opportunity to ask questions. During both the baseline and the intervention sessions (see below), two board-certified obstetricians were in the room with the students. One served as the proctor and the second scored the students. After the teaching demonstration and practice session, a baseline assessment was undertaken with each student performing an NSVD simulation. No comments were provided during the baseline simulation. The trainees were scored using a checklist that was a modification of a standardized published checklist [6]. This checklist (Figure 2) assessed cognitive (knowledge) and technical parameters (applied performance). At the completion of the baseline simulation, the students filled out a confidence survey based on a previously published Likert scale survey [6].
Sr. No. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Interventions/Checklist Introduces self to patient Asks for parity Asks for gestational age Asks for medical problems Assesses fetal tracing Assesses clinical EFW Calls for supervisor help Pelvic exam: Cervical dilatation Pelvic exam: Cervical effacement Pelvic exam: Cervical station Instructs patient how to push Controls delivery of the head Supports the perineum Notes the time of delivery of head Asks to stop pushing (after delivery of head) Checks for nuchal cord Suctions the nose and mouth Checks and delivers the anterior shoulder Delivers posterior shoulder Delivers rest of the body Clamps and cuts the cord Waits for signs of placental separation Delivers placenta using suprapubic pressure Asks for pitocin postpartum Provides fundal massage Inspects perineum for lacerations Inspects placenta Assesses and announces blood loss Asks for fetal well being and APGAR scores Counsels and talks to the patient Total score
These checklists and scoring systems were constructed based on three of Kirkpatrick’s [14] four levels of learning evaluations, an established method for judging learning processes. The levels of learning assessed were as follows: (A) Reaction (how the participants felt) – confidence score (Likert self-report scale); (B) Learning (increase in knowledge) – cognitive score; and (C) Behavior (applied learning, i.e., performance) – technical score. We did not assess the fourth of Kirkpatrick’s levels – Results, i.e., long-term outcome. During the intervention simulation performed later on the same day, each participant again performed a simulated NSVD. However, this time, based on their random group assignment, the proctor provided one of two interventions; either scripted praise or criticism (Figure 3). The scripts included comments at initial proctor greetings and then at five predetermined points during the NSVD. Comments varied depending on whether the trainee had performed the task correctly. For example, after a correct placental delivery using the traction/counter-traction method, a participant in the praise arm would be told, “Excellent, that’s the classic way to deliver the placenta.” A participant in the criticism arm would be told, “OK, that’s fine, but hurry up and keep moving.” If a participant in the praise arm performed an incorrect placental delivery she/he would be told, “Good job, but make sure you apply suprapubic pressure for counter traction,” while a participant in the criticism arm would be told, “That
Cognitive assessment
Technical assessment
/28
/22
Figure 2 Checklist 1 – Baseline simulation checklist. Assessment of students’ technical and cognitive baseline performance.
Brought to you by | University of Manitoba Authenticated Download Date | 6/6/15 1:49 AM
482 Saraf et al., Praise/criticism during obstetric simulation Sr. No
1 2 3 4
5 6 7 8 9 10
11 12 13
14 15 16 17 18 19 20 21 22 23
24 25 26 27 28 29 30
Interventions/Checklist
Cognitive assessment
Technical assessment
(Outside the room) What’s your name? Which medical school do you attend? You did/did not perform well the last time. I was satisfied/disappointed I hope you realize this is/is not going to affect your grades. Introduces self to patient Asks for parity Asks for gestational age Asks for medical problems a) Wonderful, you requested all the appropriate information b) Great introduction, it is also important to ask about… (mention what they missed) c) Keep going, you did fine, but we need to get to the point d) This is not how you do it. You have to ask specific questions, for example if she has a heart failure, are you going to let her push? Assesses fetal tracing Assesses clinical EFW Calls for supervisor help Pelvic exam: Cervical dilatation Pelvic exam: Cervical effacement Pelvic exam: Cervical station a) Excellent, that’s exactly how you do a complete pelvic exam b) Excellent start just let me show you how you could perfect your exam. c) Ok, your exam was fine, that does not mean that you can deliver the baby easily, let’s see. d) Watch out how you do it. You can cause lacerations and make her uncomfortable. ( shows) Instructs patient how to push Controls delivery of the head Supports the perineum a) Great job- That’s exactly how you support the perineum b) Overall you are doing great, but let me show you how to do this better c) Yeah OK, finally you did something right d) What do you think you are doing? you are not supporting the perineum properly and this could results in more injuries Notes the time of delivery of head Asks to stop pushing (after delivery of head) Checks for nuchal cord Suctions the nose and mouth Checks and delivers the anterior shoulder Delivers posterior shoulder Delivers rest of the body Clamps and cuts the cord Waits for signs of placental separation Delivers placenta using suprapubic pressure(traction/counter traction) a) Excellent, That’s the classic way of delivering the placenta b) Good job, but make sure you apply the suprapubic pressure for counter traction c) OK, that’s fine. But hurry up and keep moving ahead . d) This is totally wrong, without suprapubic pressure, the uterus is going to flip inside out and the mother is going to bleed to death Asks for pitocin postpartum Provides fundal massage Inspects perineum for lacerations Inspects placenta Assesses and announces blood loss Asks for fetal well being and APGAR scores Counsels and talks to the patient Total score
/28
/22
Figure 3 Checklist 2 – Intervention simulation checklist. Assessment of students’ technical and cognitive post-intervention performance. was totally wrong, without suprapubic pressure, the uterus is going to flip inside out and the mother is going to bleed to death.” Whenever technical errors were observed by trainees in either group, the proctor immediately demonstrated the correct way of performing the task. The scoring provider did not have any interaction with students during either the baseline or the intervention simulation. The same scoring checklist that was used for baseline assessment of cognition and technical skills was again utilized for the intervention assessment, and that assessment was followed by a second self-scored
Likert confidence survey. The participants were debriefed about the study details and its purpose at the end of participation. The “change in score” from baseline to intervention, obtained from the praise group, was compared with that obtained from the criticism group for cognition, technical skills and confidence. We used a video-recording device to mitigate possible grading bias by the scoring provider who heard the proctor’s comments during the second simulated NSVD and in order to determine the reliability of the scoring assessments. A separate reviewer, blinded to the
Brought to you by | University of Manitoba Authenticated Download Date | 6/6/15 1:49 AM
Saraf et al., Praise/criticism during obstetric simulation 483
participants’ group assignment, evaluated video recordings of the subset (20%) of participants. This blinded reviewer scored students’ performance by watching the simulation videos, which had been edited so that the comments were not heard, and the provider giving the comments was not seen.
Statistical analysis Demographic differences between participants in the two arms were evaluated with two-way frequency tables and Fisher exact tests. Then, for each of the three outcome measures (cognitive, technical and self-reported confidence) as dependent variable, a mixed linear model was constructed with profession (medical, nursing), assessment time (baseline, intervention), study arm (praise, criticism) and subject gender. An unstructured within-subject covariance matrix was modeled. Satterthwaite corrections were applied to denominator degrees of freedom. Non-significant fourth-order terms were excluded from models. Model residuals were inspected for outliers and skew. Reported means for outcome measures were model estimated and, therefore, properly adjusted for differences between genders and professions. Since there were no relevant studies from which to project effect size, we conducted an interim analysis of the first 25 subjects. That data suggested that 25 subjects in each group would provide 80% power to detect a significant difference for our principal outcomes (cognitive and technical scores) using two-way analysis of variance with significance level set at 0.05. One outlier with extremely low,
