assessment for learning Making sense of feedback experiences: a multi-school study of medical students’ narratives Lynn M Urquhart, Charlotte E Rees & Jean S Ker

CONTEXT Until recently, the perspective of students in the feedback process has been ignored, with strategies for improvement focusing on the tutor and feedback delivery. We employed an original narrative interviewing approach to explore how medical students make sense of their experiences of feedback. METHODS A qualitative design was adopted employing three individual and 10 group interviews to elicit narratives of feedback experiences from 53 medical students at three 5-year undergraduate programmes in the UK during 2011. Thematic analysis was undertaken of students’ understandings of feedback and of their narratives of positive and negative experiences of feedback at medical school. In addition, thematic and discourse analysis of the linguistic and paralinguistic features of talk within the narratives was conducted. RESULTS Students typically constructed feedback as a monologic process that happened ‘to’ them rather than ‘with’ them. They

shared 352 distinct narratives of feedback experiences, which were rich in linguistic and paralinguistic features of talk. Through the analysis of the interplay between the ‘whats’ and ‘hows’ of student talk, i.e. emotion, pronominal and metaphoric talk and laughter, we were able to understand how students find meaning in their experiences. Students used laughter as a coping strategy, emotion talk as a means to convince the audience of the impact of feedback, pronominal and metaphoric talk to describe their relationship (often adversarial) with their feedback providers and to communicate feelings that they might otherwise struggle to articulate. CONCLUSIONS This research extends current feedback literature by focusing on medical students’ lived experiences of feedback and their emotional impact through narrative. We go on to discuss the educational implications of our findings and to make recommendations for improvement of the feedback process for students, tutors and for institutions.

Medical Education 2014: 48: 189–203 doi: 10.1111/medu.12304 Discuss ideas arising from the article at ‘www.mededuc.com discuss’

Medical Education Institute, University of Dundee, Dundee, UK

Correspondence: Lynn Urquhart, Centre for Medical Education, Medical Education Institute, University of Dundee, The Mackenzie Building, Kirsty Semple Way, Dundee DD2 4BF, UK. Tel: 00 44 1382 381 952; E-mail: [email protected]

ª 2014 John Wiley & Sons Ltd. MEDICAL EDUCATION 2014; 48: 189–203

189

L M Urquhart et al

INTRODUCTION

Feedback is a necessary component of teaching and learning: it has been shown to be one of the most important influences upon achievement.1–3 Successful feedback allows the learner to understand how they are currently performing with regards to an accepted standard, and enables them to develop learning plans to develop areas of deficiency and build on areas of strength. This is particularly important in adult learners, where feedback should be used to empower students as self-regulated learners.4 However, there remains significant dissatisfaction amongst higher education students who report that feedback is lacking in both quality and quantity. This can be seen in the annual UK National Student Survey, which is undertaken by final-year higher education students towards the completion of their degrees.5 Students are asked to rate a number of aspects of their course: quality of teaching, assessment and feedback, academic support, organisation and management, learning resources and personal development. Consistently, feedback is rated as the least satisfactory component of each course by medical students, a phenomenon that spans the UK and includes all curriculum types. This is not simply a problem within the UK, as numerous studies within the USA have also shown student dissatisfaction with feedback.6–8 Previous attempts to improve the feedback experience have focused on improving the techniques of the feedback providers through multiple guidelines.9–14 However, despite the ready availability of these guidelines, this top-down approach appears to have had little influence upon students’ experiences of feedback. Even the definition of feedback, while variable throughout the literature, has been very tutor-centric in its origins. In 2008, Van de Ridder et al.15 sought to provide a consensus definition of feedback from the literature. Although they suggested a definition of ‘specific information about a comparison between a trainee’s observed performance and a standard, given with the intent to improve a trainee’s performance’, it must be noted that the majority of the articles were written from the position of experts and none contained the definition as conceptualised by students themselves. It has previously been suggested that the perceived lack of success of feedback is down to ‘the feedback

190

gap’. This phenomenon occurs when tutor and student perceptions of feedback given differ.16–18 Prior interventions have failed to improve the feedback experience (as opposed to feedback delivery), as they have failed to take into account the perceptions and experience of the target of that feedback, i.e. the student. A shift in emphasis has occurred within research into feedback and its success or failure to facilitate improvement in a student’s performance. Recent publications across higher education have started to explore feedback from students’ perspectives and to investigate the key influences upon how feedback is received and valued.19–22 We have started to understand that there are many factors that influence a student’s receptiveness to feedback. These include, among other things, self-esteem, credibility of the feedback provider and the emotional impact of the feedback experience.22–24 Understanding students’ perspectives The recent paper published in Medical Education by Murdoch-Eaton and Sargeant21 explored students’ perceptions of feedback at one medical school, using focus group methodology. The authors asked questions regarding previous experiences and the effect that this feedback had upon students’ motivation and the value placed upon the feedback. A number of important findings were reported, particularly regarding maturational differences in undergraduates and the effect that this had on students’ abilities to perceive or recognise feedback. In particular, they found that junior students were more likely to see feedback as a passive activity and for providing reassurance, whereas more senior students used feedback to inform their learning needs. These findings concur with those of Pokorny and Pickford19 who conducted similar research in the context of a UK business school, suggesting that this maturational issue is found across the diverse world of higher education and is not limited to medical education. We also know from a number of publications that the way feedback makes students feel has an important impact upon the success or otherwise of that feedback. Indeed, Eva and his colleagues22 suggested that the key factors influencing responsiveness to feedback include, among other things, fear. In addition, although the regulatory focus theory25 is gaining popularity in explaining the impact of feedback upon a student, Watling et al.24 criticised the proposal that it offers a fully satisfactory

ª 2014 John Wiley & Sons Ltd. MEDICAL EDUCATION 2014; 48: 189–203

Student feedback narratives explanation for the variability in feedback responses. They note that there are other influences at play besides the learner’s regulatory focus, including emotions and the perception of credibility.24

2 3

What meaningful experiences do students narrate about feedback? How do students narrate these experiences?

METHODS

Yet none of these studies has gone so far as to adopt a narrative approach that allows students to share their lived experiences of feedback in their own words. Narratives are stories with a narrator, listener(s), characters, time sequence, plot and purpose.26 By asking participants to share their stories, they can be empowered through researchers showing a genuine interest in their worlds. Rather than ask participants questions about their generalised beliefs or perceptions about feedback, a narrative approach helps to reveal participants’ views in a personally meaningful way because these are grounded in their actual lived experiences of feedback. Narrative inquiry has been employed in both education research27–29 and health care research,30,31 particularly in situations where the aim is to understand the perspectives of marginalised groups, and this aim translates well into medical education research. Indeed, recent publications in medical education have explored students’ professionalism dilemma stories.32 The strength of narrative inquiry is that it allows for analysis of not only what is said, but also how stories are shared. It is through analysis of the interplay between the whats and the hows that new insights can be achieved into the phenomenon being researched.33 Indeed, as Bleakely34 reports in his discussion of the use of illness narratives in clinical education, narratives give ‘insight into the intellectual, spiritual, and practical, as well as the emotional and relational, aspects of living with illness’. By collecting narratives in our research we sought to unearth an equally multilayered explanation of the key factors that influence the impact of feedback upon students. In particular, through analysis of emotion, pronominal and metaphoric talk and other paralinguistic features such as laughter we wished to understand the emotional impact of that feedback, as well as students’ impressions of the wider institutional culture within which the feedback is being delivered (see Box S1 for definitions of key linguistic and paralinguistic features). Through our research, the questions we wanted to answer included: 1

What are students’ solicited definitions of feedback?

Study design This study was based upon a constructionist epistemology suggesting that all knowledge and meaningful reality is contingent upon human practices being constructed in and out of human interaction. Aligned with an interpretive approach, focus group methodology using narrative interviewing techniques was chosen to explore students’ lived experiences of feedback within the social context of the interview setting.35 Focus groups can be the ideal method for understanding stakeholder perspectives and are often used to give voice to under-represented groups.36 Certainly, medical students’ opinions have been under-represented in this area to date. In addition, focus groups have benefits over individual interviews that relate to group interaction, including ‘safety in numbers’ and ‘snowballing’, wherein one participant’s comments evoke further discussion within the group.37 Narratives are intertwined with, and are shaped by, the emotions they evoke.38 A narrative interviewing approach was taken as this is a powerful approach when trying to understand participants’ experiences and how they make sense of those experiences. Narratives also allow for an exploration of what is said and how participants choose to share these experiences through, for example, laughter and emotional talk.32 Participants After receiving ethical approval, medical students from three UK medical schools were invited to participate in group or individual interviews about the topic of feedback. These schools were chosen for two reasons. First, the schools lay at opposing ends of the spectrum regarding their performance in the feedback category of the National Student Survey 2010. Second, although students across all three schools have early clinical experiences from Year 1, they also have some differences in curriculum design. Although Schools 1 and 2 have integrated curricular with systems-based approaches, School 3 adopts a problem-based learning approach with a

ª 2014 John Wiley & Sons Ltd. MEDICAL EDUCATION 2014; 48: 189–203

191

L M Urquhart et al spiral curriculum. Students from all years were invited to attend via e-mail, although we did not hold groups for all 5 years at all three schools. In total, 53 students participated in 10 group and three individual interviews (Table 1). Data collection The interviews were mostly held over lunchtime and therefore typically lasted between 45 and 60 minutes. Students consented to take part and also

Table 1 Demographic and education-related characteristics of participants Participants’ characteristics

Number

Age (years) 17–19

20

20–22

17

23–25

12

26–30

4

Gender Male

17

Female

36 50

Other

3

First Language English

50

Other

3

Place of study School 1

32

School 2

14

School 3

7

Participants’ year in medical school First

12

Second

21

Third

5

Intercalated*

2

Fourth

6

†

Fifth

7

Student has previous degree? No

43

Yes

10

* Intercalated year is between year three and year four of a medical degree in which the student can get a Bachelor degree in medical sciences. † The fifth year represents the final year for the three undergraduate programs participating in this study.

192

A discussion guide was produced as an aide memoir and to ensure that similar topics were discussed across the different interviews. The focus groups began with the question: ‘What do you understand by the word feedback?’ After this, narrative interviewing techniques were employed to elicit students’ personal incident narratives around feedback, and open-ended questions were used to explore the widest range of feedback experiences. Students tended to volunteer negative experiences of feedback so the interviewers prompted students specifically about positive experiences. The focus groups were audio-recorded and transcribed by the first author. Transcripts included paralinguistic features of talk, such as laughter, pauses and hesitations. Data analysis

Ethnicity White

for interviews to be audio-recorded. Focus groups were conducted primarily by the lead author, although two of the focus groups were co-facilitated by the second author. Interviews were conducted when only one student for that year group attended the focus group at the allocated time and venue.

Data were analysed using framework analysis.39 This framework was developed by all three authors working independently to listen to, read and analyse two to four focus groups, making notes about what they identified as key themes. Then, all three shared their analyses and compared, contrasted and negotiated themes. From this discussion, a preliminary coding framework was developed that acted as the starting point for coding all data using ATLAS-TI (Version 6.2; Berlin, Germany), which was conducted by the first author with further discussion with the other two authors. Each personal incident narrative was identified and coded in its entirety. A personal incident narrative can be described as ‘a report of a sequence of events that have entered into the biography of the speaker by a sequence of clauses that correspond to the order of the original events’.40 Although students also offered their opinion and comment around feedback, we did not analyse these data, as similar nonnarrative data have already been reported in the feedback literature.19–22 Instead, we analysed only the narratives. Each entire narrative was coded to key themes, such as ‘positive narrative of feedback’ or ‘negative narrative of feedback’, and segments within each narrative were also coded to multiple content-related themes, such as ‘feedback is verbal’ and ‘feedback is constructive’, and process-related

ª 2014 John Wiley & Sons Ltd. MEDICAL EDUCATION 2014; 48: 189–203

Student feedback narratives themes, such as ‘laughter for coping’ or ‘metaphoric talk’. This approach allowed us to code data for content-related themes (i.e. what students said about their feedback experiences) and processrelated themes (i.e. how students chose to narrate their feedback experiences).

RESULTS

We identified four content-related themes or ‘the whats’ of the feedback narratives. We also identified four process-related themes within the feedback narratives, known as ‘the hows’, i.e. how our students chose to narrate their experiences to the group (see Table 2 for illustrative quotes of both content- and process-related themes). Here we shall discuss briefly the content-related themes and delve deeper into the process-related themes, such as laughter, emotional, metaphoric and pronominal talk employed by the students in this study. Indeed, the originality of this study is in exploring both facets of the narratives and looking at the interplay between these whats and hows. In this way, we gain new insights into the lived experiences of feedback for the students in this study. Content-related themes Students’ understandings of feedback Twenty-one students responded to the question ‘What do you understand by the term feedback?’ Although not every student offered an explicit definition within the context of the focus groups, there was agreement from the other students in the group, often via non-verbal means such as nodding or by saying ‘I agree’. When no further explanations were offered, the researcher asked if anyone within the group had anything to add and if no further definitions were offered it was assumed that all definitions considered by this particular group had been exhausted. Students constructed feedback in terms of its mode (verbal or written), its type (alpha/numeric grades or comments), the feedback provider (clinical tutor or self), its setting (workplace or university), its formality (formal or informal) and purpose (formative or summative). Students typically described the purpose of feedback as for improvement, encouragement or information on performance: ‘Feedback in terms of performance in the clinical environment or coursework feedback. How well you are doing at a specific time point’ (male, fourth-year student, School 3).

Students appeared to conceptualise feedback as a one-way process, as information given to them rather than something that occurred with them and including them. Although clinical students did offer a greater range of possible types of feedback, all but one student described feedback as a monologic process, of information given to them: ‘They tell us basically how we are doing and any areas we need to work on’ (female, third-year student, School 1). Only one student, who was a senior student, described feedback as a conversation: ‘I suppose comments by doctors or whoever is assessing you that you can build upon and improve upon, whether that be written or face to face conversation’ (male, fifth-year student, School 1). In addition to students conceptualising feedback as something given to them, they also understood feedback to be something that they were required to give to the medical school, i.e. student evaluation of the curriculum. Although this was not the original focus of our study, it revealed interesting insights into how students saw the extent to which feedback (in the form of student evaluation) was accepted by their school. Furthermore, this feedback role modelling by schools was reported by students to influence their own receptiveness to the feedback they received in return. As such, students reported giving tokenistic evaluation back to their schools for fear of negatively influencing their relationship with their tutor or school: ‘I think the problem is that everybody [students] is tempted to say what you think they [schools] want to hear…I know that’s [student evaluation] not us getting feedback but it [negative student evaluation] affects your relationship with the person you are getting feedback from’ (male, third-year student, School 1). Students’ narratives of feedback experiences In total, 352 distinct narratives were shared by students: 134 positive experiences of feedback; 178 negative experiences; and 40 neutral experiences. The positive experiences were classified in terms of mode (55 verbal, 20 written), type (9 comments, seven grades), feedback provider (28 clinical tutor, 20 patient, 14 peer, 4 self, 1 non-clinical tutor) and setting (65 university, 43 workplace, 21 simulated). The negative experiences were also classified in terms of mode (36 verbal, 31 written), type (26 grades, 13 comments), feedback provider (30 clinical tutor, 9 peer, seven patient, 2 self, 2 non-clinical

ª 2014 John Wiley & Sons Ltd. MEDICAL EDUCATION 2014; 48: 189–203

193

L M Urquhart et al

Table 2 Key content- and process-related themes Themes Type

(with subthemes in brackets)

Illustrative quotes

Content

Narrative of positive experience

‘In the first few years at the end of every block you had a meeting with

of feedback (Feedback as verbal;

your facilitator… they would show you how you were progressing

student feels feedback provider

through the block… which I actually thought was really good… they

knows them)

have seen you twice a week and they are in a good position to tell you whether you know stuff or not’ (male, Year 4, School 3)

What constitutes a positive experience

‘At the very end of it all as we were leaving he shook our hand and then

of feedback? (There is a respectful

he actually gave us a personal strength and weakness as we were

dynamic between the student and

leaving. It was amazing’ (female, Year 2, School 1)

feedback giver; feedback in the workplace; feedback is individual in the group setting) Narratives of negative experiences of feedback

‘I failed that [examination] by one mark and you would think I had failed

(Feedback in the university setting;

it by about 20. I had a meeting with the two heads of our first year and

feedback is not constructive for learning;

as a graduate, as someone who had studied for exams before, I felt belittled

feedback has a negative emotional impact)

by it. It was like “you obviously haven’t been working hard enough”… and quite a lot of what I had to offer as to why I thought I had failed it wasn’t taken into account. It was quite frustrating and they didn’t actually offer any sort of constructive feedback’ (female, Year 2, School 2)

What constitutes a negative experience of

Process

‘I was asked to do a respiratory examination, I know I didn’t do it very

feedback? (Negative feedback is given in front

well but the consultant told me it was “plain awful”, that was her

of others; feedback has a negative emotional

words and this was in front of another student, the patient and the

impact; feedback in the workplace; feedback

patient’s wife. It was just horrific I just actually felt like crying’

from a clinical teacher)

(female, Year 5, School 1)

Laughter (Laughter as shared understanding)

‘We asked someone to sign us off for DOPS [direct observation of procedure skills] which they are meant to watch you do something and we were going to do an ECG and he was like “oh you do ECGs all the time you can do ECGs” and he just filled out the form signed it and went “don’t tell anyone about that” (laughs and group laugh) and like it’s pointless’ (female, Year 5, School 1)

Pronominal talk

‘Well I just think they do a lot of things, they try to do a lot of things to support us but I don’t think they are particularly successful. They give us the graphs which are pretty good the x scores they have never explained it to us’ (male, Year 2, School 2)

Metaphoric talk

‘Yes it was PBL it was a PBL facilitator and um the woman just laid into me’ (female, Year 2, School 3)

Emotion talk

‘When I had my appraisal. I found it a very intense period. It was only 10 minutes and I felt like I was being told that my personality doesn’t quite fit into their mould… and em I came away from it very deflated and very angry em- em we do a thing called the [names course] and this was during my critical appraisal, this was the time that the tutor decided to tell me that my patient had actually died and was dead when I gave my talk (pauses) and I found that extremely disappointing (male, Year 1, School 1)

194

ª 2014 John Wiley & Sons Ltd. MEDICAL EDUCATION 2014; 48: 189–203

Student feedback narratives tutor) and setting (93 university, 39 workplace, 19 simulated). Note that these numbers do not add up to the total number of narratives because not all features were apparent in every narrative.

as ‘they’, giving a ‘them and us’ adversarial feel to the feedback provider and student relationship.

There were 31 subthemes for what makes a positive experience. The most common were: feedback is constructive, feedback is specific, feedback is given on the basis of direct observation, there is a respectful dynamic between the student and the feedback giver and feedback is balanced.

Students typically used metaphors of FEEDBACK AS WAR to describe their negative experiences of feedback within their narratives. Other metaphors employed included FEEDBACK AS SPORT, FEEDBACK AS MUSIC and FEEDBACK AS A PRECIOUS METAL.

There were 34 subthemes for what makes a negative experience. The most common were: feedback has a negative emotional impact, feedback is not standardised, feedback is not constructive, the purpose or content of the feedback is unclear, feedback is absent, feedback relates to students’ personal characteristics and the student does not respect the course/feedback provider. In both the negative and positive narratives there were more examples from the university setting (perhaps reflecting our participants who were predominantly more junior students). Positive narratives were much more likely to involve verbal feedback than written feedback and this feedback was typically delivered by a clinical tutor. Negative narratives were more likely to have taken place within the university setting and involved written feedback. Feedback as grades was more common in negative than positive narratives. In Table 2 we present illustrative quotes for each of the content- and process-related themes of the narratives. During our analysis we found that similar themes were found across all three schools. Process-related themes In addition to the content-related themes we also coded for process-related themes. Examples of the four key process-related themes can also been seen in Table 2. Laughter and laughter-talk We found that students used laughter for four main purposes within the narratives. These were: for coping, for shared understanding, for power and finally during a comical or incongruent situation. Unsurprisingly, students seemed more likely to use laughter for coping during the telling of negative narratives.

Metaphoric talk

Emotion talk Negative emotion talk, e.g. ‘hurtful’, ‘tough’, ‘deflated’ and ‘angry’ was typical in the narration of negative experiences of feedback and positive emotion talk, e.g. ‘good’, ‘happy’, ‘helpful’ and ‘amazing’, was common in narratives of positive experiences. In addition, students also employed other techniques such as intensifiers, repetition and non-fluencies such as ‘um’ or pauses as well as hedging (e.g. ‘I suppose’) to narrate their stories with emotion. The interplay between the ‘whats’ and the ‘hows’ of the narratives In order to explore the interplay between content and process we present two full narratives here. We present our narratives according to Labov’s40 classical elements of a narrative. These are the abstract (summary); the orientation (time, place, participants); the complicating action (sequence of actions, turning point, problem); the most reportable event (least frequent event with greatest impact); the resolution (complicating action after the resolution); evaluation (the narrator’s commentary); and finally the coda (return to the present). In particular, by examining full narratives we can see, through the way the student has structured their story, what they construct as the important aspects of the event as well as their evaluation of the event. This has not been explored in previous studies using other qualitative methodologies that have instead focused on student opinions.19–22 These narratives have been chosen because they showcase a number of content- and process-related themes. They are also fairly typical of our broader collection of narratives across the three schools. ‘We got absolutely amazing feedback’

Pronominal talk Students typically described themselves and their student body as ‘us’ or ‘we’ and the feedback providers

Our first example is from a female, third-year student from School 3 as she narrates a positive experience of feedback (see Box 1).

ª 2014 John Wiley & Sons Ltd. MEDICAL EDUCATION 2014; 48: 189–203

195

L M Urquhart et al

Box 1

‘We got absolutely amazing feedback’: narrative of

positive experience of feedback

Abstract Well I have to say I had really good feedback when we did a student-selected component, an SSC. Orientation I did work recently Evaluation and we got absolutely amazing feedback on that. I was thinking usually on our written work we get almost no feedback, they are not allowed to write on our scripts and we get this tiny paragraph et cetera et cetera Complicating Action but for my SSC for every single section of my report I got a paragraph on how each bit was done and how it could be done better and whether… Evaluation So that was good Complicating Action for every single bit and then an overall review at the end

short narrative, such as emotion, pronominal and metaphoric talk, we can gain original insights into this student’s experience. Within this narrative we see this student use positive emotion talk (e.g. ‘good’ and ‘amazing’ are stated twice each) and this positive emotional talk is sometimes accompanied by intensifiers (e.g. ‘really good’, ‘absolutely amazing’). In her narration she uses emotion talk to differentiate her positive experience from the ‘usual’ negative experience of receiving inadequate feedback on written work. This is illustrated by her use of negatives such as ‘no feedback’ and ‘they are not allowed’. Her reference to feedback providers here and later in the narrative as ‘they’, alongside her repeated reference to feedback receivers as ‘we’, illustrates an adversarial ‘them and us’ feel to the perceived relationship between students and their feedback providers,41 alongside the solidarity she clearly feels with her fellow feedback receivers.42 Finally, this student employs the metaphor of ‘gold dust’ to describe this positive experience of feedback. Her choice of metaphor here illustrates that she conceptualises feedback as a valuable commodity. The suggestion of rarity also helps to contrast this positive experience rich in comprehensive, constructive and specific feedback with her previous ‘usual’ negative experiences of written feedback.42 ‘I went home and cried’

Evaluation which was just like amazing – it was like gold dust. If they could do that for written… if we could all get feedback on our written work people would appreciate it so much cos it was just so thorough, whereas typically you get a sort of generalised ‘it was OK’, that’s it”

In this narrative of a positive experience of feedback we can see a number of content-related themes at play. The student discusses an episode of written feedback for written work and contrasts this with her previous typical negative experiences of written feedback. We can see that for this student a positive feedback experience was created by the tutor because he or she had given her specific, comprehensive and hence constructive feedback on her work. Were we to stop our analysis at this level we might not gather any new insights into this student’s experience, but would simply confirm previous research in this area. However, by exploring the process-related themes in this

196

In our second example we can see the interplay between multiple content- and process-related themes as part of a conversation between the narrator, a female second-year student from School 2, the interviewer and another female student. This time, the narrator shares a negative experience of verbal feedback within a simulated environment (see Box 2). In this example of a negative experience of feedback we can see a number of content-related themes at play. This verbal feedback is delivered within the simulated setting. The student describes this as a negative experience because it is abusive, relates to her personal characteristics and is shared in front of others. She is told that she is ‘too tense’, ‘too square’, that she looks like ‘a pointer dog’ and that she intimidates her patients. It will be no surprise to anyone that she found this a negative experience. However, by exploring the ‘hows’ of her narrative, we can understand the emotional impact of this experience on her, as well as illustrating how she feels recounting her story.

ª 2014 John Wiley & Sons Ltd. MEDICAL EDUCATION 2014; 48: 189–203

Student feedback narratives

Box 2

‘I went home and cried’: narrative of negative experience of feedback

Abstract FS2: I had, we had done a session where you had to tell, give your [simulated] patient information Complicating Action and then your tutor stopped you and gave you feedback on how well you did in front of a room full of your peers Most Reportable Event and she told me that I, my shoulders were too tense and too square Evaluation which I can’t help the shape of my shoulders they are just how they are Most Reportable Event that I look like a pointer dog and that I intimidate my patients Evaluation and I was very upset Resolution I sat and I took it and I sat for the rest of the class and then I went home and cried Evaluation and I didn’t know what to do about that, like I know a few people felt like I should have complained about it but I didn’t really know what to say to her cos she’s a tutor and I suppose she thinks she is helping em LU: And did any of your colleagues that were there at the time comment then to you? Evaluation FS5: I was there, she was particularly and unnecessarily harsh and we all felt the same like we didn’t know where it came from; it was very unnecessary and particularly harsh like the things she was saying but again it made a very awkward atmosphere and you couldn’t really say anything at that point Evaluation FS2: Yeah everyone was really nice to me afterwards they all said ‘oh I would have cried if she had said that to me. I thought she was way harsh on you’ but um… LU: And were you able to take anything from that feedback or? Evaluation FS2: I think she might be a slightly bitter and twisted lady ((laughs)) em ((group laughs heartily))”

During the narration, the student employs negative emotional talk, including an intensifier to convey the extent of her emotional turmoil: ‘very upset’ and ‘went home and cried’. Interestingly, there are further instances of emotion talk from the narrator’s peer and also laughter from the whole group,

allowing us to understand more the benefits of group discussion over interview. This second student is also female and uses negative emotion talk, repetition and intensifiers in support of her colleague: ‘very awkward’, ‘particularly and unnecessarily harsh’ with further repetition of ‘unnecessary’

ª 2014 John Wiley & Sons Ltd. MEDICAL EDUCATION 2014; 48: 189–203

197

L M Urquhart et al and ‘particularly’. Furthermore, the use of this peer’s inclusive pronoun (‘we all felt’) provides the primary narrator with more evidence of this support. These linguistic tactics perform two purposes: 1) to show solidarity with her colleague and 2) to emphasise how negative this situation was to the narrator and the rest of the group. Following this support from her colleague the primary narrator changes to use positive emotion talk with an intensifier to describe her peers and the support that she received at the time: ‘everyone was really nice’. She does so to demonstrate her gratitude to her colleagues both contextually and non-contextually, i.e. for their support at the time of the incident as well as during her sharing of the story within this focus group setting. Interestingly, although this was clearly a very negative experience, the student uses two paralinguistic features, hedging and laughter, that show us that she is not altogether comfortable critiquing her tutor in front of the group and the researcher. She hedges by stating: ‘I suppose she thinks she is helping’. She hedges further in the following comment, which is terminated with laughter: ‘I think she is a slightly bitter and twisted lady (laughs)’. This student uses laughter for coping. She uses it contextually to deal with her discomfort at criticising her tutor as well as to deal with the emotion of telling this difficult story. Her choice of the metaphoric talk ‘bitter and twisted’ suggests vitriol, yet her use of laughter after this attempts to hide this vehemence from the group, perhaps suggesting her discomfort at criticising her tutor. Finally, as this narrative concludes the other students all join in with her laughter which has the intended purpose of showing support and shared understanding for their colleague.

DISCUSSION

In terms of our first research question ‘How do students define feedback?’, this study gives insights into medical students’ solicited definitions of feedback, thus extending previous work.19–21,43,44 Students appeared to conceptualise feedback in terms of its mode, type, provider, setting, formality and purpose, and they typically defined feedback as information on their performance. Feedback was almost exclusively understood as a one-way process; as information (examination results, grades or comments on written work) given to them rather than something occurring with them. This is in contrast to the current thinking and best practice for feed-

198

back in which feedback is conceptualised as a dialogue between the tutor and the student.4,45–47 Certainly the proportion of negative narratives around written feedback and grades was much higher than for positive narratives, supporting the results of Sargeant et al.,48 who found that monologic grades were perceived as less successful than verbal commentary. Indeed, from our narratives we can see that written feedback is most often monologic, i.e. constructed by the tutor away from the student and therefore preventing students from clarifying any points of ambiguity. In terms of our second research question ‘What types of feedback experiences do students narrate?’, we found that students were more likely to volunteer negative, rather than positive experiences of feedback. This phenomenon of preferentially sharing negative experiences has been shown in other focus group studies.49 We would suggest that it is our choice of methodology that has empowered students to share these negative stories. Although a lot of our contentrelated themes relating to our second research question supports previous research,19–22 it is our third question (i.e. the ‘whats’ and the ‘hows’) that extends current literature about feedback in medical education. We identified a number of process-related themes relating to research question three including emotional, metaphoric and pronominal talk and paralinguistic features such as laughter. We found the narratives littered with emotion talk and particularly negative emotion talk within the negative experiences. By choosing to narrate their stories with emotion our students emphasised how feedback made them feel.32 As emotion and memory are intertwined, those incidents with a strong emotional impact are more likely to be remembered and therefore shared.50,51 Negative emotion talk within the narratives was either directed towards the situation or the feedback provider. Indeed, in our second narrative our student expresses a negative emotion not about her performance when talking to the patient, but instead towards her ‘bitter and twisted’ tutor, despite over a year passing since the event took place. This suggests her negative feelings towards the tutor have persisted, and that any future feedback from this tutor may not carry with it any credibility. Credibility and supervisor engagement have recently been the subject of a number of publications and is an important factor in the success of an episode of feedback.24,52–55 Although emotion and credibility are different constructs, we would

ª 2014 John Wiley & Sons Ltd. MEDICAL EDUCATION 2014; 48: 189–203

Student feedback narratives suggest that these narratives illustrate that the emotion felt by a student during their feedback could influence their feelings towards the feedback provider and subsequent credibility judgements. In terms of pronominal talk, we see further evidence to support the content-related themes in that there is a sense of divide between students and their tutors or institutions. We can see from our example of pronominal use in Table 2 that students typically used the pronoun ‘they’ (during negative narratives) to describe feedback providers, which suggests that they feel on different sides from the feedback providers rather than part of the same team.41 This might go some way to explain why students feel that feedback is something that happens ‘to’ them rather than ‘with’ them. We found that metaphoric linguistic expressions were used by students in both positive and negative narratives of feedback. Kleres38 states that metaphors ‘allow for expression of emotional states that are otherwise inexpressible’ and that by looking at these we get a better understanding of how students conceptualise feedback within the context of their lived experiences. In our example of a positive narrative of feedback, the ‘gold dust’ metaphor has allowed us to understand not only why this was a positive experience, but perhaps more importantly, the suggestion of rarity directs us to see that this is not the norm from the student’s perspective. Within the negative narratives students often used the metaphor of FEEDBACK AS WAR, which showed that students felt on opposing sides of a battle between them and their feedback provider.42 Clearly this is not the ideal supportive environment that is required to ensure a positive feedback experience and goes further to perpetuate the sense of ‘them and us’ between the feedback provider and receiver. In terms of paralanguage, the students in this study employed a number of paralinguistic features in their narration of their feedback incidents (e.g. laughter, hedges and emphases). Laughter can be used throughout narration for a number of different purposes. It can be used for coping, both contextually and non-contextually, for power, for shared experience and in response to an incongruent or comical situation.56,57 Paralanguage was also employed to provide emphasis upon certain parts of the story that students felt were important. They also used these features, especially laughter, as part of a strategy for coping with the narration of what were at times difficult stories to recount.32,56

We can see from these narratives that there are still medical educators who deliver ineffectual and potentially harmful feedback despite readily available guidance on how to provide feedback. Previous research by Kluger and DeNisi58 has suggested that not all feedback is good feedback and that up to one-third of feedback interventions have a negative impact. It would seem from our narratives that it is the way feedback makes a student feel that has one of the biggest influences over the feedback legacy, both in terms of how memorable the event has been and perhaps by influencing the credibility judgements made by the student. This emotional issue is made even more complex by the findings of Fishbach et al.,59 who suggest that emotions experienced that are unrelated to the feedback itself can still influence how the student receives and perceives their feedback. This is an interesting and important area for consideration, as tutors and students will have to become aware of the student’s emotional state prior to the delivery of feedback and to decide whether it might be more appropriate to deliver the feedback at a later point (especially if the student is experiencing negative emotions). We can see this in our example of emotion talk from School 1 (Table 2), where the student tells us about feedback he receives around the same time as being told about the death of his patient, and he is clearly very upset about this more than 6 months after the event. We actually have no idea about the content of his feedback, although the student says that he ‘couldn’t concentrate on what they were saying’. This narrative goes to show that the legacy of this emotional event has persisted for quite some time and that in this case the student is unable to separate his negative emotional state about the way he was told about the patient’s death from the feedback itself. In summary, we found various key issues from our analysis of the interplay between content and process. We can see through the whats and hows that there is a sense of ‘them and us’ between students and feedback providers. Through our exploration of emotion, pronominal and metaphoric talk and laughter we can see that students continue to experience challenging emotions even when narrating their experiences quite some time after the events. Strengths and challenges In our research we sought to understand better students’ perspectives about feedback and to understand why feedback has been successful or not

ª 2014 John Wiley & Sons Ltd. MEDICAL EDUCATION 2014; 48: 189–203

199

L M Urquhart et al based upon our exploration of both the whats and hows of students’ narratives. Our study was a multischool study with no discernible differences found in the content-related themes between the three schools and therefore we suggest that our findings might be transferable to other institutions with undergraduate programmes. However, we do acknowledge that we had a higher proportion of junior than senior students in our study and this could have had some bearing on the results (for example we received more narratives from the classroom than from the workplace). In addition, a large proportion of our overall participants came from one school. This can be explained by our sampling process and by the venues for the focus groups. Although all students were invited to attend the focus groups, we relied upon students volunteering to participate. In School 1, the main teaching hospital was attached to the focus group venue and therefore interviews were more accessible to both preclinical and clinical students. This was not the case for Schools 2 and 3, where clinical attachments were more geographically disparate and distant from the focus group venue. Around 20% of the participants were graduate students who had previously completed another degree. We were unable to obtain data for all three schools, but for reference the percentage of graduate students at School 1 was 12% (L. M. Urquhart, personal communication). Although we note the recent publications around maturity and feedback,21 we found no difference in students’ solicited definitions of feedback or the issues expressed within the content of their narratives about whether feedback was successful or not according to stage of learning. Fishbach et al.’s59 paper suggests that rather than maturity it may be where students sit on the novice–expert continuum that affects their feedback reception. For example, consider the senior student who has taken a history many times (and feels expert) but who learns to perform arterial blood gases for the first time (and feels novice). Therefore, we do not suggest that there should be one type of feedback for pre-clinical versus clinical students, but instead the tutor and student should take into account whether the student is a novice or expert for any particular skill. As our study was cross-sectional rather than longitudinal we did not seek to address the question of whether students used these feedback episodes in their future learning. However, this would be an interesting topic for further exploration, particularly looking at the relationship between how emotionally charged an episode of feedback is with any subse-

200

quent behaviour change. Another interesting area for future research centres around Fishbach et al.’s59 work and how moving from novice to expert influences students in terms of their taking responsibility for their learning. We know that a mismatch exists between tutor and student perceptions, and that current research typically focuses on the tutor or the student rather than both. Therefore, our current research is considering the interactions between the feedback giver and receiver, including senior students using video reflexivity methodology.60 In a recent commentary by Murdoch-Eaton61 it is noted that most research to date on feedback has been retrospective in nature and is therefore open to internal and external influences. This video reflexivity, in showing feedback episodes back to students and tutors, should generate discussion from both students and tutors based upon what they actually see rather than what they remember, i.e. this is research that is ‘in the moment’. Implications We would suggest that sharing these narratives with students and tutors might improve the feedback experience through: 1

2

3

4

5

6

Sign-posting learning experiences to students that had not been previously considered as feedback and allowing tutors to recognise when this sign-posting is required. Helping students to develop a more sophisticated understanding of the dialogic nature of feedback and to feel that they are not alone in their experiences. Helping tutors understand the potential legacy that their feedback might have upon students, particularly feedback that yields a strong emotional response. Providing a platform to discuss any negative emotional response to feedback with students and to consider whether this emotional response is linked to the feedback itself or other factors. Tutors using the narratives to role model, where they might adopt aspects of other’s practice that is successful, while allowing for reflection of whether there might be anything within their own practice that might be detrimental to the feedback process. Sign-posting student discomfort around feedback, e.g. providing negative evaluation to institutions, allowing for role modelling around how to reflect upon and deal with feedback.

ª 2014 John Wiley & Sons Ltd. MEDICAL EDUCATION 2014; 48: 189–203

Student feedback narratives In summary, by adopting a narrative approach, we have unearthed new understandings of the lived feedback experiences of students. We would suggest that through sharing narratives, students, tutors and institutions might understand each other’s unique perspectives around feedback and can therefore become more engaged in the feedback dialogue.

Contributors: LMU conducted this research as part of her PhD at the University of Dundee. All authors were involved in the design of the study. LMU secured ethics approval and collected the data. CER helped with some of the data collection. LMU transcribed all data. All authors were involved in analysing part of the data and contributed to the development of the coding framework. LMU used the coding framework to code all data in Atlas-Ti. LMU wrote the paper, and this was edited by CER. JSK commented on various iterations of the paper. All authors have read and approved the final version. Acknowledgements: we would like to thank all of our participants for sharing their stories with us, as well as members of staff at all schools for helping us to contact students during the recruitment phase. We would also like to thank the reviewers for their very helpful suggestions during revision of our paper. Funding: this study was carried out as part of a PhD study by LMU and is therefore funded by the University of Dundee. Conflicts of interest: none. Ethical approval: obtained from the research ethics committee of School 1. Reciprocal ethical approval was obtained from Schools 2 and 3.

REFERENCES 1 Hattie J. Influences on student learning. Inaugural professorial address. University of Auckland 1999. 2 Hattie J, Timperley H. The power of feedback. Rev Educ Res 2007;77 (1):81–112. 3 Norcini J. The power of feedback. Med Educ 2010;44: 16–7. 4 Nicol DJ, Macfarlane-Dick D. Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Stud High Educ 2006;31 (2):199–218. 5 National Students Survey. Unistats from universities and colleges in the UK 2010. http://unistats.direct. gov.uk/. 6 De SK, Henke PK, Ailawadi G, Dimick JB, Colletti LM. Attending, house officer, and medical student perceptions about teaching in the third-year medical school general surgery clerkship. J Am Coll Surg 2004; 199 (6):932–42. 7 Irby DM. Teaching and learning in ambulatory care settings: a thematic review of the literature. Acad Med 1995;70 (10):898–931.

8 Gil DH, Heins M, Jones PB. Perceptions of medical school faculty members and students on clinical clerkship feedback. J Med Educ 1984;59 (11 Pt 1): 856–64. 9 Ende J. Feedback in clinical medical education. JAMA 1983;250 (6):777–81. 10 Hewson MG, Little ML. Giving feedback in medical education: verification of recommended techniques. J Gen Intern Med 1998;13 (2):111–6. 11 Bhattarai M. ABCDEFG IS - the principle of constructive feedback. JNMA J Nepal Med Assoc 2007;46 (167): 151–6. 12 Cantillon P, Sargeant J. Giving feedback in clinical settings. Br Med J 2008;337:a1961. 13 Branch WT Jr, Paranjape A. Feedback and reflection: teaching methods for clinical settings. Acad Med 2002; 1:1185–8. 14 McKimm J. Giving effective feedback. Br J Hosp Med 2009;70 (3):158–61. 15 Van de Ridder JMM, Stokking KM, McGaghie WC, ten Cate OTJ. What is feedback in clinical education? Med Educ 2008;42:189–97. 16 Sostok MA, Coberly L, Rouan G. Feedback process between faculty and students. Acad Med 2002;77 (3): 267. 17 Eraut M. Feedback. Learn Health Social Care 2006;5 (3): 111–8. 18 Jensen AR, Wright AS, Kim S, Horvath KD, Calhoun KE. Educational feedback in the operating room: a gap between resident and faculty perceptions. Am J Surg 2012;204 (2):248–55. 19 Pokorny H, Pickford P. Complexity, cues and relationships: student perceptions of feedback. Active Learn High Educ 2010;11 (1):21–30. 20 Poulos A, Mahony MJ. Effectiveness of feedback: the students’ perspective. Assess Eval High Educ 2008;33 (2):143–54. 21 Murdoch-Eaton D, Sargeant J. Maturational differences in undergraduate medical students’ perspectives about feedback. Med Educ 2012;46:711–21. 22 Eva KW, Armson H, Holmboe E, Lockyer J, Lovey E, Mann K, Sargeant J. Factors influencing responsiveness to feedback: on the interplay between fear, confidence, and reasoning processes. Adv Health Sci Educ Theory Pract 2012;17 (1):15–26. 23 Young P. ‘I might as well give up’: self-esteem and mature students’ feelings about feedback on assignments. J Furth High Educ 2000;24 (3):409–18. 24 Watling C, Driessen E, van der Vleuten CP, Vanstone M, Lingard L. Understanding responses to feedback: the potential and limitations of regulatory focus theory. Med Educ 2012;46:593–603. 25 Kluger AN, Van Dijk D. Feedback, the various tasks of the doctor, and the feed forward alternative. Med Educ 2010;44:1166–74. 26 Riessman C. Narrative Methods for the Human Sciences. Thousand Oaks, CA: Sage 2008. 27 Butcher SE. Narrative as a teaching strategy. J Corr Educ 2006;57 (3):195–208.

ª 2014 John Wiley & Sons Ltd. MEDICAL EDUCATION 2014; 48: 189–203

201

L M Urquhart et al 28 Johnston KE, Golombek PR. The transformative power of narrative in second language teaching. TESOL Q 2011;45 (3):486–509. 29 Atkinson D. Theorising how student teachers form their identities in initial teacher education. Br Educ Res J 2004;30 (3):379–94. 30 Lomas T, Cartwright T, Edginton T, Ridge D. ‘I was so done in that I just recognized it very plainly, ‘“you need to do something”’: men’s narratives of struggle, distress and turning to meditation. Health 2013;17 (2): 191–208. 31 Reismann C. Performing identities in illness narrative: masculinity and multiple sclerosis. Qual Res 2003;3 (1): 5–32. 32 Rees CE, Monrouuxe LV, Mcdonald LA. Narrative emotion and action. Analysing most memorable professionalism dilemmas. Med Educ 2013;47:80–96. 33 Monrouxe LV, Rees CE. “It’s just a clash of cultures”: emotional talk within medical students’ narratives of professionalism dilemmas. Adv Health Sci Educ Theory Pract 2012;17 (5):671–701. 34 Bleakely A. Stories as data, data as stories: making sense of narrative inquiry in clinical education. Med Educ 2005;39:534–40. 35 Crotty M. The Foundations of Social Research: Meaning and Perspective in the Research Process. London: Sage Publications 1998. 36 Barbour RS. Making sense of focus groups. Med Educ 2005;39:742–50. 37 Vaughn S, Schumm JS, Sinagub J. Focus Group Interviews in Education and Psychology. Thousand Oaks, CA: Sage Publications 1996. 38 Kleres J. Emotions and narrative analysis: a methodological approach. J Theor Soc Behav 2011;41 (2):182– 202. 39 Ritchie J, Spencer L. Qualitative analysis for applied policy research. In: Bryman A, Burgess RG, eds. Analysing Qualitative Data. London: Routledge 1994;173–94. 40 Labov W. Some further steps in narrative analysis. J Narrat Life Hist 1997;7 (1–4):395–415. 41 Rees CE, Monrouxe LV. ‘Is it alright if I-um-we unbutton your pyjama top now?’ Pronominal use in bedside teaching encounters. Commun Med 2008;5 (2):171–81. 42 Rees CE, Knight LV, Cleland JA. Medical educators’ metaphoric talk about their assessment relationships with students: ‘you don’t want to sort of be the one who sticks the knife in them’. Assess Eval High Educ 2009;34 (4):455–67. 43 Carless D. Differing perceptions in the feedback process. Stud High Educ 2006;31 (2):219. 44 Perera J, Lee N, Win K, Perera J, Wijesuriya L. Formative feedback to students: the mismatch between faculty perceptions and student expectations. Med Teach 2008;30 (4):395–9. 45 Nicol D. From monologue to dialogue. Improving written feedback processes in mass higher education. Assess Eval High Educ 2010;35 (5):501–17.

202

46 Jonsson A. Facilitating productive use of feedback in higher education. Active Learn High Educ 2012;14 (1): 63–76. 47 Yang M, Carless D. The feedback triangle and enhancement of dialogic feedback processes. Teach High Educ 2012;1:1–13. 48 Sargeant J, Eva KW, Armson H, Chesluk B, Dornan T, Holmboe E, Lockyer JM, Loney E, Mann KV, van der Vleuten CP. Features of assessment learners use to make informed self assessments of clinical performance. Med Educ 2011;45:636–47. 49 Geis SB, Fuller RL, Rush J. Lovers of AIDS victims – psychosocial stresses and counselling needs. Death Stud 1986;10 (1):43–53. 50 Levine LJ, Edelstein RS. Emotion and memory narrowing: a review and goal-relevance approach. Cogn Emot 2009;23 (5):833–75. 51 Laird JD, Wagener JJ, Halal M, Szegda M. Remembering what you feel - effects of emotion on memory. J Pers Soc Psychol 1982;42 (4):646–57. 52 Watling C, Driessen E, van der Vleuten CP, Lingard L. Learning from clinical work: the roles of learning cues and credibility judgements. Med Educ 2012;46 (2):192–200. 53 Bing-You RG, Paterson J, Levine MA. Feedback falling on deaf ears: residents’ receptivity to feedback tempered by sender credibility. Med Teach 1997;19 (1):40–4. 54 Gillespie M. Student–teacher connection in clinical nursing education. J Adv Nurs 2002;37 (6): 566–76. 55 Sargeant J, Mann K, Ferrier S. Exploring family physicians’ reactions to multisource feedback: perceptions of credibility and usefulness. Med Educ 2005; 39:497–504. 56 Rees C, Monrouxe L. Laughter for coping: medical students narrating professionalism dilemmas. In: Figley CR, Huggard P, Rees CE, eds. First Do No Self Harm: Understanding and Promoting Physician Stress Resilience. New York, NY: Oxford University Press. (in press) 57 Hay J. Functions of humor in the conversations of men and women. J Pragmat 2000;32 (6):709–42. 58 Kluger AN, DeNisi A. The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull 1996;119 (2):254–84. 59 Fishbach A, Ayel T, Finkelstein S. How positive and negative feedback motivate goal pursuit. Soc Person Psychol Compass 2010;4 (8):517–30. 60 Iedema RA, Jorm C, Long D, Braithwaite J, Travaglia J, Westbrook M. Turning the medical gaze in upon itself: root cause analysis and the investigation of clinical error. Soc Sci Med 2006;62 (7):1605–15. 61 Murdoch-Eaton D. Feedback: the complexity of selfperception and the transition from ‘transmit’ to ‘received and understood’. Med Educ 2012;46:538–40. 62 Oxford University Press. Oxford English Dictionaries. http://oxforddictionaries.com/definition/english/ pronoun. [Accessed 3 April 2013].

ª 2014 John Wiley & Sons Ltd. MEDICAL EDUCATION 2014; 48: 189–203

Student feedback narratives 63 Oxford University Press. Oxford English Dictionaries. http://oxforddictionaries.com/definition/english/ metaphor. [Accessed 3 April 2013]. 64 Rees CE, Knight LV, Wilkinson CE. Doctors being up there and we being down here: a metaphorical analysis of talk about student/doctor–patient relationships. Soc Sci Med 2007;65 (4):725–37. 65 Oxford University Press. Oxford English Dictionaries. http://oxforddictionaries.com/definition/english/ paralanguage. [Accessed 3 April 2013].

SUPPORTING INFORMATION

Additional Supporting Information may be found in the online version of this article: Box S1. Definitions of key process-related themes. Received 4 December 2012; editorial comments to author 4 February 2013, 6 June 2013; accepted for publication 4 July 2013

ª 2014 John Wiley & Sons Ltd. MEDICAL EDUCATION 2014; 48: 189–203

203