European Journal of Dental Education ISSN 1396-5883
Developing integrated clinical reasoning competencies in dental students using scaffolded case-based learning – empirical evidence T. C. Postma and J. G. White Department of Dental Management Sciences, University of Pretoria, Pretoria, South Africa
Keywords integrated clinical reasoning; 4C/ID-Model of Complex Learning; scaffolding; case-based learning; authenticity; part-whole-task simulation. Correspondence Thomas Corn e Postma Department of Dental Management Sciences School of Dentistry University of Pretoria Corner of Dr Savage Rd and Bophelo Road Gezina 0084 Pretoria South Africa Tel: +27 12 3192553 Fax: +27 12 3192171 e-mail: [email protected]. za Accepted: 22 May 2015 doi: 10.1111/eje.12159
Abstract Introduction: This study provides empirical evidence of the development of integrated clinical reasoning in the discipline-based School of Dentistry, University of Pretoria, South Africa. Students were exposed to case-based learning in comprehensive patient care (CPC) in the preclinical year of study, scaffolded by means of the fourcomponent instructional design model for complex learning. Methods: Progress test scores of third- to fifth-year dental students, who received case-based teaching and learning in the third year (2009–2011), were compared to the scores of preceding fourth- and fifth-year cohorts. These fourth- and fifth-year cohorts received content-based teaching concurrently with their clinical training in CPC. The progress test consisted of a complex case study and 32 MCQs on tracer conditions. Students had to gather the necessary information and had to make diagnostic and treatment-planning decisions. Results: Preclinical students who participated in the case-based teaching and learning achieved similar scores compared to final-year students who received lecture-based teaching and learning. Final-year students who participated in the case-based learning made three more correct clinical decisions per student, compared to those who received content-based teaching. Students struggled more with treatment-planning than with diagnostic decisions. Conclusion: The scaffolded case-based learning appears to contribute to accurate clinical decisions when compared to lecture-based teaching. It is suggested that the development of integrated reasoning competencies starts as early as possible in a dental curriculum, perhaps even in the preclinical year of study. Treatment-planning should receive particular attention.
Introduction There is a need in dental education to develop integrated clinical reasoning competencies to ensure the required competence at graduation. Tertiary teaching institutions are therefore obligated to continuously evaluate the ability of the students to make accurate clinical decisions (1, 2). Clinical reasoning is a difficult concept to teach (3, 4). Patients often present with multiple problems of varying degrees (5), which are complicated by their psychological health, social circumstances and other factors (4). Students should consider these factors to make accurate diagnostic and treatment-planning
ª 2015 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd
decisions, which can only be achieved through repeated practise (3, 5–7) in an authentic context (3, 5, 8). Teaching in an authentic context brings complexity into play (5, 8). To deal with complexity, teaching and learning should allow for systematically scaffolded exercises that enable students to gradually develop their knowledge structures (3, 5, 8). This would enable them to make accurate clinical decisions later on (9–12).
Institutional background Before 2009, undergraduate dental students at the School of Dentistry, University of Pretoria, South Africa, were exposed
1
Developing clinical reasoning
to clinical reasoning lectures during the fourth year of study in comprehensive patient care (CPC).1 They had to concurrently apply their knowledge during clinical training sessions. A curriculum renewal process in 2007 and 2008 revealed inadequacies in the way clinical reasoning have been taught. Therefore, the school implemented a case-based instructional design in 2009 to develop integrated clinical reasoning skills in the preclinical year of study (13). The objective was to prepare the students for clinical training. The instructional design used principles advocated in the four-component instructional design model for complex learning (4C/IDmodel) to scaffold the development of clinical reasoning skills over time (5, 8). Students had to analyse five complex case studies (Fig. 1 contains an example of a case study) for which they had to make diagnostic, aetiological, prognostic and treatment-planning decisions. The case studies were focused on tracer conditions (5), including dental caries, tooth wear, periodontal disease, pulpal and periapical conditions, tooth loss, as well as white and red oral mucosal lesions. Cardiovascular problems and the safe administration of local anaesthetics and antibiotics were also covered in the content. The tracer conditions were arranged in different configurations, affecting different teeth and anatomical structures in each case study. The case studies were sequenced in order of complexity. The first case study included dental caries, a cardiovascular problem and an oral mucosal problem. Diseases and conditions were systematically added to the subsequent case studies to increase the level of difficulty (5). The complex cases required the student to go through a systematic data collection process before decision-making could take place. Accurate decision-making is dependent on accurate data collection. This approach is synonymous with a partwhole-task approach, whereby a large part of a real-life event is being simulated to maintain the context of the exercise (4, 8). Learning relied on repetitive knowledge application followed by formative feedback (5). The hypothesis was that the preclinical students who were taught with the complex case studies would develop rudimentary knowledge structures at an early stage. It was expected that subsequent practise would lead to more accurate and precise diagnostic and treatment-planning decisions. It was further expected that those who participated in the case-based learning would have superior clinical reasoning competence compared with those who received lecture-based teaching.
Aim of the study The primary aim of this quantitative analysis was therefore to report the development of the clinical reasoning skills (diagnosis and treatment planning) over time (2009–2011) using progress test results. The secondary aim was to compare the progress test results of case-based cohorts with the results of the lecture-based cohorts.
1
Before 2009, Comprehensive Patient Care was a module in the undergraduate dental curriculum of the University of Pretoria
2
Postma & White
Methods Ethical approval and consent The Research Ethics Committee of the Faculty of Health Sciences, University of Pretoria, provided ethical clearance to conduct the research. Students gave written consent.
Target population and study design The researchers included the progress test scores of the thirdto fifth-year dental students (2009–2011) in this study (Fig. 2). Third-year cohorts A, D and E participated in the case-based intervention in 2009, 2010 and 2011, respectively. These groups served as the ‘intervention’ groups. The researchers followed the progress of Cohort A from year three (2009) to year five (2011) and the progress of Cohort D from year three (2010) to year four (2011). Cohorts B and C served as the ‘control’ groups. Cohorts B and C received lecture-based teaching in CPC in 2009 and 2008, respectively. The researchers followed the progress of Cohort B from year four (2009) to year five (2010). Fifth-year Cohort C served as an additional control.
The progress test The progress test consisted of a complex case study (Fig. 1), covering tracer conditions (5), followed by 32 multiple-choice questions (MCQs) that were designed according to institutional and scientific standards (14–17). The test cannot be made public as a large part of the original test is still in use since 2012. Figure 1 contains examples of typical questions. The progress test requires the students to systematically collect the data and to make 18 diagnostic decisions and 14 treatment-planning decisions. A thematic description of each question is displayed in Table 1. The treatment-planning decisions included complicated decisions related to four of the teeth described in the case study. The students had to determine the prognosis of the teeth and had to make decisions whether the teeth should be extracted, be restored, be treated with root canal therapy or be crowned. They also had to select the most appropriate tooth replacement strategy to replace the absent teeth; indicate how they would manage a white oral mucosal lesion; select a professional fluoride treatment regimen; select an appropriate local anaesthetic; indicate how they would manage the high blood pressure of the patient; and establish the need for prophylactic antibiotics. They also had to prioritise the treatment. These decisions were deliberately complicated by introducing a biopsychosocial context in which the decisions had to be made. Two to five subject experts evaluated and critiqued individual test questions before the test was implemented (18). The progress test was administered in the second semester with an open-book strategy to the third-, fourth- and fifth-year cohorts (2009–2011). The 3-year study period served as a trial phase for the test. A longer trial period was chosen because of the relatively small sample sizes of the cohorts (Table 2). The longer trial ª 2015 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd
Postma & White
Developing clinical reasoning
Typical case study Rapport-building conversation: During the rapport-building conversation you determine that the patient is 42 years old and employed as an executive at an IT company. She complains that she has limited free time due to the nature of her work. She refers to her previous dentist as an “idiot”. Main complaint: She requires a stronger denture because the one front tooth on the denture keeps breaking off. Another complaint is that cold and hot foodstuffs cause a sharp pain in the upper back jaw (left - and right-hand sides) for about ten seconds. She wants this discomfort relieved. The appearance o f the patient’s front teeth is important to her, and she is concerned because some have become black around the fillings. She also complains that her mouth is often quite dry. Dental history: She last visited her previous dentist two months ago to repair h er one-year-old upper plastic partial denture for the third time. Her teeth were extracted eight years ago. Medical history: The patient smokes 20 cigarettes a day. She has tried to stop but has not succeeded. She suffers from chronic sinusitis, for which she uses antihistamines, and she also uses medication to control her high blood pressure of 130/90. She had rheumatic fever as a child, but has no heart murmur or heart valve lesions as a result of the illness. She is allergic to cephalosporin. Extra-oral examination: Nothing abnormal is found. Intra-oral examination: The patient has a thick, yellow coating on the posterior third of the dorsal surface of the tongue. The buccal mucosa presents with a white thickening along the occlusal line. The following teeth are absent: 18, 12, 22, 28, 36 to 38, 46 to 48. The patient never had third molars. Periodontal examination: Clinically, a 2 mm recession is observed buccally at teeth 26, 27, 44 and 45. The patient presents with 4 mm pockets at all six locations around teeth 26 and 27. A periodontal probing depth of 5 mm is found distally at tooth 15, as well as surrounding teeth 16 and 17. There are thick layers of supra -gingival calculus adjacent to the buccal surfaces of the maxillary molars and the lingual anterior of the mandibular anterior teeth, and the palatal and lingual surfaces of the teeth are stained from smoking. The gingiva is soft and spongy, enlarged in relation to the calculus and bleeds upon probing. A radiographic examination shows horizontal bone loss adjacent to and between the maxillary molars. Hard-tissue examination: Existing restorations include a large BDP Class III restoration and a smaller MP Class III restoration on tooth 13. Furthermore, tooth 16 presents with a MOD amalgam . The following teeth present with B Class V composite restorations near the gingival margin: 15, 14, 13, 23, 24 and 25. Black marginal discolouration surrounds the Class V and Class III restorations. Most of these discoloured margins appear to be open dur ing probing, especially close to the gingiva. Both the palatal cusps of tooth 16 are fractured but the pulp is not exposed. The open dentine palatally at toot h 16 is very sensitive to cold and heat, as is tooth 26. Tooth 14 has a small cavity with soft wal ls in a pit, mesial on the occlusal surface (too small to be detected on the bitewing). Tooth 27 presents with a dark brown cavity with soft walls dista l on the occlusal surface. Tooth 27 is slightly sensitive to percussion but does not respond to cold and heat tests or an electrical pulp test. All incisors and canines have open dentine on the incisal edges. None of these lesions are sensitive to temperature stimuli. Teeth 44 and 45 have small concave lesions into the dentine buccally below the CEJ in the gingival recession areas. Small radiolucent areas are identified under the distal contact point of tooth 26 and the mesial contact poi nt of tooth 27. The lesion on tooth 26 extends into the DEJ, whereas the lesion mesial on tooth 27 extends 0,5 mm into the dentine. A large radiolucent area starting below the distal contact point of tooth 27 extends into the pulp. A small radioluc ent area is observed at the disto-buccal root apex of tooth 27 (the lamina dura cannot be followed in this area). Tooth 13 has a completed root canal treatment which appears to be sound. Self-care practices: The plaque index is 30%, with most of the plaque found interproximally. The patient informs you that she sucks on mints to soothe a dry mouth. She generally brushes her teeth in the morning using a vigorous scrubbing technique. As a rule, she does not floss, and sometimes she does not remove her dentures when brushing. She appears motivated while health promotion is being discussed.
Examples of typical questions Select the most probable diagnosis for the radiolucent distal of the 26. a. Cervical burnout b. Incipient interproximal caries c. Moderate interproximal caries d. Advanced interproximal caries e. Severe interproximal caries
Select the most probable cause for the concave lesion buccally of the 45. a. Attrition b. Abrasion c. Erosion d. Erosion and abrasion e. Root caries
Select the most probable pulpal diagnosis for tooth 27. a. Normal pulp b. Reversible pulpitis c. Irreversible pulpitis d. Necrotic pulp
Select the most appropriate treatment for tooth 27. a. Tooth extraction b. MOD amalgam restoration c. Root canal treatment (RCT) and post crown d. Periodontal treatment and crown
Fig. 1. Example of a case study and progress test questions.
period allowed the researchers to draw inferences on the performance of the test based on a larger quantity of pooled data. Quality control measures after the test included the level of difficulty (LOD), and Discrimination Index (DI) evaluations (14). LOD is defined as the percentage of students who answered a question correctly. The DI is defined as the difference of the ratio of correctly answered questions of the 25% top performing students minus the ratio of correctly answered questions for the 25% worst performing students for each question. A DI of 0.30 or higher was considered to be ideal (14). Due to the multilevel nature of the study, the researchers calculated the DI for the pooled data, the diagnostic and
ª 2015 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd
treatment-planning components of the test and for the third, fourth and fifth years of study.
Statistical methods This study reports the changes in progress test, diagnostic and treatment-planning scores with descriptive statistics. The researchers analysed the difference in LOD for Cohort A for 2009 to 2011 with the Chi-square test. They also used the Chi-square test to analyse the difference in the LOD, of individual questions, between the intervention and control groups. P < 0.05 was considered to be statistically significant. 3
Developing clinical reasoning
Postma & White
Year
2009
2010
2011
Third study year
Cohort A
Cohort D
Cohort E
Intervention group – cased-based teaching and learning
Intervention group – cased-based teaching and learning
Intervention group – cased-based teaching and learning
Fourth study year
Cohort B
Cohort A
Cohort D
Control group – lecture-based teaching and learning
Intervention group – cased-based teaching and learning
Intervention group – cased-based teaching and learning
Fifth study year
Cohort C
Cohort B
Cohort A
Control group – lecture-based teaching and learning
Control group – lecture-based teaching and learning
Intervention group – content-based teaching and learning
and 18 than Cohort B. Cohort D outperformed students from Cohort B in terms of questions 8, 9, 11, 16, 20, 21, 27, 30 and 31. Cohort D only achieved a statistically lower score for Question 2 when compared to Cohort B. Only two statistically significant differences were found between the two-fourth-year intervention groups, cohorts A and D. These differences were observed for questions 2 and 19.
Differences between the intervention and control groups – fifth year of study Cohort A achieved statistically higher scores for questions 4, 5, 6, 7, 9, 16, 19 and 31 when compared to Cohort B (Table 4) whilst Cohort A outperformed students from Cohort C in terms of questions 1, 4, 6, 11, 12, 13, 16, 26, 27 and 31. Five statistical differences were found between the control groups, cohorts B and C.
Properties of discrimination of the individual test questions Fig. 2. Quasi-experimental design of the study.
Results Changes in progress test scores over time Progress test scores generally improved over time for all the cohorts who wrote the test more than once (Table 2). The standard deviations (SD) of the progress test scores of the intervention groups became smaller over time, whilst the SD of the control group, Cohort B, increased over time. The same trends were observed for the diagnostic and treatment-planning component scores (Table 2). Table 3 contains the progression of Cohort A in terms of the individual progress test questions over the 3-year period. Cohort A achieved a 12% improvement from year three to year five. Significant improvements were observed for questions 2, 4, 6, 9, 12, 26, 27, 29 and 31.
Twenty-three of the thirty-two questions displayed a DI of 0.30 or higher in at least one of the different contexts for which it was tested (Table 1), whilst Question 28 achieved a DI of 0.29. Questions 3, 15, 18, 20 and 22 displayed a maximum DI, ranging from 0.22 to 0.25. Both questions 14 and 23 had a maximum DI of 0.18. Question 17 showed a maximum DI of 0.13. For two-thirds of the questions, the properties of discrimination were higher for the third-year cohorts compared to the fifth-year cohorts (Table 1).
Discussion The study empirically illustrates the development of integrated clinical reasoning competencies in a discipline-based dental school from the preclinical to the final year of study. What makes this educational intervention unique is the adoption of the 4C/ID-model (5, 8) to scaffold case-based learning and the use of a new variant of the progress test to measure changes in clinical reasoning over time.
Differences between the diagnostic and treatment-planning component scores
The development of clinical reasoning competencies
The diagnostic component scores were higher compared with the treatment-planning component scores (Table 2). The SDs for the treatment-planning component were in turn higher than the SDs for the diagnostic component. Questions 4, 10, 14 and 17 were the easiest questions for the final-year students in the diagnostic component (Table 4). Questions 20, 22, 23 and 30 were the most difficult questions in the treatment-planning component.
The results of this study (Tables 2 and 3) illustrate that the diagnostic and treatment-planning decisions of the intervention groups increased in accuracy over time. These findings were strengthened by the decrease in SD and DI from year three to five, which suggest a decrease in the variability of the decisionmaking amongst students over time. Literature indeed suggests that clinicians with more experience possess more integrated knowledge structures compared with less experienced clinicians. The more experienced clinician is therefore able to make more correct decisions compared with the less experienced clinician (9–12). Challenging third-year dental students with complicated case studies, for the first time, is therefore destined to elicit variable and inaccurate decision-making. However, through repetitive practice, their reasoning should become more accurate over time (3, 5–7).
Differences between the intervention and control groups – fourth year of study Cohort A achieved statistically higher scores for questions 8, 21, 25, 27, 30 and 31 when compared to Cohort B (Table 5). Cohort A achieved statistically lower scores for questions 15 4
ª 2015 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd
Postma & White
Developing clinical reasoning
TABLE 1. Description of content of MCQ stems, test component, ratio of correctly answered questions and properties of discrimination (Please refer to the key at the bottom of the table) Discrimination Index (Pooled data) Q
Thematic Description
Cmp
LOD Y5
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
Diagnosis of an oral mucosal lesion Diagnosis of a pulpal condition Diagnosis of a pulpal condition Diagnosis of a periapical condition Diagnosis of a tooth-wear lesion (attrition) Diagnosis of a tooth-wear lesion (attrition and erosion) Diagnosis of a tooth-wear (abrasion)/caries lesion Diagnosis of periodontitis Diagnosis of interproximal dental caries Diagnosis of a tooth fracture Management of an oral mucosal lesion Aetiology of tooth wear (attrition/opposing porcelain crown) Aetiology of tooth wear (attrition and erosion) Aetiology of tooth wear (abrasion) Aetiology of dental caries Aetiology of tooth wear (attrition) Caries risk assessment Aetiology of periodontitis Identification of aetiology of pulpal condition Selection of treatment option pertaining to attrition Determination of prognosis of a compromised tooth and subsequent treatment Selection of tooth-loss treatment based on patient’s bio-psychosocial status Determination of prognosis of a compromised tooth and subsequent treatment Determination of prognosis of a compromised tooth and subsequent treatment Determination of prognosis of a compromised tooth and subsequent treatment Treatment prioritisation Treatment prioritisation Determination of prognosis of a compromised tooth and subsequent treatment Selection of appropriate therapeutic treatment option Decision for the safe administration of local anaesthetic Decision to give prophylactic antibiotic (based on patient’s medical history) Decision to manage a patient with high blood pressure safely
Dx Dx Dx Dx Dx Dx Dx Dx Dx Dx Rx Dx Dx Dx Dx Dx Dx Dx Dx Rx Rx Rx Rx Rx Rx Rx Rx Rx Rx Rx Rx Rx
0.55 0.80 0.78 0.90 0.86 0.56 0.62 0.72 0.45 0.98 0.75 0.74 0.69 0.89 0.68 0.72 0.98 0.80 0.58 0.14 0.71 0.26 0.38 0.70 0.79 0.80 0.77 0.80 0.92 0.33 0.45 0.48
selection selection selection selection
selection
PT
Dx
0.37 0.34 0.17 0.28 0.28 0.53 0.52 0.35 0.50 0.21 0.41 0.41 0.39 0.12 0.13 0.39 0.08 0.17 0.26 0.18 0.39 0.17 0.09 0.31 0.17 0.37 0.40 0.23 0.26 0.23 0.56 0.28
0.33 0.37 0.23 0.34 0.35 0.59 0.63 0.44 0.48 0.23
Rx
Y3 0.34 0.51 0.22 0.51 0.46 0.39 0.63 0.44 0.47 0.32 0.50 0.31 0.33 0.18 0.05 0.34 0.13 0.25 0.30 0.06 0.42 0.14 0.01 0.30 0.22 0.26 0.60 0.29 0.36 0.45 0.54 0.31
0.42 0.54 0.43 0.15 0.16 0.45 0.10 0.22 0.37 0.25 0.47 0.23 0.18 0.40 0.31 0.38 0.58 0.25 0.26 0.46 0.64 0.42
Y4
Y5
0.45 0.13 0.16 0.08 0.16 0.45 0.42 0.47 0.45 0.13 0.39 0.42 0.42 0.18 0.18 0.34 0.05 0.03 0.34 0.21 0.47 0.16 0.08 0.18 0.29 0.29 0.29 0.21 0.11 0.26 0.50 0.24
0.47 0.32 0.18 0.19 0.17 0.49 0.46 0.22 0.47 0.06 0.42 0.41 0.33 0.13 0.22 0.57 0.06 0.17 0.19 0.25 0.33 0.12 0.01 0.18 0.06 0.30 0.32 0.18 0.16 0.14 0.71 0.44
Key: Q, Question number; Cmp, Progress test component (Dx, Diagnosis; Rx: Treatment planning); LOD, Level of difficulty; Y5, Year 5; PT, Progress Test; Y4, Year 4; Y3, Year 3.
TABLE 2. Progress test scores over time Cohort
A
N 39 Study year 3 Year 2009 Case studies Yes Progress test X 63.7 SD 12.45 Diagnostic component X 70.37 SD 15.73 Treatment component X 55.13 SD 14
A
A
B
B
C
D
D
E
39 4 2010 Yes
36 5 2011 Yes
49 4 2009 No
48 5 2010 No
48 5 2009 No
54 3 2010 Yes
48 4 2011 Yes
49 3 2011 Yes
66.59 10.69
75.69 6.93
58.74 9.33
63.67 11.14
64.97 9.71
62.27 14.52
68.95 10.4
61.54 11.57
73.79 12.99
83.64 7.96
67.35 12.45
68.87 14.66
71.75 11.34
67.18 16.45
73.38 12.71
67.01 12.9
57.33 12.53
65.47 11.13
47.67 9.82
56.99 14.4
56.25 13.87
55.95 19.01
63.24 13.66
54.52 14.56
X: mean; SD: Standard deviation.
ª 2015 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd
5
Developing clinical reasoning
Postma & White
TABLE 3. Ratio of correctly answered questions for Cohort A – progression over time
Question
2009 Year 3 (n = 39)
2010 Year 4 (n = 39)
2011 Year 5 (n = 36)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
0.59 0.62 0.67 0.72 0.85 0.62 0.56 0.62 0.36 0.95 0.74 0.64 0.82 0.82 0.85 0.87 1.00 0.59 0.54 0.18 0.74 0.36 0.15 0.56 0.59 0.74 0.67 0.87 0.82 0.38 0.49 0.41
0.69 0.69 0.74 0.95 0.85 0.64 0.46 0.85 0.54 0.90 0.77 0.67 0.85 0.95 0.51 0.85 0.95 0.62 0.59 0.15 0.59 0.26 0.21 0.59 0.72 0.82 0.85 0.74 0.95 0.33 0.62 0.44
0.78 0.83 0.81 1.00 0.97 0.83 0.75 0.78 0.61 0.97 0.89 0.89 0.83 0.94 0.69 0.89 1.00 0.75 0.72 0.22 0.75 0.22 0.33 0.72 0.72 0.94 0.89 0.78 0.97 0.31 0.81 0.61
Difference: Year 3 to Year 5 Chi-square test P ns*
Developing integrated clinical reasoning competencies in dental students using scaffolded case-based learning – empirical evidence T. C. Postma and J. G. White Department of Dental Management Sciences, University of Pretoria, Pretoria, South Africa
Keywords integrated clinical reasoning; 4C/ID-Model of Complex Learning; scaffolding; case-based learning; authenticity; part-whole-task simulation. Correspondence Thomas Corn e Postma Department of Dental Management Sciences School of Dentistry University of Pretoria Corner of Dr Savage Rd and Bophelo Road Gezina 0084 Pretoria South Africa Tel: +27 12 3192553 Fax: +27 12 3192171 e-mail: [email protected]. za Accepted: 22 May 2015 doi: 10.1111/eje.12159
Abstract Introduction: This study provides empirical evidence of the development of integrated clinical reasoning in the discipline-based School of Dentistry, University of Pretoria, South Africa. Students were exposed to case-based learning in comprehensive patient care (CPC) in the preclinical year of study, scaffolded by means of the fourcomponent instructional design model for complex learning. Methods: Progress test scores of third- to fifth-year dental students, who received case-based teaching and learning in the third year (2009–2011), were compared to the scores of preceding fourth- and fifth-year cohorts. These fourth- and fifth-year cohorts received content-based teaching concurrently with their clinical training in CPC. The progress test consisted of a complex case study and 32 MCQs on tracer conditions. Students had to gather the necessary information and had to make diagnostic and treatment-planning decisions. Results: Preclinical students who participated in the case-based teaching and learning achieved similar scores compared to final-year students who received lecture-based teaching and learning. Final-year students who participated in the case-based learning made three more correct clinical decisions per student, compared to those who received content-based teaching. Students struggled more with treatment-planning than with diagnostic decisions. Conclusion: The scaffolded case-based learning appears to contribute to accurate clinical decisions when compared to lecture-based teaching. It is suggested that the development of integrated reasoning competencies starts as early as possible in a dental curriculum, perhaps even in the preclinical year of study. Treatment-planning should receive particular attention.
Introduction There is a need in dental education to develop integrated clinical reasoning competencies to ensure the required competence at graduation. Tertiary teaching institutions are therefore obligated to continuously evaluate the ability of the students to make accurate clinical decisions (1, 2). Clinical reasoning is a difficult concept to teach (3, 4). Patients often present with multiple problems of varying degrees (5), which are complicated by their psychological health, social circumstances and other factors (4). Students should consider these factors to make accurate diagnostic and treatment-planning
ª 2015 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd
decisions, which can only be achieved through repeated practise (3, 5–7) in an authentic context (3, 5, 8). Teaching in an authentic context brings complexity into play (5, 8). To deal with complexity, teaching and learning should allow for systematically scaffolded exercises that enable students to gradually develop their knowledge structures (3, 5, 8). This would enable them to make accurate clinical decisions later on (9–12).
Institutional background Before 2009, undergraduate dental students at the School of Dentistry, University of Pretoria, South Africa, were exposed
1
Developing clinical reasoning
to clinical reasoning lectures during the fourth year of study in comprehensive patient care (CPC).1 They had to concurrently apply their knowledge during clinical training sessions. A curriculum renewal process in 2007 and 2008 revealed inadequacies in the way clinical reasoning have been taught. Therefore, the school implemented a case-based instructional design in 2009 to develop integrated clinical reasoning skills in the preclinical year of study (13). The objective was to prepare the students for clinical training. The instructional design used principles advocated in the four-component instructional design model for complex learning (4C/IDmodel) to scaffold the development of clinical reasoning skills over time (5, 8). Students had to analyse five complex case studies (Fig. 1 contains an example of a case study) for which they had to make diagnostic, aetiological, prognostic and treatment-planning decisions. The case studies were focused on tracer conditions (5), including dental caries, tooth wear, periodontal disease, pulpal and periapical conditions, tooth loss, as well as white and red oral mucosal lesions. Cardiovascular problems and the safe administration of local anaesthetics and antibiotics were also covered in the content. The tracer conditions were arranged in different configurations, affecting different teeth and anatomical structures in each case study. The case studies were sequenced in order of complexity. The first case study included dental caries, a cardiovascular problem and an oral mucosal problem. Diseases and conditions were systematically added to the subsequent case studies to increase the level of difficulty (5). The complex cases required the student to go through a systematic data collection process before decision-making could take place. Accurate decision-making is dependent on accurate data collection. This approach is synonymous with a partwhole-task approach, whereby a large part of a real-life event is being simulated to maintain the context of the exercise (4, 8). Learning relied on repetitive knowledge application followed by formative feedback (5). The hypothesis was that the preclinical students who were taught with the complex case studies would develop rudimentary knowledge structures at an early stage. It was expected that subsequent practise would lead to more accurate and precise diagnostic and treatment-planning decisions. It was further expected that those who participated in the case-based learning would have superior clinical reasoning competence compared with those who received lecture-based teaching.
Aim of the study The primary aim of this quantitative analysis was therefore to report the development of the clinical reasoning skills (diagnosis and treatment planning) over time (2009–2011) using progress test results. The secondary aim was to compare the progress test results of case-based cohorts with the results of the lecture-based cohorts.
1
Before 2009, Comprehensive Patient Care was a module in the undergraduate dental curriculum of the University of Pretoria
2
Postma & White
Methods Ethical approval and consent The Research Ethics Committee of the Faculty of Health Sciences, University of Pretoria, provided ethical clearance to conduct the research. Students gave written consent.
Target population and study design The researchers included the progress test scores of the thirdto fifth-year dental students (2009–2011) in this study (Fig. 2). Third-year cohorts A, D and E participated in the case-based intervention in 2009, 2010 and 2011, respectively. These groups served as the ‘intervention’ groups. The researchers followed the progress of Cohort A from year three (2009) to year five (2011) and the progress of Cohort D from year three (2010) to year four (2011). Cohorts B and C served as the ‘control’ groups. Cohorts B and C received lecture-based teaching in CPC in 2009 and 2008, respectively. The researchers followed the progress of Cohort B from year four (2009) to year five (2010). Fifth-year Cohort C served as an additional control.
The progress test The progress test consisted of a complex case study (Fig. 1), covering tracer conditions (5), followed by 32 multiple-choice questions (MCQs) that were designed according to institutional and scientific standards (14–17). The test cannot be made public as a large part of the original test is still in use since 2012. Figure 1 contains examples of typical questions. The progress test requires the students to systematically collect the data and to make 18 diagnostic decisions and 14 treatment-planning decisions. A thematic description of each question is displayed in Table 1. The treatment-planning decisions included complicated decisions related to four of the teeth described in the case study. The students had to determine the prognosis of the teeth and had to make decisions whether the teeth should be extracted, be restored, be treated with root canal therapy or be crowned. They also had to select the most appropriate tooth replacement strategy to replace the absent teeth; indicate how they would manage a white oral mucosal lesion; select a professional fluoride treatment regimen; select an appropriate local anaesthetic; indicate how they would manage the high blood pressure of the patient; and establish the need for prophylactic antibiotics. They also had to prioritise the treatment. These decisions were deliberately complicated by introducing a biopsychosocial context in which the decisions had to be made. Two to five subject experts evaluated and critiqued individual test questions before the test was implemented (18). The progress test was administered in the second semester with an open-book strategy to the third-, fourth- and fifth-year cohorts (2009–2011). The 3-year study period served as a trial phase for the test. A longer trial period was chosen because of the relatively small sample sizes of the cohorts (Table 2). The longer trial ª 2015 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd
Postma & White
Developing clinical reasoning
Typical case study Rapport-building conversation: During the rapport-building conversation you determine that the patient is 42 years old and employed as an executive at an IT company. She complains that she has limited free time due to the nature of her work. She refers to her previous dentist as an “idiot”. Main complaint: She requires a stronger denture because the one front tooth on the denture keeps breaking off. Another complaint is that cold and hot foodstuffs cause a sharp pain in the upper back jaw (left - and right-hand sides) for about ten seconds. She wants this discomfort relieved. The appearance o f the patient’s front teeth is important to her, and she is concerned because some have become black around the fillings. She also complains that her mouth is often quite dry. Dental history: She last visited her previous dentist two months ago to repair h er one-year-old upper plastic partial denture for the third time. Her teeth were extracted eight years ago. Medical history: The patient smokes 20 cigarettes a day. She has tried to stop but has not succeeded. She suffers from chronic sinusitis, for which she uses antihistamines, and she also uses medication to control her high blood pressure of 130/90. She had rheumatic fever as a child, but has no heart murmur or heart valve lesions as a result of the illness. She is allergic to cephalosporin. Extra-oral examination: Nothing abnormal is found. Intra-oral examination: The patient has a thick, yellow coating on the posterior third of the dorsal surface of the tongue. The buccal mucosa presents with a white thickening along the occlusal line. The following teeth are absent: 18, 12, 22, 28, 36 to 38, 46 to 48. The patient never had third molars. Periodontal examination: Clinically, a 2 mm recession is observed buccally at teeth 26, 27, 44 and 45. The patient presents with 4 mm pockets at all six locations around teeth 26 and 27. A periodontal probing depth of 5 mm is found distally at tooth 15, as well as surrounding teeth 16 and 17. There are thick layers of supra -gingival calculus adjacent to the buccal surfaces of the maxillary molars and the lingual anterior of the mandibular anterior teeth, and the palatal and lingual surfaces of the teeth are stained from smoking. The gingiva is soft and spongy, enlarged in relation to the calculus and bleeds upon probing. A radiographic examination shows horizontal bone loss adjacent to and between the maxillary molars. Hard-tissue examination: Existing restorations include a large BDP Class III restoration and a smaller MP Class III restoration on tooth 13. Furthermore, tooth 16 presents with a MOD amalgam . The following teeth present with B Class V composite restorations near the gingival margin: 15, 14, 13, 23, 24 and 25. Black marginal discolouration surrounds the Class V and Class III restorations. Most of these discoloured margins appear to be open dur ing probing, especially close to the gingiva. Both the palatal cusps of tooth 16 are fractured but the pulp is not exposed. The open dentine palatally at toot h 16 is very sensitive to cold and heat, as is tooth 26. Tooth 14 has a small cavity with soft wal ls in a pit, mesial on the occlusal surface (too small to be detected on the bitewing). Tooth 27 presents with a dark brown cavity with soft walls dista l on the occlusal surface. Tooth 27 is slightly sensitive to percussion but does not respond to cold and heat tests or an electrical pulp test. All incisors and canines have open dentine on the incisal edges. None of these lesions are sensitive to temperature stimuli. Teeth 44 and 45 have small concave lesions into the dentine buccally below the CEJ in the gingival recession areas. Small radiolucent areas are identified under the distal contact point of tooth 26 and the mesial contact poi nt of tooth 27. The lesion on tooth 26 extends into the DEJ, whereas the lesion mesial on tooth 27 extends 0,5 mm into the dentine. A large radiolucent area starting below the distal contact point of tooth 27 extends into the pulp. A small radioluc ent area is observed at the disto-buccal root apex of tooth 27 (the lamina dura cannot be followed in this area). Tooth 13 has a completed root canal treatment which appears to be sound. Self-care practices: The plaque index is 30%, with most of the plaque found interproximally. The patient informs you that she sucks on mints to soothe a dry mouth. She generally brushes her teeth in the morning using a vigorous scrubbing technique. As a rule, she does not floss, and sometimes she does not remove her dentures when brushing. She appears motivated while health promotion is being discussed.
Examples of typical questions Select the most probable diagnosis for the radiolucent distal of the 26. a. Cervical burnout b. Incipient interproximal caries c. Moderate interproximal caries d. Advanced interproximal caries e. Severe interproximal caries
Select the most probable cause for the concave lesion buccally of the 45. a. Attrition b. Abrasion c. Erosion d. Erosion and abrasion e. Root caries
Select the most probable pulpal diagnosis for tooth 27. a. Normal pulp b. Reversible pulpitis c. Irreversible pulpitis d. Necrotic pulp
Select the most appropriate treatment for tooth 27. a. Tooth extraction b. MOD amalgam restoration c. Root canal treatment (RCT) and post crown d. Periodontal treatment and crown
Fig. 1. Example of a case study and progress test questions.
period allowed the researchers to draw inferences on the performance of the test based on a larger quantity of pooled data. Quality control measures after the test included the level of difficulty (LOD), and Discrimination Index (DI) evaluations (14). LOD is defined as the percentage of students who answered a question correctly. The DI is defined as the difference of the ratio of correctly answered questions of the 25% top performing students minus the ratio of correctly answered questions for the 25% worst performing students for each question. A DI of 0.30 or higher was considered to be ideal (14). Due to the multilevel nature of the study, the researchers calculated the DI for the pooled data, the diagnostic and
ª 2015 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd
treatment-planning components of the test and for the third, fourth and fifth years of study.
Statistical methods This study reports the changes in progress test, diagnostic and treatment-planning scores with descriptive statistics. The researchers analysed the difference in LOD for Cohort A for 2009 to 2011 with the Chi-square test. They also used the Chi-square test to analyse the difference in the LOD, of individual questions, between the intervention and control groups. P < 0.05 was considered to be statistically significant. 3
Developing clinical reasoning
Postma & White
Year
2009
2010
2011
Third study year
Cohort A
Cohort D
Cohort E
Intervention group – cased-based teaching and learning
Intervention group – cased-based teaching and learning
Intervention group – cased-based teaching and learning
Fourth study year
Cohort B
Cohort A
Cohort D
Control group – lecture-based teaching and learning
Intervention group – cased-based teaching and learning
Intervention group – cased-based teaching and learning
Fifth study year
Cohort C
Cohort B
Cohort A
Control group – lecture-based teaching and learning
Control group – lecture-based teaching and learning
Intervention group – content-based teaching and learning
and 18 than Cohort B. Cohort D outperformed students from Cohort B in terms of questions 8, 9, 11, 16, 20, 21, 27, 30 and 31. Cohort D only achieved a statistically lower score for Question 2 when compared to Cohort B. Only two statistically significant differences were found between the two-fourth-year intervention groups, cohorts A and D. These differences were observed for questions 2 and 19.
Differences between the intervention and control groups – fifth year of study Cohort A achieved statistically higher scores for questions 4, 5, 6, 7, 9, 16, 19 and 31 when compared to Cohort B (Table 4) whilst Cohort A outperformed students from Cohort C in terms of questions 1, 4, 6, 11, 12, 13, 16, 26, 27 and 31. Five statistical differences were found between the control groups, cohorts B and C.
Properties of discrimination of the individual test questions Fig. 2. Quasi-experimental design of the study.
Results Changes in progress test scores over time Progress test scores generally improved over time for all the cohorts who wrote the test more than once (Table 2). The standard deviations (SD) of the progress test scores of the intervention groups became smaller over time, whilst the SD of the control group, Cohort B, increased over time. The same trends were observed for the diagnostic and treatment-planning component scores (Table 2). Table 3 contains the progression of Cohort A in terms of the individual progress test questions over the 3-year period. Cohort A achieved a 12% improvement from year three to year five. Significant improvements were observed for questions 2, 4, 6, 9, 12, 26, 27, 29 and 31.
Twenty-three of the thirty-two questions displayed a DI of 0.30 or higher in at least one of the different contexts for which it was tested (Table 1), whilst Question 28 achieved a DI of 0.29. Questions 3, 15, 18, 20 and 22 displayed a maximum DI, ranging from 0.22 to 0.25. Both questions 14 and 23 had a maximum DI of 0.18. Question 17 showed a maximum DI of 0.13. For two-thirds of the questions, the properties of discrimination were higher for the third-year cohorts compared to the fifth-year cohorts (Table 1).
Discussion The study empirically illustrates the development of integrated clinical reasoning competencies in a discipline-based dental school from the preclinical to the final year of study. What makes this educational intervention unique is the adoption of the 4C/ID-model (5, 8) to scaffold case-based learning and the use of a new variant of the progress test to measure changes in clinical reasoning over time.
Differences between the diagnostic and treatment-planning component scores
The development of clinical reasoning competencies
The diagnostic component scores were higher compared with the treatment-planning component scores (Table 2). The SDs for the treatment-planning component were in turn higher than the SDs for the diagnostic component. Questions 4, 10, 14 and 17 were the easiest questions for the final-year students in the diagnostic component (Table 4). Questions 20, 22, 23 and 30 were the most difficult questions in the treatment-planning component.
The results of this study (Tables 2 and 3) illustrate that the diagnostic and treatment-planning decisions of the intervention groups increased in accuracy over time. These findings were strengthened by the decrease in SD and DI from year three to five, which suggest a decrease in the variability of the decisionmaking amongst students over time. Literature indeed suggests that clinicians with more experience possess more integrated knowledge structures compared with less experienced clinicians. The more experienced clinician is therefore able to make more correct decisions compared with the less experienced clinician (9–12). Challenging third-year dental students with complicated case studies, for the first time, is therefore destined to elicit variable and inaccurate decision-making. However, through repetitive practice, their reasoning should become more accurate over time (3, 5–7).
Differences between the intervention and control groups – fourth year of study Cohort A achieved statistically higher scores for questions 8, 21, 25, 27, 30 and 31 when compared to Cohort B (Table 5). Cohort A achieved statistically lower scores for questions 15 4
ª 2015 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd
Postma & White
Developing clinical reasoning
TABLE 1. Description of content of MCQ stems, test component, ratio of correctly answered questions and properties of discrimination (Please refer to the key at the bottom of the table) Discrimination Index (Pooled data) Q
Thematic Description
Cmp
LOD Y5
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
Diagnosis of an oral mucosal lesion Diagnosis of a pulpal condition Diagnosis of a pulpal condition Diagnosis of a periapical condition Diagnosis of a tooth-wear lesion (attrition) Diagnosis of a tooth-wear lesion (attrition and erosion) Diagnosis of a tooth-wear (abrasion)/caries lesion Diagnosis of periodontitis Diagnosis of interproximal dental caries Diagnosis of a tooth fracture Management of an oral mucosal lesion Aetiology of tooth wear (attrition/opposing porcelain crown) Aetiology of tooth wear (attrition and erosion) Aetiology of tooth wear (abrasion) Aetiology of dental caries Aetiology of tooth wear (attrition) Caries risk assessment Aetiology of periodontitis Identification of aetiology of pulpal condition Selection of treatment option pertaining to attrition Determination of prognosis of a compromised tooth and subsequent treatment Selection of tooth-loss treatment based on patient’s bio-psychosocial status Determination of prognosis of a compromised tooth and subsequent treatment Determination of prognosis of a compromised tooth and subsequent treatment Determination of prognosis of a compromised tooth and subsequent treatment Treatment prioritisation Treatment prioritisation Determination of prognosis of a compromised tooth and subsequent treatment Selection of appropriate therapeutic treatment option Decision for the safe administration of local anaesthetic Decision to give prophylactic antibiotic (based on patient’s medical history) Decision to manage a patient with high blood pressure safely
Dx Dx Dx Dx Dx Dx Dx Dx Dx Dx Rx Dx Dx Dx Dx Dx Dx Dx Dx Rx Rx Rx Rx Rx Rx Rx Rx Rx Rx Rx Rx Rx
0.55 0.80 0.78 0.90 0.86 0.56 0.62 0.72 0.45 0.98 0.75 0.74 0.69 0.89 0.68 0.72 0.98 0.80 0.58 0.14 0.71 0.26 0.38 0.70 0.79 0.80 0.77 0.80 0.92 0.33 0.45 0.48
selection selection selection selection
selection
PT
Dx
0.37 0.34 0.17 0.28 0.28 0.53 0.52 0.35 0.50 0.21 0.41 0.41 0.39 0.12 0.13 0.39 0.08 0.17 0.26 0.18 0.39 0.17 0.09 0.31 0.17 0.37 0.40 0.23 0.26 0.23 0.56 0.28
0.33 0.37 0.23 0.34 0.35 0.59 0.63 0.44 0.48 0.23
Rx
Y3 0.34 0.51 0.22 0.51 0.46 0.39 0.63 0.44 0.47 0.32 0.50 0.31 0.33 0.18 0.05 0.34 0.13 0.25 0.30 0.06 0.42 0.14 0.01 0.30 0.22 0.26 0.60 0.29 0.36 0.45 0.54 0.31
0.42 0.54 0.43 0.15 0.16 0.45 0.10 0.22 0.37 0.25 0.47 0.23 0.18 0.40 0.31 0.38 0.58 0.25 0.26 0.46 0.64 0.42
Y4
Y5
0.45 0.13 0.16 0.08 0.16 0.45 0.42 0.47 0.45 0.13 0.39 0.42 0.42 0.18 0.18 0.34 0.05 0.03 0.34 0.21 0.47 0.16 0.08 0.18 0.29 0.29 0.29 0.21 0.11 0.26 0.50 0.24
0.47 0.32 0.18 0.19 0.17 0.49 0.46 0.22 0.47 0.06 0.42 0.41 0.33 0.13 0.22 0.57 0.06 0.17 0.19 0.25 0.33 0.12 0.01 0.18 0.06 0.30 0.32 0.18 0.16 0.14 0.71 0.44
Key: Q, Question number; Cmp, Progress test component (Dx, Diagnosis; Rx: Treatment planning); LOD, Level of difficulty; Y5, Year 5; PT, Progress Test; Y4, Year 4; Y3, Year 3.
TABLE 2. Progress test scores over time Cohort
A
N 39 Study year 3 Year 2009 Case studies Yes Progress test X 63.7 SD 12.45 Diagnostic component X 70.37 SD 15.73 Treatment component X 55.13 SD 14
A
A
B
B
C
D
D
E
39 4 2010 Yes
36 5 2011 Yes
49 4 2009 No
48 5 2010 No
48 5 2009 No
54 3 2010 Yes
48 4 2011 Yes
49 3 2011 Yes
66.59 10.69
75.69 6.93
58.74 9.33
63.67 11.14
64.97 9.71
62.27 14.52
68.95 10.4
61.54 11.57
73.79 12.99
83.64 7.96
67.35 12.45
68.87 14.66
71.75 11.34
67.18 16.45
73.38 12.71
67.01 12.9
57.33 12.53
65.47 11.13
47.67 9.82
56.99 14.4
56.25 13.87
55.95 19.01
63.24 13.66
54.52 14.56
X: mean; SD: Standard deviation.
ª 2015 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd
5
Developing clinical reasoning
Postma & White
TABLE 3. Ratio of correctly answered questions for Cohort A – progression over time
Question
2009 Year 3 (n = 39)
2010 Year 4 (n = 39)
2011 Year 5 (n = 36)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
0.59 0.62 0.67 0.72 0.85 0.62 0.56 0.62 0.36 0.95 0.74 0.64 0.82 0.82 0.85 0.87 1.00 0.59 0.54 0.18 0.74 0.36 0.15 0.56 0.59 0.74 0.67 0.87 0.82 0.38 0.49 0.41
0.69 0.69 0.74 0.95 0.85 0.64 0.46 0.85 0.54 0.90 0.77 0.67 0.85 0.95 0.51 0.85 0.95 0.62 0.59 0.15 0.59 0.26 0.21 0.59 0.72 0.82 0.85 0.74 0.95 0.33 0.62 0.44
0.78 0.83 0.81 1.00 0.97 0.83 0.75 0.78 0.61 0.97 0.89 0.89 0.83 0.94 0.69 0.89 1.00 0.75 0.72 0.22 0.75 0.22 0.33 0.72 0.72 0.94 0.89 0.78 0.97 0.31 0.81 0.61
Difference: Year 3 to Year 5 Chi-square test P ns*
