really good stuff What was tried? Service learning has been shown to benefit students, improving interpersonal and communication skills, clinical skills and understanding of community issues.1 However, service learning has not yet been incorporated into the formal curriculum in India. A pilot study was carried out with 30 final-year medical students who volunteered to conduct community health visits over a 4-month period in a peri-urban village adopted by the Army College of Medical Sciences, New Delhi, India. The goal was to educate students to improve anaemia prevention practices in the community. They learned to diagnose anaemia clinically and to confirm it using a field test. They evaluated community perceptions related to anaemia and used health education to effect change in the knowledge, attitudes and practices of community women. We conducted two communication skills workshops and assessed student knowledge using a 10item questionnaire before and focus group discussion after the service-learning activity. Sixty-nine married women from the community consented to participate. We developed a questionnaire and tested the women’s knowledge, attitudes and practices in relation to anaemia before and 3 months after the intervention. Students worked under faculty staff supervision to test the women and their children for anaemia and carried out deworming procedures and administered iron and folic acid supplementation in standard doses. They also organised lectures with discussion sessions and street plays for the women. What lessons were learned? During the follow-up focus group, students recognised themselves as change agents within the community, and reported feeling empowered, having improved communication skills, being more aware of community, and believing they could make a difference. Overall, the community women’s knowledge, attitudes and practices related to anaemia prevention showed statistically significant improvements as a result of these simple interventions. More women were able to identify pallor as a sign of anaemia (8.7% pre; 87.3% post) and multiple pregnancies as a cause of anaemia (20.3% pre; 84.1% post). Questions such as ‘Do you think that anaemia can affect academic performance in schoolchildren?’ and ‘Do you think anaemia is a significant health issue?’ showed a change in positive responses from 75.4% and 71.0% to 97.1% and 91.3%, respectively. The project did not run for sufficient time to record any improvement in haemoglobin values. Overall, the results indicate that service learning can benefit both the provider and the recipient. We plan to sustain and improve the project by measur-
ing changes in haemoglobin to establish that service learning improves health. In resource-poor settings, medical students can become agents of change in order to provide simple yet impactful community health initiatives that align with national health needs. REFERENCE 1 Buckner AV, Ndjakani TD, Banks B, Blumenthal DS. Using service-learning to teach community health: the Morehouse School of Medicine Community Health Course. Acad Med 2010;85 (10):1645–51. Correspondence: Kavita Sahai, Department of Pathology, Armed Forces Medical College, Sholapur Road, Pune 411040, India. Tel: 00 91 9673 110702; E-mail: [email protected] doi: 10.1111/medu.12683
Formative evaluation of a Master of Public Health curriculum Meerjady S Flora & Shannon Marquez What problems were addressed? Bangladesh is one of the most densely populated countries in the world (1021 people per square kilometre). A quarter of the population is poor and semi-literate, and three-quarters live in rural areas. In recent years, the patterns of disease have changed to reflect a shift from acute infectious and deficiency diseases to chronic non-communicable diseases.1 Frequent outbreaks, caused by new disease agents (e.g. Chikungunya virus), new strains of agents of infection or other agents (e.g. pesticides), have been reported. Although Bangladesh has achieved much in public health, there are areas in which competent public health professionals are needed to meet changing global and national demands. Surveys conducted in 2010 and 2011 revealed a lack of competencies in epidemiology in Bangladeshi public health graduates; thus the Master of Public Health (MPH) curriculum was revised in 2012. The new programme extends the duration of education from 12 to 18 months, organises the content into modules, and adds an extended research period. We conducted a formative evaluation to identify remaining gaps in the curriculum and to develop recommendations to further align the curriculum with national needs. What was tried? We conducted this formative evaluation in two steps using a mixed-methods approach: (i) a needs assessment conducted from the perspec-
ª 2015 John Wiley & Sons Ltd. MEDICAL EDUCATION 2015; 49: 513–541
519
really good stuff tives of three different stakeholder groups, and (ii) a review of the MPH epidemiology curriculum. The study samples included 10 faculty members, the first cohort of graduates (n = 15) to complete the new curriculum, and five heads of different epidemiology institutions. Data were collected through a review of the curriculum document, by collating information from stakeholders obtained through focus group discussions and in-depth interviews, and a literature review. We reviewed the literature and stakeholder needs assessment data with the curriculum objectives to identify areas of alignment and gaps. We also reviewed curriculum content to identify the skills and competencies in applied epidemiology and public health practice required by graduates. What lessons were learned? The formative assessment revealed that much of the revised curriculum was effective; for example, students are competent in estimating disease burdens and in trend analysis and the handling of epidemiological data, and in planning and conducting epidemiological studies. However, there were a few major gaps. For instance, MPH graduates were neither sufficiently competent nor confident in setting up epidemiological surveillance systems and investigating outbreaks, using epidemiological data in policy and planning, or conducting risk assessments. Accordingly, these skills need to be strengthened to accommodate the changes in disease patterns. An in-depth analysis of the curriculum revealed an overall learning goal to conceptualise the principles of epidemiology; we found, however, that although this aim covered all issues important in general, it had no specific learning objectives. Neither faculty members nor students were aware of thematic or session objectives. This gap allowed faculty members the freedom to create objectives which they viewed as important, but also allowed the omission of significant components. The curriculum analysis also revealed that the epidemiological issues were mostly taught using a theory-based approach. Thus the MPH graduates were not competent or confident to perform epidemiological activities in real life. This information is being used to develop a competencybased curriculum so that these graduates can contribute to improving public health in Bangladesh. REFERENCE 1 Karar ZA, Alam N, Streatfield PK. Epidemiological transition in rural Bangladesh, 1986–2006. Glob Health Action 2009;19:2. Correspondence: Meerjady S Flora, Department of Epidemiology, National Institute of Preventive and Social Medicine, Mohakhali,
520
Dhaka 1212, Bangladesh. Tel: 00 880 1713 083893; E-mail: [email protected] doi: 10.1111/medu.12702
Setting short-answer question standards using borderline regression Katharine J Reid, Agnes E Dodds & Michael A Fink What problems were addressed? Setting defensible assessment standards in medical courses is challenging. The purpose of the assessment standard must be determined, an appropriate method selected and standard setting implemented within the context of the resources and time available. Methods of standard setting are well described for multiple-choice questions (MCQs) and for performance assessments such as the objective structured clinical examination (OSCE). There is little research on setting standards for other written assessments, such as short-answer questions (SAQs), in which students demonstrate understanding of diagnosis, investigations or management of relevant conditions by generating prose responses to clinical scenarios. We have implemented the Angoff method for MCQ standard setting in our medical school; this requires significant face-to-face training and independent standard setting by examiners. We sought an empirical approach to standard setting for SAQs that demanded few additional resources, but showed evidence of rigour as a method for establishing achievement standards. What was tried? Borderline regression1 has been widely implemented for OSCEs. We explored the feasibility of adapting the method for SAQs. For each SAQ marked, examiners awarded a total mark out of 20 based on a comprehensive marking guide. The same examiner also assigned a global competency rating (using a 5-point scale on which 1 = fail, 2 = borderline, 3 = pass, 4 = very good and 5 = excellent). We fitted a linear regression model predicting total score from global competency ratings and calculated the SAQ standard (cut score) by substituting the borderline rating value into the regression equation. We calculated total SAQ cut scores by summing individual cut scores for the six SAQs. To enhance marking consistency, we developed a detailed scoring rubric which described typical responses for each of the five global competency ratings in the domains of knowledge, interpretation and synthesis. Examiners also marked a sample of 10 SAQs
ª 2015 John Wiley & Sons Ltd. MEDICAL EDUCATION 2015; 49: 513–541
Copyright of Medical Education is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
ing changes in haemoglobin to establish that service learning improves health. In resource-poor settings, medical students can become agents of change in order to provide simple yet impactful community health initiatives that align with national health needs. REFERENCE 1 Buckner AV, Ndjakani TD, Banks B, Blumenthal DS. Using service-learning to teach community health: the Morehouse School of Medicine Community Health Course. Acad Med 2010;85 (10):1645–51. Correspondence: Kavita Sahai, Department of Pathology, Armed Forces Medical College, Sholapur Road, Pune 411040, India. Tel: 00 91 9673 110702; E-mail: [email protected] doi: 10.1111/medu.12683
Formative evaluation of a Master of Public Health curriculum Meerjady S Flora & Shannon Marquez What problems were addressed? Bangladesh is one of the most densely populated countries in the world (1021 people per square kilometre). A quarter of the population is poor and semi-literate, and three-quarters live in rural areas. In recent years, the patterns of disease have changed to reflect a shift from acute infectious and deficiency diseases to chronic non-communicable diseases.1 Frequent outbreaks, caused by new disease agents (e.g. Chikungunya virus), new strains of agents of infection or other agents (e.g. pesticides), have been reported. Although Bangladesh has achieved much in public health, there are areas in which competent public health professionals are needed to meet changing global and national demands. Surveys conducted in 2010 and 2011 revealed a lack of competencies in epidemiology in Bangladeshi public health graduates; thus the Master of Public Health (MPH) curriculum was revised in 2012. The new programme extends the duration of education from 12 to 18 months, organises the content into modules, and adds an extended research period. We conducted a formative evaluation to identify remaining gaps in the curriculum and to develop recommendations to further align the curriculum with national needs. What was tried? We conducted this formative evaluation in two steps using a mixed-methods approach: (i) a needs assessment conducted from the perspec-
ª 2015 John Wiley & Sons Ltd. MEDICAL EDUCATION 2015; 49: 513–541
519
really good stuff tives of three different stakeholder groups, and (ii) a review of the MPH epidemiology curriculum. The study samples included 10 faculty members, the first cohort of graduates (n = 15) to complete the new curriculum, and five heads of different epidemiology institutions. Data were collected through a review of the curriculum document, by collating information from stakeholders obtained through focus group discussions and in-depth interviews, and a literature review. We reviewed the literature and stakeholder needs assessment data with the curriculum objectives to identify areas of alignment and gaps. We also reviewed curriculum content to identify the skills and competencies in applied epidemiology and public health practice required by graduates. What lessons were learned? The formative assessment revealed that much of the revised curriculum was effective; for example, students are competent in estimating disease burdens and in trend analysis and the handling of epidemiological data, and in planning and conducting epidemiological studies. However, there were a few major gaps. For instance, MPH graduates were neither sufficiently competent nor confident in setting up epidemiological surveillance systems and investigating outbreaks, using epidemiological data in policy and planning, or conducting risk assessments. Accordingly, these skills need to be strengthened to accommodate the changes in disease patterns. An in-depth analysis of the curriculum revealed an overall learning goal to conceptualise the principles of epidemiology; we found, however, that although this aim covered all issues important in general, it had no specific learning objectives. Neither faculty members nor students were aware of thematic or session objectives. This gap allowed faculty members the freedom to create objectives which they viewed as important, but also allowed the omission of significant components. The curriculum analysis also revealed that the epidemiological issues were mostly taught using a theory-based approach. Thus the MPH graduates were not competent or confident to perform epidemiological activities in real life. This information is being used to develop a competencybased curriculum so that these graduates can contribute to improving public health in Bangladesh. REFERENCE 1 Karar ZA, Alam N, Streatfield PK. Epidemiological transition in rural Bangladesh, 1986–2006. Glob Health Action 2009;19:2. Correspondence: Meerjady S Flora, Department of Epidemiology, National Institute of Preventive and Social Medicine, Mohakhali,
520
Dhaka 1212, Bangladesh. Tel: 00 880 1713 083893; E-mail: [email protected] doi: 10.1111/medu.12702
Setting short-answer question standards using borderline regression Katharine J Reid, Agnes E Dodds & Michael A Fink What problems were addressed? Setting defensible assessment standards in medical courses is challenging. The purpose of the assessment standard must be determined, an appropriate method selected and standard setting implemented within the context of the resources and time available. Methods of standard setting are well described for multiple-choice questions (MCQs) and for performance assessments such as the objective structured clinical examination (OSCE). There is little research on setting standards for other written assessments, such as short-answer questions (SAQs), in which students demonstrate understanding of diagnosis, investigations or management of relevant conditions by generating prose responses to clinical scenarios. We have implemented the Angoff method for MCQ standard setting in our medical school; this requires significant face-to-face training and independent standard setting by examiners. We sought an empirical approach to standard setting for SAQs that demanded few additional resources, but showed evidence of rigour as a method for establishing achievement standards. What was tried? Borderline regression1 has been widely implemented for OSCEs. We explored the feasibility of adapting the method for SAQs. For each SAQ marked, examiners awarded a total mark out of 20 based on a comprehensive marking guide. The same examiner also assigned a global competency rating (using a 5-point scale on which 1 = fail, 2 = borderline, 3 = pass, 4 = very good and 5 = excellent). We fitted a linear regression model predicting total score from global competency ratings and calculated the SAQ standard (cut score) by substituting the borderline rating value into the regression equation. We calculated total SAQ cut scores by summing individual cut scores for the six SAQs. To enhance marking consistency, we developed a detailed scoring rubric which described typical responses for each of the five global competency ratings in the domains of knowledge, interpretation and synthesis. Examiners also marked a sample of 10 SAQs
ª 2015 John Wiley & Sons Ltd. MEDICAL EDUCATION 2015; 49: 513–541
Copyright of Medical Education is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
