JOURNAL OF PALLIATIVE MEDICINE Volume 18, Number 8, 2015 ª Mary Ann Liebert, Inc. DOI: 10.1089/jpm.2014.0392

Development and Validation of a Set of Palliative Medicine Entrustable Professional Activities: Findings from a Mixed Methods Study Jeff Myers, MD,1 Paul Krueger, PhD,2 Fiona Webster, PhD,2 James Downar, MD,1 Leonie Herx, MD,3 Christa Jeney, MD,1 Doreen Oneschuk, MD,4 Cori Schroder, MD,5 Giovanna Sirianni, MD,1 Dori Seccareccia, MD,1 Tara Tucker, MD,6 and Alan Taniguchi, MD 7

Abstract

Background: Entrustable professional activities (EPAs) are routine tasks considered essential to a professional practice. An EPA can serve as a performance-based outcome that a clinical supervisor would progressively entrust a learner to perform. Objective: Our aim was to identify, develop, and validate a set of EPAs for the palliative medicine discipline. Methods: The design was a sequential qualitative and quantitative mixed methods study. A working group was convened to develop a set of EPAs. Focus groups and surveys were used for validation purposes. Palliative medicine educators and content experts from across Canada participated in both the working group as well as the focus groups. Attendees of the 2014 Canadian Society of Palliative Care Physicians (CSPCP) annual conference completed surveys. A questionnaire was used to collect survey participant sociodemographic, clinical, and academic information along with ratings of the importance of the EPAs individually and collectively. Cronbach’s alpha examined internal consistency of the set of EPAs. Results: Focus group participants strongly endorsed the 12 EPAs. Virtually all survey participants rated the individual EPAs as being ‘‘fairly/very important’’ (range 94% to 100%). Of the participants, 97% agreed that residents able to perform the set of EPAs would be practicing palliative medicine and 87% indicated strong agreement that this collective set of EPAs captures activities that all palliative medicine physicians must be able to perform. A Cronbach’s alpha of 0.841 confirmed good internal consistency. Conclusions: Near uniform agreement from a national group of palliative medicine physicians provides strong validation for the set of 12 EPAs. Introduction

M

edical education is undergoing substantial reform with widespread adoption of competency-based curricula and a focus for learner assessment on outcomes.1 A potential pitfall, however, is a detailed list of competencies with outcomes limited to a set of characteristics of a physician that lack a coherent conceptualization of practice.2 With the ultimate outcome being patient care, assessment of learner competence would ideally extend beyond characteristics of an individual and include the interplay of the individual and his or her clinical environment.3,4 The complex task for

clinical educators is determining how to effectively assess learners using methods that are valid, reliable, confirm the attainment of competence, and represent the likely impact individuals will have on their environment.5 As a method of learner assessment, entrustable professional activities (EPAs) hold particular promise, as EPAs may serve as a more direct link to clinical practice.6 An EPA is a core task or responsibility for which a learner must become sufficiently competent that an assessor would trust the learner to perform the activity unsupervised, that is, ‘‘entrustment.’’7 An EPA involves both the acquisition and integration of multiple competencies and results in a

1

Division of Palliative Care, 2Department of Family Medicine, University of Toronto, Ontario, Canada. University of Calgary, Calgary, Alberta, Canada. 4 University of Alberta, Edmonton, Alberta, Canada. 5 Queen’s University, Kingston, Ontario, Canada. 6 University of Ottawa, Ottawa, Ontario, Canada. 7 McMaster University, Hamilton, Ontario, Canada. Accepted May 2, 2015. 3

682

VALIDATION OF A SET OF PALLIATIVE MEDICINE EPAS

measurable outcome.8 Taken collectively, a set of EPAs could succinctly represent the routine and essential professional work that defines and distinguishes an individual discipline. This has particular relevance for palliative medicine as previous work outlining competence for the discipline was challenged by overall views on the elements that constitute a competent palliative medicine physician.9 The critical first step in developing an EPA-based assessment system is to identify activities that are considered essential to clinical competence.10 Therefore the primary objective for this study was to identify, develop, and validate a set of palliative medicine EPAs.

683

Working group members then developed each EPA into a two-page document comprising a title, description, required knowledge, skills and attitudes, methods of assessing progress, proposed entrustment conditions, and approach to arriving at an entrustment decision. Each of the knowledge, skill, and attitude requirements from the National Standards of Accreditation for Canadian palliative medicine residency programs12 was included in at least one EPA. Additionally, for each EPA, working group members identified the most relevant domains of competence from the CanMEDS competency framework.13 EPA validation: Process

Methods

A sequential qualitative and quantitative mixed methods study design was used to develop and validate the EPAs (Fig. 1). To summarize, a working group identified and subsequently developed a draft set of EPAs. Preliminary input was obtained through a series of focus groups and validation was then sought through a national survey of palliative medicine physicians. Approval for the study was obtained through the Research Ethics Board of the University of Toronto. EPA development: Process Working group. The process began by convening a working group comprising 10 palliative medicine clinical educators and content experts from across Canada. All had palliative medicine residency program director experience and collectively the group represented 6 of 13 palliative medicine residency-training programs in Canada. EPA identification and development. A face-to-face meeting of the working group was held with the aim of identifying an initial draft set of EPAs. The process outlined in Mulder et al. was used to guide EPA development.11 Through task analysis of palliative medicine physicians’ routine professional work, working group members were to identify a maximum of 15 EPAs.8 Over the subsequent 5 months, all 10 working group members participated in monthly teleconferences to refine, revise, and synthesize the EPAs until consensus was reached on the initial draft set.

FIG. 1.

Focus groups. The two aims for the focus groups were to obtain preliminary input on the draft set of EPAs and inform the development of questionnaire items. Target participants were academic palliative medicine leaders from across the country. Potential participants were purposively selected based on recommendations of working group members using a snowball sampling approach.14 Individuals were contacted by e-mail and following a brief study description the request was made to reply if interested in focus group participation. A convenience sample of 16 individuals participated in four focus groups. All focus groups were professionally transcribed. A semi-structured interview guide was developed and organized into three sections. First, to provide context, the EPA construct was briefly described. Next, participants were asked to identify and discuss professional activities thought to be specific to palliative medicine. Finally, participants were provided with the draft set of EPAs and asked to provide overall impressions and discuss both the importance and relevance of individual EPAs. Qualitative analysis. Two investigators independently reviewed the transcripts and then met to compare their emergent coding. These codes were then applied by hand to the transcripts. Over a series of three meetings and using an iterative process of analysis and review, codes were grouped into themes that corresponded to the interview topics. Themes were used to revise the wording of EPAs, to inform the process of developing questionnaire items, and as initial

Key process elements to develop and validate the set of EPAs.

684

validity evidence for the EPAs. Rigor was achieved through detailed meeting notes to ensure transparency and having multiple researchers review each transcript. Questionnaire development. The aim of the questionnaire was to further establish validity of the EPAs. Using an iterative process and beginning with focus group data, individual items were developed and refined by a team consisting of a content expert and two methodological experts (one qualitative and one quantitative) who met four times over a 5-month period. The team assessed each item for issues of validity, reliability, readability, and potential for bias. Pretesting of the questionnaire was completed by the two methodologists (i.e., process experts) to identify design, implementation, or analysis issues. Pretesting was also completed by content experts from the working group as well as three ‘‘typical respondents’’ (i.e., palliative medicine physicians) who were unfamiliar with both EPAs and the overall project. Revisions were made based on the feedback. Both a paper-based and a web-based format were developed. All features of the web-based format were tested prior to its use including proper function of the e-mail system, respondent tracking, and the option to opt out. On the final questionnaire, participants were asked to rate for each EPA the level of importance (using a 5-point Likert scale from ‘‘not at all important’’ to ‘‘very important’’) that by the end of training a palliative medicine resident could be trusted to perform it unsupervised. Participants were also asked to rate their level of agreement (from ‘‘disagree strongly’’ to ‘‘agree strongly’’) that residents able to perform the set of activities collectively (including an adequate level of clinical complexity) would be practicing palliative medicine. Participants were asked to select what they felt were the ‘‘top three’’ most important EPAs for a palliative medicine physician, and using a scale from 1 to 10 participants were asked to rate how well the collective set of EPAs captures activities that all palliative medicine physicians must be able to perform. Participants were invited to suggest activities all palliative medicine physicians must be able to perform that were not listed, as well as identify activities from the list that they felt should not be included. Finally, the questionnaire included items addressing participants’ sociodemographic information and characteristics of current clinical and academic practice. Survey implementation. The target group chosen to validate the EPAs was physicians maintaining the focus of their clinical practice on palliative medicine. In early 2014, the Canadian Society of Palliative Care Physicians (CSPCP) had a total of 270 active members. Two hundred thirty-five individuals were registered for the 2014 annual meeting and conference, which was determined would serve as a convenient vehicle for participation. A paper copy of the questionnaire was included in conference registration packages and potential participants were invited to complete and return questionnaires. An incentive in the form of a draw for a gift card was included. Those who did not participate were subsequently invited by e-mail to complete the web-based format of the questionnaire using a modified Dillman approach.15 Quantitative analysis. Descriptive statistics (numbers and percentages or means and standard deviations) were

MYERS ET AL.

calculated for all variables measured. For the item addressing the level of agreement that a resident performing the set of activities would be practicing palliative medicine, bivariate analyses were conducted (v2 tests for categorical variables and t tests for continuous variables) with participant sociodemographic information. The purpose was to confirm uniformity or identify response variation among subgroups based on sociodemographic characteristics. To evaluate internal consistency among the set of EPAs, Cronbach’s alpha was calculated. Results EPA development

The final set of EPAs comprised 12 individual activities. Titles, brief summaries, and associated observable and measurable tasks for each are outlined in Appendix 1. EPA validation Focus groups: Qualitative data. Overall impressions of the set of EPAs were extremely consistent and participants expressed strong endorsement illustrated by the following quote: ‘‘I completely agree that all of these are very, very important’’ (participant 3, male, palliative medicine physician). Participants indicated, ‘‘having a tool for assessing trainees was important but also useful as something to say this is what I do’’ (participant 2, female, palliative medicine physician; see Appendix 2 for a list of additional supporting quotes). Three themes outline distinct approaches participants used to describe the specificity of clinical activities to palliative medicine (Table 1). For certain activities, participants suggested case complexity distinguishes the role for palliative medicine. For these activities and associated EPAs, it was considered essential that physicians in other disciplines maintain adequate competence to provide care to patients with less complex needs. Participants described other activities as being fundamental for all physicians. Palliative medicine physicians were described as having particular comfort and skill with the corresponding EPAs. Lastly, the concept of discipline-specific was described with the ‘‘complete a palliative medicine consultation’’ EPA felt to be most applicable. Survey findings: Quantitative data. Of the 235 individuals invited to participate, 176 (75%) completed the questionnaire. Sociodemographic information and practice characteristics of participants are outlined in Table 2. Virtually all participants’ felt each of the 12 EPAs was ‘‘fairly/very important’’ that a palliative medicine resident be trusted to perform unsupervised by the end of training (Table 3). Of participants, 97% agreed (‘‘agree/strongly agree’’) that residents able to perform this set of EPAs collectively (including an adequate level of clinical complexity) would be practicing palliative medicine, with 60% indicating strong agreement. For all bivariate analyses conducted, it was found responses to this item did not vary significantly based on any sociodemographic, academic, or practice characteristic, suggesting high levels of uniformity among all participant subgroups. Each of the 12 EPAs was selected as being one of the top three most important by at least some participants. The four EPAs most frequently identified as one of the top three were

VALIDATION OF A SET OF PALLIATIVE MEDICINE EPAS

685

Table 1. Qualitative Themes, Example Quotes, Clinical Activities, and Related EPA Topics Theme

Example quote

Clinical activities

Related EPA topics

Cases complexity ‘‘It’s the type of situation distinguishes the role where we might get called in for palliative medicine as back up or a second opinion and for extra help’’ (participant 11, female)

Care at the end of life Goals of care/ACP Symptom management Awareness and advocacy Care in the home setting Conflict & Decision-making

Manage the care of a dying patient Conduct a family conference or meeting Address symptoms Educate about a palliative approach to care Manage patients in the community Approach an ethical issue

Fundamental for all physicians

Collegial collaboration Interprofessional care Self-care Communication skills Care transitions

Collaborate with referring teams Integrate in to an interprofessional team Maintain resiliency as a physician Provide telephone advice and management Serve as Most Responsible Physician

‘‘Something that everyone in every branch and every area of medicine could and should be doing.’’ (participant 5, male)

Discipline-specific

Palliative Medicine Consultation

ACP, advance care planning; EPA, entrustable professional activity.

also the four EPAs participants most frequently rated as being ‘‘very important’’ (see Table 3). In rating how well the set captures activities that all palliative medicine physicians must be able to perform, 87% of participants provided a rating of 8, 9, or 10. This indicates strong agreement that taken collectively these activities are considered essential and central to clinical competence in palliative medicine physicians. When asked to suggest activities palliative medicine physicians must be able to perform that were not listed, each suggestion provided was in fact a subcomponent of one of the 12 EPAs and an element of the corresponding two-page document. When asked if any of the activities from the list should not be included, 94% responded ‘‘No.’’ The remaining 10 participants each identified one EPA with five (2.8%) indicating the ‘‘serves as Most Responsible Physician’’ EPA and two (1.1%) indicating the ‘‘maintain resiliency’’ EPA. For this set of 12 EPAs, Cronbach’s alpha was calculated to be 0.841, demonstrating good internal consistency. Discussion

This report outlines the process of developing and validating a set of palliative medicine EPAs. Although the EPA construct is new to the discipline, the content of the reported EPAs is by no means unfamiliar. An example is found with Morrison et al. and a comprehensive set of hospice palliative medicine competency-based outcomes.9 With the exception of the EPA ‘‘providing telephone advice and management,’’ the remaining 11 EPAs are addressed with varying degrees of specificity by a number of the outcomes described.9 This validated set of EPAs meets several important needs. First, the EPAs outline what learners should be entrusted to perform by the end of a palliative medicine-training program. These should not be viewed as an alternative to competencybased outcomes but rather a means to translate competencies into clinical practice with potential as meaningful units of learner assessment.7 An EPA-based assessment system organizes existing curricula into a format enabling focus on tasks that are both observable and measurable. This is congruent with the direction in which both program accreditation and learner credentialing are evolving.16 Similarly, EPAs should not be viewed as replacing existing assessment methods (e.g., Mini-CEX, structured written or oral assess-

ments). These methods may be important in assessing progress toward entrustment.4 Although rarely formalized, ‘‘trust’’ is a fundamental element of health professionals’ education and reflects a dimension of competence that extends beyond observed ability. Because of its subjective nature, however, trust has historically been viewed as challenging in learner assessment. Educators should be encouraged to view trust as an important element of meaningful assessment.17 The EPAs can serve as a step toward incorporating the subjective concept of ‘‘entrustability’’ into palliative medicine learner assessment. Finally, an important role for the set of EPAs relates to palliative care as an essential element of a system-wide strategy to address current unmet and future care needs of an aging population. Given the total amount of palliative care to be delivered is far greater than what specialized palliative medicine physicians could provide,9 this validated set of EPAs could help clarify what is specific to the discipline and, perhaps more importantly, what is not or should not be specific to palliative medicine. How might it be reconciled that 97% of participants agreed physicians able to perform this collective set of EPAs would be practicing palliative medicine, yet only one EPA was felt to be ‘‘discipline-specific’’? For six of the EPAs, it was suggested clinical complexity should determine the specific palliative medicine role and thus circumscribes potential relevance to other disciplines. Sets of EPAs have recently been published for critical care medicine, pulmonary medicine, and geriatric medicine.18,19 Each includes ‘‘Provide palliative care’’ as a specific EPA. Although all three disciplines must be applauded for identifying palliative care as being essential, this may not adequately outline a measurable task. As an alternative, any of the six EPAs for which case complexity distinguishes the role for palliative medicine could be incorporated into other disciplines’ EPA sets. The five EPAs identified as fundamental to all physicians could be dismissed as nonspecific and generically emerging from either the Accreditation Council for Graduate Medical Education (ACGME) or CanMEDS core competency frameworks.20,13 Closer examination of the clinical activity for each reveals an element that is somewhat unique to palliative medicine. For example, ‘‘collaboration with referring health care teams’’ (EPA 5, Appendix 1): one could

686

MYERS ET AL.

Table 2. Sociodemographic and Practice Characteristics of Survey Participants Characteristic

Number (%)

Characteristic

Number (%)

Gender (n = 173) Male Female Age (n = 168) Mean (SD) Minimum/maximum Marital status (n = 168) Married/partner Single/widowed/separated/divorced Born in Canada (n = 171) Yes No Largest income source (n = 171) Capitation Fee for service Salary/stipend/research award Sessional payment Alternate payment/funding plan Other Years since undergraduate medicine (n = 168) Mean (SD) Minimum/maximum Postgraduate education completed (n = 176) Family medicine trained Specialty or subspecialty trained Trained in the UK Trained in the US (ABHPM certified) Rotating internship Other

Palliative medicine residency 59 (34.1%) Yes 67 (38.0%) 114 (65.9%) Years licensed for independent practice (n = 175) Less than 1 year 6 (3.4%) 49 (11.5) 1–5 years 29 (16.6%) 30/73 6–10 years 29 (16.6%) 11–15 years 19 (10.9%) 146 (86.9%) 16 or more years 92 (52.6%) 22 (13.1%) Years of palliative medicine practice (n = 175) Mean (SD) 12 (8.2) 119 (69.6%) Less than 1 year 6 (3.4%) 52 (30.4%) 1–5 years 45 (25.7%) 6–10 years 39 (22.3%) 18 (10.5%) 11–15 years 36 (20.6%) 52 (30.4%) 16 or more years 49 (28.0%) 22 (12.9%) % time focused on palliative medicine (n = 173) 13 (7.6%) Less than 25% 20 (11.6%) 58 (33.9%) 25–49% 14 (8.1%) 8 (4.8%) 50–74% 20 (11.6%) 75–99% 18 (10.4%) 22 (12.2) 100% 101 (58.3%) 2/39 Currently teach palliative medicine residents (n = 176) Yes 128 (72.7%) 121 (68.8%) Current teaching activities (n = 128) 30 (17.0%) Direct clinical supervision 112 (87.5%) 7 (4.0%) Formal teaching rounds 83 (64.8%) 5 (2.8%) Informal teaching rounds 82 (64.1%) 4 (2.3%) Academic leadership or mentorship 51 (39.8%) 9 (5.1%) Small or large group teaching 76 (59.4%) Project supervision 22 (17.2%)

ABHPM, American Board of Hospice and Palliative Medicine; SD, standard deviation.

Table 3. Ratings of Importance that Palliative Medicine Residency Training Program Graduates e Trusted to Perform Each Activity Unsupervised by the End of Their Training (n = 176) EPA title 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.

Complete a palliative medicine consultation Manage the care of a dying patient in the last days, and final hours Conduct a family conference or meeting Address a difficult to manage symptom using medications and appropriate route of administration specific to the palliative medicine settinga Collaborate as a palliative medicine physician with referring health care teams Educate patients, families, and colleagues about ‘‘palliative care’’ as an approach or philosophy Integrate into an interprofessional specialized palliative care team Manage the palliative care of a patient in the community setting Maintain resiliency in practice as a palliative medicine physician Provide palliative medicine telephone advice and management Serve as Most Responsible Physician for a patient admitted to a designated palliative care bed Describe an approach to managing a controversial palliative medicine ethical issue for a patient

% very Important % fairly/very Important 98.7% 93.8% 93.2% 91.5%

100% 100% 99.5% 100%

85.2%

99.5%

85.2%

98.9%

78.4% 77.3% 71.6% 71.0% 70.5%

97.2% 98.9% 97.7% 95.5% 93.8%

68.8%

97.7%

a A final revision to this EPA title was based on comments provided by survey participants (see Appendix 1). EPA, entrustable professional activity.

VALIDATION OF A SET OF PALLIATIVE MEDICINE EPAS

hypothesize palliative medicine physicians consider collegial collaboration and consultation etiquette beyond what is typical or average. As well, palliative medicine physicians are professionalized to deeply revere interprofessional teambased care (EPA 7, Appendix 1). Palliative medicine physicians tend to have particular appreciation for the significance of self-care (EPA 9, Appendix 1). Considering emerging models of shared and consultative care, expert communication skills applied to providing telephone advice and management is becoming increasingly important (EPA 10, Appendix 1). Finally, as Most Responsible Physician (EPA 1, Appendix 1), important expertise relates to care transitions. Although collegial collaboration, interprofessional care, self-care, communication skills, and care transitions were considered fundamental for any physician, achieving and maintaining a certain competence level for each was considered essential for a palliative medicine physician. This is underscored by previous work further describing each of these five clinical activities for the palliative medicine context.21–25 As suggested by Head et al.,26 an important opportunity for palliative medicine is to capitalize on our expertise for each activity and lead in advancing related educational experiences. What then is specific to the palliative medicine discipline? The answer is, the discipline itself. As outlined by Morrison et al.,9 collective competence for all topics is expected of no other discipline. This is likely to explain why agreement was virtually uniform that this set of palliative medicine EPAs represents what a palliative medicine physician does. Although content validity of the set of EPAs is achieved through expert opinion and evidence of construct validity is demonstrated by the good internal consistency, limitations of this study include criterion validity and certain types of reliability not being assessed. Use of a one-time survey limited the assessment of reliability including both rater agreement and stability over time. Survey participation was limited to palliative medicine physicians attending the 2014 CSPCP annual meeting, and it is possible those who did not attend may have responded differently. The high response rate and the absolute number of participants relative to the organization’s membership makes it less likely opinions of nonparticipants would substantially change the observed findings. In keeping with the mixed methods design, the focus groups were kept relatively structured. Future qualitative research might explore more openly physicians’ understandings and experiences of the concepts of expertise and specialist care. Future directions include the complex next step of EPA implementation. Faculty and learners should be engaged in designing and evaluating EPAs with a focus for skills training on direct observation and effective feedback. In addition, seeking opportunities to collaborate with other disciplines in developing their EPA sets may be an important educational strategy in building palliative care capacity. Conclusion

This report outlines a mixed methods design for developing and validating a set of palliative medicine EPAs. Validity evidence is demonstrated by the near uniform agreement among a national group that this set of activities collectively represents the work of a palliative medicine physician. These

687

performance-based learner outcomes can be used as a method of learner assessment and have the potential and capacity to confirm the attainment of competence. Acknowledgments

This study was funded through a University of Toronto Education Development Fund grant. Author Disclosure Statement

No competing financial interests exist. References

1. Long DM: Competency-based residency training: The next advance in graduate medical education. Acad Med 2000;75: 1178–1183. 2. Frank J, Snell L, ten Cate O, et al.: Competency-based medical education: Theory to practice. Med Teach 2010;32: 638–645. 3. Carraccio C, Englander R: From Flexner to competencies: Reflections on a decade and the journey ahead. Acad Med 2013;88:1067–1073. 4. ten Cate O, Snell L, Carraccio C: Medical competence: The interplay between individual ability and the health care environment. Med Teach 2010;32:669–675. 5. Litzelman D, Cottingham A: The new formal competencybased curriculum and informal curriculum at Indiana University School of Medicine: Overview and five-year analysis. Acad Med 2007;82:410–421. 6. ten Cate O: Entrustability of professional activities and competency-based training. Med Educ 2005;39:1176–1177. 7. ten Cate O, Scheele F: Competency-based postgraduate training: Can we bridge the gap between theory and clinical practice? Acad Med 2007;82:54–57. 8. ten Cate O: Nuts and bolts of entrustable professional activities. J Grad Med Educ 2013;5:157–158. 9. Morrison L, Scott J, Block S, et al.: Developing initial competency-based outcomes for the hospice and palliative medicine subspecialist: Phase I of the Hospice and Palliative Medicine Competencies Project. J Palliat Med 2007; 10:313–330. 10. Hauer K, Kohlwes J, Cornett P, et al.: Identifying entrustable professional activities in internal medicine training. J Grad Med Educ 2013;5:54–59. 11. Mulder H, ten Cate O, Daalder R, Bervkens J: Building a competency-based workplace curriculum around entrustable professional activities: The case of physician assistant training. Med Teach 2010;32:e453–e459. 12. Conjoint Palliative Medicine Residency Program, National Standards of Accreditation, 2010. www.cfpc.ca/red_book/ (Last accessed July 10, 2014.) 13. Frank JR: The CanMEDS 2005 physician competency framework: Better standards, better physicians, better care. Ottawa: Royal College of Physicians and Surgeons in Canada, 2005. 14. Patton, M: Qualitative Research and Evaluation Methods. Thousand Oaks, CA: Sage Publications, Inc., 2002. 15. Dillman DA, Smyth JD, Christian LM: Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method. Hoboken, NJ: John Wiley & Sons, 2009. 16. Long DM: Competency-based residency training: The next advance in graduate medical education. Acad Med 2000; 75:1178–1183.

688

17. Sterkenburg A, Barach P, Kalkman C, et al.: When do supervising physicians decide to entrust residents with unsupervised tasks? Acad Med 2010;85:1408–1417. 18. Fessler H, Addrizzo-Harris D, Beck J, et al.: Entrustable professional activities and curricular milestones for fellowship training in pulmonary and critical care medicine: Report of a multi-society working group. Chest 2014;146: 813–834. 19. Leipzig RM, Sauvigne´ K, Granville LJ, et al.: What Is a geriatrician? American Geriatrics Society and Association of Directors of Geriatric Academic Programs End-ofTraining Entrustable Professional Activities for Geriatric Medicine. J Am Geriatr Soc 2014;62(5):924–929. 20. Accreditation Council for Graduate Medical Education. ACGME Common Program Requirements. www.acgme.org/ acgmeweb/tabid/429/ProgramandInstitutionalAccreditation/ CommonProgramRequirements.aspx (Last accessed December 22, 2014). 21. von Gunten CF, Weissman DE: Consultation etiquette in palliative care. J Palliat Med 2013;16:578–579. 22. Meier D, Beresford L: The palliative care team. J Palliat Med 2008;11:677–681.

MYERS ET AL.

23. Kim HC, Rapp E, Gill A, Myers J: An innovative self-care module for palliative care medical learners. J Palliat Med 2013;16:603–608. 24. Dy SM, Apostol C, Martinez KA, Aslakson RA: Continuity, coordination, and transitions of care for patients with serious and advanced illness: A systematic review of interventions. J Palliat Med 2013;16:436–445. 25. Carr CH, McNeal H, Regalado E, Nelesen RA, et al.: PALMED CONNECT: A telephone consultation hotline for palliative medicine questions. J Palliat Med 2013;16:263–267. 26. Head BA, Schapmire T, Hermann C, et al.: The Interdisciplinary Curriculum for Oncology Palliative Care Education (iCOPE): Meeting the challenge of interprofessional education. J Palliat Med 2014;17:1107–1114.

Address correspondence to: Jeff Myers, MD 2075 Bayview Avenue, Room H336 Toronto, Ontario Canada, M4N 3M5 E-mail: [email protected]

(Appendices follow /)

VALIDATION OF A SET OF PALLIATIVE MEDICINE EPAS

689

Appendix 1. Palliative Medicine EPAs EPA title 1. Complete a palliative medicine consultation

Summary Multidimensional assessment and synthesis of information to formulate an individualized management plan for a patient with serious life-limiting illness and his or her family

Observable and measurable tasks 

     

2. Manage the care of a dying patient in the last days, and final hours

3. Conduct a family conference or meeting

4. Address difficult to manage symptoms through pharmacological and nonpharmacological modalities appropriate for the palliative medicine setting

Recognize clinical signs and symptoms of patients approaching death. Bereavement practices, identifying caregivers at risk for pathological grief, and when possible, advocacy for any cultural or spiritual practices near and at the time of death Lead a meeting with a family/ caregivers and/or the patient and often includes colleagues and members of the interprofessional team Comprehensive assessment and formulation of a symptom management plan considering medications, routes of administration and therapeutic and interventional procedures

5. Collaborate as a palliative medicine physician with referring health care teams

Effective working relationships with and education of members of referring health care teams when either direct or indirect consultation is being sought

6. Educate about ‘‘palliative care’’ as an approach or philosophy

Educate patients, family, informal caregivers, formal health care providers/ teams, and the public Communicate with and advocate for interprofessional team members

7. Integrate into an interprofessional specialized palliative care team







Assesses physical, social, psychological, spiritual, and functional domains (including appropriate use of assessment tools) Communication skills used to develop a therapeutic relationship Facilitates a goals-of-care discussion (includes illness understanding) Decision-making processes are based on care goals Assesses caregiver distress and resiliency Selects and interprets tests and interventions Communicates effectively using oral, written, and electronic methods Appropriately uses interventions as well as educates around the inappropriate use of other interventions (e.g., blood work, vital signs) Supports and effectively educates informal and formal caregivers on common and expected clinical manifestations of an imminently dying person Facilitates the diverse range of grief reactions and responses

  

Ensures perspective of all participants is heard Communicates using conflict resolution skills Facilitates a goals of care discussion (includes illness understanding)  Develops a plan of care Symptoms to include (but not limited to): Pain, nausea, vomiting, breathlessness, cough, constipation, diarrhea, anorexia, cachexia, weakness, fatigue, edema, bleeding, thrombosis, anxiety, depression, spiritual or existential distress; at end-of-life: agitation, respiratory and oropharyngeal secretions, and the constellation of symptoms associated with delirium  Therapeutic symptom management procedures (e.g., paracentesis)  Clinical processes associated with palliative sedation therapy (PST)  Negotiates to determine how the care of the patient will be shared with the referring team  Direct consultations represents direct involvement in assessing the patient/family  Indirect consultations represents discussing clinical issues with colleagues from a different service  Educates referring team members with the aim of building capacity in the provision of quality primary level palliative care  Incorporates the principles of health literacy, adult learning, and best practices in patient and family member education  Advocates for palliative care provision as appropriate  Advocates for the contribution of each profession comprising an interprofessional palliative care team  Active contributes to palliative care team function  Appropriately participates in resolving team conflict (continued)

690

MYERS ET AL.

Appendix 1. (Continued) EPA title 8. Manage the palliative care of a patient in the community setting

Summary Collaborate with interprofessional members of community-based palliative care teams

Observable and measurable tasks     

9. Maintain resiliency in practice as a palliative medicine physician

10. Provide palliative medicine telephone advice and management

11. Serve as Most Responsible Physician for a patient admitted to a designated palliative care bed

12. Describe an approach to managing a controversial palliative medicine ethical issue for a patient

Maintain an individualized approach to resiliency (i.e., self-care) aimed at enhancing both the well-being of the clinician him or herself as well as the individuals with whom he/she interacts within a professional context Communicate by telephone with patient, family member, caregiver, or clinical colleague for the purpose of addressing clinical issues, symptoms, or practical concerns

 

Maintain overall responsibility for a patient admitted to a palliative care bed (for any setting) from the time admission until the time of death, discharge, or transfer of responsibility Articulate how he/she ‘‘should’’ proceed for a given context, involves consideration of the relevant laws, policies, and ethical principles that govern medical practice in the jurisdiction



 

     

  

Documents the care plan Informs caregivers of changes in status Identifies care needs for the patient as well as his or her caregivers Appropriately utilizes community resources When appropriate facilitates decision making re: transfers in care setting Applies continuously throughout training program Incorporates measures to attempt to achieve balance between the physical, emotional, intellectual, relational, and spiritual aspects Method should have a self-awareness-based approach (e.g., reflective writing and mindfulness meditation) Exposure to and experience with a variety of self-care practices to facilitate determining which ones work best for him or her for ongoing maintenance Gathers relevant clinical information/history Provision of information/advice Arranges appropriate resources and follow-up Documents the encounter Communicates to other team members involved in the care of the patient as required Telephone advice constitutes the clinical management of the patient and equates to the provision of direct patient care Manages the clinical, operational, and administrative elements of the patient’s care Discharge or care transition planning Appropriately utilizes institutional resources Accountable for operational and clinical quality indicators

‘‘Controversial’’ ethical issues include (but not limited to):  Nondisclosure (diagnoses, prognoses)  Discordance or confusion about a patient’s autonomous wishes  Withholding or withdrawing medical therapies or artificial nutrition/hydration  Patient/family requests: euthanasia, assisted suicide

EPA, entrustable professional activities.

Appendix 2. Supplementary Focus Group Quotes        

‘‘These look brilliant’’ (participant 4, female, palliative medicine physician) ‘‘They’re all important things that we do.’’ (participant 6, male, palliative medicine physician) ‘‘They’re all excellent’’ (participant 12, female, palliative medicine physician) ‘‘I think being able to have something that you use for assessing your trainees is great. But also as something to say this is what I do’’ (participant 2, female, palliative medicine physician) ‘‘I feel confident they have a lot of face validity’’ (participant 2, female, palliative medicine physician) ‘‘I think these are the right EPAs’’ (participant 8, male, palliative medicine physician) ‘‘My initial reaction would be (the list is) pretty comprehensive’’ (participant 14, female, palliative medicine physician) ‘‘Overall (the set) captures what a palliative medicine physician would need to do competently’’ (participant 11, male, palliative medicine physician) EPA, entrustable professional activities.