Measuring what matters: top-ranked quality indicators for hospice and palliative care from the American Academy of Hospice and Palliative Medicine and Hospice and Palliative Nurses Association.

Vol. 49 No. 4 April 2015

Journal of Pain and Symptom Management 773

Special Article

Measuring What Matters: Top-Ranked Quality Indicators for Hospice and Palliative Care From the American Academy of Hospice and Palliative Medicine and Hospice and Palliative Nurses Association Sydney Morss Dy, MD, MSc, Kasey B. Kiley, MPH, Katherine Ast, MSW, LCSW, Dale Lupu, PhD, Sally A. Norton, PhD, RN, FAAN, Susan C. McMillan, PhD, ARNP, FAAN, Keela Herr, PhD, RN, AGSF, FAAN, Joseph D. Rotella, MD, MBA, FAAHPM, and David J. Casarett, MD, MA Johns Hopkins Sidney Kimmel Cancer Center (S.M.D.), Baltimore, Maryland; Johns Hopkins Hospital (K.B.K.), Baltimore, Maryland; American Academy of Hospice and Palliative Medicine (K.A., D.L.), Chicago, Illinois; University of Rochester School of Nursing (S.A.N.), Rochester, New York; University of South Florida College of Nursing (S.C.M.), Tampa, Florida; University of Iowa College of Nursing (K.H.), Iowa City, Iowa; Hosparus (J.D.R.), Louisville, Kentucky; and University of Pennsylvania Perelman School of Medicine (D.J.C.), Philadelphia, Pennsylvania, USA

Abstract Context. Measuring quality of hospice and palliative care is critical for evaluating and improving care, but no standard U.S. quality indicator set exists. Objectives. The Measuring What Matters (MWM) project aimed to recommend a concise portfolio of valid, clinically relevant, cross-cutting indicators for internal measurement of hospice and palliative care. Methods. The MWM process was a sequential consensus project of the American Academy of Hospice and Palliative Medicine (AAHPM) and Hospice and Palliative Nurses Association (HPNA). We identified candidate indicators mapped to National Consensus Project (NCP) Palliative Care Guidelines domains. We narrowed the list through a modified Delphi rating process by a Technical Advisory Panel and Clinical User Panel and ratings from AAHPM and HPNA membership and key organizations. Results. We narrowed the initial 75 indicators to a final list of 10. These include one in the NCP domain Structure and Process (Comprehensive Assessment), three in Physical Aspects (Screening for Physical Symptoms, Pain Treatment, and Dyspnea Screening and Management), one in Psychological and Psychiatric Aspects (Discussion of Emotional or Psychological Needs), one in Spiritual and Existential Aspects (Discussion of Spiritual/Religious Concerns), and three in Ethical and Legal Aspects (Documentation of Surrogate, Treatment Preferences, and Care Consistency with Documented Care Preferences). The list also recommends a global indicator of patient/family perceptions of care, but does not endorse a specific survey instrument. Conclusion. This consensus set of hospice and palliative care quality indicators is a foundation for standard, valid internal quality measurement for U.S. settings. Further development will assemble implementation tools for quality measurement and benchmarking. J Pain Symptom Manage 2015;49:773e781. Ó 2015 American Academy of Hospice and Palliative Medicine. Published by Elsevier Inc. All rights reserved. Key Words Palliative care, pain measurement, hospice care, quality of health care, quality indicators, advance care planning, patient satisfaction

Address correspondence to: Sydney Morss Dy, MD, MSc, Johns Hopkins Sidney Kimmel Cancer Center, Room 609, 624 N. Broadway, Baltimore, MD 21205, USA. E-mail: sdy@ jhsph.edu Ó 2015 American Academy of Hospice and Palliative Medicine. Published by Elsevier Inc. All rights reserved.

Accepted for publication: January 21, 2015.

0885-3924/$ - see front matter http://dx.doi.org/10.1016/j.jpainsymman.2015.01.012

774

Dy et al.

Introduction Far too many patients and families in the U.S. experience unnecessary physical and emotional suffering during serious and life-threatening illnesses.1 Recent studies have demonstrated gaps in quality of care in domains such as pain and symptom management, communication, and care planning across settings such as hospitals, nursing homes, and ambulatory care.2e4 Palliative care is defined as care that provides relief from symptoms and supports quality of life for patients with serious advanced illness and their families, and hospice care is an approach focused on patients with limited life expectancy and their families.1 Accumulating research demonstrates that interventions such as ambulatory palliative care clinics, structured goals of care discussions in critical care, and outpatient nurse-led interventions targeting patient/family pain management can improve outcomes such as patient and family satisfaction and health care utilization.5 Measuring the quality of care delivery is integral to hospice and palliative care programs, as specified by the National Consensus Project (NCP) Clinical Practice Guidelines and the Joint Commission Advanced Certification Program for Palliative Care6 and the Centers for Medicare & Medicaid Services (CMS) for the hospice programs.7 A critical first step in improving care is demonstrating where quality deficits exist compared with national benchmarks and determining where quality improvement initiatives are most needed and most likely to be beneficial, but no national U.S. (or universal) standards for measurement or databases that would support benchmarking yet exist. Quality indicator sets for a variety of relevant populations and settings have been developed and tested8e11 and the National Quality Forum (NQF) has endorsed a group of palliative care indicators11 suitable for accountability, such as public reporting. However, many of the indicators intended for accountability may not be the most appropriate for internal use in clinical programs, and a standard, concise, cross-cutting set of indicators that can be used internally for benchmarking, comparison across programs, and quality improvement is needed. Many palliative care programs are not yet routinely measuring quality, many are using locally developed, non-validated indicators, and for those who wish to measure quality, there are no nationally used sets with benchmarking that apply across populations and settings. The Quality and Practice Standards Committee of the American Academy of Hospice and Palliative Medicine (AAHPM), therefore, initiated the Measuring What Matters (MWM) consensus project, which was joined by the Hospice and Palliative Nurses Association (HPNA) Research Advisory Council. The overall


goal of the project was to recommend a concise portfolio of valid, clinically relevant, cross-cutting performance indicators for internal measurement for hospice and palliative care programs. The intent was to develop a common core set from which programs could select, to help create standards for quality measurement of palliative care in the U.S. and allow for national benchmarking. The aspirational goal was an initial set of process and outcome indicators that apply regardless of diagnosis, organizational structure, or setting, although the MWM team recognized that such indicators may not always be appropriate or not yet exist and that some included indicators as currently defined may not apply across populations and settings. The MWM team acknowledged that promoting high-quality hospice and palliative care indicators for accountability with CMS and other groups is also an important goal,1 but may be less relevant for program development or for improving the patient and family experience of care; this was, therefore, not a goal of the MWM project. In this article, we describe the MWM methodology and the initial core consensus set of quality indicators.

Methods MWM was a sequential consensus project, directed by a partnership between AAHPM and HPNA with a modified Delphi rating process by first a Technical Advisory Panel (TAP) and then a Clinical User Panel (CUP), followed by input solicited from AAHPM and HPNA membership and from external organizations and patient advocacy groups to obtain the final indicator set (Table 1). The project goals described above guided the MWM process. In particular, the MWM team aligned the set with the NCP guidelines, including organizing by the eight domains, and other existing quality initiatives whenever possible. The team envisioned the indicator set for populationlevel measurement, and it may be insufficient for measurement targeting specific settings or populations. The MWM process also evaluated whether denominators of existing indicators should be considered for future modification to be more inclusive and crosscutting across settings or populations.

Identifying Indicators The MWM project began by identifying existing U.S. process and outcome indicators relevant to hospice and palliative care, available in the public domain as of October 2013 and specified for U.S. data sources and developed through a rigorous process and/or tested for reliability and validity in English. Sources included indicators endorsed by the NQF and/or in


MWM in Hospice and Palliative Care

Table 1 Steps in the Measuring What Matters (MWM) Process for Selecting the Initial Consensus Set of Quality Indicators 1. The MWM team identified existing indicators a. Relevant to U.S. hospice and palliative care, b. Available in the public domain as of October 2013, and c. Developed through a rigorous process and/or tested for reliability and validity. 2. The Technical Advisory Panel (TAP) rated indicators on their scientific soundness and referred a set of indicators (n ¼ 34) for review by the Clinical User Panel (CUP). 3. The CUP rated those indicators based on three dimensions: a. How meaningful is this for patients/families? b. How actionable is this for providers/organizations? c. How large is the potential impact? 4. The CUP achieved consensus on the top 12 indicators for further input. 5. The draft set of 12 indicators was posted on the American Academy of Hospice and Palliative Medicine (AAHPM)’s web site to elicit feedback from AAHPM and Hospice and Palliative Nurses Association (HPNA) members and their interdisciplinary teams, and asked them to reduce the list of 12 measures down to 10 (or less) of the best measures. 6. The draft set of 12 indicators was distributed to key relevant external organizations and patient advocacy groups for a final set of ratings. 7. The MWM team compiled the input from the membership, teams, organizations, and advocacy groups for the final consensus set of 10 indicators.

the Agency for Healthcare Research and Quality (AHRQ) Quality Measures database, as well as indicators included in the PEACE Palliative Care and Hospice,8,12 Cancer Quality-ASSIST supportive oncology,2,13 American Society for Clinical Oncology (ASCO) Quality Oncology Practice Initiative (QOPIÒ) Care at the End of Life,10 and Assessing Care of Vulnerable Elders (ACOVE) Palliative Care and End of Life14,15 sets, indicators developed by the National Hospice and Palliative Care Organization (NHPCO)16,17 and American Medical Associatione convened Physician Consortium for Performance Improvement (PCPI),18 and indicators used by the CMS. Indicators were considered relevant to hospice and palliative care based on the numerator. If the numerator or topic of the indicator was related to the NCP domains or preferred practices of palliative care, the indicator was considered for inclusion. Members of the project steering committee then reviewed the compiled list and identified any missing indicators. From these sources, the MWM team identified 75 existing indicators that met criteria, and classified them by NCP domain and compiled existing specifications and evidence about each indicator. Almost all indicators had reliability and validity testing in palliative care populations.

Panel Ratings and Public Comment Technical Advisory Panel. The key purpose of the TAP was to judge the technical strength of existing indicators. The MWM team recruited 11 key researchers on

775

quality measurement in hospice and palliative medicine for the TAP and co-chairs from AAHPM and HPNA. After an introductory instructional webinar, all members of the TAP were given detailed information on the indicators and associated evidence, including their reliability and validity testing, and asked to review and rate the indicators on a scale from one to nine, where one indicated low and nine high scientific soundness (Appendix). When rating each indicator, in addition to using the specifications and evidence provided about each indicator, the TAP members considered the AHRQ desirable attributes of quality indicators: Importance Scientific soundness: clinical logic Scientific soundness: indicator properties, including evidence for reliability and validity Feasibility The TAP members rated each indicator twice: once for the indicator as specified and once for the indicator concept (the numerator only), and also could provide free-text comments. The purpose of the indicator concept rating was to determine if the numerator was a good candidate for future indicator development using a global all-palliative care population denominator to be developed in the future. The TAP members then participated in a conference call to discuss the indicators, focusing particularly on those with disagreement, defined as at least one rating lower than four or at least three ratings lower than seven. Members could discuss issues raised and advocate for or against indicators. The TAP members then rated the indicators a second time using the same process as the first round. This process identified 34 indicators sufficiently highly rated for scientific soundness to advance to the CUP rating process. Clinical User Panel. The purpose of the CUP process was to select the indicators that were the most clinically important and usable. The goal was to form a concise measurement set of approximately 10 indicators, although it was recognized that the final number of indicators would be responsive to the concerns of both comprehensiveness and brevity. The MWM team recruited co-chairs from AAHPM and HPNA and 27 members for the CUP to represent diversity in settings where hospice and palliative services are provided; patient populations and illnesses; and providers including physicians, nurses, social workers, chaplains, and grief counselors, as well as consumer groups and quality organizations. The CUP reviewed the 34 indicators forwarded by the TAP using a similar process as the TAP. The CUP rated the indicators on meaningfulness for patients and families, actionability for providers and organizations (including both

776

Dy et al.

feasibility of measurement and possibility of making changes based on results), and potential impact (prevalence of problem and/or amount of possible improvement; Appendix). After the survey and discussion webinar and further consolidation/prioritization from the CUP co-chairs considering all feedback and comments, consensus was reached on 12 crosscutting indicators from a variety of sources to submit for the last steps in the process. In the final steps, the MWM team distributed information on the project and on the 12 indicators to the AAHPM and HPNA membership and their teams and to key relevant stakeholders for further ratings and comment on which indicators were highest priority and which should not be included, before winnowing down to a final list of 10 indicators. In total, feedback was received from 264 individuals, including 131 physicians, 106 nurses, 10 social workers, four pharmacists and three chaplains, and 27 organizations, including the American Academy of Neurology, American Academy of Pain Medicine, American Academy of Pediatrics, American College of Emergency Physicians, American Geriatrics Society, Association of Professional Chaplains, Center to Advance Palliative Care, NHPCO, National Palliative Care Research Center, Oncology Nursing Society, Social Work Hospice & Palliative Network, and The Joint Commission.

Results Description of Selected Indicators The final MWM quality indicator set includes 10 indicators (Table 2). These include one in the NCP domain of Structure and Process of Care (Comprehensive Assessment), three in the domain of Physical Aspects of Care (Screening for Physical Symptoms, Pain Treatment, and Dyspnea Screening and Management), one in Psychological and Psychiatric Aspects of Care (Discussion of Emotional or Psychological Needs), one in Spiritual and Existential Aspects of Care (Discussion of Spiritual/Religious Concerns), and three in Ethical and Legal Aspects of Care (Documentation of Surrogate, Treatment Preferences, and Care Consistency with Documented Care Preferences). In addition, the MWM list recommends that a global indicator of care from the patient/family perspective be included, but the MWM process did not endorse a specific measure. Measures of the patient/caregiver experience that were considered and met the initial MWM criteria for the candidate set are described in Table 3. No indicators were included in the domains of Social Aspects of Care, Cultural Aspects of Care, and Care at the End of Life, highlighting gaps in areas where indicators may not be


supported by current evidence or may potentially not be appropriate for evaluating care. None of the MWM indicators are disease-specific and almost all apply to any patients receiving palliative or hospice care services, although one is for vulnerable elders (people older than 65 years at increased risk for death or functional disability). Most are from the PEACE data set and apply to hospitalized or hospice patients; one (Discussion of Spiritual/Religious Concerns) is for hospice only and one (Dyspnea Screening and Treatment) is for ambulatory care (by physicians). Most address processes of care. All indicators would require medical record review or duplicate data entry into a quality database, except for patient/ family perceptions of care. Only two are currently endorsed by the NQF (Discussion of Spiritual/ Religious Concerns and Treatment Preferences).

Discussion In summary, the MWM project, led by the AAHPM and HPNA with input from a wide variety of stakeholder and patient advocacy groups, assembled a consensus-based concise set of process and outcome quality indicators for internal current use for quality measurement and improvement in settings caring for hospice and palliative care patients. These indicators cover five of the eight domains of the NCP Palliative Care Guidelines and most are applicable to broad hospice and palliative care populations and more than one setting (hospital and hospice care); only one is for the ambulatory setting. Most are process indicators and require medical record abstraction or separate data entry. The MWM list also includes a global indicator for patient/family experience of care, although no consensus developed for a specific measurement tool (the measure for hospices, the Family Evaluation of Hospice Care, was being replaced at the time of this project by the Consumer Assessment of Healthcare Providers and Systems hospice survey) and the group recommended that additional measure development was needed. Some of these indicators have been used and tested more widely than others, and further refinement and evaluation are needed. Many supporting tools already exist for the PEACE indicators,12 and further development of accompanying tools for the indicator set, including details on standardizing data abstraction, measurement specifications, benchmarking, and evidence-based quality improvement, as well as potential recommendations for modifications of the indicators based on experience with them, is needed. The MWM panel recognized that quality indicators are needed both to evaluate the quality of hospice and palliative care programs and to evaluate the quality



777

Table 2 Measuring What Matters Top 10 Quality Indicators Set National Consensus Project Domain 1. Structure and Processes of Care

2. Physical Aspects of Care

3. Psychological and Psychiatric Aspects of Care 4. Social Aspects of Care 5. Spiritual, Religious, and Existential Aspects of Care

Indicator Comprehensive Assessment Hospice: Percentage of patients enrolled for more than seven days for whom a comprehensive assessment was completed within five days of admission (documentation of prognosis, functional assessment, screening for physical and psychological symptoms, and assessment of social and spiritual concerns). Seriously ill patients receiving specialty palliative care in an acute hospital setting: Percent of all patients admitted for more than one day who had comprehensive assessment (screening for physical symptoms and discussion of the patient/family’s emotional or psychological needs) completed within 24 hours of admission. Screening for Physical Symptoms Percentage of seriously ill patients receiving specialty palliative care in an acute hospital setting for more than one day or patients enrolled in hospice for more than seven days who had a screening for physical symptoms (pain, dyspnea, nausea, and constipation) during the admission visit. Pain Treatment For seriously ill patients receiving specialty palliative care in an acute hospital setting for more than one day or patients enrolled in hospice for more than seven days who screened positive for moderate-to-severe pain on admission, the percent with medication or non-medication treatment, within 24 hours of screening. Dyspnea Screening and Management Percentage of patients with advanced chronic or serious life-threatening illnesses who are screened for dyspnea. For those who are diagnosed with moderate or severe dyspnea, a documented plan of care to manage dyspnea exists (Ambulatory physician care). Discussion of Emotional or Psychological Needs Percentage of seriously ill patients receiving specialty palliative care in an acute hospital setting for more than one day or patients enrolled in hospice for more than seven days with chart documentation of a discussion of emotional or psychological needs. No indicators Discussion of Spiritual/Religious Concerns Percentage of hospice patients with documentation of a discussion of spiritual/ religious concerns or documentation that the patient/caregiver/family did not want to discuss. No indicators

Source PEACE Set25,26

PEACE Set25,26

PEACE Set25,26

AMA-PCPI/NCQA18

PEACE Set25,26

Deyta, LLC/NQF# 1647

6. Cultural Aspects of Care 7. Care of the Patient at No indicators the End of Life 8. Ethical and Legal Documentation of Surrogate PEACE Set25,26 Aspects of Care Percentage of seriously ill patients receiving specialty palliative care in an acute hospital setting for more than one day or patients enrolled in hospice for more than seven days with name and contact information for surrogate decision maker in the chart or documentation that there is no surrogate. Treatment Preferences PEACE Set/25,26NQF# 1641 Percentage of seriously ill patients receiving specialty palliative care in an acute hospital setting for more than one day or patients enrolled in hospice for more than seven days with chart documentation of preferences for life-sustaining treatments. Care Consistency with Documented Care Preferences ACOVE Palliative Care and If a vulnerable elder has specific treatment preferences (for example, a DNR order, End of Life14,15 no tube feeding, or no hospital transfer) documented in a medical record, then these treatment preferences should be followed. Global Measure Although no specific global measure was endorsed by the MWM process, the committee, panels, membership, and stakeholders agreed that patient and/or family assessments of the quality of care are a key part of measuring quality for any setting caring for palliative or hospice patients. Global measures that met the MWM selection criteria are described in Table 3. AMA-PCPI ¼ American Medical Association Physician Consortium on Performance Improvement; NCQA ¼ National Committee on Quality Assurance; NQF ¼ National Quality Forum; DNR ¼ Do Not Resuscitate; ACOVE ¼ Assessing Care of Vulnerable Elders14,15 Palliative Care and End of Life. More details on the measures are available at http://aahpm.org/quality/measuring-what-matters. The PEACE measures and supporting materials are available at http://www.med.unc.edu/pcare/resources/PEACE-Quality-Measures.

of palliative care provided more broadly within settings by all providers to patients with palliative care needs, and further testing should evaluate both of

these applications. Because the purpose of the project was to define cross-cutting indicators, initiatives targeting quality measurement in specific populations (e.g.,

778

Dy et al.


Table 3 Candidate Global Measures of Patient and Family Experience Considered by the Measuring What Matters Project Measure Family Evaluation of Hospice Care17 (FEHC; www.nhpco.org/fehcsurvey-materials) Family Evaluation of Palliative Care (FEPC; www.nhpco.org/fepcsurvey-materials)

Bereaved Family Survey (BFS; www.cherp.research.va.gov/ PROMISE/)

Consumer Assessments and Reports of End of Life27 (CARE; www.chcr. brown.edu/pcoc/toolkit.htm)

Description

Source

The FEHC survey is an after-death survey administered to bereaved family caregivers of individuals who died while enrolled in hospice A post-death survey that captures family members’ perceptions about the quality of the palliative care that their loved ones receiveddby a hospital-based consultant service or by a hospice program offering palliative care The purpose of this measure is to assess families’ perceptions of the quality of care that Veterans received from the VA in the last month of life. The BFS consists of 19 items (17 structured and two open-ended). The CARE survey is a mortality follow-back survey that is administered to the bereaved family members of adult persons (aged 18 years and older) who died of a chronic progressive illness receiving services for at least 48 hours from a home health agency, nursing homes, hospice, or acute care hospital.

Comments

NHPCO/NQF# 0208

A new measure for CMS reporting is currently being developed

NHPCO

The questions on the FEPC survey are based on those in the FEHC survey, with wording modifications appropriate to palliative care service delivery.

PROMISE Center/NQF# 1623

Used throughout the VA; includes questions specific to Veterans

Center for Gerontology and Health Care Research, Brown University/NQF# 1632

Requires administration by an interviewer.

NHPCO ¼ National Hospice and Palliative Care Organization; NQF ¼ National Quality Forum; CMS ¼ Centers for Medicare & Medicaid Services; PROMISE ¼ Performance Reporting and Outcomes Measurement to Improve the Standard of care at End-of-life; VA ¼ Veterans Administration.

cancer) or settings (e.g., nursing homes) may benefit from using more targeted indicators. In addition, the purpose of the MWM project was not to identify or test indicators for public reporting, which requires a different evaluation process; the NQF, which evaluates indicators for public reporting, has endorsed a number of indicators relevant to palliative and hospice care (only two of the MWM set are endorsed by the NQF).11 Finally, the MWM team emphasizes that quality measurement is only one small component of ensuring the best hospice and palliative caredattention to structure, program development, evidence-based practice, education, and quality improvement are all also key to improving care for patients and families. The MWM list includes cross-cutting indicators in five of the eight NCP domains, leaving social aspects of care, cultural aspects of care, and care of the imminently dying without any defined indicators in this set. Although a number of indicators in these domains were considered by the expert panels, the lack of included indicators in these domains in the final MWM set reflects the lack of scientific evidence and insufficient documentation in the medical record for feasible abstraction. In addition, some of these aspects, such as cultural aspects of care, are less practical for process and outcome quality measurement,

which requires issues that can be objectively measured in standard ways that reflect quality across populations. Potentially, structural indicators, such as the availability or penetration of chaplaincy services, may be more appropriate for these domains. Some of the domains, such as physical and ethical and legal aspects of care (which have the most indicators), are more likely to be documented and easier to objectively measure than others, such as social aspects of care. For example, whether or not a patient has lifesustaining treatment preferences documented in the chart is easier to objectively measure than whether or not the provider developed a comprehensive social care plan, addressing issues such as relationships and caregiver stress, as recommended by the NCP guidelines.6 The lack of included indicators in the domain of care of the imminently dying is a clear gap critical to hospice and palliative care, where more objective markers exist but further development is needed. Eight of the nine indicators considered in this domain were outcome measures of health care utilization (the ninth was family evaluation of bereavement services; the global indicators discussed previously also address this domain from the patient/caregiver perspective). In particular, two NQF-endorsed Quality Oncology Practice Initiative indicators for cancer patients,



whether or not a patient received hospice for less than three days10,19 and proportion with more than one emergency room visit in the last 30 days of life,19 were considered throughout the MWM process. However, many concerns about these two Quality Oncology Practice Initiative indicators were raised by the panels, membership, and stakeholder groups. These measures were tested using Medicare claims data and have been benchmarked only for cancer patients; and they require knowing about patients’ outcomes outside a health system and who were within the last 30 days of life, which is generally challenging data to obtain for palliative care providers not in hospice settings. Outcome indicators also are generally riskadjusted and have exclusions, which have not yet been developed for these indicators and could significantly affect outcomes. They also could be very dependent on local structure, such as availability of urgent care and the efficiency of hospice admission processes. Finally, these indicators may not accurately reflect patient and family perceptions of quality of care, especially across cultures, as studies have found that most caregivers feel that the length of stay in hospice was sufficient20 and that caregivers in Japan did not rank emergency department use highly as an indicator of quality of care.21 Ideal indicators for care at the end of life could include patient and family preferences for hospice care and the timing of initial hospice referral and admission accounting for whether any lag between referral and admission was patient preference or a system performance issue. On the other hand, outcomes such as hospice length of stay can be integrative and identify potential issues within a setting that need to be addressed (e.g., very low hospice stays could indicate that information is not being provided early enough, referral processes are too slow, or coordination with hospices is poor). However, these would need further detailed evaluation to identify the source of the issue before attempts to improve quality can be made. The MWM process and the indicator set have a number of limitations. The MWM set is based on an extensive process of expert consensus and diverse input, but will require ongoing efforts to further evaluate feasibility for routine data collection, usability for quality improvement projects, links between process and outcome indicators, and updates as new and revised indicators become available. Using indicators from the set will require careful monitoring for unintended negative consequences, such as focusing initiatives only on such areas where indicators are robust and neglecting important areas where performance may be more difficult to measure. Not all indicators may apply to all settings or populations, and

779

programs will need to select those that are most appropriate to their internal population, needs, and opportunities for improvement. Further research with benchmarking and evaluation of the impact of contextual factors, such as electronic health record system, academic status, or quality program structure, are needed to improve the usability of the indicators. None of the indicators have been evaluated in pediatrics, and evaluation specific to geriatric populations is limited. Patient/caregiver evaluations of care are critical to hospice and palliative care, as medical record review process indicators address only a small fraction of care and cannot measure many important issuesdincluding the quality of communication, coordination, respect, and attention to patient preferences. However, there is no universally agreed-on measure in palliative care; different measures have been developed and are used in different settings, and no current indicators we identified that met eligibility criteria crossed settings. The consensus and stakeholder input processes, therefore, did not endorse any specific measure, and further measure development is needed, but some process for assessing relevant patient/caregiver input should be included in settings providing care to palliative or hospice patients.

Conclusion In conclusion, the MWM Project of the AAHPM and HPNA has assembled a list of 10 consensus indicators for internal measurement of quality in settings serving hospice and palliative patients. Although this set serves as an initial starting place for standardizing U.S. measurement and benchmarks for care, more development and partnerships with organizations to develop benchmarking is needed, and additional or revised indicators may be needed for particular settings or populations. Further research is needed to evaluate how best to define palliative care populations of interest for quality measurement, and how to measure the quality of hospice and palliative care both overall in different settings and for patients served by hospice and palliative care services. Finally, as evaluation of this national U.S. indicator set occurs, consideration of including structural and newly developed indicators and coordination with lessons learned from individual hospice and palliative care programs in the U.S., relevant quality programs such as the ASCO QOPIÒ10 and other emerging international palliative care quality measurement projects, such as those in Australia,22 Belgium,23 and The Netherlands,24 will help to maximize the value of quality measurement in the U.S.

780

Dy et al.

Disclosures and Acknowledgments This project was funded by the American Academy of Hospice and Palliative Medicine, with additional support from the Hospice and Palliative Nurses Association. The authors acknowledge the members of the Technical Advisory Panel: Marie Bakitas, Teresa Craig, Mary Ersek, Chris Feudtner, Laura Hanson, Arif Kamal, Lisa Lindley, Karl Lorenz, Carol Spence, Martha Tecca, and Joan Teno; and Clinical User Panel: Michael Balboni, Patricia Berry, Cynthia Boyd, Caitlin Brennan, Janet Bull, Ira Byock, Barb Daley, Kenneth Doka, Jennifer Eurek, Joy Goebel, Elizabeth Gunderson, Krista Harrison, Joan Harrold, Jean Kutner, Thomas Lee, Suzana Makowski, Kelly McCutcheon Adams, Deirdre Mylod, Marsha Nelson, Ritika Oberoi-Jassal, Lynn Reinke, Michael Reynolds, Christine Ritchie, Eugenia Smither, Lisa Stephens, Rodney Tucker, Deborah Waldrop, and Joanne Wolfe. The authors also acknowledge June Lunney and Steve Smith for their support and guidance.

References 1. Institute of Medicine. Dying in America: Improving quality and honoring individual preferences near the end of life. Washington, DC: The National Academies Press, 2014. 2. Walling AM, Tisnado D, Asch SM, et al. The quality of supportive cancer care in the veterans affairs health system and targets for improvement. JAMA Intern Med 2013;173: 2071e2079. 3. Fisch MJ, Lee JW, Weiss M, et al. Prospective, observational study of pain and analgesic prescribing in medical oncology outpatients with breast, colorectal, lung, or prostate cancer. J Clin Oncol 2012;30:1980e1988. 4. Teno JM, Clarridge BR, Casey V, et al. Family perspectives on end-of-life care at the last place of care. JAMA 2004;291:88e93. 5. Dy S, Aslakson R, Wilson R, et al. Improving health care and palliative care for advanced and serious illness. Closing the quality gap: revisiting the state of the science. Evidence Report No. 208. (Prepared by the Johns Hopkins University Evidence-based Practice Center under Contract No. 2902007-10061-I.) AHRQ Publication No. 12(13)-E014-EF. Rockville, MD: Agency for Healthcare Research and Quality, 2012. 6. National Consensus Project for Quality Palliative Care. Clinical Practice Guidelines for Quality Palliative Care, 3rd ed. 2013; Available at: http://www.nationalconsensusproject.org. Accessed September 1, 2014.


9. Nelson JE, Mulkerin CM, Adams LL, Pronovost PJ. Improving comfort and communication in the ICU: a practical new tool for palliative care performance measurement and feedback. Qual Saf Health Care 2006;15:264e271. 10. Campion FX, Larson LR, Kadlubek PJ, Earle CC, Neuss MN. Advancing performance measurement in oncology: quality oncology practice initiative participation and quality outcomes. J Oncol Pract 2011;7:31se35s. 11. National Quality Forum. Palliative and end-of-life care performance measures. Available at: www.qualityforum.org. Accessed September 1, 2014. 12. The Carolinas Center for Medical Excellence (CCME). PEACE Hospice and Palliative Care quality measures. Available at: http://www.med.unc.edu/pcare/resources/PEACEQuality-Measures. Accessed May 23, 2014. 13. Dy SM, Lorenz KA, O’Neill SM, et al. Cancer qualityASSIST supportive oncology quality indicator set: feasibility, reliability, and validity testing. Cancer 2010;116:3267e3275. 14. Lorenz KA, Rosenfeld K, Wenger N. Quality indicators for palliative and end-of-life care in vulnerable elders. J Am Geriatr Soc 2007;55:S318eS326. 15. Walling AM, Asch SM, Lorenz KA, et al. The quality of care provided to hospitalized patients at the end of life. Arch Intern Med 2010;170:1057e1063. 16. Connor SR, Tecca M, LundPerson J, Teno J. Measuring hospice care: the National Hospice and Palliative Care Organization national hospice data set. J Pain Symptom Manage 2004;28:316e328. 17. Connor SR, Teno J, Spence C, Smith N. Family Evaluation of Hospice Care: results from voluntary submission of data via website. J Pain Symptom Manage 2005;30:9e17. 18. Wenger N, Dy SM, Casarett D, et al. American Medical Association. Physician Consortium on Performance Improvement palliative and end of life care physician performance measurement set 2008; Available at: http://www.amaassn.org/ama/pub/physician-resources/physician-consortiumperformance-improvement/pcpi-measures.page. Accessed September 4, 2014. 19. Earle CC, Park ER, Lai B, et al. Identifying potential indicators of the quality of end-of-life cancer care from administrative data. J Clin Oncol 2003;21:1133e1138. 20. Teno JM, Shu JE, Casarett D, Spence C, Rhodes R, Connor S. Timing of referral to hospice and quality of care: length of stay and bereaved family members’ perceptions of the timing of hospice referral. J Pain Symptom Manage 2007;34:120e125. 21. Miyashita M, Morita T, Ichikawa T, et al. Quality indicators of end-of-life cancer care from the bereaved family members’ perspective in Japan. J Pain Symptom Manage 2009;37:1019e1026.

7. Center for Medicare & Medicaid Services. Hospice quality reporting. Available at: http://www.cms.gov/Medicare/ Quality-Initiatives-Patient-Assessment-Instruments/HospiceQuality-Reporting/index.html. Accessed September 23, 2014.

22. Currow DC, Allingham S, Yates P, et al. Improving national hospice/palliative care service symptom outcomes systematically through point-of-care data collection, structured feedback and benchmarking. Support Care Cancer 2015;23: 307e315.

8. Hanson LC, Scheunemann LP, Zimmerman S, Rokoske FS, Schenck AP. The PEACE project review of clinical instruments for hospice and palliative care. J Palliat Med 2010;13:1253e1260.

23. Leemans K, Deliens L, Francke AL, et al. Quality indicators for palliative care services: mixed-method study testing for face validity, feasibility, discriminative power and usefulness. Palliat Med 2015;29:71e82.



24. Claessen SJ, Francke AL, Sixma HJ, de Veer AJ, Deliens L. Measuring relatives’ perspectives on the quality of palliative care: the Consumer Quality Index Palliative Care. J Pain Symptom Manage 2013;45:875e884. 25. Hanson LC, Rowe C, Wessell K, et al. Measuring palliative care quality for seriously ill hospitalized patients. J Palliat Med 2012;15:798e804.

781

26. Schenck AP, Rokoske FS, Durham D, Cagle JG, Hanson LC. Quality measures for hospice and palliative care: piloting the PEACE measures. J Palliat Med 2014;17:769e775. 27. Teno JM, Clarridge B, Casey V, Edgman-Levitan S, Fowler J. Validation of Toolkit After-death bereaved family member Interview. J Pain Symptom Manage 2001;22: 752e758.

Appendix Example Questions From the TAP and CUP Surveys

Developing a care model for hospice and palliative nurses.

Hospice and palliative care: differences matter.

The Island Hospice model of palliative care.

Hospice and Palliative Care for African Americans: Overcoming Disparities.

Patients' perceptions of palliative care quality in hospice inpatient care, hospice day care, palliative units in nursing homes, and home care: a cross-sectional study.

Are geriatric medicine fellows prepared for the important skills of hospice and palliative care?

American Board of Radiology subspecialty certification in hospice and palliative medicine.

Quality measures for hospice and palliative care: piloting the PEACE measures.

Palliative medicine education: bridging the gap between acute care and hospice.

Using Agency Nurses to Fill RN Vacancies Within Specialized Hospice and Palliative Care.

Palliative care development in the Asia-Pacific region: an international survey from the Asia Pacific Hospice Palliative Care Network (APHN).

Cost savings and enhanced hospice enrollment with a home-based palliative care program implemented as a hospice-private payer partnership.

Benchmarking the provision of palliative rehabilitation within the hospice setting.

[The current state, the possibilities and difficulties of palliative and hospice care in Hungary].

Hospice palliative care volunteers: the benefits for patients, family caregivers, and the volunteers.

Use of palliative care and hospice among surgical and medical specialties in the Veterans Health Administration.

Prevalence and Predictors of Burnout Among Hospice and Palliative Care Clinicians in the U.S.

Pediatric hospice and palliative care: designing a mobile app for clinical practice.

Antimicrobial use for symptom management in patients receiving hospice and palliative care: a systematic review.

An overview of hospice and palliative care nursing models and theories.

Integration of legal aspects and human rights approach in palliative care delivery-the Nyeri Hospice model.

The Influence of Nursing Unit Characteristics on RN Vacancies in Specialized Hospice and Palliative Care.

Functional Assessment and Intervention by Nursing Assistants in Hospice and Palliative Care Inpatient Care Settings: A Quality Improvement Pilot Study.

Hearing Loss in Hospice and Palliative Care: A National Survey of Providers.